Cloud application health checks determine whether a service receives traffic, remains running, and has finished starting. This guide explains readiness, liveness, and startup probes; compares HTTP, TCP, and command checks; provides provider-neutral configuration examples; and gives you a repeatable workflow for diagnosing failed checks, restart loops, and unsafe rollout behavior.
Overview
Health checks are simple signals with significant consequences. A failed readiness check can remove an instance from service without stopping it. A failed liveness check can cause the platform to restart an otherwise recoverable process. A startup check can prevent slow initialization from being mistaken for a dead application.
The most important distinction is intent:
- Readiness asks, “Can this instance safely receive traffic now?”
- Liveness asks, “Is this process still functioning, or should it be restarted?”
- Startup asks, “Has this process finished its initial startup?”
These checks should measure conditions the platform can act on. A readiness endpoint may verify that essential dependencies are available, while a liveness endpoint should usually avoid checking every external dependency. If a database has a short outage, removing an instance from traffic may be appropriate; repeatedly restarting every instance may make the outage worse.
Health checks are one part of a broader cloud app development workflow. Configuration changes, secret injection, image contents, permissions, and deployment behavior can all affect probe results. For related configuration practices, see the Kubernetes configuration management guide and the CI/CD pipeline failure troubleshooting guide.
What to track
Probe purpose and expected action
| Probe type | What it should measure | Typical platform action | Design guidance |
|---|---|---|---|
| Readiness | Whether the instance can serve requests correctly | Stop routing traffic to the instance | Include required service conditions, but keep the check fast and deterministic. |
| Liveness | Whether the process is responsive and able to make progress | Restart the instance or container | Do not make routine dependency failures trigger restarts unless that is deliberate. |
| Startup | Whether initial boot and application initialization have completed | Delay liveness and readiness evaluation | Allow for migrations, cache warming, framework initialization, or large dependency loading. |
HTTP, TCP, and command checks
HTTP checks are generally the clearest option for web applications. They can target a dedicated path such as /health/ready and return a status code that represents the result. Keep the response small and avoid authentication requirements that depend on a user session. If the endpoint exposes diagnostic details, restrict those details rather than returning credentials, tokens, connection strings, or sensitive exception data.
TCP checks confirm that a port accepts connections. They are useful when an application does not expose an HTTP endpoint, but they provide limited evidence. A listening socket does not prove that requests are processed correctly, that dependencies are available, or that the application is ready for production traffic.
Command checks run a process-local command or script. They can test a worker, inspect a local file, or invoke an application-specific diagnostic. They also introduce shell behavior, executable permissions, runtime dependencies, and quoting problems. Keep command checks short, version-controlled, and tested inside the same image or runtime environment as the application.
Settings worth recording
For every probe, document the path or command, port, expected success condition, initial delay, check interval, timeout, and failure threshold. A practical starting point for a typical web service is a short interval, a timeout comfortably above normal endpoint latency, and several consecutive failures before action. These are starting assumptions, not universal defaults. Measure startup time and normal response behavior in your own environment before tuning them.
Track these operational signals alongside probe status:
- Time from process launch to successful startup.
- Readiness transitions during deployments and dependency incidents.
- Restart count and the time between restarts.
- Probe response latency, status codes, and timeout frequency.
- Application logs immediately before and after each transition.
- Deployment revision, image version, configuration revision, and node or host placement.
Example deployment configuration
The following Kubernetes-style example illustrates the separation of responsibilities. Field names and supported options should be checked against the version of the platform you operate.
containers:
- name: web
image: registry.example.invalid/orders:stable
ports:
- name: http
containerPort: 8080
startupProbe:
httpGet:
path: /health/startup
port: http
periodSeconds: 5
timeoutSeconds: 2
failureThreshold: 24
readinessProbe:
httpGet:
path: /health/ready
port: http
periodSeconds: 10
timeoutSeconds: 2
failureThreshold: 3
livenessProbe:
httpGet:
path: /health/live
port: http
periodSeconds: 15
timeoutSeconds: 3
failureThreshold: 3
terminationGracePeriodSeconds: 30
In this example, the startup budget is approximately the startup interval multiplied by its failure threshold, subject to platform behavior. The application should also handle termination signals, stop accepting new work during shutdown, and allow in-flight requests to finish within the available grace period.
Cadence and checkpoints
Health checks should be reviewed when a service changes, not only when an incident occurs. Use the following cadence as a lightweight operating routine.
At every deployment
- Confirm that each probe path or command exists in the new image.
- Verify that the configured port, scheme, and hostname match the application.
- Check startup duration under realistic resource limits.
- Watch readiness transitions, restarts, and rollout progress.
- Confirm that the previous version drains connections before termination.
Monthly or quarterly
Review probe failures, false positives, slow starts, and restarts by service and deployment version. Compare probe latency with application latency. A probe that normally completes close to its timeout is fragile even if it has not failed yet. Check whether endpoints still reflect the service contract and whether operators can distinguish a dependency outage from a process failure.
Use a small review record containing the service name, probe type, endpoint or command, current thresholds, observed startup time, last change, owner, and next review date. This turns health checks into maintainable configuration rather than undocumented deployment folklore.
After platform or framework changes
Recheck syntax, default behavior, termination handling, probe scheduling, and event reporting after upgrading the orchestrator, cloud runtime, web framework, server, or base image. Refresh examples in internal runbooks at least every six months and sooner after a major release. Do not assume that a configuration copied from another platform has identical semantics.
How to interpret changes
When a health check is failing, start with the action the platform took. A service removed from traffic but still running points first toward readiness. Repeated process restarts point toward liveness, startup timeout, application crashes, resource pressure, or infrastructure termination.
Provider-neutral troubleshooting decision tree
- Is the instance being restarted? If yes, inspect liveness and startup results, process exit codes, out-of-memory events, CPU throttling, and termination reasons.
- Is the instance running but receiving no traffic? Inspect readiness status, endpoint response codes, service selectors, registration state, and rollout conditions.
- Does the endpoint respond locally inside the instance? If no, investigate application startup, binding address, route registration, dependency initialization, and local logs.
- Does it respond locally but fail from the platform? Compare port names and numbers, network policy, security rules, TLS expectations, DNS, sidecars, and the platform's source address.
- Does it fail only during startup? Increase startup allowance only after measuring the cause. Check migrations, secret loading, configuration parsing, image size, external calls, and resource availability.
- Does it fail intermittently? Correlate timestamps with latency spikes, dependency errors, garbage collection, connection pool exhaustion, node events, autoscaling, and deployment activity.
Application failure and infrastructure failure often look similar at the probe layer. Application logs can reveal a failed bind, invalid configuration, or unhandled exception. Platform events can reveal an image pull problem, permission error, failed attachment, eviction, or resource limit. If the application cannot retrieve a secret or configuration value, investigate both sides: application startup logs and the identity or permission path. The cloud IAM permission errors guide can help separate authorization failures from health endpoint defects.
For API services, avoid using a health endpoint as a substitute for contract validation. A service can be “ready” while returning incompatible responses. Pair operational checks with automated API tests, such as the practices described in the OpenAPI contract testing guide.
When to revisit
Revisit health-check settings after any event that changes startup time, request handling, dependencies, or the platform's ability to reach the process. The most important triggers are:
- A new runtime, framework, base image, server, or application architecture.
- A database, queue, cache, identity provider, or third-party integration change.
- A migration, cache-warming step, large model load, or other longer initialization task.
- Repeated rollout failures, restart loops, false alarms, or unexpected traffic removal.
- Changes to CPU, memory, networking, autoscaling, service mesh, ingress, or security policy.
- A major orchestrator or cloud-platform release that may alter probe syntax or defaults.
- A monthly or quarterly reliability review, even when no incident occurred.
Use this practical review sequence: first record the current configuration and recent probe metrics; then reproduce the endpoint locally and inside the deployed image; next inspect application logs and platform events for the same timestamps; finally change one threshold or behavior at a time and observe a complete rollout. Treat increased thresholds as a temporary experiment unless measurements justify making them permanent.
Before closing the review, confirm that readiness prevents bad traffic, liveness detects genuine process failure, startup protects slow initialization, and shutdown drains work safely. Record the reason for each setting, assign an owner, and set the next review date. This keeps cloud application health checks aligned with the service they are meant to protect.