Readiness Probes in Kubernetes

Readiness Probes in Kubernetes

A plain-English guide to telling Kubernetes when your application is truly ready for visitors, so users never meet a half-started server or a struggling pod.

Imagine a restaurant that unlocks its front door at nine in the morning, exactly as the sign says. But the kitchen staff are still arriving, the ovens are cold, and half the tables are not laid. A customer walks in at five past nine, sits down, and waits forty minutes for a glass of water. The restaurant was technically open. It was not ready.

Software has the same problem. A container can be running, and its process can be alive, and yet the application inside may still be loading data, warming up its caches, connecting to a database, or simply catching its breath after a heavy burst of work. If Kubernetes sends visitors to such a pod, they get errors, slow pages, or dropped connections.

A readiness probe solves this problem. It is a small, regular question that Kubernetes asks your container: “Are you ready to serve visitors right now?” As long as the answer is yes, traffic flows. When the answer becomes no, Kubernetes politely stops sending traffic until the answer changes back. In this guide you will learn exactly how this works, how to design good answers, and how to avoid the classic mistakes. No earlier knowledge is needed, and every new word is explained the first time it appears.

1Why “Running” Does Not Mean “Ready”

Being alive and being ready to help are two very different states, and Kubernetes cannot guess the difference on its own.

When Kubernetes starts a container, it can easily see whether the program has begun running. It can also see whether the program has crashed. What it cannot see, without your help, is what the program is doing inside. Is it still reading a large file? Is it waiting for a database to answer? Is it in the middle of a long clean-up? To the outside, all of these look the same: a healthy, running process.

Without further information, Kubernetes assumes that a container is ready as soon as it starts. That is a safe guess for a tiny program that starts in a blink. It is a poor guess for the typical real application, which needs seconds or even minutes before it can serve its first visitor properly.

Everyday analogy

Think of a taxi that has just arrived at the pick-up point. The engine is on, so the taxi is “running”. But the driver is still finishing a phone call, the meter has not started, and the trunk is full of the last passenger’s luggage. If the dispatcher sends a customer to that taxi now, the customer has a bad experience. The taxi needs a green light on the roof that says “available”. The readiness probe is that green light.

Common reasons a running pod is not ready

Startup

Still loading

The application is reading settings, loading large files, or building an index. It cannot answer requests properly yet.

Startup

Cold caches

The program works but is very slow until its memory of recent data warms up. Early visitors would feel the delay.

Dependencies

Waiting for a database

The application needs another service that is still starting. Without it, every request would fail.

Overload

Temporarily swamped

A running pod may be so busy that it cannot take another request. Sending more traffic makes things worse.

Maintenance

Doing a clean-up

The program is compacting data or rebuilding something and asks visitors to wait elsewhere for a while.

Shutdown

Finishing up

The pod is about to stop. It should stop accepting new visitors and finish the ones it has.

i
Good to know

Readiness is not a one-time event. A pod can be ready, become unready, and become ready again many times in its life. It is a continuing conversation, not a single gate.

2What Is a Readiness Probe?

A readiness probe is a repeated health question that decides whether a pod may receive traffic.

A probe is a small test that Kubernetes runs against your container on a schedule. The word comes from a doctor’s probe: a tool used to check on something inside a body without opening it up. A readiness probe is the kind of probe whose only job is to answer one question: should this pod be receiving requests from visitors right now?

You describe the test in the pod’s description. You say what to check, such as “ask this web address” or “try this network port”. You also say how often, how long to wait for an answer, and how many failures to tolerate. Kubernetes then runs the test over and over, for as long as the container lives, and acts on the results.

Everyday analogy

Consider the “open” sign on a shop door. The owner flips it to “closed” when the shop is restocking, when the till is broken, or when it is too crowded inside. Customers who walk by simply go to another shop. Nobody is angry, and nobody’s day is ruined. The shop is not demolished, and the owner is not fired. The sign just says: not right now. A readiness probe is that sign, flipped automatically.

What it is, why it exists, where it is used

What it is: a repeated check, defined per container, that marks a pod as ready or not ready for traffic.

Why it exists: so that visitors reach only the pods that can serve them well, and so that updates and restarts do not cause errors.

Where it is used: in nearly every production web service, API, and worker that receives traffic. It is one of the most important settings in a reliable Kubernetes system.

!
Common trap

A failing readiness probe does not restart the container. It only removes the pod from traffic. Beginners often expect a restart and are surprised when the unready pod just sits there. Restarting is the job of a different probe, which you will meet shortly.

3A Quick Kubernetes Refresher

A few basic words are enough to follow the rest of this guide.

Kubernetes is a system that runs your containers across many machines and keeps them healthy. You say what you want, and Kubernetes makes it true and keeps it true. Here are the parts that matter for probes.

Building block

Cluster

The whole group of machines managed together. Think of it as a large restaurant chain.

Building block

Node

One machine in the cluster. Each node is a single restaurant branch.

Building block

Pod

The smallest unit Kubernetes runs. A pod holds one or more containers. Think of it as a single serving station.

Building block

kubelet

The manager program on every node. It starts containers and, importantly here, runs the probes.

Building block

Service

A stable front desk that spreads visitors across a group of matching pods. It keeps a list of pods that are ready.

Building block

Deployment

A manager that keeps the right number of identical pods running and updates them gradually when you release a new version.

How a Service finds its pods

A Service does not memorize pod addresses. It uses a list, maintained automatically, called the endpoints of the Service. Each entry in the list is a pod that matches the Service and is currently able to receive traffic. When a pod becomes unready, its entry is marked as not ready, and the Service stops sending it visitors. This list is the bridge between a readiness probe and real traffic, and you will see it again and again in this guide.

Three kinds of probes

Kubernetes offers three probes. They look alike but have very different jobs. The readiness probe decides whether a pod receives traffic. The liveness probe decides whether a container should be restarted. The startup probe gives slow-starting applications extra time before the other two begin. The next chapters treat each in turn, with the readiness probe as the star.

4Architecture and Components

A readiness answer starts inside one container and ends up changing where visitors are sent across the whole cluster.

The diagram below shows the path. The kubelet on the node asks the container the health question. It reports the answer to the API server, the cluster’s front door. Other parts of Kubernetes notice the change and update the list of ready pods. Finally, the components that route traffic read that list and adjust.

flowchart LR
    subgraph Node["Worker Node"]
        Kubelet["kubelet prober"] -->|asks the health question| App["Container"]
        Kubelet -->|reports Ready or not| API
    end
    API["API Server"] --> Slice["EndpointSlice controller"]
    Slice --> Proxy["kube-proxy and load balancers"]
    Proxy -->|sends traffic only to ready pods| App
    Client["Users"] --> Proxy
    
Figure 1: The kubelet asks, the control plane records, and traffic follows the answer.

Meet the components

On the node

kubelet

Runs the probe on schedule, counts successes and failures, and reports the pod’s readiness. It is the only part that talks to the container directly.

In the pod

Your container

Answers the probe. It might serve a small health address, accept a network connection, or run a check command.

Control plane

API server

Stores the pod’s status, including whether it is ready, and tells interested parties when it changes.

Control plane

Endpoint controller

Watches pod status and keeps each Service’s list of ready pods correct.

Traffic layer

kube-proxy

Sets up forwarding rules on every node using only the ready entries from the list.

Traffic layer

Ingress and load balancers

Front doors that route outside visitors. Many follow the same ready list, and some add their own health checks on top.

i
Key idea

The probe never talks to visitors. It only talks to the kubelet. The rest of the change happens because the control plane reacts to the kubelet’s report.

5Readiness, Liveness, and Startup Probes Compared

Three probes, three different questions, three different consequences.

ProbeThe question it asksWhat happens on failureRuns when
ReadinessCan you serve visitors right now?Pod is removed from traffic. Nothing is restartedThroughout the container’s life
LivenessAre you stuck or broken beyond recovery?The container is killed and restartedThroughout the container’s life
StartupHave you finished starting up?The container is restarted if it takes too longOnly at the beginning, until it first passes
Everyday analogy

Think of a chef in a restaurant kitchen. The startup check asks whether the chef has arrived and finished setting up. The readiness check asks whether the chef can take another order right now, which may be no during a rush. The liveness check asks whether the chef is conscious. If the answer to the last question is no, an ambulance is called, which is the restart. You would never call an ambulance just because the chef is busy.

flowchart TD
    Start["Container starts"] --> SP{"Startup probe configured"}
    SP -->|Yes| SW["Wait for the startup probe to pass"]
    SP -->|No| RUN["Liveness and readiness begin"]
    SW -->|Passes| RUN
    SW -->|Fails for too long| KILL["Container is restarted"]
    RUN --> LV["Liveness failing means restart"]
    RUN --> RD["Readiness failing means no traffic"]
    
Figure 2: The startup probe holds the other two back until the application has finished starting.

Why keeping them separate matters

Mixing up these probes causes some of the nastiest incidents in Kubernetes. If a liveness probe is used for something that is really a readiness matter, such as a slow database, Kubernetes restarts healthy pods again and again, which makes the slowdown worse. If a readiness probe is used where a liveness probe is needed, a truly frozen program sits idle forever, never restarted. Each probe must ask the right question.

!
Watch out

Never point the liveness probe at something that depends on another service. If the database is down and every pod’s liveness probe asks the database, every pod is restarted at once, turning a small outage into a large one.

6Four Ways to Ask the Question

Kubernetes can ask your container the health question in four different styles. Pick the one that fits your program.

StyleWhat Kubernetes doesCounts as success whenBest for
HTTP requestSends a web request to an address and port in the containerThe answer code is 200 or above and below 400Web services and APIs
TCP connectionTries to open a network connection to a portThe connection opensDatabases, caches, and other non-web servers
CommandRuns a small program inside the containerThe program exits with the number zeroPrograms with a built-in status tool
gRPC callCalls the standard health service of a gRPC serverThe service reports that it is servingServices that use gRPC

HTTP: the most common choice

Most web applications offer a tiny address dedicated to health, often named something like “ready” or “health”. Kubernetes sends a request to it. If the application replies with a success code, the probe passes. If it replies with an error code, or does not reply in time, the probe fails. This style is popular because it lets the application itself decide what “ready” means.

TCP: only checks that a door is open

A TCP probe simply tries to connect to a port. If something is listening, it passes. This is a weak test, because a program may accept connections long before it can do real work. It is fine for simple servers, but be aware that it only proves the door is open, not that the shop is ready.

Command: for programs without a web address

Some programs have a small tool that reports their status. Kubernetes can run that tool inside the container and read its exit number, where zero means success. This style costs more, since a new process starts at every check. Use it when HTTP or TCP cannot express readiness.

gRPC: a native option

Services built on the gRPC framework follow a standard health-checking convention. Kubernetes can call it directly, with no extra adapter program needed. It is available in current versions of Kubernetes.

Everyday analogy

Imagine checking on a friend who is cooking. You could shout “Is dinner ready?” and listen for an answer, which is the HTTP style. You could check whether the kitchen light is on, which is the TCP style: it proves someone is there, but not that dinner is done. You could send a helper into the kitchen to inspect the oven, which is the command style. Each works. Some tell you more than others.

i
Practical hint

Prefer the HTTP style when you can. It is cheap, it is easy to test from your own machine, and it lets your application report a meaningful answer rather than just an open port.

7The Timing Settings

Five numbers control how patient or nervous a probe is. Getting them right is half the art.

SettingPlain meaningDefaultEffect if too smallEffect if too large
Initial delaySeconds to wait after the container starts before the first check0Early checks fail while the app is still startingPod is slow to receive traffic
PeriodSeconds between checks10More load, faster reactionSlow reaction to changes
TimeoutSeconds Kubernetes waits for one answer1Slow but healthy answers count as failuresA stuck pod holds traffic longer
Success thresholdPasses in a row needed to become ready again1A single lucky answer flips the statePod is slow to return to service
Failure thresholdFailures in a row needed to become not ready3One hiccup removes a good podA failing pod keeps receiving traffic
flowchart TD
    S["Container starts"] --> D["Wait for the initial delay"]
    D --> P["Probe runs every period"]
    P --> Q{"Probe passes"}
    Q -->|Yes so count a success| SU{"Reached the success threshold"}
    SU -->|Yes| R["Pod is marked Ready"]
    SU -->|No| P
    Q -->|No so count a failure| F{"Reached the failure threshold"}
    F -->|Yes| N["Pod is marked Not Ready"]
    F -->|No| P
    
Figure 3: Consecutive results, not single results, change the pod’s state.

Doing the arithmetic

To find how quickly a pod is removed from traffic after it breaks, multiply the period by the failure threshold. With the defaults, that is ten seconds times three, which is about thirty seconds. Add up to a few more seconds for the report to travel through the cluster. During those seconds, the broken pod still receives traffic. If thirty seconds of errors is too much, shorten the period or lower the threshold, at the price of more sensitivity to random blips.

Everyday analogy

It is like a smoke alarm. If it goes off at the first tiny wisp, you get false alarms every time someone makes toast. If it waits until the room is thick with smoke, you are in danger before it warns you. The thresholds are the alarm’s sensitivity dial.

!
Common trap

The default timeout of one second is short. A busy application, or a pod whose processor time is heavily limited, may take longer to answer even when it is perfectly healthy. Slow answers then look like failures, and pods flap in and out of service for no real reason.

8How It Works Internally

Between one failed check and a change in traffic, a short chain of events unfolds.

sequenceDiagram
    participant K as kubelet
    participant C as Container
    participant A as API Server
    participant E as EndpointSlice controller
    participant P as kube-proxy
    K->>C: Readiness check
    C-->>K: Fails with an error
    K->>K: Count failures until the threshold
    K->>A: Report pod not Ready
    A->>E: Pod status changed
    E->>A: Mark the endpoint not ready
    A->>P: Updated endpoint list
    P->>P: Stop sending traffic to this pod
    K->>C: Readiness check again
    C-->>K: Succeeds
    K->>A: Report pod Ready
    A->>E: Pod status changed
    E->>A: Mark the endpoint ready
    A->>P: Updated endpoint list
    P->>P: Resume sending traffic
    
Figure 4: A failure and a recovery, from the first check to the change in traffic.
1

The kubelet runs a worker for each probe

For every probe defined in every pod on the node, the kubelet keeps a small worker that repeats the check on schedule and remembers the latest results.

2

Results are counted, not trusted one by one

The worker tracks how many results in a row were failures or successes and compares them with the thresholds. One odd result changes nothing.

3

The container is marked ready or not ready

When a threshold is reached, the kubelet flips the readiness of that container. A pod is fully ready only when all of its containers are ready.

4

The pod’s condition is updated

The pod carries a small set of named conditions. The kubelet sets the one named Ready and the one named ContainersReady, then reports the new status to the API server.

5

The endpoint list is rewritten

The endpoint controller sees the changed condition and marks that pod’s entry as not ready in every Service that includes it.

6

Traffic rules are refreshed

kube-proxy and other traffic components read the updated list and rewrite their rules. Visitors are no longer sent to that pod.

i
Key idea

There is a short delay between a failure and the moment traffic stops. It is the sum of the probe timing, the reporting steps, and the time for rules to spread. Design your application to tolerate a few seconds of extra traffic after it begins to fail.

9What Readiness Controls: Traffic, Updates, and Order

Readiness affects far more than visitors. It also steers software updates and startup order.

Traffic

Services

A Service sends visitors only to ready pods. An unready pod stays in the group but receives nothing.

Traffic

Ingress and load balancers

Front doors that follow the Service’s ready list stop routing to unready pods. Some cloud load balancers also run their own separate health checks.

Updates

Rolling updates

A Deployment only counts a new pod as available once it is ready. It removes old pods only as new ones become ready.

Order

StatefulSets

In the default ordered mode, the next member starts only after the previous one is running and ready. Readiness sets the pace.

Scaling

Autoscalers

Unready pods are typically ignored when measuring load, so the autoscaler does not misjudge a pod that is still warming up.

Status

Dashboards and tools

Tools that show “how many copies are available” count only ready pods, so readiness shapes what you see.

An unready pod still exists

A pod that fails its readiness probe is not deleted, not restarted, and not moved. It simply sits outside the traffic. Its logs remain available, and you can still look inside it. This is very useful for debugging, since the broken pod stays around for investigation while healthy pods serve users.

Direct access still works

Readiness controls traffic that goes through Services and the components built on them. It does not build a wall around the pod. Anyone who knows the pod’s own address can still call it directly. Readiness is a routing signal, not a security fence.

!
Watch out

If all pods behind a Service become unready at once, the Service has nobody to send visitors to, and users see errors. A readiness design that can turn every pod off together is dangerous. The next chapters explain how to avoid this.

10The Life of a Pod’s Readiness

A pod moves between a few clear states during its life, and the probe is the switch.

stateDiagram-v2
    [*] --> NotReady: Pod starts
    NotReady --> Ready: Probe passes enough times
    Ready --> NotReady: Probe fails enough times
    Ready --> Terminating: Pod is deleted
    NotReady --> Terminating: Pod is deleted
    Terminating --> [*]
    
Figure 5: A pod can move between Ready and Not Ready many times before it ends.
1

The pod is created and its containers start

At first, the pod is not ready. With a readiness probe configured, it stays that way until the probe passes. Without one, it is considered ready the moment its containers run.

2

The application initializes

It loads data, opens connections, and warms up. The probe fails during this time, which is exactly the point. Visitors are kept away.

3

The probe passes and the pod becomes ready

After the required number of passes, the pod is added to the ready list, and traffic begins to arrive.

4

Normal service

The probe keeps running quietly in the background. The pod handles visitors and remains in the list.

5

Trouble: temporary unreadiness

A dependency slows down, a burst of work arrives, or the app pauses to clean up. Probes fail, the pod leaves the list, and traffic goes to its siblings.

6

Recovery

When the problem passes, probes succeed again, and the pod returns to the list without any restart.

7

Shutdown

When the pod is deleted or replaced, Kubernetes marks it as terminating and removes it from the list. The application then has a grace period to finish its work and exit.

i
Practical hint

Notice that the pod never restarts in steps five and six. Readiness lets an application take a breath and come back, which a restart cannot do. This makes it ideal for temporary trouble.

11Readiness and Rolling Updates

Readiness is what makes a software release smooth instead of scary.

When you release a new version of an application, Kubernetes does not switch everything at once. A Deployment replaces the old pods gradually. It starts a new pod, waits for it to be ready, and only then removes an old one. This gentle handover is called a rolling update, and readiness is the signal that makes it safe.

flowchart LR
    A["Old pods serve all traffic"] --> B["A new pod starts"]
    B --> C{"New pod passes readiness"}
    C -->|Yes| D["Traffic shifts and one old pod is removed"]
    D --> E["Repeat until every pod is new"]
    C -->|No for a long time| F["Rollout stalls and old pods keep serving"]
    
Figure 6: Readiness protects users during a release, because a bad new version never takes over.
Everyday analogy

Imagine a relay race where the incoming runner must hold the baton firmly before the outgoing runner lets go. If the handover happens before the new runner is ready, the baton falls. The readiness probe is the moment the new runner says, “I have it.” Only then does the old runner step out of the lane.

The settings that shape a rollout

SettingWhat it controlsDefault
Maximum surgeHow many extra pods may exist above the target during the updateA quarter of the desired count
Maximum unavailableHow many pods may be missing or unready during the updateA quarter of the desired count
Minimum ready secondsHow long a new pod must stay ready before it counts as availableZero
Progress deadlineHow long the Deployment waits for progress before reporting a failureTen minutes

What a stuck rollout looks like

If a new version has a bug that stops it from ever becoming ready, the rollout stops. The new pods stay unready. The old pods keep serving users, so nobody notices an outage, but the release never finishes. After the progress deadline, the Deployment reports that it has failed to make progress. This is the system working exactly as designed: a broken release is quarantined by readiness before it can hurt anyone.

!
Watch out

Without a readiness probe, Kubernetes treats every new pod as ready the moment it starts. A broken version then replaces the good one at full speed, and users see errors. A missing readiness probe turns your safe rollout into a gamble.

i
Practical hint

Use the minimum ready seconds setting to require that a new pod stays ready for a short time before the rollout moves on. It catches applications that pass the first check and fail a moment later.

12Designing What “Ready” Should Mean

The hardest part of a readiness probe is not the settings. It is deciding what the application should check.

Your application decides what to answer when the probe arrives. There are two schools of thought, and a good engineer understands both.

Shallow check: “am I able to run?”

  • Answers yes if the program itself is up and can handle a request.
  • Does not contact other services.
  • Fast, cheap, and hard to break.
  • Cannot detect a broken database connection.

Deep check: “can I really do my job?”

  • Also verifies the services the program needs, such as its database.
  • Detects problems a shallow check would miss.
  • Slower, and depends on other systems.
  • Risk: one shared failure can make every pod unready at once.

The cascade danger

Imagine twenty copies of a web service, all depending on one database. If the readiness probe asks the database, then a brief database slowdown makes all twenty copies unready together. The Service now has nobody to send visitors to. Users see errors, even though the web service itself is perfectly healthy and would have shown a friendly message or served cached pages. The deep check turned a partial problem into a total one.

A useful rule is this: check what is local to the pod, and be cautious about checking what is shared by all pods. If a shared dependency is down, no pod can fix it by leaving the traffic. Removing them all only makes matters worse. Instead, let the application handle the failure gracefully, with clear errors, cached data, or retries.

Everyday analogy

Suppose a chain of shops all depend on one delivery truck. If the truck is late, should every shop close its doors? No. Each shop should stay open, sell what it has, and apologize for the missing items. If every shop shut whenever the truck was late, customers would have nowhere to go at all. A shop should close only when that shop has a problem, such as a broken till.

What belongs in a good readiness check

Include

Startup completion

Has the application finished loading its data and settings? Report not ready until it has.

Include

Capacity to accept work

Is the pod overloaded past a safe point? Reporting not ready briefly can let it recover, while others take the load.

Include

Essential local resources

Are local files, caches, or connection pools that this pod needs in a working state?

Include

Shutdown signal

Has the pod started shutting down? Report not ready at once so that no new visitors arrive.

Be careful

Shared databases

Check them only if the pod is truly useless without them, and only in ways that will not fail every pod together.

Avoid

Slow or heavy work

The check runs constantly. It must be quick and light, never a complex query or a big calculation.

13Graceful Shutdown and the Last Visitors

Readiness matters at the end of a pod’s life as much as at the start.

When a pod is deleted, for example during an update or when the cluster shrinks, Kubernetes does two things at nearly the same moment. It tells the traffic layer to stop sending visitors, and it tells the container to shut down. Those two messages travel by different routes and take different amounts of time. The result can be a small race: the container may start shutting down while some visitors are still being sent to it.

Everyday analogy

Think of a shop closing for the night. The manager turns the “open” sign to “closed” and starts stacking chairs. But some customers had already left home heading toward the shop, and a few arrive after the chairs are stacked. A good shop keeps its door unlocked for a few minutes after flipping the sign, serves the stragglers, and only then locks up.

The steps of a graceful goodbye

1

The pod is marked as terminating

Kubernetes records that the pod is going away. It is treated as not ready, and the traffic layer starts removing it from the list.

2

An optional pre-stop pause runs

A pod can define a small action to run before shutdown begins. Many teams use a short sleep here, giving the rest of the cluster a few seconds to stop sending traffic.

3

The container receives a polite stop signal

The application is told to shut down. A well-built program stops accepting new connections and finishes the requests it already has.

4

The grace period counts down

By default the pod has thirty seconds to finish. It can exit early if it is done.

5

A forced stop if time runs out

If the container is still running when the grace period ends, it is stopped by force, and any unfinished work is lost.

How to make it smooth

  • Handle the polite stop signal in your application so it finishes current requests before exiting.
  • Make the readiness answer flip to “not ready” as soon as shutdown begins.
  • Add a short pre-stop pause to cover the delay before all traffic components notice.
  • Set the grace period long enough for your slowest normal request to finish.
  • Keep long-lived connections in mind, such as chat or streaming, and close them cleanly.
i
Good to know

The most common cause of a few stray errors during every release is not a bad probe but a missing graceful shutdown. If you see a small burst of failed requests at each rollout, look here first.

14Slow Starters and the Startup Probe

Some applications take minutes to start. The startup probe gives them time without weakening the other probes.

Big applications, such as those that load large models, build big indexes, or run many start-up checks, may need a long time before they can answer even the simplest question. If you set a very long initial delay on the readiness and liveness probes to accommodate them, you lose sharpness later: a failing pod would also be detected slowly once the application is running normally.

The startup probe solves this. While the startup probe has not yet passed, the other two probes are held back. It can be given a generous budget, such as checking every ten seconds up to thirty times, giving the application five minutes to finish starting. As soon as it passes once, it steps aside for good, and the readiness and liveness probes take over with their normal, tighter timing.

ApproachHow it handles a slow startDownside
Long initial delay on readinessWaits a fixed time before the first checkWastes time if the app starts quickly. Fails if it starts slower than expected
Generous failure threshold on readinessTolerates many failed checks at the startAlso delays detection of failures later on
Startup probeHolds back other probes until first success, with its own patient budgetOne more setting to maintain
Everyday analogy

A hospital gives a newborn baby a special, patient schedule of checks in the first days, with generous allowances for a slow start. Once the baby is settled, the normal, more frequent routine of everyday check-ups begins. The startup probe is that special early schedule.

!
Watch out

If the startup probe never passes within its budget, Kubernetes restarts the container. Size the budget for your slowest realistic start, including moments when the node is busy or the application must download data.

15Readiness Gates: Adding Outside Opinions

Sometimes a pod should not be considered ready until something outside the pod agrees.

A readiness probe is the pod’s own opinion of itself. But in some situations another system must confirm that the pod is ready. For example, a cloud load balancer may need to register the pod and confirm that it can reach it before any traffic should be routed there. Kubernetes supports this through readiness gates.

A readiness gate is an extra named condition listed in the pod’s description. The pod is ready only when all its containers are ready and every gate condition has been set to true by whoever owns it. An outside controller sets the condition when its work is finished. Until then, the pod remains not ready, even if the container passes every probe.

Everyday analogy

A new restaurant may be fully set up, with the kitchen ready and staff in place, but it still cannot open until the health inspector signs the certificate. The restaurant’s own readiness is not enough. A gate is that outside sign-off.

Use case

Load balancer registration

Controllers for cloud load balancers can hold a pod not-ready until the load balancer has registered it and marked it healthy. This prevents dropped requests during rollouts.

Use case

Service mesh readiness

A mesh proxy may need to receive its configuration before traffic is safe. A gate can wait for that.

Use case

Custom checks

Teams can write their own controllers to confirm data replication, license checks, or compliance tests before opening a pod to traffic.

i
Practical hint

Most beginners never write a readiness gate. But if you use a cloud load balancer controller that mentions pod readiness gates, this is what it means, and it is worth turning on to smooth your releases.

16Performance and Scalability

A probe is cheap once and expensive when repeated forever across thousands of pods.

10 s
default time between readiness checks
1 s
default time allowed for each answer
3
default consecutive failures before not ready

The load a probe adds

Each check is one small request to the application. Every pod is checked by its own node’s kubelet, so the load does not concentrate in one place. For normal applications, this is invisible. But if the check is heavy, such as running a database query, and the period is short, the checks themselves become a meaningful share of the application’s work.

Where trouble shows up

Load

Heavy check logic

A check that performs real work adds constant load. Keep it to a few quick operations.

Timeouts

Starved of processor time

If a pod’s processor limit is tight, the health request may wait so long that it times out. A healthy pod then flaps in and out of service.

Command style

Costly process starts

Command probes launch a new process every period. On nodes with many pods, this adds up. Prefer HTTP or TCP when possible.

Scale

Thousands of pods

Each pod adds a probe worker on its node. At large scale, choose sensible periods rather than the shortest possible.

Churn

Frequent flapping

Every state change triggers updates across the cluster’s endpoint lists and traffic rules. Constant flapping wastes control-plane effort.

Cold start

Thundering herd on recovery

When many pods become ready together after an outage, they can all be hit with a sudden flood of traffic. Plan for it.

!
Common trap

A pod that is slow because it is overloaded will fail its readiness probe, leave the traffic, and pass the load to its siblings, which then become overloaded and fail as well. This chain reaction is called a cascading failure. Give the system spare capacity, and add enough copies so that losing a few does not push the rest over the edge.

17Reliability and Availability

Readiness is a tool for staying available, but a careless setup can do the opposite.

How readiness helps availability

  • Unhealthy or warming-up pods are kept away from users.
  • Releases stop by themselves when a new version is broken.
  • Brief trouble in one pod is absorbed by its siblings.
  • Pods recover without a disruptive restart.

How readiness can hurt availability

MistakeWhat goes wrongHow to avoid it
Checking a shared dependencyOne outage removes every pod at onceCheck only what is local, and degrade gracefully
Thresholds that are too tightRandom blips remove healthy pods, causing flappingUse a failure threshold of several checks and a sensible timeout
Thresholds that are too looseBroken pods keep receiving traffic for a long timeCompute the detection time and compare it with your tolerance
Too few copiesLosing one pod loses a large share of capacityRun several copies, spread across nodes and zones
No readiness probeNew and warming-up pods get traffic too earlyAlways define one for services that receive traffic

Working together with other protections

Readiness alone does not make a system highly available. It works best alongside spare capacity, spreading pods across machines and zones, disruption budgets that limit how many pods maintenance may remove at once, and sensible autoscaling. Think of readiness as the traffic warden, and the others as the road, the lanes, and the signals.

i
Key idea

A readiness probe should reflect whether this pod, and only this pod, can serve. Its power is in removing single bad pods, not in switching the whole service off.

18Security Considerations

A health endpoint is a small door into your application. Keep it small and quiet.

Exposure

Keep answers minimal

Reply with a simple status. Do not list internal details such as versions, addresses, or dependency names. Attackers love free information.

Exposure

Do not leak secrets

Never include tokens, passwords, or personal data in health responses or in their logs.

Access

Consider a separate port

Serving health on a different port from the public application makes it easier to keep it away from outside visitors.

Access

Do not require login

The probe is sent by the kubelet and carries no user credentials. Do not protect the health address with a password the kubelet cannot supply. Restrict who can reach it by network rules instead.

Commands

Careful with command probes

A command probe runs a program inside the container. Make sure it is a fixed, trusted program that cannot be tricked into doing more.

Network

Allow the kubelet through

Probes arrive from the node. If you use strict network rules, make sure they allow this traffic, or your healthy pods will appear unready.

!
Watch out

If the health address is also exposed on the public internet, anyone can call it repeatedly. A heavy check behind a public address becomes an easy way to overload your application. Keep checks light, and keep them off public routes.

19Monitoring, Logging, and Metrics

Readiness problems are invisible to users until they become outages. Watch for them early.

Events

Probe failure events

When a probe fails, Kubernetes records an event with the reason, such as a connection refused or an error code returned. These are the first clue.

Metrics

Pod ready status

Cluster state metrics show, for every pod, whether it is ready. Graph the count of ready pods per application.

Metrics

Probe results from the kubelet

The kubelet counts probe successes and failures by type. Rising failure counts warn of trouble before pods leave service.

Metrics

Unavailable replicas

Deployment metrics show how many copies are missing or unready. It should be zero except during releases.

Metrics

Endpoint counts

The number of ready endpoints behind each Service is the most direct picture of available capacity.

Logs

Application health logs

Log state changes in your health logic, not every check. A message when readiness flips gives you a timeline of what happened.

Alerts worth setting up

  • The number of ready pods for an application falls below a safe minimum.
  • A pod flips between ready and not ready several times in a short period, which signals flapping.
  • A Deployment has had unavailable copies for longer than a normal release.
  • A rollout has passed its progress deadline.
  • Probe failures rise sharply across many pods together, which suggests a shared cause.

20Advantages, Disadvantages, and Trade-offs

Readiness probes are among the most valuable settings in Kubernetes, and among the most misused.

Advantages

  • Users meet only healthy pods. Errors from half-started applications disappear.
  • Safe releases. Broken versions are stopped before they replace good ones.
  • Gentle recovery. Pods can rest and return without a restart.
  • Automatic. No person needs to flip switches.
  • Works with everything. Services, ingress, rollouts, and autoscalers all understand it.
  • Debug friendly. Unready pods stay alive for investigation.

Disadvantages

  • Easy to misdesign. Bad checks cause outages instead of preventing them.
  • Detection is not instant. There is always a delay before traffic stops.
  • Flapping risk. Tight settings can remove healthy pods on random blips.
  • Extra work for developers. The application must provide a meaningful answer.
  • Hidden coupling. Checks that depend on other systems link failures together.
  • False comfort. A pod that passes may still be broken in ways the check does not test.

The main trade-offs

Sensitivity versus stability. A twitchy probe catches problems fast and removes healthy pods by mistake. A patient probe is calm and lets broken pods linger. You choose the point that matches your tolerance for errors and for false alarms.

Depth versus safety. A deep check finds more problems, and it can make every pod fail together. A shallow check is safe, and it misses real issues. Most teams settle on a mostly shallow check with a few carefully chosen local tests.

Speed of rollout versus caution. Requiring pods to stay ready for a while before proceeding makes releases slower and safer. Skipping this makes them faster and riskier.

A readiness probe is a promise your application makes: “When I say I am ready, I really am.”

21Design Patterns and Anti-patterns

Some habits produce calm, boring deployments. Others produce surprises.

Patterns that work

Pattern

Separate health addresses

Offer one address for readiness and a different one for liveness, each answering its own question.

Pattern

Startup probe for slow apps

Give slow starters a patient startup probe and keep readiness and liveness sharp.

Pattern

Flip on shutdown

The application reports not ready as soon as it is told to stop, then finishes current requests.

Pattern

Warm-up before ready

Preload caches and open connections before reporting ready, so the first visitors get full speed.

Pattern

Degrade gracefully

When a shared dependency fails, stay ready and return a helpful fallback instead of leaving traffic.

Pattern

Minimum ready time in rollouts

Require new pods to stay ready for a short period before old ones are removed.

Anti-patterns to avoid

ANTI-PATTERN 01 · NO READINESS PROBE AT ALLAvoid
What people do

They deploy services without any probe, because everything works in a quick test.

Why it hurts

Every pod counts as ready the moment it starts. Rollouts send traffic to half-started pods, and a broken new version replaces the working one at full speed.

What to do instead

Define a readiness probe for every service that receives traffic, with a health address that reflects real readiness.

ANTI-PATTERN 02 · ONE ADDRESS FOR LIVENESS AND READINESSAvoid
What people do

They point both probes at the same address that checks everything, including the database.

Why it hurts

When the database is slow, the liveness probe fails too, and Kubernetes restarts pods that only needed to wait. The restart makes the slowdown longer and can wipe helpful warm caches.

What to do instead

Keep liveness simple, checking only that the process is not stuck. Use readiness for the richer question of whether to take traffic.

ANTI-PATTERN 03 · A PROBE THAT ALWAYS SAYS YESAvoid
What people do

They add a health address that returns success no matter what, only to satisfy a checklist.

Why it hurts

It gives a false sense of safety. The pod is treated as ready during startup, during overload, and even when the application is broken, exactly like having no probe.

What to do instead

Make the answer depend on real signals: startup finished, capacity available, and not shutting down.

ANTI-PATTERN 04 · HEAVY CHECKS AT HIGH SPEEDAvoid
What people do

They run expensive queries or long calculations inside the health address and set a very short period.

Why it hurts

The checks steal capacity from real visitors, time out under load, and push healthy pods out of service exactly when they are busiest.

What to do instead

Keep the check tiny. If a deeper test is valuable, run it in the background and let the health address report its latest result.

22Best Practices and Common Mistakes

A handful of habits prevents most readiness problems.

Best practices

Do

Always define readiness

Every service that takes traffic deserves a probe that reflects real readiness.

Do

Keep it light and fast

The check should complete in a few milliseconds and use almost no resources.

Do

Calculate detection time

Multiply period by failure threshold and confirm the result matches your tolerance for errors.

Do

Use realistic timeouts

Set a timeout comfortably above normal answer time, including busy moments.

Do

Test failure on purpose

Make the check fail in a test environment and watch what the traffic does. Confirm the behavior you expect.

Do

Pair with graceful shutdown

Handle the stop signal, flip to not ready, and drain requests before exit.

Common mistakes

Expecting a restart

Believing a failing readiness probe restarts the container. It only removes the pod from traffic. Use liveness for restarts.

Wrong port or wrong path

Pointing the probe at an address that does not exist. The probe fails forever, and the pod never becomes ready, even though the application is fine.

Default timeout on a busy pod

Keeping the one-second timeout for an application that sometimes needs longer, causing needless flapping.

Ignoring startup time

Setting no initial delay or startup probe for a slow application, so early checks fail and confuse everyone.

Forgetting network rules

Blocking the kubelet’s probe traffic with strict network policies, so healthy pods look unready.

Trusting the probe as a security tool

Thinking that an unready pod is unreachable. It can still be called directly by anyone who knows its address.

23Real-World and Industry Examples

The same ideas appear in systems of every size and industry.

Web shops and busy APIs

Online stores and public APIs use readiness probes so that new pods, which need a moment to load product data and warm caches, never receive customers until they are truly prepared. This keeps checkout pages fast during releases, even at busy hours.

Cloud load balancer integrations

Controllers that connect Kubernetes to managed load balancers can use readiness gates so a pod is only treated as ready after the load balancer has registered it and confirmed it is healthy. Teams adopt this to avoid dropped requests during every rollout.

Databases and stateful systems

Database clusters on Kubernetes use readiness to hold a member out of service until it has caught up with its peers. Serving stale data would be worse than serving nothing, so readiness includes a check on replication progress.

Machine learning inference

Services that load large models take a long time to start. They use startup probes for the long load and readiness to report when the model is in memory and ready to answer questions, keeping slow starts from harming users.

Continuous delivery pipelines

Automated release systems watch rollout progress, which depends on readiness. If new pods never become ready, the pipeline stops and can roll back automatically, protecting users without a person watching.

Traffic peaks and autoscaling

When traffic surges, autoscalers add new pods. Readiness ensures those new pods take their share only after they are warmed up, so the extra capacity really helps instead of adding slow responses.

i
Pattern to notice

In every setting, readiness protects the moments of change: starting, releasing, scaling, and stopping. Steady running is easy. Transitions are where systems fail.

24When It Does Not Work: A Troubleshooting Path

Match the symptom to its likely cause, then apply the fix.

SymptomLikely causeWhat to checkTypical fix
Pod runs but never becomes readyProbe points at the wrong port or path, or the app never reports readyThe event text of the failed probe, and the address by hand from inside the podCorrect the address, or fix the application’s health answer
Pod flaps between ready and not readyTimeout too short, thresholds too tight, or an overloaded podAnswer time under load, processor limits, probe settingsRaise the timeout and threshold, add capacity or processor allowance
Rollout is stuckNew pods never pass readinessEvents and logs of the new podsFix the new version, or roll back
Errors during releasesTraffic reaches pods before ready or after shutdown beginsWhether readiness and graceful shutdown are configuredAdd a proper probe, a pre-stop pause, and shutdown handling
All pods become unready togetherCheck depends on a shared serviceWhat the health address testsRemove the shared dependency from readiness
Healthy app but pod shows not readyNetwork rules block the kubelet, or a readiness gate is not satisfiedNetwork policies and pod conditionsAllow probe traffic, or check the gate’s owner
Container keeps restartingThis is probably the liveness or startup probe, not readinessWhich probe named in the failure eventFix that probe, and give slow starters a startup probe
1

Read the pod’s events

The message from a failed probe names the address tried and the error received. It often solves the case immediately.

2

Try the health address yourself

Step inside the pod and call the same address the probe uses. If you get an error, the application is the cause. If it works, suspect timing or network rules.

3

Compare answer time with the timeout

If answers arrive close to the timeout, the probe is fragile. Loosen the setting or speed up the check.

4

Look at the pod’s conditions

See which condition is false: the container’s readiness, or an extra gate. This tells you which system to blame.

5

Check the Service’s endpoints

Confirm which pods are listed as ready. Compare with what you expected.

6

Change one setting at a time

Adjust a single value, observe, and repeat. Changing many at once hides the cause.

25A Hands-On Walkthrough in Plain Words

Let us set up a readiness probe for a small web application, describing each step in words rather than code.

Suppose you have a small web service that needs about twenty seconds after starting to load its data. You run three copies behind a Service. Here is how a careful engineer adds readiness.

1

Give the application a health address

Add a small address, for example named “ready”, that answers with a success code only when the data is loaded and the app is not shutting down, and with an error code otherwise.

2

Describe the probe in the pod

In the pod’s description, add a readiness probe of the HTTP style pointing to that address and the application’s port.

3

Choose the timing

Set an initial delay of about ten seconds, a period of five seconds, a timeout of two seconds, and a failure threshold of three. Detection of a broken pod then takes around fifteen seconds.

4

Add a liveness probe separately

Give the application a different, simpler address that answers if the process is not stuck. Do not let it depend on the data or on other services.

5

Roll it out and watch

Apply the change. Observe the new pods: they should show as running but not ready for about twenty seconds, then flip to ready.

6

Confirm the Service list

Check that each new pod appears in the Service’s ready list only after it flips to ready. Old pods leave the list as they are removed.

7

Test a failure

In a test environment, make one pod report an error. Confirm it leaves the traffic within the calculated time, that its siblings keep serving, and that it returns when you fix it.

8

Test a release

Deploy a version that never becomes ready. Confirm that the rollout stalls, the old pods keep serving, and users notice nothing.

What this walkthrough teaches

The order matters: first make the application able to answer honestly, then tell Kubernetes how often to ask, then test both good and bad cases on purpose. A probe that has never been seen failing has never been proved to work.

26Frequently Asked Questions

Quick answers to the questions beginners ask most often.

Q1What is the difference between readiness and liveness probes?
A readiness probe decides whether a pod should receive traffic, and a failure only removes it from the traffic. A liveness probe decides whether a container is broken beyond recovery, and a failure restarts it.
Q2Does a failing readiness probe restart my container?
No. The pod stays alive and simply stops receiving traffic from Services. It returns to service automatically when the probe passes again.
Q3What happens if I do not define a readiness probe?
Kubernetes treats the pod as ready as soon as its containers are running. Traffic may reach it before the application can handle requests, and rollouts cannot tell good pods from broken ones.
Q4How long before a broken pod stops getting traffic?
Roughly the period times the failure threshold, plus a few seconds for the change to spread through the cluster. With defaults, that is about thirty seconds or a little more.
Q5Should the readiness probe check my database?
Be cautious. If all pods depend on the same database, checking it can make all pods unready at once and cause a total outage. Prefer checking local health and letting the application handle a database failure gracefully.
Q6Which probe style should I use?
Use an HTTP request when your application serves web traffic. Use a network connection check for simple servers, a command for programs with a status tool, and the gRPC style for gRPC services.
Q7Why does my pod show running but not ready?
The container has started but its readiness probe has not yet passed enough times. Read the pod’s events to see why the probe is failing.
Q8Can I use readiness and startup probes together?
Yes, and it is recommended for slow-starting applications. The startup probe holds back readiness and liveness until the application has finished starting.
Q9Does readiness protect me from security threats?
No. It controls routing through Services, not access. Anyone who knows a pod’s address can still call it directly. Use network policies and authentication for security.
Q10Why do I get errors during every release even with a probe?
The most likely cause is missing graceful shutdown. Old pods may receive requests just after they begin stopping. Handle the stop signal, report not ready at once, and add a short pause before exit.
Q11What is a readiness gate?
It is an extra condition, set by an outside controller, that must be true before a pod counts as ready. Load balancer controllers use it to confirm that a pod is registered before it receives traffic.
Q12Should readiness and liveness use the same address?
Usually not. Readiness may include broader conditions, such as capacity and startup progress. Liveness should be simple and independent of other services, so restarts happen only when the process itself is stuck.

27Summary and Key Takeaways

What to remember about readiness probes

  • Running is not the same as ready. A readiness probe lets Kubernetes ask each pod whether it can serve visitors right now.
  • Failure removes traffic, not the pod. A failing readiness probe takes the pod out of Service lists without a restart. Restarts belong to the liveness probe.
  • Three probes, three jobs. Startup gives slow applications time, readiness controls traffic, and liveness rescues stuck containers.
  • Timing is arithmetic. Period times failure threshold tells you how fast a bad pod leaves service. Set a realistic timeout so healthy pods do not flap.
  • Readiness makes releases safe. Rolling updates proceed only as new pods become ready, and a broken version stalls instead of replacing a good one.
  • Design the answer with care. Check local health, avoid tying all pods to a shared dependency, keep the check light, and flip to not ready when shutdown begins.
  • Test it on purpose. Watch a probe fail, recover, and stall a bad release in a test environment, so you can trust it in production.