Pod Lifecycle in Kubernetes

Pod Lifecycle in Kubernetes

From the moment you ask for a pod to the moment it disappears: every phase, state, health check, and shutdown step explained.

Every living thing has a life story. It is born, it grows up, it does its work, and one day it retires. A pod in Kubernetes lives a story just like that, only much faster. Some pods live for a few seconds, such as a job that resizes a photo. Others live for months, such as a web server answering customers around the clock. In both cases, the pod moves through the same well-defined stages, and Kubernetes writes down exactly which stage it is in.

Why does this matter? Because almost every problem you meet in Kubernetes is really a problem at one stage of a pod’s life. A pod that never starts, a pod that keeps crashing, a pod that refuses to stop, a pod that runs but receives no traffic: each one points to a specific stage. Once you know the stages, you can read the signs and find the cause quickly. In this guide we will follow a pod from birth to death, one step at a time, and explain every word you meet along the way.

1What Is the Pod Lifecycle?

Before we go deep, let us agree on what a pod is and what we mean by its lifecycle.

The short answer

A pod is the smallest unit that Kubernetes runs. It wraps one or more containers, which are lightly packaged boxes that each hold a program and everything the program needs. The containers in a pod share one network address and can share storage, so they behave like a tiny household living under one roof. A cluster is the group of machines, called nodes, on which pods run.

The pod lifecycle is the complete journey of one pod: it is requested, accepted, given a home on a node, prepared, started, kept healthy, and finally stopped and removed. Kubernetes tracks this journey in three layers of information, which we will study one by one: the pod phase, the state of each container, and a list of conditions.

Analogy

Think of an airline flight. First the flight is scheduled and passengers are waiting (Pending). Then the plane takes off and travels (Running). At the end it either lands safely (Succeeded) or has to abort with a problem (Failed). While that happens, the airport board shows a status for the flight, each crew member has their own status, and a pre-flight checklist tracks whether the plane is ready. A pod has the same three layers: the overall phase, the individual container states, and the conditions checklist.

The most important idea: pods do not come back

A pod is ephemeral, which means short-lived and disposable by design. Kubernetes never repairs a broken pod and never moves a running pod from one node to another. If a pod is lost, a controller creates a brand-new pod to take its place. The new pod has a new name suffix, a new unique ID, and usually a new IP address. This is why your applications must never depend on one particular pod staying alive.

5
phases a pod can report
30 sec
default time a pod gets to shut down
5 min
longest wait between crash restarts
Layer 1

Pod phase

One word that gives the big picture: Pending, Running, Succeeded, or Failed.

Layer 2

Container state

What each container inside the pod is doing right now: Waiting, Running, or Terminated.

Layer 3

Pod conditions

A checklist of true or false answers, such as “has it been scheduled?” and “is it ready for traffic?”

i
Good to know

When you run the usual pod listing command and see a status word such as CrashLoopBackOff or ContainerCreating, that word is not an official phase. It is a friendly summary that the tool builds from all three layers together.

2Anatomy of a Pod

To understand how a pod’s life unfolds, it helps to know what a pod is made of and who looks after it.

A pod is described by a small document called the pod specification, usually written in YAML, a simple text format for structured data. The specification says what you want: which container images to run, how much CPU and memory they need, which storage to attach, how to check their health, and what to do if they stop. Kubernetes stores this wish and then works to make it true.

Next to the specification lives the status. The specification is written by you; the status is written by Kubernetes. It records what is actually happening: which node the pod landed on, its IP address, its phase, its conditions, and the state of each container. The whole lifecycle is really a story of the status catching up with the specification.

What lives inside a pod

  • Init containers run first, one at a time, to prepare the ground. Each must finish successfully before the next begins.
  • App containers are the main programs. They start after all init containers are done.
  • Sidecar containers are helper programs that run alongside the main program, such as a log shipper or a network proxy.
  • Volumes are folders that containers in the pod can share and that can outlive a single container restart.

Who looks after a pod?

You can create a lone pod by hand, called a bare pod, but in real systems pods are almost always created and watched by a controller. A controller is a small program that keeps comparing what you asked for with what exists, and fixes any difference.

Stateless apps

Deployment

Keeps a chosen number of identical pods running and replaces them gradually during updates.

Stateful apps

StatefulSet

Gives each pod a stable name and its own storage, and starts and stops pods in a fixed order.

One per node

DaemonSet

Runs exactly one pod on every node, often for logging or monitoring helpers.

Run to finish

Job and CronJob

Run pods until the work is done, once or on a schedule, and then let them end.

!
Watch out

A bare pod has no controller behind it. If its node fails or someone deletes it, nobody creates a replacement. Use bare pods only for quick experiments.

Every pod also carries an identity. It has a name, a namespace (a named area used to organize resources), and a unique ID that is never reused. When a controller replaces a pod, the ID changes. That is how Kubernetes tells “the old pod” and “its replacement” apart.

3The Five Pod Phases

The phase is the simplest summary of a pod’s life. It answers one question: roughly where in its journey is this pod?

PhaseWhat it meansTypical duration
PendingThe cluster has accepted the pod, but not all its containers are running yet. It may be waiting for a node, or for images to download.Seconds, sometimes minutes
RunningThe pod is on a node and every container has been created. At least one is running, starting, or restarting.As long as the app lives
SucceededAll containers finished successfully and will not be restarted.Final
FailedAll containers have stopped, and at least one ended with an error or was killed.Final
UnknownAn old label for “the state cannot be read”. Modern clusters rarely set it.Rare
stateDiagram-v2
    [*] --> Pending: Pod accepted by the API server
    Pending --> Running: Placed on a node and containers started
    Pending --> Failed: Could not be started
    Running --> Succeeded: All containers exit with code 0
    Running --> Failed: A container fails and will not restart
    Succeeded --> [*]
    Failed --> [*]
    
Figure 1: The main phases of a pod and how it moves between them

Read the diagram from the top. Every pod is born into Pending. From there it normally moves to Running. Pods that are meant to run forever, like web servers, stay in Running until someone or something removes them. Pods that are meant to finish, like a nightly report job, move on to Succeeded when they end cleanly, or to Failed if something goes wrong. Succeeded and Failed are terminal: once a pod reaches one of them, it never goes back to Running.

Analogy

The phase is like the big status board at the airport: Scheduled, In Air, Landed, or Cancelled. It tells you the headline, but not why. To learn why a flight is delayed you must look at the detailed notes, which for a pod means the container states and conditions.

Words that look like phases but are not

Terminating is shown in the pod list when a pod has been asked to stop, but it is not a phase. The pod is usually still in the Running phase while it shuts down. Completed is just the friendly name shown for a pod in the Succeeded phase. Evicted describes a pod that the node removed to protect itself; its phase is Failed, with a reason attached. And if a node is lost, the pod list may print Unknown in the status column, even though the modern way to express this is through the Ready condition, which we cover soon.

i
Note

The phase is a rough summary, not a strict state machine. Many important events, such as a crashing container or a failing health check, happen while the phase still says Running. Always look deeper than the phase.

4Container States

A pod is a group. Each member of the group has its own small life, and Kubernetes tracks it separately.

Each container in a pod is always in exactly one of three states. The pod’s phase is calculated from these individual states.

State 1

Waiting

The container is not running yet, because something it needs is still being prepared, such as downloading the image or mounting storage. A reason is always given.

State 2

Running

The container is executing without problems. The state records the moment it started.

State 3

Terminated

The container has finished, either successfully or with an error. The state keeps the exit code, the reason, and the start and stop times.

Analogy

Imagine actors in a theatre. Some are in the wings waiting for their cue (Waiting), some are on stage performing (Running), and some have finished their scene and left (Terminated). The play, meaning the pod, is only over when the last actor has left.

The reasons behind the states

The status words you see every day are really reasons attached to these three states.

You seeStateWhat is happening
ContainerCreatingWaitingThe node is still preparing the network, storage, or image
ErrImagePull, ImagePullBackOffWaitingThe image could not be downloaded, so Kubernetes retries with growing delays
CreateContainerConfigErrorWaitingA setting the container needs, such as a secret, is missing
CrashLoopBackOffWaitingThe container keeps crashing, and Kubernetes is pausing before the next restart
CompletedTerminatedThe container exited cleanly with code 0
ErrorTerminatedThe container exited with a non-zero code
OOMKilledTerminatedThe container used more memory than its limit and was killed

Reading exit codes

When a container terminates, it leaves behind an exit code, a number that explains how it ended. Code 0 means success. Code 1 and most small numbers mean the application reported an error. Numbers above 128 mean a signal stopped it: 137 means the process was forcibly killed, which is what happens on an out-of-memory kill, and 143 means it received a polite stop request and exited. Kubernetes also remembers the previous state and the total restart count, so you can see what happened before the latest restart.

5Pod Conditions

If the phase is the headline, conditions are the checklist that explains it.

A condition is a single question with a true, false, or unknown answer, together with a reason, a message, and the time it last changed. Together they show exactly how far the pod has progressed on its way to being useful.

ConditionTrue means
PodScheduledThe scheduler has chosen a node for the pod
PodReadyToStartContainersThe node has prepared the pod’s private space and network (present in recent versions)
InitializedAll init containers have finished successfully
ContainersReadyEvery container in the pod reports that it is ready
ReadyThe pod can serve traffic, and Services will send requests to it
Analogy

Before an aircraft leaves the gate, the crew runs through a checklist: crew on board, fuel loaded, doors closed, cabin secure. Each item is a condition. Only when everything is ticked does the plane count as ready to go. The Ready condition is the final tick that says “you may now put passengers on this plane”.

Running is not the same as Ready

This is the most useful lesson in the whole guide. A pod can be in the Running phase, with every container alive, and still be not ready. Perhaps the application is still loading data, or a health check is failing. Kubernetes keeps such a pod alive but stops sending it traffic. Once the checks pass, the Ready condition turns true and traffic starts to flow. This is what lets a new version of an app warm up safely before users reach it.

Readiness gates

Sometimes a pod is not truly ready until something outside the pod is also finished. A readiness gate lets you add an extra condition that an external program must set to true. For example, the AWS Load Balancer Controller can use a readiness gate so that a pod only counts as ready after the cloud load balancer has registered it and marked it healthy. This avoids the gap where Kubernetes thinks a pod is ready but the load balancer has not caught up.

i
Tip

When something looks wrong, read the conditions from top to bottom. The first one that is false tells you which stage of the lifecycle is stuck.

6Birth: Admission and Scheduling

A pod’s life starts long before any container runs. It begins with a request, a series of checks, and a search for a home.

1

The request arrives

A person or a controller sends the pod description to the API server, the front door of the cluster. The server first checks who is asking and whether they are allowed to create pods.

2

Admission

The request passes through admission controllers. Some fill in default values or add helpers, such as an automatic sidecar. Others enforce rules, for example refusing pods that run as the administrator account.

3

Validation and saving

The pod description is checked for mistakes and then saved in etcd, the cluster’s database. The pod now exists, in the Pending phase, but it has no node yet.

4

Scheduling

The scheduler notices the pod without a node. It removes every node that cannot host it, then scores the remaining nodes and picks the best.

5

Binding

The scheduler records its choice in the API server. The PodScheduled condition turns true, and the pod now has a home.

Analogy

Think of hiring a new employee. First the application is received (request). Then it is screened against company rules (admission). Then it is filed in the records (saved). Finally the manager decides which team and which desk the person will get (scheduling and binding). Nobody has started work yet.

How the scheduler decides

Scheduling has two steps. Filtering throws out nodes that cannot work: those with too little free CPU or memory, those marked with a taint the pod does not tolerate, or those that do not match the labels the pod asked for. Scoring then ranks what is left, favouring nodes that balance load or spread copies of the same app across different machines and zones. The highest score wins.

Why a pod stays Pending

If no node passes the filters, the pod remains in Pending, and its PodScheduled condition says false with the reason Unschedulable. The scheduler keeps trying whenever the cluster changes. The pod’s event list tells you exactly why.

Cause

Not enough resources

No node has enough free CPU or memory to satisfy the pod’s requests.

Cause

Untolerated taints

Every suitable node carries a “keep out” mark the pod has no permission to ignore.

Cause

Label mismatch

The pod demands a node label or zone that no node offers.

Cause

Storage not ready

The pod needs a volume claim that has not yet been bound to real storage.

Cause

Pod slots used up

Each node allows only a limited number of pods, by default 110.

Cause

Waiting for a new node

An autoscaler may be starting a new machine, and the pod stays Pending until it joins.

The scheduler has one more trick. Every pod can have a priority. If an important pod cannot fit anywhere, the scheduler may decide to remove less important pods from a node to make room. This is called preemption, and we return to it when we discuss how pods end.

7Coming to Life on the Node

Once a pod has a home, the node’s local manager, the kubelet, takes over and turns the plan into running programs.

The kubelet is a background program on every node. It watches the API server for pods assigned to its node and makes them real. It does this with the help of a container runtime, the software that downloads images and starts containers. Here is the full path from request to a pod that receives traffic.

sequenceDiagram
    participant U as User or Controller
    participant A as kube-apiserver
    participant E as etcd
    participant S as kube-scheduler
    participant K as kubelet
    participant R as Container runtime
    participant EP as EndpointSlice controller
    U->>A: Create Pod
    A->>A: Authenticate, authorize, admit, validate
    A->>E: Save Pod as Pending
    S->>A: Watch for unscheduled pods
    S->>A: Bind Pod to a node
    K->>A: Notice Pod assigned to my node
    K->>R: Create sandbox and attach network
    K->>R: Pull images
    K->>R: Run init containers one by one
    K->>R: Start app containers
    K->>A: Report Running
    K->>K: Run startup and readiness probes
    K->>A: Report Ready
    A->>EP: Pod is Ready
    EP->>A: Add Pod to Service endpoints
    
Figure 2: From a create request to a pod that is receiving traffic

Notice how many separate helpers take part. The API server records, the scheduler chooses, the kubelet prepares, the runtime executes, and finally the endpoint controller adds the pod to the list of destinations behind its Service. No single component does everything, which is what keeps the system reliable.

What the node prepares, in order

1

The sandbox

The runtime first builds a private space for the pod, including a tiny “pause” container that holds the pod’s network address in place for its whole life.

2

The network

A network plugin gives the pod its own IP address and connects it to the rest of the cluster.

3

The storage

Volumes are attached and mounted. Secrets and configuration files are placed exactly where the containers expect them.

4

The images

Container images are downloaded from a registry, an online store for images, unless the node already has them.

5

The containers

Init containers run first, then the app containers start, and the pod’s phase changes to Running.

Image pull policy

Downloading images is often the slowest step. The image pull policy decides when the node should download.

PolicyBehaviourDefault when
IfNotPresentDownload only if the node does not already have this imageThe image has a specific version tag
AlwaysCheck the registry every time the container startsThe tag is “latest” or missing
NeverNever download; use only what is already on the nodeOnly if you set it
!
Watch out

If an image cannot be downloaded because of a typo, a missing tag, or missing credentials, the container sits in Waiting with ErrImagePull, then ImagePullBackOff. Kubernetes retries with longer and longer pauses, up to five minutes.

8Init Containers, Sidecars, and Lifecycle Hooks

A pod’s startup is not one leap. It is a short, ordered sequence, and you can hook your own steps into it.

Init containers

An init container is a short-lived helper that runs before the main program and must finish first. Typical jobs include waiting until a database is reachable, downloading a configuration file, or preparing a folder with the right permissions. Init containers run strictly one after another. If one fails, it is retried according to the pod’s restart policy, and the app containers never start until every init container has succeeded. While this happens, the pod list shows statuses such as Init:0/2, meaning none of two init containers has finished.

Analogy

Init containers are like the setup crew before a concert. They set up the stage, test the microphones, and check the lights. Only after every task is signed off does the band walk on. The setup crew then goes home, and the audience never sees them.

Sidecar containers

A sidecar is a helper that runs for the whole life of the pod, next to the main program. Common examples are a proxy that handles encrypted traffic, or an agent that ships logs. Sidecars used to cause trouble: an ordinary sidecar might start after the main program, and in a Job it could keep the pod alive forever after the main work ended. Recent Kubernetes versions solve this with native sidecars, which are init containers marked to keep running. They start before the app containers, stay up for the whole life of the pod, and are stopped after the app containers have exited.

Lifecycle hooks

A hook is a small action that Kubernetes runs at a chosen moment in a container’s life. There are two.

At start

postStart

Runs right after a container is created. It can register the container somewhere or warm up a cache. If it fails, the container is killed and handled by the restart policy.

Before stop

preStop

Runs just before the container is told to stop. It is the place to finish requests, deregister from other systems, or simply wait a few seconds.

i
Tip

Do not put long or fragile work in postStart. Nothing guarantees it runs before the main program begins, and a failure kills the container. Use an init container for anything the app truly depends on.

9Health Probes

A running container is not always a healthy one. Probes let Kubernetes ask the application how it feels.

A probe is a small check that the kubelet repeats at regular intervals. It can send a web request to a path, try to open a network port, run a command inside the container, or call a gRPC health service. Depending on the answer, Kubernetes takes different actions. There are three kinds of probe, and each has its own job.

ProbeQuestion it asksIf it fails
StartupHas the application finished starting?After too many failures, the container is killed and restarted. Until it passes, the other two probes stay switched off.
LivenessIs the application stuck or dead?The container is killed and restarted according to the restart policy.
ReadinessCan the application handle requests right now?The container is not restarted. The pod is removed from Service destinations until it passes again.
Analogy

Picture a nurse doing rounds. The startup check is “has the patient woken up yet?” The liveness check is “is the patient still responding at all?” If not, call for emergency help (restart). The readiness check is “is the patient well enough for visitors right now?” If not, visitors wait outside, but the patient is not sent to emergency care.

The settings that matter

Each probe has a few knobs. The initial delay waits before the first check. The period sets how often to check, by default every 10 seconds. The timeout sets how long to wait for an answer, by default 1 second. The failure threshold sets how many misses in a row count as a real failure, by default 3. Together they decide how quickly a problem is noticed and how likely a healthy but busy application is to be wrongly punished.

Good probe habits

  • Use a startup probe for slow-starting programs instead of a huge initial delay on liveness.
  • Make readiness reflect whether this pod can serve requests now.
  • Keep checks fast, light, and cheap to run.
  • Give probes realistic timeouts based on measured behaviour.

Probe traps

  • A liveness check that calls the database can restart every pod when the database hiccups.
  • Using the same strict check for liveness and readiness hides which problem is real.
  • Timeouts that are too tight restart healthy pods under heavy load.
  • No readiness probe means traffic arrives before the app is ready.
!
Important

Liveness should answer only “is this process stuck?” It should never depend on other systems. Restarting a healthy pod because a neighbour is slow makes outages bigger, not smaller.

10Restart Policy and CrashLoopBackOff

Programs crash. What Kubernetes does next depends on one small setting and one clever delay.

Every pod has a restart policy. It tells the kubelet what to do when a container stops.

Default

Always

Restart the container whenever it stops, even after a clean exit. Right for servers that should never end.

Batch work

OnFailure

Restart only if the container exits with an error. A clean exit ends the pod as Succeeded.

One shot

Never

Never restart. The pod ends as Succeeded or Failed after the first run.

An important detail: the restart policy applies to containers, not to the pod. The kubelet restarts the failed container inside the same pod, on the same node. The pod itself is not rescheduled elsewhere just because a container crashed.

Why crash loops get slower

Imagine a container that crashes the moment it starts, for example because of a wrong setting. If Kubernetes restarted it instantly, over and over, it would waste CPU and flood the logs. Instead the kubelet uses exponential backoff: it waits 10 seconds before the first restart, then 20, then 40, doubling each time until it reaches a ceiling of five minutes. During those waits the pod shows CrashLoopBackOff. If the container then runs healthily for ten minutes, the timer resets.

Analogy

It is like pressing the snooze button on an alarm clock, except that every press makes the next wake-up later. The system keeps trying, but it gives you increasing time to notice and fix the real problem.

i
Debugging tip

CrashLoopBackOff is a symptom, not a cause. Read the logs of the previous run of the container, since the current one may not have started yet. The exit code and the last-state reason usually name the real problem, such as a bad setting, a missing file, or an out-of-memory kill.

11Graceful Termination

A good ending matters as much as a good start. Kubernetes gives every pod time to finish its work before it is stopped.

Pods end for many reasons: a new version is rolled out, a scaling rule removes extra copies, a person deletes it, or a node is drained for maintenance. In every case the same shutdown routine applies.

sequenceDiagram
    participant U as User or Controller
    participant A as kube-apiserver
    participant EP as EndpointSlice controller
    participant K as kubelet
    participant C as Container
    U->>A: Delete Pod
    A->>A: Set deletion time and grace period
    par Traffic side
        A->>EP: Pod is terminating
        EP->>A: Remove Pod from Service endpoints
    and Node side
        K->>A: Notice Pod is terminating
        K->>C: Run preStop hook
        K->>C: Send SIGTERM
    end
    Note over K,C: Wait up to the grace period, 30 seconds by default
    K->>C: Send SIGKILL if still running
    K->>A: Confirm containers stopped
    A->>A: Remove Pod object
    
Figure 3: The graceful shutdown sequence of a pod
1

Deletion is requested

The API server marks the pod with a deletion time and a grace period, 30 seconds by default. The pod list now shows Terminating.

2

Two things happen at once

The pod is removed from Service destinations so new requests stop arriving, while the kubelet begins stopping the containers.

3

The preStop hook runs

If defined, it runs first. Its time counts against the grace period.

4

SIGTERM is sent

A signal is a small message sent to a program. SIGTERM politely asks the main process to finish up and exit.

5

SIGKILL if needed

If the container is still alive when the grace period ends, it is forcibly killed. It cannot refuse or delay this.

6

The pod disappears

Once containers are gone and storage is cleaned up, the pod object is removed from the database.

Analogy

A shop announces “we close in thirty minutes”. First the doors stop admitting new customers (endpoint removal). The staff finish serving those already inside (handling SIGTERM). When time is up, the lights go off whether or not everyone has finished (SIGKILL).

The classic race, and how to avoid it

Removing the pod from Service destinations and stopping the container happen in parallel, so for a brief moment some routers may still send a request to a pod that has just begun shutting down. Those requests fail. The usual cure is a short preStop pause of a few seconds, giving the network time to catch up before SIGTERM arrives. Your application should also stop accepting new work on SIGTERM, finish requests already in progress, close connections, and exit.

!
Watch out

Some programs ignore SIGTERM, or run under a wrapper script that never passes it on. They always wait the full grace period and are then killed abruptly. Check that your main process really receives the signal.

Stuck in Terminating

A pod may remain in Terminating for a long time. Common reasons are a finalizer, which is a marker that stops deletion until some cleanup finishes, a node that is unreachable so no one can confirm the containers have stopped, or a volume that will not detach. A force delete removes the record immediately without waiting for confirmation. Use it rarely, because it can leave a container still running on a lost node, and for stateful applications that can mean two copies writing to the same data.

12Eviction, Preemption, and Disruption

Not every pod ends because you asked. Some are removed to protect the node, to make room for something more important, or by plain bad luck.

flowchart TB
    P["Running Pod"]
    P --> V["Voluntary disruptions - planned"]
    P --> I["Involuntary disruptions - unplanned"]
    V --> V1["Rolling update replaces the Pod"]
    V --> V2["Node drain evicts the Pod"]
    V --> V3["Scale down removes the Pod"]
    V --> V4["Higher priority Pod preempts it"]
    I --> I1["Node hardware or VM failure"]
    I --> I2["Node pressure eviction by the kubelet"]
    I --> I3["Availability zone outage"]
    I --> I4["Cloud provider reclaims a spot machine"]
    V2 --> PDB["PodDisruptionBudget can slow or block it"]
    
Figure 4: The common reasons a running pod is removed
Node protection

Node-pressure eviction

When a node runs low on memory or disk, the kubelet removes pods to save itself, starting with those using more than they asked for. The evicted pod ends as Failed with the reason Evicted.

Priority

Preemption

A high-priority pod that cannot fit anywhere may push lower-priority pods off a node. The victims still get their normal grace period.

Maintenance

API-initiated eviction

Draining a node asks the API server to evict each pod politely, respecting the safety rules you defined.

Failure

Taint-based eviction

When a node stops responding, its pods are marked for removal after a default wait of five minutes.

The safety net: PodDisruptionBudget

A PodDisruptionBudget is a promise you make to the cluster: “no matter what maintenance you do, keep at least this many copies of my app running.” The cluster will pause a drain rather than break that promise. It only protects against voluntary disruptions. Hardware failure, of course, ignores every budget.

Example: upgrading nodes without an outage

A team runs six copies of a checkout service and sets a budget of at least five available. When the operations team drains a node, the cluster evicts one checkout pod, waits until its replacement is Ready on another node, and only then evicts the next. Customers never notice.

Cleaning up the leftovers

Pods that reach Succeeded or Failed are not deleted at once, because you may want to read their logs. Eventually a garbage collector removes the oldest ones when the count grows too large; by default, that limit is 12,500 finished pods. Jobs can also set a time-to-live so that finished pods clean themselves up.

13Pods and Their Controllers

Individual pods come and go. Controllers turn that constant churn into a stable, reliable service.

Most of the time you never create pods directly. You create a Deployment, and it creates a ReplicaSet, which in turn creates and watches the pods. The ReplicaSet has one job: make sure the requested number of matching pods exists. If a pod disappears for any reason, it creates another. This is called a reconciliation loop: compare wish to reality, fix the gap, repeat.

Analogy

A bakery promises to always have twelve loaves on the shelf. Loaves sell out, and some get dropped on the floor. The baker does not repair a dropped loaf. They simply bake a new one. The shelf always shows twelve, even though every individual loaf is temporary.

Rolling updates

When you change the pod template, for example to use a new image version, the Deployment creates a second ReplicaSet and gradually shifts pods from the old one to the new one. Two settings control the pace: maxSurge says how many extra pods may exist above the target, and maxUnavailable says how many may be missing. Both default to 25 percent. A new pod must become Ready before an old pod is removed, so the readiness probe from Chapter 9 is what makes rolling updates safe. If the new version never becomes ready, the rollout stalls and the old pods keep serving users.

ControllerHow it treats pod lifecycle
DeploymentReplaces pods freely and gradually; pods are interchangeable
StatefulSetNames pods with numbers such as web-0 and web-1, creates them in order, and stops them in reverse order; a replacement keeps the same name and storage
DaemonSetKeeps one pod per node, adding a pod when a node joins
JobRuns pods to completion, retrying failed ones up to a limit, by default six times
CronJobCreates a new Job, and so new pods, on a schedule
i
Key idea

Design every application as if any one of its pods could vanish at any second, because one eventually will. Pods are cattle, not pets.

14Troubleshooting by Lifecycle Stage

Now that you know the stages, every symptom becomes a clue that points to one of them.

SymptomStageCommon causesWhere to look
Pending, no nodeSchedulingNot enough resources, taints, label mismatch, unbound volumePod events
ContainerCreating for longNode preparationSlow image pull, volume mount failure, network plugin troublePod events
ErrImagePull, ImagePullBackOffImage downloadWrong name or tag, missing registry credentials, registry unreachablePod events
Init:Error, Init:CrashLoopBackOffInit containersA prerequisite is not available, or the init step has a bugInit container logs
CrashLoopBackOffRunningApplication error, bad configuration, missing filePrevious container logs, exit code
OOMKilledRunningMemory limit too low, or a memory leakLast state reason
Running but not ReadyReadinessProbe path or port wrong, app still loading, dependency downConditions and probe events
Terminating for longShutdownFinalizer, app ignoring SIGTERM, unreachable nodePod metadata and node status
EvictedNode protectionNode ran short of memory or diskNode conditions

A calm debugging routine

1

Read the status word

Note whether it points to scheduling, image download, init, running, or shutdown.

2

Read the events

The describe command for a pod lists recent events in time order. This is usually where the answer sits in plain words.

3

Read the conditions

The first false condition tells you which stage is stuck.

4

Read the logs

Look at both the current and the previous instance of the container. The previous one often holds the crash message.

5

Check the node

If several pods on one node misbehave, the node itself, not the pods, is the likely problem.

i
Tip

Events are short-lived: by default the cluster keeps them for about one hour. If you are investigating a problem that happened last night, the events may already be gone, which is a strong reason to ship them to long-term storage.

15Security and Observability Across the Lifecycle

Every stage of a pod’s life is a chance to enforce safety, and every stage leaves traces you can measure.

Security at each stage

At admission

Policy checks

Built-in Pod Security Admission applies one of three levels, Privileged, Baseline, or Restricted, to each namespace. Pods that break the rules are refused before they exist.

At start

Least privilege

Run containers as a non-root user, drop unneeded Linux powers, and keep the root filesystem read-only where possible.

At image pull

Trusted images

Pull from a private registry with pull secrets, use fixed version tags or digests, and scan images before they ship.

While running

Short-lived identity

Pods receive time-limited service account tokens. Give each application its own account with only the permissions it needs.

While running

Network limits

Use network policies so that a pod can talk only to the pods it truly needs.

At shutdown

Clean exit

Short-lived pods limit exposure. Secrets held in memory or in temporary volumes disappear when the pod is removed.

Observability at each stage

SignalWhere it comes fromWhat it tells you
EventsThe cluster, per podScheduling, image pull, and probe problems in plain words
Container logsContainer output, collected by log agentsApplication errors, including from the previous instance
Restart countsPod statusHow unstable a container is
Pods by phasekube-state-metricsSudden growth of Pending or Failed pods across the fleet
Waiting reasonskube-state-metricsCluster-wide ImagePullBackOff or CrashLoopBackOff trends
Pod start durationKubelet metricsSlow image pulls or node trouble

Good alerts focus on things users would feel: pods stuck in Pending for more than a few minutes, restart counts rising quickly, and deployments whose ready pod count is below target. Alerting on every single restart creates noise, because restarts are a normal, healthy part of Kubernetes life.

16Design Patterns and Anti-patterns

Teams that handle pod lifecycles well share a few habits. Teams that struggle share a few mistakes.

Patterns that work

Pattern

Shutdown-aware applications

Handle SIGTERM, stop taking new work, finish in-flight requests, and exit inside the grace period.

Pattern

Ready before traffic

Use a readiness probe so a pod only receives requests when it can truly serve them.

Pattern

Init containers for prerequisites

Move waiting, downloading, and preparing into init containers so the app can assume its world is ready.

Pattern

Several replicas with a budget

Run at least two or three copies spread across nodes, protected by a PodDisruptionBudget.

Pattern

Honest resource requests

Set requests and limits from measured usage so scheduling and eviction behave predictably.

Anti-patterns to avoid

ANTI-PATTERN 01Avoid
What people do

They run real workloads as bare pods with no controller.

Why it hurts

If the node fails or the pod is evicted, nothing replaces it. The service simply disappears until someone notices.

Do this instead

Always use a Deployment, StatefulSet, Job, or DaemonSet so a controller owns the pod’s replacement.

ANTI-PATTERN 02Avoid
What people do

They write a liveness probe that checks the database or other services the app depends on.

Why it hurts

When the database is slow, every pod fails liveness and restarts together. The outage grows and recovery becomes harder.

Do this instead

Keep liveness limited to the process itself. Let readiness reflect dependencies, so pods simply pause receiving traffic.

ANTI-PATTERN 03Avoid
What people do

They deploy images tagged “latest” and leave the pull policy to chance.

Why it hurts

Two pods of the same Deployment may run different code, and a restart can quietly pull a new, untested version.

Do this instead

Use unique version tags or image digests, so every pod created from a template runs exactly the same thing.

17Best Practices and Common Mistakes

A short checklist that applies to almost any workload, followed by the slips that catch teams most often.

Best practices

1

Let controllers own your pods

Never depend on a single hand-made pod.

2

Use all three probes wisely

Startup for slow boots, liveness for stuck processes, readiness for traffic.

3

Test your shutdown

Delete a pod under load and confirm that no requests fail.

4

Set the grace period to fit reality

If requests can take up to 60 seconds, a 30-second grace period will cut them off. Measure and adjust.

5

Protect availability

Combine several replicas, spread across nodes and zones, with a PodDisruptionBudget.

6

Keep evidence

Send events and logs to long-term storage so that you can investigate after the pod is gone.

Common mistakes

Mistake

Storing data inside the container

Anything written to the container’s own filesystem or a temporary volume disappears with the pod. Use persistent volumes for data that matters.

Mistake

Assuming a stable IP

A replaced pod gets a new address. Always reach pods through a Service name.

Mistake

Skipping the readiness probe

Without it, users can hit pods that are still starting and see errors during every deployment.

Mistake

Reaching for force delete

It hides a real problem and can leave two copies of a stateful app running at once.

18Real-World and Industry Examples

These examples show how the lifecycle ideas matter when systems get large.

Niantic and Pokémon GO

Pokémon GO launched in 2016 with its backend on Google Kubernetes Engine, and demand far exceeded the plan. Growth like that only works because pods are disposable and identical: capacity is added by starting more pods, and updates can be rolled out gradually while the game stays online.

Spotify

Spotify moved its microservices from an in-house orchestrator to Kubernetes. Every team then gets the same lifecycle rules for free: the same probes, rolling updates, and graceful termination, rather than each team inventing its own way to start and stop services.

The New York Times

The Times has publicly described moving from its own data centres to Google Cloud and running many services on Kubernetes. For a news site, a breaking story can send traffic soaring within minutes, so pods that start quickly and become ready safely are essential.

Scenario: a flash sale at an online shop

Imagine a shop announces a one-hour sale. Traffic jumps tenfold. The autoscaler creates dozens of new pods; those first wait in Pending while new nodes join, then download images, pass their startup probes, and become Ready. When the sale ends, extra pods receive SIGTERM, finish their current orders, and exit. Because the application handles each stage correctly, no customer loses a shopping cart.

19Frequently Asked Questions

Q1Is a pod the same as a container?
No. A container is one packaged program. A pod is a wrapper that holds one or more containers and gives them a shared network address and storage.
Q2Can a pod restart itself?
Containers inside a pod can be restarted by the kubelet, on the same node. The pod itself is never restarted. If it is lost, a controller creates a new pod with a new identity.
Q3What is the difference between Running and Ready?
Running means the containers have started. Ready means the pod passes its readiness checks and may receive traffic. A pod can be Running but not Ready.
Q4Why is my pod stuck in Pending?
Usually the scheduler cannot find a suitable node because of missing resources, taints, label rules, or unbound storage. The pod’s events state the exact reason.
Q5What does CrashLoopBackOff mean?
A container keeps crashing, and Kubernetes waits longer between each restart attempt, up to five minutes. Check the previous logs and the exit code to find the cause.
Q6How long does a pod take to shut down?
Up to 30 seconds by default, counted from the moment deletion is requested. You can change this with the termination grace period setting.
Q7Does a pod keep its IP address?
A pod keeps its address for as long as it exists, even when its containers restart. A replacement pod gets a new address, so use Services instead of pod addresses.
Q8What happens to data when a pod dies?
Data in the container filesystem and in temporary volumes is lost. Data on persistent volumes survives and can be attached to the next pod.
Q9Can I move a running pod to another node?
No. Pods are bound to their node for life. To “move” one, delete it and let its controller create a replacement, which the scheduler may place elsewhere.
Q10What is the difference between liveness and readiness probes?
A failing liveness probe restarts the container. A failing readiness probe only stops traffic from reaching the pod until it recovers.

20Summary and Key Takeaways

Key Takeaways

  • A pod’s life has three layers: the phase gives the headline, container states give the detail, and conditions give the checklist.
  • Pods are disposable. They are never repaired or moved. A controller replaces a lost pod with a new one that has a new identity.
  • Birth is a chain of helpers: the API server admits, the scheduler chooses, the kubelet prepares, and the runtime starts the containers.
  • Running is not Ready. Probes decide when a pod receives traffic and when a stuck container is restarted.
  • Crashes slow down on purpose. CrashLoopBackOff uses growing delays of up to five minutes, so read the previous logs and exit code.
  • Endings should be graceful. Handle SIGTERM, finish in-flight work, and stay within the grace period, 30 seconds by default.
  • Every symptom maps to a stage. Pending, image errors, init failures, crashes, not-ready, and stuck-terminating each point to one step of the lifecycle.