Let us start with the plain meaning of a Job and the words we need to describe it.

The short answer

A Job is a Kubernetes object that runs one or more pods until a chosen number of them finish successfully. A pod is the smallest unit Kubernetes runs; it wraps one or more containers, which are lightly packaged boxes holding a program and everything it needs. Kubernetes runs pods on nodes, the machines that make up the cluster. A Job sits above the pods and takes responsibility for them: if a pod fails, the Job starts another. When enough pods have succeeded, the Job marks itself complete and stops creating new ones.

What Jobs are good for

Choosing the right tool starts with one question: is this work supposed to end?

Why not use a Deployment?

A Deployment believes that a finished program is a broken program. It keeps its pods running with a restart rule called Always, so when your task finishes and exits cleanly, the Deployment happily starts it again, and again, without end. That is perfect for a web server and wrong for a nightly report.

Why not use a bare pod?

A lone pod does run your task, but nobody is responsible for it. If its node crashes halfway through, the pod is simply lost. No one retries it, and no one records whether the work was completed. A Job adds exactly the missing pieces: retries, counting, and a final verdict.

A Job is described by a small document with a handful of settings. Learn these settings and you can read almost any Job.

Like other Kubernetes objects, a Job is written in a simple structured text format called YAML and submitted to the cluster. The description has two main parts. The first is the pod template, a recipe for the pods the Job will create: which container image to run, what command to give it, how much CPU and memory it needs, and where to find its settings. The second part is a set of rules that control how many pods run, how many must succeed, and what to do when things go wrong.

flowchart TB
    CJ["CronJob (optional)"] -->|"creates on schedule"| JOB["Job"]
    USER["Person or pipeline"] -->|"creates directly"| JOB
    JOB -->|"creates and tracks"| P1["Pod 1"]
    JOB --> P2["Pod 2"]
    JOB --> P3["Pod 3"]
    P1 --> C1["Container runs the task"]
    P2 --> C2["Container runs the task"]
    P3 --> C3["Container runs the task"]
    JOB -->|"records"| RES["Result: succeeded and failed counts"]
    

The diagram shows the chain of ownership. A person, a pipeline, or a CronJob creates the Job. The Job creates and tracks pods. Each pod runs the container that does the real work. The Job keeps a running count of how many pods succeeded and how many failed, and this count decides when it is finished. Because the Job owns its pods, deleting the Job also deletes them.

Two details worth remembering

First, Kubernetes creates the Job’s labels and selector automatically, so that the Job can tell its own pods apart from everyone else’s. You should almost never edit them by hand. Second, the pod template of a Job cannot be changed after the Job is created. If you need a different image or command, create a new Job. This rule protects you from a Job whose pods are half old and half new.

Let us follow one Job from the moment it is created until it disappears.

sequenceDiagram
    participant U as User
    participant A as kube-apiserver
    participant J as Job controller
    participant S as kube-scheduler
    participant K as kubelet on a node
    participant T as TTL controller
    U->>A: Create Job
    A->>J: New Job noticed
    J->>A: Create Pod
    S->>A: Bind Pod to a node
    K->>A: Notice Pod on my node
    K->>K: Run the container
    K->>A: Report Pod Failed
    J->>J: Count the failure and wait for the backoff delay
    J->>A: Create replacement Pod
    K->>A: Report Pod Succeeded
    J->>A: Mark Job Complete
    T->>A: Delete Job after the TTL expires
    

A Job reports how it is doing through a few counters and a short list of conditions.

stateDiagram-v2
    [*] --> Active: Job created
    Active --> Suspended: suspend set to true
    Suspended --> Active: suspend set to false
    Active --> Complete: Enough pods succeeded
    Active --> Failed: Retry limit or deadline reached
    Complete --> [*]
    Failed --> [*]
    

An active Job is still working: pods are running or being created. A suspended Job is paused; no pods run until you resume it. A complete Job reached its goal. A failed Job gave up. Complete and Failed are final: a finished Job never wakes up again.

Real tasks fail. A network hiccup, a busy database, or a node crash should not ruin a night of work. Retrying is the heart of a Job.

Two ways a failure can be retried

The restart policy in the pod template decides how a failure is handled.

The backoff limit and the growing delay

The backoff limit is the number of failures the Job will tolerate, six by default. Once that many failures have been counted, the Job is marked Failed. Between retries, the Job controller waits a little longer each time: about 10 seconds, then 20, then 40, and so on, up to a ceiling of six minutes. This is called exponential backoff. It gives a struggling system, such as an overloaded database, time to recover instead of hammering it with instant retries.

A time limit for the whole Job

A Job that retries forever can hide a real problem. Set activeDeadlineSeconds to cap the total time. When the deadline arrives, every running pod is stopped and the Job fails with the reason DeadlineExceeded. The deadline wins over the backoff limit, so even if retries remain, the Job stops.

Smarter rules for failures

Not all failures deserve the same reaction. If a node is drained for maintenance and your pod is evicted, that is not your program’s fault, yet by default it still counts against the retry limit. A pod failure policy lets you write rules based on the container’s exit code or on why the pod was removed. For example, you can say: “if the exit code is 42, which my program uses for a bad input file, fail the whole Job immediately because retrying is pointless” and “if the pod was evicted by the cluster, ignore that failure and do not count it”. Pod failure policies work with the restart policy Never.

One pod at a time is fine for small tasks. Large tasks need many workers. Jobs support several patterns, chosen by two numbers.

The two numbers are completions, the total successes needed, and parallelism, the number of pods allowed to run together. Depending on how you set them, a Job behaves in one of four ways.

How indexed Jobs work

In indexed completion mode, Kubernetes gives every pod a unique number, available to the program as an environment variable called JOB_COMPLETION_INDEX. Your program reads its number and decides which slice of the data to handle. This avoids needing a queue, because the split is decided in advance. If pod 17 fails, only slice 17 is retried. Newer Kubernetes versions can also count retries separately for each index, so one stubborn slice does not use up the retry budget of the whole Job.

A single Job is simple. Hundreds of Jobs and thousands of pods need a little more planning.

Tell the cluster what you need

Every pod in a Job should declare its resource requests, the minimum CPU and memory it needs, and limits, the maximum it may use. The scheduler uses requests to decide where a pod fits. Without them, it may pile too many pods onto one node, and the node may run out of memory and start killing pods. Batch work is often hungry, so honest numbers are especially important.

Hold Jobs back until there is room

If you submit 500 heavy Jobs to a small cluster, most pods will sit in Pending while a few run. That is wasteful and noisy. The better approach is a queue. Kubernetes lets you create Jobs in the suspended state, and projects such as Kueue, which is part of the Kubernetes community, watch the cluster’s free capacity and unsuspend Jobs in a fair order. Another project, Volcano, adds batch-oriented scheduling features such as scheduling a whole group of pods at once, which matters when a training job needs all its workers to start together or not at all.

Many tasks repeat: every night, every hour, every first day of the month. That is the job of a CronJob.

A CronJob is a Kubernetes object that creates a new Job whenever its schedule says so. It contains the same Job recipe you already know, plus a schedule. The name comes from cron, the classic scheduler on Unix computers.

Reading a schedule

A schedule is written as five fields separated by spaces: minute, hour, day of the month, month, and day of the week. A star means “every”.

By default, the schedule follows the time zone of the cluster’s control plane, which is often UTC. Newer versions let you set a timeZone field such as a named region, so “9:00 in the morning” means the same thing for your team everywhere and daylight saving changes are handled for you. The shortest interval is one minute.

What if the last run is still going?

Suppose a report takes 70 minutes but is scheduled every hour. When the next hour arrives, the previous Job is still running. The concurrency policy decides what happens.

flowchart TB
    T["Scheduled time arrives"] --> Q{"Is the previous Job still running?"}
    Q -->|"No"| N["Create a new Job"]
    Q -->|"Yes, policy Allow"| N
    Q -->|"Yes, policy Forbid"| SK["Skip this run"]
    Q -->|"Yes, policy Replace"| R["Stop the old Job then create a new one"]
    R --> N
    N --> H["Old finished Jobs trimmed by history limits"]
    

Behind every Job there is a small team of controllers quietly doing their rounds.

All the smarts of Jobs live in the kube-controller-manager, a program in the control plane that hosts many controllers. Each controller runs a simple, endless loop known as reconciliation: look at the desired state, look at the actual state, and act to close the gap.

Keeping an honest count

How does the Job controller know exactly how many pods succeeded, even if some pods are deleted or the controller itself restarts? It attaches a small marker called a finalizer to each of its pods. A finalizer tells the cluster “do not fully delete this pod until I have recorded its result”. Only after the Job controller has added the pod to its count does it remove the marker. This prevents a success from being counted twice, or a failure from being forgotten.

A Job’s pods are temporary. The results of the work must go somewhere that lasts.

Where do the results go?

When a pod ends, everything written inside its own filesystem is lost when the pod is removed. So a Job should send its output to a place outside the pod.

Settings and credentials should come in through ConfigMaps and Secrets rather than being baked into the image. And do not forget logs: they disappear when finished pods are deleted, so a log collector should copy them elsewhere.

Idempotency: safe to repeat

A task is idempotent if doing it twice has the same effect as doing it once. Retries, node failures, and the “at least once” nature of CronJobs mean your task will sometimes run again over work that is already partly or fully done.

Jobs are excellent at what they do, and they are not meant to do everything.

Choosing between related tools

Jobs often touch valuable data and run with real credentials, so they deserve the same care as any service.

Short life is a security advantage. Because Job pods exist only for the time needed, credentials they hold live in the cluster only briefly. Combine that with a time-to-live, and few traces are left behind once the work is done.

A Job that fails at 3 a.m. is only a problem if nobody notices. Good monitoring turns silent failures into clear alerts.

Alerts that help

• Job failed: page someone, or open a ticket, as soon as a Job enters the Failed state.
• Missed schedule: alert when a CronJob has not started a run for longer than expected.
• Too slow: alert when a Job runs far beyond its usual duration, before the deadline kills it.
• Too many leftovers: alert when finished Jobs or pods grow past a sensible number.

Finally, remember that logs from finished pods vanish with the pods. Ship logs to a central store as they are produced. It is also useful to have the task print a short summary at the end, such as items processed, items skipped, and time taken, so that a quick look at the logs tells the story.

When a Job misbehaves, the symptom usually points straight at the cause.

A calm routine

Teams that run batch work well keep repeating the same good habits. Trouble also has repeating shapes.

Patterns that work

Anti-patterns to avoid

A short checklist that suits nearly every Job, followed by the slips seen most often.

Best practices

Common mistakes

Batch work on Kubernetes shows up in many well-known places.