Before bringing Kubernetes into the picture, it’s worth being precise about what problem dynamic provisioning solves.

The static agent’s fundamental limitation

A static Jenkins agent is a machine — physical or virtual — that an administrator sets up once and leaves running continuously, whether or not any build is actually using it at a given moment. This arrangement is simple to understand, but it creates an unavoidable mismatch between capacity and demand: during quiet periods, that machine sits idle, consuming resources and, if it’s a paid cloud instance, money, for no benefit. During a sudden burst of activity — many commits landing at once, or a large team all pushing changes near the end of a workday — that same fixed pool of static agents can become a bottleneck, forcing builds to queue and wait their turn even though, in principle, more computing capacity exists somewhere in the world that simply isn’t being used.

Dynamic provisioning as the alternative philosophy

A dynamic agent flips this arrangement entirely: no build machine exists in advance at all. Instead, the moment a build actually needs to run, Jenkins requests that a new agent be created, specifically and exactly for that one build, and once the build finishes, that agent is torn down and its resources are released back to whatever underlying platform provided them. Kubernetes happens to be an extremely well-suited platform for this kind of on-demand provisioning, because creating and destroying small, isolated units of compute — pods — is precisely the core capability Kubernetes was built to provide efficiently and reliably at scale.

With the general philosophy of dynamic provisioning clear, the next chapter looks specifically at why Kubernetes, among the many platforms capable of running containers, became such a natural and popular choice for implementing this pattern in Jenkins.

Kubernetes brings several capabilities together that make it an especially strong fit for on-demand build capacity.

Scheduling is Kubernetes’ core competency

At its heart, Kubernetes is a scheduler: given a description of what a workload needs — how much processing power, how much memory, which container images to run — it finds a suitable machine somewhere in the cluster with available capacity and places the workload there, entirely automatically. This is exactly the capability Jenkins needs when it wants a build agent to simply appear: it doesn’t need to know or care which specific physical or virtual machine in the cluster actually ends up running the build, only that Kubernetes will find somewhere suitable.

Elastic scaling of the underlying cluster itself

Many organizations run their Kubernetes clusters on top of cloud infrastructure configured to automatically add or remove underlying machines based on overall demand across the entire cluster, not just Jenkins’ workloads. This means that during a burst of build activity, the cluster itself can grow additional capacity if needed, and during quiet periods, that same underlying infrastructure can shrink back down, extending the same elastic philosophy one further level down the stack.

A shared platform many organizations already run

A large number of organizations already operate Kubernetes clusters for running their actual production applications, meaning the operational expertise, monitoring tooling, and infrastructure investment already exist. Extending that same, already-familiar platform to also host Jenkins’ build agents avoids introducing an entirely separate, unfamiliar system purely for CI capacity.

With the rationale for choosing Kubernetes established, the next chapter turns to the specific architecture Jenkins uses to actually talk to a cluster and request these dynamic agents.

Several distinct pieces cooperate to translate a pipeline’s request for an agent into an actual, running pod inside a Kubernetes cluster.

How these pieces relate to each other

The Kubernetes plugin sits between Jenkins’ own scheduling logic and the Kubernetes cluster’s API, translating Jenkins’ internal notion of “I need an agent matching this label” into an actual API request to create a pod matching a specific pod template. That pod template, in turn, describes everything about what should run inside the resulting pod — most importantly, a special agent container that establishes a connection back to the Jenkins controller, plus any additional containers the pipeline’s own build steps actually need to use, such as one equipped with a particular language runtime or build tool.

flowchart TD
    A[Pipeline Requests Agent by Label] --> B[Kubernetes Plugin]
    B --> C[Cloud Configuration: Cluster Connection]
    C --> D[Kubernetes API Server]
    D --> E[Pod Scheduled Onto a Cluster Node]
    E --> F[Agent Container Connects to Jenkins Controller]
    E --> G[Additional Build-Tool Containers in Same Pod]
    F --> H[Pipeline Steps Execute Inside the Pod]
        

Why a pod, rather than a single container, is the natural unit

Because a pod can contain more than one container running side by side, sharing the same network address and, optionally, shared storage, Jenkins can structure a build agent as a small team of specialized containers working together — one dedicated purely to communicating with the controller, and one or more others equipped with exactly the tools a particular pipeline needs, such as a container built specifically for compiling a certain language, running alongside a general-purpose container for everyday shell commands.

With the architecture named, the next chapter walks through exactly what happens internally, step by step, from a pipeline declaring it needs a Kubernetes agent to that agent actually being ready to run steps.

A closer look at the sequence of operations behind provisioning a Kubernetes-based dynamic agent.

Step 1 — A pipeline declares it needs an agent with a certain label

When a pipeline reaches a stage requiring a specific kind of agent, it declares that requirement by referencing a label — a name that identifies which pod template should be used to satisfy this particular request.

Step 2 — Jenkins checks whether existing capacity can satisfy the request

Jenkins first checks whether any currently idle, already-provisioned agent matches the requested label. If dynamic provisioning is properly configured and no matching idle capacity exists, Jenkins turns to the Kubernetes plugin to request a brand-new one.

Step 3 — The Kubernetes plugin builds a pod specification

Using the matching pod template’s configuration, the plugin constructs a full pod specification — a detailed, structured description of every container the pod should run, how much processing power and memory each should be allocated, and any additional configuration such as environment variables or mounted volumes.

Step 4 — The specification is submitted to the Kubernetes API

The plugin submits this pod specification to the Kubernetes cluster’s API server, which is the central point through which all requests to create, modify, or query cluster resources pass.

Step 5 — Kubernetes schedules the pod onto a suitable node

Kubernetes’ own scheduler evaluates the pod’s resource requirements against the current available capacity across the cluster’s machines, called nodes, and places the pod onto whichever node has sufficient free resources to run it.

Step 6 — Containers start, and the agent container connects back

Once scheduled, the pod’s containers start running on the chosen node. The dedicated agent container specifically establishes an outbound connection back to the Jenkins controller, registering itself as available and ready to receive the actual build steps.

Step 7 — The pipeline’s steps execute inside the pod

With the connection established, Jenkins runs the pipeline’s steps inside the pod’s containers exactly as it would on any other kind of agent, and once the stage or build finishes, the pod is torn down, releasing its resources back to the cluster.

sequenceDiagram
    participant PL as Pipeline
    participant JC as Jenkins Controller
    participant KP as Kubernetes Plugin
    participant API as Kubernetes API Server
    participant NODE as Cluster Node
    PL->>JC: Request agent with label
    JC->>KP: No idle capacity, provision new agent
    KP->>KP: Build pod specification from template
    KP->>API: Submit pod creation request
    API->>NODE: Schedule pod
    NODE-->>API: Pod running
    NODE->>JC: Agent container connects
    JC->>NODE: Send pipeline steps to execute
    NODE-->>JC: Step results
    JC->>API: Delete pod after use
        

Understanding this sequence for a single build is useful, but the pod template itself deserves a closer, dedicated look, since it’s where most of the meaningful configuration decisions actually live — the subject of the next chapter.

The pod template is the single most important piece of configuration in this entire system — it defines exactly what a dynamic agent actually looks like.

Container definitions

A pod template lists one or more containers, each specifying which container image to use, and each image effectively determines what tools are available inside that particular container — a Java-focused image, a Node.js-focused image, or a general-purpose utility image, for example. A pipeline’s steps can be directed to run inside a specific named container within the pod, letting a single pod host several different toolchains side by side without needing several separate agents.

Resource requests and limits

Each container can specify how much processing power and memory it expects to need, called a resource request, and optionally a hard ceiling on how much it’s allowed to consume, called a resource limit. These values directly influence which cluster node Kubernetes considers suitable for scheduling the pod, and setting them thoughtfully — neither too low, risking a starved, slow build, nor unnecessarily high, wasting cluster capacity — is one of the more important tuning exercises in operating this system well.

Labels as the connection point back to the pipeline

Every pod template is tagged with one or more labels, and a pipeline requests an agent by naming one of these labels rather than referring to the template directly by some internal identifier. This indirection means the same label can, in principle, be satisfied by different underlying pod template definitions over time, as long as whoever maintains the templates keeps the label pointing at something suitable — the pipeline itself doesn’t need to change.

With the pod template’s structure understood, it’s worth following a single build’s pod through its complete lifecycle from creation to teardown, which the next chapter covers.

Following one pod from birth to teardown shows exactly how ephemeral this whole system really is.

Why this lifecycle is deliberately short

Keeping a pod’s entire lifetime scoped tightly to a single build or stage is a deliberate design choice, not an incidental limitation. It guarantees that every build starts from a genuinely clean, freshly created environment, free of any leftover state from a previous, unrelated build that might otherwise have run in the same location — directly extending the same environment-consistency benefit that Docker agents provide, at the scale and elasticity Kubernetes adds on top.

With a single pod’s lifecycle in view, it’s worth zooming out to see how the cluster as a whole scales its total capacity up and down in response to overall demand across many builds — the subject of the next chapter.

One of the most compelling properties of this architecture is its ability to scale genuinely down to nothing when there’s no work to do.

Scaling up in response to demand

As multiple builds trigger in quick succession, Jenkins requests multiple pods, and Kubernetes schedules each one onto available capacity. If the cluster’s existing nodes don’t have enough room for all the requested pods at once, and the cluster is configured with node-level auto-scaling, additional underlying machines can be provisioned automatically to absorb the extra demand, allowing genuine concurrent build capacity to grow in near real time as needed.

Scaling back down to true idle

Once a burst of build activity subsides, pods are torn down as their builds finish, and if the cluster’s auto-scaling is configured accordingly, underlying nodes that are no longer needed can themselves be removed, shrinking the cluster’s actual footprint back toward — potentially all the way to — a genuinely minimal baseline, rather than a fixed, always-on pool sitting idle regardless of demand.

This elastic scaling behavior naturally shapes the different ways teams actually configure pod templates for different needs, which the next chapter categorizes.

Jenkins offers a few distinct ways to define a pod template, each suited to a different level of complexity or reuse.

Whichever configuration style a team adopts, it’s worth comparing this whole Kubernetes-based approach directly against the static and Docker-on-a-static-host agent models covered elsewhere, which the next chapter does.

Kubernetes-based agents sit at one end of a spectrum of provisioning models, each with genuinely different trade-offs.

Why the ceiling matters as much as the floor

The most distinctive advantage Kubernetes-based agents bring isn’t just their ability to scale down toward zero cost during idle periods — a Docker agent on a single static host also avoids per-build configuration drift, after all — it’s their ability to scale up across an entire cluster’s worth of capacity, rather than being limited to whatever a single fixed host machine can physically provide. A sudden burst of a hundred concurrent builds can, in principle, be absorbed by a Kubernetes cluster spreading those pods across many nodes, in a way a single Docker host simply cannot match.

These comparative strengths carry directly into the broader advantages and disadvantages of adopting Kubernetes-based dynamic agents overall, which the next chapter lays out in full.

This architecture solves real capacity and consistency problems, but running it well takes genuine operational investment.

Because this architecture involves a cluster’s API, containers, and shared infrastructure, it’s worth examining the security dimension closely, which the next chapter addresses in depth.

Granting Jenkins the ability to create pods on a Kubernetes cluster is a real, meaningful capability that deserves careful boundaries.

Scoping Jenkins’ cluster permissions tightly

The credentials Jenkins uses to talk to the Kubernetes API should be scoped, wherever possible, to only what creating and managing build agent pods genuinely requires — typically confined to a specific namespace dedicated to CI workloads, rather than broad, cluster-wide administrative access that could reach entirely unrelated production workloads running on the same cluster.

The pod’s own service account matters too

Separately from Jenkins’ own cluster credential, each pod itself runs under a Kubernetes service account that determines what that specific pod is permitted to do from inside the cluster — for example, whether it can read secrets, create other resources, or reach other namespaces. Assigning build pods a minimal, purpose-specific service account, rather than a broad default one, limits what a compromised build step could actually reach even if it tried.

Once access is properly scoped both at the Jenkins level and the pod level, knowing exactly what happened during a given build’s pod lifecycle becomes the next priority, which the monitoring chapter that follows addresses.

Because pods are ephemeral by design, knowing what to look at while they’re running — and immediately after — matters more here than for a permanent, always-inspectable static agent.

Why ephemeral infrastructure needs deliberate log capture

Because a pod disappears the moment a build finishes, any diagnostic information not already captured in Jenkins’ own build log or a properly configured centralized logging system is genuinely lost once that pod is deleted. This makes it especially important to ensure build output and relevant diagnostic detail are captured into Jenkins’ persistent build record, rather than relying on being able to inspect a still-running pod after the fact, since it usually no longer exists by the time anyone goes looking.

Good visibility into pod lifecycle and cluster health naturally leads into recognizing which structural patterns keep this whole system healthy and efficient over time, and which quietly create waste or confusion — the subject of the next chapter.

Some ways of structuring pod templates and their usage age well as an organization scales; others quietly become sources of waste or confusion.

These principles translate directly into the concrete, actionable checklist covered in the next chapter.

A short, practical checklist for anyone setting up or operating Kubernetes-based dynamic agents in Jenkins.

With the checklist in place, it’s worth seeing how organizations actually deploy this architecture in real production settings, which the next chapter explores.

Kubernetes-based dynamic agents show up across a wide range of organizations, particularly those already invested in Kubernetes for other purposes.

Before closing out, it’s worth addressing the questions that come up most often once teams start adopting Kubernetes-based dynamic agents in practice.

No — while running the Jenkins controller inside the same cluster is a common and convenient setup, the Kubernetes plugin can connect to and provision agents on a cluster the controller itself is not running inside of, as long as network connectivity and proper credentials are configured.

Yes — a pod template can define multiple containers within the same pod, and a pipeline’s steps can target a specific named container, letting one pod host several different toolchains side by side for a single build.

Without cluster auto-scaling, the pod remains pending until capacity frees up, delaying the build; with cluster auto-scaling configured, additional underlying nodes can be added automatically to accommodate the request within a reasonable amount of time.

Not by default — since each pod is freshly created and later deleted, nothing persists automatically between separate builds unless a pipeline deliberately uses external caching or persistent storage mechanisms designed for that purpose.

Both provide fresh, isolated environments per build, but Kubernetes-based agents can be scheduled across an entire cluster’s worth of machines rather than being limited to a single fixed host’s capacity, and can scale the underlying infrastructure itself up and down, not just the containers running on it.

Basic usage through Jenkins’ own simplified declarative pod template syntax requires relatively modest Kubernetes knowledge, though deeper customization using raw YAML pod specifications, along with properly securing and tuning the underlying cluster, does benefit from more substantial Kubernetes familiarity.

Kubernetes-based dynamic agents turn build capacity from a fixed, always-on cost into something that closely tracks actual demand, appearing as a freshly scheduled pod the moment a build needs it and disappearing the instant that build finishes. This is made possible by Kubernetes’ own core strength as a scheduler, combined with the Kubernetes plugin translating Jenkins’ agent requests into pod specifications built from reusable, well-designed templates. The real payoff — genuine elasticity, strong environment isolation, and the ability to absorb large bursts of concurrent demand across an entire cluster — comes with real operational responsibility: tightly scoped cluster permissions, carefully measured resource requests, and deliberate log capture to compensate for infrastructure that, by design, doesn’t stick around long enough to be inspected after the fact.