Let’s establish a precise, shared vocabulary before comparing the two approaches in depth.

A persistent agent is a long-lived machine — physical, virtual, or otherwise — that stays registered with the Jenkins controller continuously, ready to accept new builds at any time, and reused across many builds over its lifetime, often for weeks, months, or years without ever being torn down.

An ephemeral agent, by contrast, is created fresh specifically to run one build (or a small handful of builds), and is destroyed immediately afterward. It typically exists as a container or a cloud-provisioned virtual machine, spun up on demand by a cloud plugin, used briefly, and then discarded entirely — leaving nothing behind for the next build to accidentally inherit.

Understanding the historical and practical pressures behind each model clarifies why this is a genuine trade-off, not a solved problem.

Persistent agents were Jenkins’ original, default model — in the early days of continuous integration, provisioning a fresh machine for every single build simply wasn’t practical or fast enough. Teams registered a fixed pool of build machines once, and those machines quietly ran build after build for as long as they remained useful. This model is simple to reason about and avoids any per-build provisioning delay.

As containerization and cloud infrastructure matured, a new pressure emerged: growing awareness that long-lived machines slowly accumulate “environment drift” — leftover files, subtly different installed tool versions, cached state — that can cause a build to behave inconsistently depending purely on which specific agent happened to run it. Ephemeral agents emerged directly as a response to this problem, using the same disposability that made containers popular elsewhere in software infrastructure to guarantee a clean, identical starting environment on every single build.

Both models share the same underlying controller-agent communication mechanism, but differ sharply in how the agent itself comes to exist.

Underneath both models, the same remoting communication channel connects the agent to the controller once it’s up and running — the architectural difference lies entirely in how and when that agent comes into existence and how long it’s expected to stick around afterward.

The internal mechanics reveal exactly where the “speed vs. cleanliness” trade-off actually comes from.

For a persistent agent, once initially registered, Jenkins simply waits for that specific, known machine to have a free executor. When a matching build needs to run, it’s dispatched immediately — there’s no provisioning step at all, since the machine has been sitting ready the entire time.

For an ephemeral agent, the moment a build needs an agent that no existing idle machine can satisfy, the controller’s cloud plugin sends a provisioning request to the underlying provider, which creates a brand-new container or virtual machine from a template. Only once that new resource finishes starting and connects back to the controller can the waiting build actually begin.

flowchart TD
    A[Build needs an agent] --> B{Persistent agent free & matching?}
    B -- Yes --> C[Build starts immediately, no provisioning]
    B -- No --> D{Ephemeral cloud configured for this label?}
    D -- Yes --> E[New container/VM provisioned from template]
    E --> F[Agent connects back to controller]
    F --> G[Build starts on freshly created agent]
    D -- No --> H[Build waits in queue]
        

This single difference — whether a provisioning step exists at all — is the root cause of nearly every practical distinction between the two models. Persistent agents trade the guarantee of a clean environment for the guarantee of no per-build startup delay; ephemeral agents make the opposite trade.

Looking at the two life cycles next to each other makes the differences concrete.

The key structural difference in this lifecycle comparison is where the “reset” happens. A persistent agent is reset — if at all — only through deliberate maintenance actions performed by a human or a scheduled job. An ephemeral agent is reset automatically and completely, simply by virtue of being destroyed and recreated from scratch for the next build.

Each model shines in different circumstances, and understanding both sides clearly is essential for making a good choice.

The two models carry meaningfully different security profiles, worth weighing explicitly.

A persistent agent, reused across many different builds over a long period, accumulates a kind of shared trust surface: any credentials, cached files, or leftover processes from one build could, in principle, be visible to a later, unrelated build running on that same machine. An ephemeral agent largely eliminates this specific risk, since nothing survives from one build to the next.

Whichever model is chosen, credential handling deserves equal care — persistent agents need periodic credential rotation and access review, while ephemeral agent templates need careful control over exactly which credentials get injected into every freshly-provisioned instance.

Each model benefits from a distinct monitoring emphasis, reflecting where its particular risks tend to appear.

Mature Jenkins operators tend to converge on a shared set of strategies for getting the best of both models.

A handful of consistent habits keep either model — or a mix of both — running smoothly.

A direct, side-by-side comparison makes the decision easier to reason about for a specific workload.

In practice, most mature Jenkins installations don’t pick one model exclusively — they blend both deliberately.

A common, well-tested pattern is running a small, carefully maintained pool of persistent agents for specific, well-understood needs — a handful of machines with specialized hardware, or agents dedicated to unusually long-running jobs where a cold start would be wasteful — while routing the overwhelming majority of ordinary build traffic to an elastic pool of ephemeral agents.

flowchart LR
    A[Incoming builds] --> B{Needs specialized hardware or is latency-critical?}
    B -- Yes --> C[Persistent agent pool]
    B -- No --> D[Ephemeral cloud-provisioned pool]
        

This hybrid approach captures much of the best of both worlds: the reliability and low-latency of persistent agents where it genuinely matters, and the elastic scalability and guaranteed cleanliness of ephemeral agents everywhere else. It does, however, require maintaining two distinct sets of operational practices simultaneously, which is itself a real cost worth acknowledging rather than ignoring.

Both models — and their combination — show up constantly across real production Jenkins environments.

Yes — this is extremely common and is exactly what a hybrid fleet setup looks like, with different labels routing different kinds of builds to whichever agent type fits best.

To some extent — techniques like external dependency caching can recover much of the speed benefit persistent agents get from local caching, though the fundamental per-build provisioning delay generally remains.

They eliminate drift between builds, but the underlying template or base image itself can still become outdated over time, so it still needs regular maintenance — just at the template level rather than on individual machines.

Many teams starting with modest build volume and simple infrastructure find persistent agents easier to set up initially, then migrate toward ephemeral agents as build volume, team size, or consistency requirements grow.

No — many organizations successfully pair ephemeral agents with a shared, external caching layer, getting most of the speed advantage of local caching without sacrificing the guarantee of a clean environment.

Ephemeral and persistent agents represent two genuinely different philosophies for solving the same underlying problem: giving Jenkins somewhere to actually run its builds. Persistent agents offer speed and simplicity by staying always-on and reused, at the cost of gradually accumulating drift and paying for idle capacity. Ephemeral agents offer guaranteed cleanliness and elastic scaling by being created fresh and destroyed after use, at the cost of per-build provisioning delay and the need for more sophisticated supporting infrastructure. Neither model is universally superior — the right answer depends on workload characteristics, and most mature organizations end up deliberately blending both, using each where its particular strengths matter most.