Before we talk about blue and green, we need to understand the problem they solve: putting new software in front of real users without breaking anything.

What does “deploying” mean?

An application is a program that people use, such as a shopping website or a banking app. Developers keep changing it by fixing bugs, adding features, and making it faster. Each time a new version is ready, it must be placed on the servers where real users can reach it. That step is called a deployment, and the place where real users live is called production.

In Kubernetes, applications run inside containers. A container is a small sealed package that holds your program and everything it needs, so it behaves the same on every computer. Kubernetes is a system that runs many containers across many computers and keeps them healthy. A deployment in Kubernetes therefore means telling the cluster, which is the group of computers Kubernetes manages, to start running the new version of your containers.

Why is deploying risky?

A new version can behave very differently in production than it did on a developer’s laptop. Real users send strange inputs, real databases hold messy old data, and real traffic is far heavier than any test. Some bugs simply cannot be seen until the code meets these real conditions.

There are two costs to worry about. The first is downtime, the period when users cannot use the application at all. The second is what we can call bad time, when the application is technically running but giving errors or wrong answers. Bad time is often worse than downtime, because it can quietly corrupt data, charge customers twice, or show private information to the wrong person.

Every deployment strategy is really an answer to two questions. How do we keep users happy while we change things? And how quickly can we undo the change if we made a mistake? The ability to undo a change is called a rollback, and a good strategy makes rollback cheap, fast, and boring.

A quick map of deployment strategies

Blue-green is one of several ways to release software. Knowing the others helps you understand why blue-green exists and when it is the right tool. Here are the main strategies you will meet in Kubernetes work.

Two identical environments, one live and one idle, with a switch that decides which one users see.

The idea in plain words

In a blue-green deployment you always keep two copies of your production environment. One copy, the live one, serves all user traffic. The other copy sits idle and is called the standby. When you want to release a new version, you install it on the idle copy, test it thoroughly without any user seeing it, and then flip a switch so that users are sent to the new copy instead.

After the flip, the roles are reversed. The copy that just received the new version is now live, and the old copy becomes the standby. You leave the old copy running for a while, untouched, as your safety net. If the new version misbehaves, you flip the switch back and users return to the version that was working a moment ago.

Where do the names come from?

The colours are just labels. The pattern is usually credited to Dan North and Jez Humble, who described it while working at ThoughtWorks, and it was spread widely through Jez Humble and Dave Farley’s book on continuous delivery and through writing by Martin Fowler. Blue and green were chosen because they are neutral. Neither colour says “old” or “bad”, which matters because the roles swap every release.

Some organisations use different names for the same idea. Netflix, for example, has described a very similar approach using the words red and black in its Spinnaker delivery platform. The names do not matter. What matters is the shape of the idea: two full environments and a controlled switch between them.

The vocabulary you will use

Blue-green in Kubernetes is built from a handful of ordinary objects, so let us meet each one before we combine them.

Kubernetes does not have a button labelled blue-green. Instead, it gives you a small set of building blocks, and clever teams combine them to get the behaviour. Understanding each block on its own makes the later chapters feel obvious rather than magical.

Labels and selectors: the secret ingredient

If you remember only one thing from this chapter, remember labels and selectors. A label is a small key and value pair, such as “version: green”, attached to a pod. A selector is a rule that picks objects by their labels. The link between a Service and its pods is not a fixed list of addresses. It is simply a selector saying “send traffic to every ready pod carrying these labels”.

This loose connection is exactly what makes blue-green possible. If the Service’s selector can be changed from “blue” to “green”, then the very same Service, with the very same address, starts sending traffic to a completely different set of pods. Nothing else about the clients needs to change.

Service: a phone number that never changes

Pods are temporary. They are created, destroyed, and moved between computers all the time, and each new pod gets a new network address. If clients had to remember pod addresses, they would break constantly. A Service solves this by giving your application one stable name and one stable virtual address that never changes, no matter how many pods come and go behind it.

Readiness and liveness: are you open, and are you alive?

Kubernetes uses two different health checks. A liveness probe asks, “Is this pod still alive, or is it stuck?” If the answer is no, Kubernetes restarts it. A readiness probe asks, “Is this pod ready to serve users right now?” If the answer is no, Kubernetes keeps the pod running but stops sending it traffic.

Readiness is the one that matters most for blue-green. A pod may take thirty seconds to load data or connect to a database. During that time it is alive but not ready, and it must not receive user requests. The Service only sends traffic to pods that are ready, so a good readiness probe is what guarantees that green is truly prepared before it ever meets a user.

Kubernetes ships with two built-in update styles, and neither is blue-green. Seeing why makes it clear what we have to add ourselves.

Rolling update: the default

When you change the container image in a Deployment, Kubernetes performs a rolling update. It creates a few new pods, waits for them to become ready, removes a few old pods, and repeats until every pod runs the new version. Two settings control the pace. One says how many extra pods may exist temporarily, and the other says how many pods may be unavailable at any moment.

Rolling updates are efficient because they need very little extra capacity, and users usually notice nothing. The downside is that for a while both versions serve real users at the same time, mixed together, and you cannot test the new version privately first. Undoing the change is also a second rolling update, which takes as long as the first one.

Recreate: the blunt option

The recreate strategy deletes all old pods first and only then starts the new ones. It is easy to understand and guarantees that two versions never run together, which can matter if the old and new versions cannot share a database. The price is downtime, from the moment the old pods stop until the new pods are ready.

Why blue-green is not built in

A Kubernetes Deployment thinks about one journey at a time, from the current version to the next version, one step after another. It has no idea that you might want two complete, fully running environments side by side and a single atomic switch between them. There is also no built-in object called a “traffic switch” that moves every user at once and can be undone instantly.

To get blue-green, we compose it from parts that already exist, such as two Deployments and a Service, or we install an extra controller that automates the choreography. The following chapters cover both approaches. First, here is how the three strategies compare side by side.

There is no single official recipe. Teams use five common patterns, ranging from the simplest trick in core Kubernetes to a fully automated platform.

All five patterns share the same skeleton. There are two groups of pods, blue and green, each managed by its own Deployment. Above them sits a traffic switch that decides who receives user requests. Below them sit shared dependencies such as a database, a cache, or a message queue. The figure below shows this skeleton.

flowchart LR
    U["Users"] --> S["Traffic switch: Service or Ingress"]
    S -->|"100 percent live"| B["Blue pods: version 1"]
    S -.->|"0 percent until cutover"| G["Green pods: version 2"]
    B --> D[("Shared database")]
    G --> D
        

Read the figure from left to right. Users always talk to the traffic switch and never to blue or green directly. The solid arrow shows the live path, where every request goes to blue. The dotted arrow shows the waiting path, where green is running but receives nothing. Notice that both colours point at the same database. That shared dependency is the hardest part of blue-green, and Chapter 8 is devoted to it.

The components you will run

Pattern 1: Two Deployments and one Service

This is the simplest pattern and needs nothing beyond core Kubernetes. You create one Deployment whose pods carry the label “blue” and another whose pods carry the label “green”. A single Service selects pods using the application name plus a colour label. To cut over, you edit the Service so that its colour selector reads green instead of blue.

The strengths are simplicity and transparency, because anyone who knows Services can understand it. The limits are that the switch is strictly all or nothing, there is no built-in gradual shift, and a separate preview Service is needed so testers can reach the idle colour. You also have to make sure the two Deployment definitions do not slowly drift apart.

Pattern 2: Switching at the Ingress or Gateway

Here you keep two Services, one for blue and one for green, and you change which Service the Ingress or Gateway sends traffic to. Because the switch happens at the web layer, you can make richer decisions. For example, you can send everyone to blue but send requests carrying a special test header to green, which lets your testers use the real public address while ordinary users are unaffected.

The Gateway API is the newer standard for describing this kind of routing, and it supports weighted backends directly. The limit is that the exact features depend on which ingress controller you run, and older Ingress objects often rely on controller-specific annotations that are not portable.

Pattern 3: Routing with a service mesh

A service mesh, such as Istio or Linkerd, places small proxies next to your pods and controls every request between services. A routing rule in the mesh can say “send one hundred percent to blue and zero to green”, and flipping the numbers performs the cutover. The mesh can also route by header for private testing, retry failed calls, and encrypt traffic between pods.

The cost is complexity and extra resource use, because you now operate the mesh as well. A useful side effect is that once you have weights, you can shift from blue-green toward canary by choosing values like ninety and ten instead of one hundred and zero.

Pattern 4: A progressive delivery controller

Tools such as Argo Rollouts and Flagger automate the choreography. Argo Rollouts replaces the ordinary Deployment with a Rollout object that has a blue-green strategy built in. It starts the new ReplicaSet, points a preview Service at it, waits for approval or for automatic checks to pass, promotes it by moving the active Service, and scales the old ReplicaSet down after a delay you choose. Flagger works on top of a mesh or ingress controller and automates similar progressive release flows using metrics.

The benefit is that the release becomes a repeatable, observable process instead of a series of manual edits. The cost is one more component to install, learn, secure, and keep upgraded.

Pattern 5: Whole-cluster blue-green

Sometimes blue and green are not two sets of pods but two entire clusters. This is common when upgrading the Kubernetes version itself, changing the network layer, or moving between regions. Traffic is switched outside the cluster using DNS or a global load balancer, and the old cluster is kept until the new one has proved itself.

This pattern gives the strongest isolation and the most dramatic rollback, but it is also the most expensive because you pay for two full clusters. DNS changes can be slow to spread because resolvers cache answers for the time set in the record, so many teams prefer a weighted load balancer for the switch.

The cutover looks instant from the outside, but a small chain of events runs inside the cluster. Knowing that chain explains every strange behaviour you will ever see.

The journey of an ordinary request

First, let us follow a normal request before any switching happens. A user types your web address. DNS, the internet’s phone book, turns the name into the address of a cloud load balancer. The load balancer forwards the request to an ingress controller running inside your cluster. The ingress controller inspects the request and forwards it to a Service. Finally, forwarding rules on the computer receiving the request pass it to one of the ready pods behind that Service.

The important detail is that the Service is not a program that sits in the middle handling every packet. It is a set of forwarding rules programmed into every node of the cluster. A component called kube-proxy watches Kubernetes and keeps those rules current. Depending on configuration, the rules live in the kernel’s packet filtering tables, in a virtual server table, or in the newer nftables system.

The chain of events at cutover

Now let us flip the selector on a Service from blue to green, and follow what happens. The sequence diagram below shows the actors and the order of messages.

sequenceDiagram
    participant O as Operator
    participant A as API Server
    participant E as EndpointSlice Controller
    participant K as kube-proxy on each node
    participant U as Users
    participant B as Blue Pods
    participant G as Green Pods
    O->>A: Change Service selector to green
    A->>E: Service change event
    E->>A: Update EndpointSlices with ready green pods
    A->>K: Watch event with new endpoints
    K->>K: Rewrite forwarding rules
    U->>G: New connections now reach green
    U-->>B: Existing connections finish on blue
        

Why the switch is not truly instant

Because each node updates on its own schedule, there is a short window, usually a second or a few seconds, when some nodes still send traffic to blue and others already send it to green. During that window, users may be served by either version. This is called eventual consistency, and it is normal in distributed systems.

The practical lesson is that blue and green must be able to live together for a moment. A request that begins on green must not break because the next request lands on blue, and the same data must make sense to both. That requirement, called backward compatibility, is the theme of Chapter 8.

Graceful shutdown when blue finally goes away

When blue pods are eventually removed, Kubernetes sends each one a polite termination signal, and the pod is expected to finish the work it is doing. A setting called the termination grace period gives the pod a fixed number of seconds to finish before Kubernetes forcefully stops it. An optional pre-stop hook can add a short delay so that the forwarding rules have time to stop sending new requests to the pod.

Applications that ignore the signal, or that take longer to finish than the grace period, drop requests during cleanup. A well-behaved application stops accepting new work, completes what is in flight, closes its connections, and then exits.

A blue-green release is a loop with clear stages, decision points, and an escape hatch at every step.

Every release follows the same lifecycle, and the discipline of following it each time is what makes the process safe. The flowchart below shows the stages and the two decision points where the release either continues or turns back.

flowchart TD
    A["Blue is live"] --> B["Deploy new version to green"]
    B --> C["Wait for green pods to be ready"]
    C --> D["Run smoke tests on the preview address"]
    D --> E{"Tests pass?"}
    E -->|"No"| F["Fix and redeploy green"]
    F --> C
    E -->|"Yes"| G["Switch traffic to green"]
    G --> H["Watch metrics during soak period"]
    H --> I{"Healthy?"}
    I -->|"No"| J["Switch back to blue"]
    I -->|"Yes"| K["Scale down blue, keep it ready for next release"]
        

Stateless pods are easy to duplicate. Anything that remembers things, such as databases, caches, and queues, is where blue-green gets difficult.

Duplicating pods is cheap because pods forget everything when they stop. But your application also depends on things that remember. A database holds customer records. A cache holds frequently used answers. A queue holds work waiting to be done. You almost never duplicate these, so blue and green usually share them, and that sharing is where most blue-green problems are born.

The database schema problem

A schema is the shape of a database, meaning its tables and columns. Suppose the new version needs a column renamed. If you rename it and cut over, green works fine, but blue, which is still your rollback target, now fails because it looks for the old name. Your safety net has quietly disappeared.

The cure is a discipline called expand and contract, also known as parallel change. You never make a breaking database change in the same release as the code that needs it. Instead, you spread the change across several releases so that every version of the code can work with every version of the schema that might exist at the same time.

flowchart LR
    A["Step 1: Expand the schema, old and new both work"] --> B["Step 2: Deploy green, compatible with both shapes"]
    B --> C["Step 3: Switch traffic to green"]
    C --> D["Step 4: Soak, blue kept as safety net"]
    D --> E["Step 5: Contract, remove old parts in a later release"]
        

Other kinds of state and how they behave

The biggest gift of blue-green is a safe place to prove the new version works. It is only a gift if you actually use it.

Because green is fully running yet invisible to users, you can test it in conditions that are very close to production. This is far more realistic than a smaller staging environment, since green runs on the same cluster, with the same network rules, the same configuration style, and often the same real dependencies. Many bugs that only appear in production show up here first.

How testers reach green

Testers need an address that leads to the idle colour and nowhere else. Common choices are a preview Service that always selects the idle pods, a separate internal hostname handled by the ingress, or a special request header that the router recognises. Whichever you pick, it must be reachable by your tests but not by the public.

What to test

Warm-up matters

A freshly started application is often slower than one that has been running for an hour. Caches are empty, programs written in languages that compile as they run have not yet optimised their hot paths, and database connection pools are still small. If you cut over a cold green environment, the first real users pay the price with slow responses. A short warm-up with synthetic traffic avoids this.

Blue-green is powerful, but nothing in engineering is free. Here is an honest look at what you gain and what you pay.

Thinking about cost honestly

The cost of blue-green is mostly the cost of the second environment. If your application uses ten large pods, you need room for twenty during the release. That extra room may already be free in your cluster, or your cluster autoscaler may need to add computers for a while, which takes time and money. The good news is that the cost lasts only for the hours or days of the soak period, and you can scale the idle side to zero afterwards.

Compare that to the cost of an outage or a bad release. For a business where an hour of downtime costs more than a month of extra computers, blue-green is very cheap insurance. For a small internal tool used by ten people, it is probably more machinery than it deserves.

How it compares with the other strategies

Two environments and a powerful switch create new things to protect. Security has to be designed in, not added afterwards.

A blue-green setup has two special features from a security point of view. First, there are twice as many running workloads, so any weakness is doubled. Second, the switch is a very powerful control, because whoever can change it can decide what code real users run. Both deserve deliberate protection.

A secure delivery path

The safest way to reach the switch is through a controlled path where every step is checked and recorded. A popular approach is called GitOps, where the desired state of the cluster is stored in a Git repository and a controller inside the cluster continuously makes reality match it. People request a cutover by changing a file and getting it approved, rather than by typing commands into a production cluster.

flowchart LR
    A["Developer"] --> B["Git repository"]
    B --> C["CI build and scan"]
    C --> D["Signed image in registry"]
    D --> E["GitOps controller"]
    E --> F["Idle colour in cluster"]
    F --> G["Approved switch"]
        

Each stage in the figure is a checkpoint. The build stage scans and signs the image. The registry stores only what passed. The controller deploys only what Git says. The final switch needs an approval, human or automatic, that is written to an audit record. If any step fails, the chain stops before real users are affected.

A blue-green release is only as safe as your ability to see what is happening. The switch is quick, but noticing a problem is up to your dashboards.

The four golden signals

Site reliability engineers often summarise service health with four signals. You should be able to see all four for blue and green separately, and side by side.

Label everything with colour and version

The most useful habit in blue-green monitoring is to attach the colour and the version to every metric, log line, and trace. When the dashboards show a spike, you can immediately ask whether it belongs to blue or green. Without that label, you can only see one blended picture, and you lose the ability to compare the old version with the new one at the same moment.

What to watch, phase by phase

Logs and traces

Write logs in a structured form, such as one record per line with named fields, and include the colour, version, and a request identifier in every record. A request identifier lets you follow one user action across many services. Distributed tracing, supported by open standards such as OpenTelemetry, draws the whole path of a request and shows exactly which step became slow after the release.

Also remember that logs from blue disappear from the cluster when blue pods are deleted, unless they were shipped elsewhere. Send logs to a central store as they are written, and keep them long enough to investigate problems that surface days after the release.

Some habits make blue-green smooth and boring. Others turn it into a source of midnight incidents.

Patterns that work well

Anti-patterns to avoid

A short field guide collected from the way experienced teams run blue-green, and from the ways beginners get hurt.

Best practices

• Make readiness probes honest — they should check that the application can really serve requests, including its critical dependencies.
• Generate both colours from one template — only the colour label and version should differ.
• Pin every image — use exact versions or digests so what you tested is what runs.
• Size green like blue before cutover — if blue is running twenty pods under load, green must be ready to carry the same load the instant it receives traffic.
• Set resource requests and limits — this lets the scheduler place both environments properly and protects neighbours from noisy pods.
• Use disruption budgets — a PodDisruptionBudget stops cluster maintenance from removing too many pods at once during a soak.
• Plan for graceful shutdown — configure grace periods and pre-stop pauses so retiring blue does not drop in-flight requests.
• Keep data changes backward compatible — follow expand and contract in every release that touches the schema.
• Define the soak period and rollback rules in advance — remove guesswork from stressful moments.
• Practise rollback — run drills on a normal day so the first rollback is not also the first time anyone has tried it.
• Reserve cluster capacity — confirm quotas and autoscaler limits allow a full second copy before the release starts.
• Record every cutover — note the time, the version, and who approved it, so graphs and incidents can be matched to changes.

Common mistakes

The pattern is older than Kubernetes, and it shows up wherever downtime is expensive and rollback speed matters.

Quick answers to the questions that come up most often when teams adopt blue-green in Kubernetes.

No. In most cases blue and green are two Deployments inside one cluster, often in the same namespace. Two clusters are only needed for whole-cluster blue-green, such as during a Kubernetes version upgrade or a region move.

Neither is better in every case. Blue-green gives a clean switch, private testing, and instant rollback, but exposes everyone to the new version at once. Canary limits how many users see a problem, but needs weighted routing and takes longer. Many teams combine them.

Long enough that a problem would very likely have shown itself. For a busy service that may be a few hours; for one with weekly or monthly cycles, such as billing runs, it may be days. Choose the period based on how your own problems tend to appear.

Only for the duration of the release and soak. Afterwards you can scale the idle side to zero and keep its definition ready. Compare that short cost with the cost of downtime or a slow, risky rollback.

It is far harder. Databases hold the truth, so two live copies would disagree. Teams usually apply blue-green to the stateless application tier and manage databases with careful backward-compatible changes, replication, and separate migration procedures.

Each Deployment has its own autoscaler. Before cutover, make sure green is scaled to at least the size blue currently needs. Otherwise green may be overloaded in the seconds before its autoscaler reacts.

Usually not, provided sessions are stored outside the pods, the versions are compatible for the short overlap, and long-lived connections are drained gracefully. Users on WebSockets or streams may reconnect once.

Yes. A common approach is to install the same chart twice with different colour values, and let the pipeline change which one the live Service or route points to. The important point is that both installs come from the same chart so they cannot drift.

If blue can still read and safely use that data, the rollback is fine. This is why data must stay backward compatible during the soak. If green wrote data in a shape blue cannot understand, a rollback may cause errors and needs a recovery plan.

A rolling rollback has to start and warm up old pods again, which takes as long as the original rollout, while both versions serve users. A blue-green rollback moves traffic to pods that are already running and warm, so it is much faster and more predictable.

No. It fits critical, user-facing services where downtime or a slow rollback is costly. A small internal tool or a low-risk change is often better served by an ordinary rolling update, which is simpler and cheaper.

Yes, and it usually should be. Progressive delivery controllers can run tests, check metrics, promote automatically, and roll back on failure. Keep an approval gate for high-risk releases and make sure every automatic action is logged.

Blue-green deployment is a simple idea with deep consequences. You keep two identical environments, prepare and test the new version on the idle one, and then move users across with one switch. Because the old version is still running, undoing a mistake is as quick as flipping the switch back. Kubernetes does not provide this as a single button, but its Services, labels, readiness probes, and routing layers give you every piece you need, and controllers such as Argo Rollouts can automate the choreography.

The technique is not magic. It costs extra capacity, it makes shared state the central challenge, and it demands honest health checks, good monitoring, and disciplined data changes. Teams that respect those requirements turn releases from stressful events into ordinary, repeatable routines.