Before touching encryption at all, we need a solid mental model of what an “overlay network” even is, and why containers need one in the first place.

What problem are we even solving?

A container is a lightweight, isolated process running on a physical or virtual machine. When you run one container on one laptop, networking is trivial — it’s all happening inside a single operating system. The trouble starts the moment you have more than one machine. Say you have three physical servers, and you want a container running on Server A to talk to a container running on Server C, as if they were sitting right next to each other on the same local network — even though, in reality, they are separated by routers, firewalls, and possibly thousands of miles of fiber-optic cable. This is the exact problem that container orchestration platforms like Docker Swarm and Kubernetes had to solve, and the solution they landed on is called an overlay network.

Why isn’t the tunnel encrypted by default in every system?

Building the tunnel (routing packets correctly across machines) and securing the tunnel (making sure nobody can read or tamper with what’s inside it) are two separate engineering problems. Docker solved the routing problem first, using a technology called VXLAN, which we will unpack in the next chapter. Encryption is a deliberately optional layer on top of that routing, because encrypting and decrypting every single packet costs CPU cycles, and not every environment needs it — a cluster running entirely inside a single, physically secured data center rack has a very different risk profile than a cluster whose nodes are spread across three different cloud regions operated by three different providers.

The three ingredients that make this possible

Everything described in this article rests on three foundational ingredients working together:

Notice the layering: IPsec doesn’t encrypt your application’s data directly. It encrypts the VXLAN envelope that is already carrying your application’s data. This is a crucial distinction that will make the rest of this article click into place, so hold onto it.

With the concepts in hand, let’s map out every moving part that participates in building and securing an overlay network.

Swarm mode: the orchestration layer underneath it all

Docker’s overlay network encryption only exists inside Swarm mode, Docker’s built-in clustering and orchestration system. When you initialize a swarm, one or more machines become manager nodes, responsible for making decisions about the cluster’s desired state, and any number of machines become worker nodes, responsible for actually running containers. Manager nodes talk to each other using a consensus algorithm called Raft, the same family of algorithm used by systems like etcd (which itself powers Kubernetes). Raft guarantees that even if some managers crash or lose network connectivity temporarily, the surviving managers still agree on a single, consistent version of the truth — including which encryption keys are currently active.

The gossip protocol: how workers learn what managers know

Worker nodes don’t participate in the Raft consensus directly — that would not scale well to hundreds or thousands of workers. Instead, control-plane information (like network topology and, critically, encryption keys) is spread across the cluster using a gossip protocol, conceptually similar to how a rumor spreads through an office: node A tells a few nearby nodes, those nodes tell a few more, and within a handful of rounds, everyone has the same information, without any single node having to talk to all the others directly.

flowchart TB
    subgraph Managers["Manager Nodes - Raft Consensus"]
        M1["Manager 1 (Leader)"]
        M2["Manager 2"]
        M3["Manager 3"]
    end
    M1  M2
    M2  M3
    M1  M3
    M1 -- gossip control data --> W1["Worker Node A"]
    M1 -- gossip control data --> W2["Worker Node B"]
    W1 -- gossip relay --> W3["Worker Node C"]
    W2 -- gossip relay --> W4["Worker Node D"]
    style Managers fill:#111111,stroke:#dc2626,color:#ffffff
    

The core building blocks, named

Where the kernel comes in

An important, often-overlooked fact: Docker itself does not implement encryption in its own application code. Instead, it configures the Linux kernel’s own, battle-tested IPsec subsystem (via the xfrm framework) to do the encrypting and decrypting. This matters enormously for performance and trust — the kernel’s IPsec implementation is decades old, used by essentially every VPN product on earth, and runs extremely close to the network hardware, rather than in a slower, user-space application. Docker’s job is reduced to being an orchestrator: it generates the keys, distributes them to the right hosts, and tells the kernel’s IPsec engine, “encrypt everything moving between these two IP addresses on this VXLAN port.”

Now we go one level deeper: what literally happens, step by step, when Docker turns on encryption for an overlay network.

Step one: key generation, not key invention

When you create an encrypted overlay network, the swarm’s leading manager node generates a symmetric encryption key — a single shared secret that both the sending and receiving host will use to scramble and unscramble data. This isn’t done casually; the manager uses a cryptographically secure random number generator, and the key itself is designed for use with the AES-256-GCM cipher, an industry-standard, authenticated encryption algorithm also used to protect, among other things, TLS connections for a large share of the modern web.

Step two: secure key distribution over the control plane

The key itself must never travel across the network in the clear. Docker solves this by piggybacking key distribution on the swarm’s own control-plane traffic, which is itself protected by mutual TLS (mTLS) — every manager and worker node in a swarm holds a unique cryptographic certificate, issued when it joins the cluster, and all manager-to-worker communication is authenticated and encrypted using those certificates. So the encryption key for your overlay network rides inside an already-encrypted channel to reach every host that needs it. This is a subtle but important point: the swarm’s control plane is always encrypted via mTLS, regardless of whether you turn on data-plane (overlay) encryption — data-plane encryption is a separate, opt-in decision layered on top.

Step three: the kernel configures a Security Association for every host pair

Once a host has the key, Docker instructs the Linux kernel’s IPsec subsystem to establish a Security Association with every other host it needs to talk to on that overlay network. Practically, this means: “for any VXLAN packet leaving this machine destined for host X on UDP port 4789, wrap it in an ESP header and encrypt the payload using this specific key before sending; for any incoming ESP packet from host X, decrypt it using this key before handing the inner VXLAN packet up to the networking stack.” This configuration happens automatically and invisibly — no administrator ever manually configures an IPsec tunnel the way they might for a traditional site-to-site VPN.

Step four: the actual encapsulation, encryption, and transmission

Here is the full technical picture, expressed as a sequence of wrapping steps, from the inside out:

• Original packet — the raw data your application sends (for example, an HTTP request from one microservice to another).
• Ethernet frame — the container’s virtual network interface wraps this in a standard Ethernet frame, as if it were on a normal local network.
• VXLAN encapsulation — the VTEP wraps that entire Ethernet frame inside a UDP packet (destination port 4789 by default), adding a VXLAN header that identifies which virtual network this traffic belongs to.
• IPsec ESP encryption — before this UDP packet leaves the physical network interface, the kernel’s IPsec engine encrypts its payload using AES-256-GCM and wraps it with an ESP header, which also includes an authentication tag to detect any tampering.
• Physical transmission — the now-encrypted packet travels across the real network — a switch, a router, the public internet, a cloud provider’s backbone — completely opaque to anyone intercepting it.

flowchart LR
    A["App Data"] --> B["Ethernet Frame"]
    B --> C["VXLAN + UDP Encapsulation"]
    C --> D["IPsec ESP Encryption AES-256-GCM"]
    D --> E["Encrypted Packet on the Wire"]
    E --> F["Remote Host: IPsec Decrypt"]
    F --> G["VXLAN Decapsulation"]
    G --> H["Ethernet Frame Delivered to Container"]
    

On the receiving host, this entire sequence runs in reverse: the ESP header is verified and decrypted, the VXLAN header is inspected to identify the correct virtual network, the inner Ethernet frame is extracted, and finally the original application data is delivered to the destination container — a process that, end to end, typically adds only a fraction of a millisecond of latency on modern hardware, thanks to CPUs that include dedicated AES instruction sets (AES-NI) built directly into the processor.

Let’s trace a single request across its entire lifecycle, and separately look at how keys themselves live and die.

Why key rotation exists at all

Every symmetric cipher has a theoretical amount of data it can safely encrypt under a single key before certain cryptographic guarantees start to weaken — a concept cryptographers call “key exhaustion.” AES-256-GCM is extremely robust, but good security hygiene doesn’t wait for a theoretical limit to become a practical one. Rotating keys on a schedule also limits the “blast radius” of a compromised key: if a key were somehow leaked, it would only be useful to an attacker for the remaining hours before the next scheduled rotation, rather than for the lifetime of the cluster.

Lifecycle at the packet level, one more time

It helps to zoom back out. Every single packet that crosses the encrypted overlay, from the very first one to the very last, follows the identical six-stage journey described in the previous chapter: original data, Ethernet framing, VXLAN encapsulation, IPsec encryption, physical transmission, and full reversal on arrival. What changes over the lifetime of the network is not this per-packet journey, but the key being used to protect it, and the set of hosts that are currently allowed to participate at all.

No engineering decision is free. Turning on overlay encryption is a trade — here is exactly what you’re trading.

When the trade genuinely favors turning it on

The calculus tips firmly toward enabling encryption whenever your swarm’s nodes are not all sitting behind the exact same, fully trusted physical network boundary — for example, nodes spread across multiple cloud availability zones, multiple data centers connected by a WAN link you don’t fully control, or any environment subject to regulatory data-protection requirements. In these cases, the modest CPU cost is a small price for closing off an entire category of network-level eavesdropping and tampering attacks.

When it’s reasonable to leave it off

If every node in your swarm lives inside a single rack, connected by switches you physically control, inside a data center with its own strict physical access controls, and your compliance requirements don’t explicitly demand transit encryption, the marginal security benefit may not justify the CPU overhead — particularly for extremely latency-sensitive, high-throughput workloads such as certain real-time trading or scientific computing systems. Even then, many teams still enable it as a defense-in-depth default, treating the performance cost as acceptable insurance.

This is the heart of the topic — what exactly are we defending against, and how well does the defense hold up?

The specific threats this feature addresses

Overlay encryption is a direct answer to network-level eavesdropping and tampering threats, sometimes grouped under the umbrella term “man-in-the-middle” attacks. Concretely, it defends against:

What it does not protect against

It’s just as important to be precise about the limits of this protection as it is to understand its strengths. Overlay network encryption does not protect against an attacker who has already gained a foothold inside a container or host on the network — at that point, the attacker is “inside the armored truck” and can see the same unencrypted, decrypted data any legitimate process on that host would see. It also does nothing to protect application-layer secrets that are poorly handled to begin with — hardcoded passwords, tokens logged in plaintext, or weak authentication between services are entirely separate problems that overlay encryption cannot solve.

Mutual TLS: the control plane’s own, always-on protection

It’s worth restating clearly: the swarm’s management traffic — the conversations managers and workers have about cluster state, scheduling decisions, and yes, encryption key distribution — is protected by mutual TLS at all times, from the moment you run the command to initialize a swarm. Every node authenticates every other node using X.509 certificates, automatically issued and automatically rotated by Docker’s built-in certificate authority. This means the key that unlocks your data-plane encryption is itself always delivered over an already-secure channel — there is no “bootstrapping problem” where the very first key exchange happens in the clear.

Interviewer’s Corner

A question that comes up often in system design interviews: “Why does Docker use IPsec instead of just running everything over a full VPN mesh like WireGuard?” A strong answer notes that IPsec’s xfrm framework is already built into every Linux kernel Docker supports, requires no additional daemon or user-space process, integrates natively with the existing VXLAN data path Docker already uses for routing, and has decades of production hardening — making it the pragmatic, lowest-friction choice for a feature that needs to work identically across every supported Linux distribution.

You can’t operate what you can’t observe. Here’s how teams keep confidence that encryption is actually working as intended, day after day.

What “healthy” looks like

Because overlay encryption operates deep in the kernel’s networking stack, most application-level monitoring tools never directly see it — which is exactly the point of a transparent security layer. Operational visibility instead comes from a small number of specific signals.

Why encrypted traffic complicates traditional packet capture

Teams used to running a packet-sniffing tool to debug connectivity issues will notice something important the first time they try it on an encrypted overlay network: the captured packets show only opaque, encrypted ESP payloads — you can see that two hosts are talking, and roughly how much data is flowing, but not what is actually being said. This is, again, precisely the intended behavior, but it does mean troubleshooting workflows have to shift toward the kernel’s own IPsec state and Docker’s network metadata, rather than reading raw packet contents.

Practical operational checklist

• Periodically confirm that networks intended to be encrypted still show the encrypted flag set — configuration drift during network re-creation is a real, if uncommon, risk.
• Track key rotation events over time to confirm rotation is happening on schedule, rather than silently failing.
• Baseline CPU usage before and after enabling encryption on production-representative traffic, so any future spike is easy to recognize as abnormal.
• Alert on any unexpected drop in the number of active Security Associations, which can indicate a node losing connectivity or falling out of the swarm unexpectedly.

Patterns that experienced platform teams gravitate toward, and traps that trip up newcomers.

The “trust the cloud provider” anti-pattern

A particularly common trap is assuming that because a cluster runs entirely inside one cloud provider’s virtual private network, overlay encryption is redundant. This overlooks that a VPC boundary protects against outside attackers, but does very little against a misconfigured security group, a compromised neighboring workload with unexpected network access, or the cloud provider’s own internal infrastructure issues. Overlay encryption is a control you own and verify yourself, independent of any provider’s shared-responsibility promises.

Concrete, field-tested guidance for anyone about to flip the switch on a real cluster.

How this pattern shows up, in spirit if not always in this exact implementation, across the industry.

The common thread across all of these examples is not any single company’s specific configuration, but the underlying principle: as soon as a system’s trust boundary extends across more than one physically secured location, encrypting the traffic that crosses that boundary stops being optional and starts being table stakes.