Before comparing drivers, it helps to know exactly what job a storage driver is hired to do.

A Docker image, as covered by the concept of layering, is a stack of independent, read-only pieces. A running container needs two things a plain stack of files cannot provide on its own: it needs that stack to look and behave like one single, ordinary filesystem to any program running inside it, and it needs somewhere to write new files and changes without disturbing the original, shared image underneath. The component responsible for solving both problems — presenting a unified view and handling writes safely — is the storage driver, sometimes called the graph driver.

Several storage drivers have existed across Docker’s history — overlay2, aufs, devicemapper, btrfs, zfs, vfs, and fuse-overlayfs among them — and each one solves the same two problems using a different underlying filesystem technology, with different trade-offs in speed, disk efficiency, and compatibility. overlay2 has been the default and recommended storage driver on modern Linux systems for several years, because it offers the best balance of performance, stability, and kernel support for the overwhelming majority of workloads. Understanding overlay2 deeply, and understanding why the alternatives exist at all, is the goal of this article.

The term “graph driver” is older terminology that still appears throughout documentation, source code, and command-line tooling output, and it refers to exactly the same component as “storage driver.” The name comes from an early internal design where the relationships between image layers were modeled as a graph, with each layer pointing back to its parent. Even though the surrounding implementation has evolved considerably since then, the name stuck, and a beginner reading Docker’s own diagnostic output for the first time should not be thrown off by seeing “graph driver” and “storage driver” used interchangeably to describe the same underlying concept.

The storage driver is one layer inside a taller stack of software, sitting directly between the container runtime and the host’s real filesystem.

flowchart TB
    A["Application Process Inside Container"] --> B["Container Runtime (e.g. containerd, runc)"]
    B --> C["Storage Driver (overlay2, devicemapper, btrfs, ...)"]
    C --> D["Host Kernel Filesystem (ext4, xfs, btrfs, zfs)"]
    D --> E["Physical or Virtual Disk"]
        

For overlay2 specifically, four named directories do all the work, and their names are worth learning because every piece of documentation and every troubleshooting guide uses them directly. The lowerdir is the stack of read-only image layers, ordered from oldest to newest. The upperdir is the single writable layer created fresh for one specific container. The workdir is an internal scratch area the kernel’s overlay filesystem implementation uses to safely prepare changes before they are committed into the upperdir — it is invisible to the container itself and exists purely for the mechanics of atomic updates. The merged directory is the final, combined view: the single unified folder tree that actually gets mounted into the running container, and the only one of the four the container’s own processes ever interact with.

overlay2 is, itself, a thin layer of Docker-specific logic sitting on top of a filesystem feature already built directly into the Linux kernel, called OverlayFS. Docker did not invent stacked, copy-on-write filesystems; it adopted a mature kernel capability and wrote the bookkeeping layer that maps “this image has these layers, in this order” onto “mount these kernel directories in this arrangement.” This is an important architectural fact: overlay2’s performance characteristics are, in large part, the Linux kernel’s own OverlayFS performance characteristics, not something Docker independently reimplemented from scratch.

This lineage also explains why overlay2 replaced an earlier driver named simply “overlay.” The original overlay driver supported only a single lower layer, which meant Docker had to work around that limitation with awkward, inefficient techniques to represent an image’s full stack of many layers. Once the Linux kernel itself gained native support for multiple stacked lower directories, Docker built overlay2 specifically to take advantage of that capability directly, mapping an image’s entire layer history onto the kernel’s multi-layer lowerdir feature in one clean step rather than working around a single-layer restriction. This is a useful pattern to notice: several of Docker’s storage-driver decisions over the years have been downstream of, and directly enabled by, capabilities the Linux kernel itself gained over time.

Two operations explain almost everything about how overlay2 behaves: reading a file, and writing to one that already exists in a lower layer.

When a process inside a container asks to read a file, the kernel’s OverlayFS implementation searches the stacked directories from the top down — starting at the upperdir, then working backward through the lowerdir layers from newest to oldest — and returns the very first copy of that file it finds. If the file has never been modified inside this particular container, the search finds it in one of the shared, read-only lower layers, and the read is served directly from there, at essentially native filesystem speed, with no copying involved at all.

When a process asks to write to a file that already exists only in a lower, read-only layer, something more expensive has to happen first, called copy-on-write, or “copy-up.” The kernel must first copy the entire file, in full, up from wherever it currently lives in the lower layers into the upperdir, and only then apply the requested change to that new upper copy. From that point forward, every future read and write to that file is served directly from the upperdir copy, and the original in the lower layer is left completely untouched, forever.

Renaming a file behaves similarly to a write for this purpose, because a rename that moves a file from a lower layer into a new location is, under the hood, implemented as a copy-up of that file followed by the rename operation applied to the newly copied-up version. This is a subtle but practical detail: a workload that appears to only be “renaming” large files, without touching their contents at all, can still trigger the same expensive copy-up behavior as an outright content modification, simply because the rename itself has to happen against a writable copy rather than the original read-only one.

Deleting a file that exists in a lower layer works through a related mechanism called a whiteout. Because the lower layers are strictly read-only, a delete cannot physically remove the file from them. Instead, the kernel creates a special marker file inside the upperdir with the same name, which tells the merged view “treat this path as deleted, even though a copy still technically exists further down the stack.” This is the exact same underlying mechanism, at the container’s writable-layer level, that makes image-layer deletions behave the way Chapter 5 of any layering discussion describes: hidden from view, but not physically gone until the whole layer is discarded.

A closely related concept, worth knowing even at a conceptual level, is the opaque directory. If a whole directory is deleted and then a new, different directory is created at that exact same path, the kernel needs a way to say “do not merge this new directory’s contents with whatever used to be at this path in a lower layer — treat it as an entirely fresh start.” It marks the new upperdir directory as opaque to signal exactly that, preventing old, deleted contents from a lower layer from confusingly reappearing underneath a directory that was deliberately replaced. Beginners rarely need to manipulate this directly, but understanding that it exists explains why directory replacement, not just file replacement, behaves predictably under a layered filesystem.

Following one image through its full lifecycle shows exactly when the storage driver gets involved, and when it deliberately steps aside.

First, when an image is pulled, the storage driver unpacks each downloaded layer archive onto the host’s disk, in the location the driver manages, and records how those layers stack together. Second, when a container is created, the storage driver allocates a brand-new, empty upperdir and workdir specifically for that container, and mounts the full lowerdir-upperdir-workdir arrangement into a single merged view. Third, while the container runs, every file operation from the application inside it flows through the read and copy-up behavior described in Chapter 3. Fourth, when the container is removed, its upperdir and workdir are deleted entirely — every change that container ever made simply vanishes, while the shared, read-only lowerdir layers remain completely untouched and ready to be reused by the next container started from that same image.

sequenceDiagram
    participant Img as Image Layers (lowerdir)
    participant Drv as Storage Driver
    participant C as Container
    Img->>Drv: Layers unpacked and cached on pull
    Drv->>C: New upperdir + workdir created on container start
    Drv->>C: Merged view mounted (read from lowerdir, write to upperdir)
    C->>Drv: Container removed
    Drv->>Drv: upperdir and workdir deleted; lowerdir untouched and reused
        

This disposability is precisely why containers are described as ephemeral by default, and it is exactly where volumes enter the picture, deliberately stepping outside the storage driver’s territory. A volume is a directory managed directly by the container engine, living outside the layered upperdir/lowerdir system entirely, and mounted directly into a container’s filesystem at a chosen path. Because a volume is not part of the copy-on-write layering, writes to it never trigger a copy-up, and its contents survive even after the container that used it is removed. Databases, user-uploaded files, and any data meant to outlive a single container’s life belong in a volume, precisely so they never have to pass through the storage driver’s layering machinery at all.

Two closely related mechanisms are worth distinguishing here, since both bypass the storage driver but serve different purposes. A named volume is fully managed by the container engine itself, with its storage location decided and tracked by the engine, and is the recommended default for most persistent application data because it is portable and easy to back up, inspect, or move between hosts. A bind mount, by contrast, maps a specific, pre-existing path on the host’s own filesystem directly into the container, giving a container direct access to a folder that already exists outside Docker’s management entirely — commonly used for local development, where a developer wants a container to see live edits made directly on their own machine. Both sit outside the storage driver’s copy-on-write layering, but a named volume is the engine’s own persistent storage, while a bind mount is simply a window into a folder the host already had.

overlay2 is the right default for most workloads, but it is a default, not a universal answer — knowing the alternatives clarifies why.

It is worth pausing on why vfs still exists at all, given how clearly it loses on both speed and disk efficiency compared to every copy-on-write driver in the table above. vfs performs no copy-on-write sharing whatsoever: every single layer is fully, physically duplicated for every container that uses it, which is slow to set up and wasteful of disk space. Its value is narrow but real — it works reliably on essentially any filesystem, including ones that lack copy-on-write support entirely, which makes it a dependable fallback and a useful tool for isolating whether a performance problem is actually caused by copy-on-write layering at all, by comparing behavior against a driver that has none.

One overlay2 advantage deserves a closer look because it is easy to underestimate: shared page-cache usage. When the operating system reads a file into memory for caching purposes, and two different containers both read that same underlying file from a shared lower layer, the kernel is able to reuse the exact same cached memory pages for both of them, rather than caching two separate copies. Storage drivers that fully duplicate layer data per container, such as vfs, lose this benefit entirely — every container ends up with its own physically separate copy of what should be identical, shared data, both on disk and in memory. On a host densely packed with many containers built from the same base image, this single difference can be the deciding factor in how many containers that host can comfortably run at once.

The storage driver is part of the isolation boundary between a container and its host, and between one container and another — its correctness is a security property, not just a performance one.

Because every container’s writable upperdir is created fresh and deleted along with the container, the storage driver is directly responsible for making sure one container’s writes never leak into another container’s view, and never leak back into the shared, supposedly-immutable image layers. A storage driver bug or a misconfiguration that breaks this isolation could allow a process in one container to see or modify data belonging to a completely different container sharing the same underlying image, which is precisely the kind of cross-container isolation break that container security depends on never happening.

Rootless container setups — where the entire container engine and every container it runs operate without any root privileges on the host at all — deliberately cannot use the standard kernel overlay2 driver in many configurations, because mounting kernel overlay filesystems has traditionally required elevated privileges. This is precisely why fuse-overlayfs exists: it reimplements overlay-style layering in ordinary userspace, using the FUSE (filesystem in userspace) mechanism, so that a completely unprivileged user can still get layered, copy-on-write container storage. Choosing fuse-overlayfs is therefore often not a performance decision at all, but a direct consequence of a security decision to run rootless in the first place.

Mandatory access control systems such as SELinux and AppArmor add another layer of interaction worth knowing about. These systems apply their own security labels and policy rules to files and processes independently of the storage driver, and a correctly configured host must ensure those labels are applied consistently to files as they move through copy-up operations and across the merged view a container actually sees. A storage-driver or configuration mismatch here can either silently weaken intended isolation or, just as disruptively, cause legitimate container operations to be unexpectedly denied. This is one of the more common sources of confusing, hard-to-diagnose permission errors when moving a containerized workload between hosts with different security-module configurations, and it is a direct reminder that the storage driver does not operate in isolation from the rest of the host’s security stack.

Storage driver problems tend to appear as vague symptoms elsewhere — slow writes, mysterious disk pressure — so knowing exactly what to watch turns guesswork into diagnosis.

The first and most direct signal is simply disk usage growth under the storage driver’s own data directory on the host, tracked over time and separately from any data sitting in dedicated volumes. Because every stopped-but-not-removed container leaves its upperdir sitting on disk, and because unused, dangling image layers are not automatically reclaimed the instant they become unreferenced, a host that never runs cleanup can quietly accumulate enormous amounts of storage-driver-managed disk usage that has nothing to do with active application data at all.

It is worth explicitly naming why this matters more than it first appears: a host can be running perfectly healthy applications, with no errors in any application log, while its available disk space steadily shrinks purely because of accumulated storage-driver overhead. By the time that shrinking disk space actually causes a visible failure — a new container failing to start because there is no room left to create its writable layer — the underlying cause has often been building quietly for weeks. Tracking this metric proactively converts a sudden, confusing outage into a routine, predictable maintenance task scheduled well in advance.

Second is write latency, ideally measured from inside representative containers rather than assumed from host-level disk benchmarks alone. A workload that suddenly starts feeling slow on writes, despite the underlying disk hardware being healthy and fast, is a classic symptom of repeated, expensive copy-up operations against large files that keep getting freshly modified inside the writable layer instead of living in a dedicated volume.

Third is layer count and stack depth per running image. Some storage-driver and kernel combinations impose a hard ceiling on how many layers can be stacked in a single overlay mount; approaching that ceiling can cause container start failures that look, at first glance, completely unrelated to layering at all. Tracking maximum layer depth across an organization’s images as a simple metric catches this class of problem long before it causes a production incident.

Most container engines also expose a built-in, human-readable disk-usage breakdown on request, separating space consumed by images, by stopped containers’ writable layers, and by volumes into distinct categories. This single, readily available breakdown is often the fastest first step in any storage investigation, because it immediately answers the most important triage question — is this unexpected disk usage coming from the images themselves, from accumulated container writable layers, or from data sitting in volumes — before any deeper, layer-by-layer inspection is needed at all.

These are the proven, repeatable configuration patterns experienced platform teams reach for, roughly in order of impact.

Together, these five patterns describe a coherent operational posture: trust the well-tested default, keep expensive writes out of the copy-on-write path entirely, keep the layer stack itself lean, never let cleanup become optional, and let the security posture of a host decide its storage driver rather than the other way around. None of them requires exotic tooling — they are disciplines, not products.

Every one of these anti-patterns is the exact mirror image of a pattern from Chapter 8, which is why recognizing the pattern makes the anti-pattern obvious.

A condensed, practical checklist — what to consistently do, and where teams most often slip.

Storage driver choice and tuning is a well-documented, practical lever that large engineering organizations have pulled to solve concrete production problems.

For the large majority of workloads on modern Linux kernels, yes. It offers the best general-purpose mix of speed, stability, and support. The exceptions are specific: hosts needing native filesystem snapshotting may prefer btrfs or zfs, and rootless setups often need fuse-overlayfs instead, since the standard kernel overlay driver has traditionally required elevated privileges to mount.

This is almost always copy-on-write cost. If those files originally came from the image’s read-only layers and are large, the very first write to each one pays the cost of copying the entire file up into the writable layer before the change can be applied. Once copied up, later writes to that same file are fast again. Moving that data path onto a dedicated volume removes the copy-up cost entirely.

Everything written into that container’s writable layer is deleted along with the container itself, because the writable layer is created fresh per container and is never part of the shared, persistent image layers. Any data meant to survive the container’s removal has to live in a volume, which is intentionally stored outside this disposable writable layer.

No, and this is a common point of confusion. The storage driver manages the layered image filesystem and each container’s disposable writable layer. A volume driver is a separate mechanism for managing persistent storage that lives outside that layered system entirely, and it can even be backed by a completely different underlying storage technology, such as networked or cloud-based storage, independent of whichever storage driver the host happens to be using.

Generally, yes, in the sense that existing images and any data sitting inside container writable layers are stored in a format specific to the driver that created them, and switching drivers on a host typically means starting from an empty storage area under the new driver. This is exactly why storage-driver choice is treated as a foundational, host-provisioning-time decision rather than something casually changed on a running production system, and why volumes — kept driver-independent — are the correct home for anything that must survive such a change.

Not dramatically for ordinary read operations under overlay2’s typical stacking depths, since the kernel’s lookup is efficient. But it is not free either, and extremely deep stacks can approach hard limits on some driver-and-kernel combinations, in addition to simply making an image harder to reason about and maintain. Keeping layer stacks reasonably shallow, as covered in Chapter 8, remains good discipline even though the per-layer cost of ordinary reads is small.

Almost always for one specific capability overlay2 does not natively provide. btrfs and zfs offer native, filesystem-level snapshotting and strong built-in data-integrity checks that some teams want directly at the storage layer rather than layered on top separately. devicemapper’s role today is mostly legacy, kept running on older systems that adopted it before overlay2 matured, rather than chosen fresh for new deployments. In every case, the decision should be traceable to a specific requirement overlay2 does not meet, not a general preference.

A single running container engine instance typically operates under one active storage driver at a time for its managed images and containers, so mixing drivers usually means running genuinely separate engine instances or separating workloads onto different hosts entirely, rather than switching per container. This is one more reason standardizing on one driver per host, as covered in Chapter 8, keeps operations simpler than trying to run a mixed configuration.