Ephemeral Inline Volumes in Kubernetes
Storage that is born with a Pod and disappears with it. A complete, step-by-step guide to scratch space, memory-backed folders, inline CSI volumes, and generic ephemeral volumes: how each one works, when to use it, and how to keep it safe and fast.
Imagine you check in to a hotel for the weekend. The hotel gives you a room, a small safe, and a notepad. You can store things, write notes, and spread your papers across the desk. When you check out, the staff clears the room, empties the safe, and throws away the notepad pages. You never worry about cleaning up, because the room was always meant to be temporary. Nobody expects the notepad to be there when the next guest arrives.
Many programs running in Kubernetes need exactly this kind of temporary space. A video converter needs a large work area for half-finished files. A web server needs a small folder to write temporary files while its main filesystem is locked as read-only for safety. A security tool needs a private folder holding a secret that must vanish the moment the program stops. These needs are served by ephemeral volumes, which share the life of a Pod: created when the Pod starts, removed when the Pod is gone.
In this guide you will learn every kind of ephemeral volume Kubernetes offers, from the humble emptyDir folder to inline CSI volumes and generic ephemeral volumes that are backed by real storage systems. You will see how each is created, how the kubelet and controllers cooperate behind the scenes, how storage limits and evictions work, how to choose the right type, and how to avoid the mistakes that cause Pods to be evicted or to hang. Every new word is explained the first time it appears, so you can read from top to bottom with no earlier knowledge.
1The Big Idea: What Are Ephemeral Inline Volumes?
Storage declared inside the Pod itself, with a life that matches the Pod.
A volume in Kubernetes is a folder that a container can read and write. Containers by themselves have a private filesystem that disappears when the container restarts, so volumes exist to give containers somewhere to keep files that must survive a restart, be shared between containers, or come from outside. Volumes come in two big families. Persistent volumes outlive any Pod and keep data for as long as you want. Ephemeral volumes live and die with the Pod that uses them.
The word inline describes how the volume is declared. An inline volume is written directly inside the Pod’s specification, in the same file as the containers. You do not need to create a separate object ahead of time, ask an administrator for a claim, or clean up afterward. You describe the volume next to the container that uses it, and Kubernetes does the rest. When the Pod is deleted, the volume goes with it.
Think of a hotel room versus a rented home. A rented home (a persistent volume) is yours until you decide to leave, and you have to arrange it separately from your trip. A hotel room (an ephemeral volume) is booked as part of your trip itself, comes with your reservation, and is cleaned out when you leave. Inline means the room is written on the same booking form as your flight.
The four main kinds in one sentence each
The simple emptyDir gives a blank folder on the node. The configMap, secret, downwardAPI, and projected volumes fill a folder with configuration or identity data. The CSI ephemeral volume asks a storage driver to prepare a folder, with driver-specific options written inline. The generic ephemeral volume asks Kubernetes to create a real, dynamically provisioned storage claim for the Pod and delete it afterwards.
Ephemeral does not mean useless. It means the data does not need to outlive the Pod. If losing the data when the Pod ends is acceptable, ephemeral volumes are often the simplest, cheapest, and safest choice.
2Why Ephemeral Volumes Exist: The Problems They Solve
Containers are born with an empty, private, short-lived filesystem. Real programs need more than that.
When a container starts, it gets a thin writable layer on top of its image. This layer is private to the container, is lost when the container is recreated, and is usually slower and less controlled than a proper volume. For many jobs, that is not enough. Let us look at the common needs one by one.
Scratch space
Sorting a huge file, unpacking an archive, or building a cache needs room to work. The data is useless after the job ends.
Sharing between containers
An init container downloads data, and the main container reads it. Both need a common folder inside the Pod.
Read-only root filesystem
Security rules often lock the container’s main filesystem. Programs that still need a temporary folder get one through an ephemeral volume.
Configuration files
Applications expect settings as files. A volume can turn a ConfigMap into a folder of files.
Secrets that must not touch disk
Passwords and keys should live only in memory and disappear when the Pod ends.
Big, fast, per-Pod disks
Some jobs need much more space than the node’s local disk offers, or need a particular speed or encryption, and the disk should vanish with the Pod.
Why not just use persistent volumes for everything?
A persistent volume needs a separate object, a claim, which someone must create, name, and delete. If a job creates a thousand claims and forgets to delete them, you pay for a thousand idle disks. Persistent volumes also tie a workload to a specific piece of storage, which makes scaling and rescheduling harder. For temporary data, all of this ceremony is wasted effort. Ephemeral volumes remove the paperwork by letting the Pod itself carry the storage request.
If you only need a table for a one-hour meeting, you do not buy a house and rent a lawyer. You book a meeting room that is cleaned automatically afterwards. Persistent volumes are the house. Ephemeral volumes are the meeting room.
Pipeline tools such as Argo Workflows and Tekton use emptyDir volumes heavily to pass files between the steps of a pipeline inside one Pod. Many machine learning and data processing teams use ephemeral volumes as scratch space for training data shards, intermediate files, and model caches.
3Core Vocabulary for Ephemeral Storage
Twelve short words that you will meet in every storage conversation.
Pod
The smallest deployable unit. One or more containers that share a network and their volumes.
Volume
A folder made available to containers in a Pod, listed once in the Pod and mounted into containers where needed.
Mount
Attaching a volume to a path inside a container, so the container sees the folder at that path.
Kubelet
The program on every node that starts containers and prepares volumes for Pods.
CSI
Container Storage Interface. A standard way for storage vendors to plug their systems into Kubernetes.
CSI driver
A program from a storage vendor that follows the CSI standard. It runs on nodes and, often, in the control plane.
PersistentVolumeClaim
A request for storage with a size and a class. Also called a PVC.
PersistentVolume
An actual piece of storage that satisfies a claim. Also called a PV.
StorageClass
A recipe that says which driver, which disk type, and which options to use when storage is created on demand.
Dynamic provisioning
Creating storage automatically when it is requested, rather than having an administrator prepare it in advance.
Ephemeral storage
In Kubernetes accounting, the node-local space used by a Pod for writable layers, logs, and ephemeral volumes.
Eviction
The kubelet removing a Pod from a node because it is using too many resources, such as disk space.
Do not confuse these two similar names
An ephemeral volume is storage that lives with a Pod. An ephemeral container is a completely different feature: a temporary debugging container that you attach to a running Pod. They share a word, not a purpose. Whenever you read “ephemeral” in Kubernetes, check which of the two is meant.
A hotel safe and a hotel handyman are both temporary guests of your stay, but one stores things and the other fixes things. Ephemeral volumes are the safe. Ephemeral containers are the handyman who visits to debug a problem.
4The Family of Ephemeral Volumes at a Glance
A map of the choices before we explore each one in depth.
flowchart TB
POD["Pod specification"] --> ED["emptyDir"]
POD --> CFG["configMap, secret, downwardAPI, projected"]
POD --> CSI["CSI ephemeral inline volume"]
POD --> GEN["Generic ephemeral volume"]
POD --> PVC["PersistentVolumeClaim reference"]
ED --> N1["Folder on node disk or in memory"]
CFG --> N2["Files filled by the kubelet from API objects"]
CSI --> N3["Folder prepared by a CSI driver on the node"]
GEN --> N4["Real claim created and deleted with the Pod"]
PVC --> N5["Persistent storage that outlives the Pod"]
| Type | Who prepares it | Where the data lives | Typical use |
|---|---|---|---|
| emptyDir | Kubelet | Node disk, or memory if requested | Scratch space, shared folder, temporary files |
| configMap, secret, downwardAPI, projected | Kubelet | Node, typically memory-backed | Configuration, credentials, Pod information, tokens |
| CSI ephemeral (inline) | CSI driver on the node | Driver decides | Secrets from vaults, certificates, special mounts |
| Generic ephemeral | Controller plus any dynamic provisioner | Any storage system with a StorageClass | Big or fast per-Pod scratch disks |
| PersistentVolumeClaim | Administrator or user | Any storage system | Data that must outlive the Pod |
Think of the many places a traveler can keep belongings: a pocket (memory-backed), a backpack (emptyDir on disk), a hotel safe prepared by the front desk (CSI driver), and a rented locker with its own key and billing (generic ephemeral). A storage unit across town (persistent volume) is for things you want after the trip is over. You pick the place based on how long, how big, and how secure.
Start with the simplest type that meets your need. Most Pods need only emptyDir or a configMap or secret. Reach for CSI or generic ephemeral volumes only when you need something that simple types cannot give, such as a certain disk speed, a certain size, or integration with an outside vault.
5emptyDir Deep Dive: The Everyday Scratch Folder
The simplest ephemeral volume, and by far the most common.
An emptyDir volume starts as a blank folder, created when the Pod is assigned to a node. Every container in the Pod can read and write it, so it is also the standard way to share files between containers in one Pod. If a container crashes and restarts, the folder and its contents remain, because the Pod is still alive. The folder is deleted for good only when the Pod is removed from the node for any reason: deletion, eviction, or node loss.
The Pod is placed on a node
The scheduler assigns the Pod to a node. At this point no folder exists yet.
The kubelet creates the folder
Before starting the containers, the kubelet makes an empty directory under its own working area on the node, or a memory-backed filesystem if requested.
Containers mount it
Each container that lists the volume sees the same folder at the path it chooses. Files written by one are visible to the others.
Containers may restart, data stays
A crash or restart of one container does not touch the folder. This is the main difference from a container’s private filesystem.
The Pod ends and the folder is removed
The kubelet deletes the directory and everything in it as part of cleaning up the Pod.
Disk-backed versus memory-backed
By default, an emptyDir uses the node’s disk. You can ask for the memory medium instead, which creates a tmpfs, a filesystem held in RAM. It is very fast and never written to disk, which makes it good for small sensitive files and for latency-sensitive caches. But memory is scarce, and everything written there counts against the container’s memory limit. A runaway program filling a memory-backed emptyDir can cause the container to be killed for using too much memory.
| Setting | Disk-backed (default) | Memory-backed |
|---|---|---|
| Speed | Depends on the node disk | Very fast |
| Capacity | Limited by node disk space | Limited by memory available to the Pod |
| Counted against | Ephemeral storage limits | Memory limits |
| Survives a node reboot | No, Pod is gone anyway | No |
| Written to physical disk | Yes | Not normally |
| Best for | Large scratch files and shared data | Small sensitive files and fast caches |
A disk-backed emptyDir is a notebook on your desk: roomy and cheap, but slower to flip through. A memory-backed one is a whiteboard on the wall: instantly readable and wiped clean when you leave, but small, and everything you write takes space from the room’s limited wall.
The size limit
An emptyDir can have a size limit. For the memory medium, the limit shapes the size of the tmpfs. For the disk medium, the limit is checked periodically by the kubelet, and if the Pod’s use exceeds it, the Pod is evicted. This is not instant, so a fast writer can briefly overshoot before the check catches it.
An emptyDir without a size limit can fill the whole node disk, hurting every Pod on that node. Always set a size limit and a matching ephemeral storage limit for Pods that write substantial temporary data.
Production example: read-only root filesystems
Security-conscious teams run containers with a read-only root filesystem, so that an attacker who breaks in cannot change program files. Programs still need places to write temporary files, so teams mount small emptyDir volumes at paths such as the temporary directory. This pattern is widely used and recommended in container hardening guides.
6ConfigMap, Secret, DownwardAPI, and Projected Volumes
Volumes that the kubelet fills with data from the Kubernetes API.
configMap
Turns the keys of a ConfigMap into files. Each key becomes a file name and its value becomes the content.
secret
Does the same for a Secret. The kubelet keeps secret volumes in memory-backed storage so they are not written to the node’s disk.
downwardAPI
Exposes facts about the Pod itself, such as its name, labels, or resource limits, as files.
projected
Merges several sources, such as a ConfigMap, a Secret, and a token, into one folder.
How they stay up to date
These volumes are different from emptyDir because the kubelet keeps them in sync with the source objects. If you edit a ConfigMap, the files in running Pods change after a short delay, which depends on the kubelet’s refresh period and cache. The kubelet updates files atomically, so a program never reads a half-written file. There is one important exception: if a container mounts a single file from such a volume using a sub-path, that file is not updated. Programs that must pick up changes should mount the whole folder and watch for changes.
Imagine a notice board in the office that the receptionist keeps updated from the head office memos. People read the board whenever they like. When a new memo arrives, the receptionist swaps the whole sheet at once, so nobody ever sees half of an old memo and half of a new one.
Projected service account tokens
A very important use of projected volumes is the service account token. Pods use it to prove their identity to the Kubernetes API or to other systems. A projected token has a short, configurable lifetime, is limited to a chosen audience, and is automatically refreshed by the kubelet before it expires. It is also bound to the Pod, so it becomes invalid when the Pod is deleted. This is much safer than the old long-lived tokens that never expired.
| Feature | Benefit |
|---|---|
| Short lifetime with automatic refresh | A stolen token is useful only briefly |
| Audience restriction | A token meant for one service cannot be used for another |
| Bound to the Pod | Deleting the Pod invalidates its token |
| Immutable ConfigMaps and Secrets option | Protects from accidental edits and lowers load on the API server |
Cloud workload identity features, such as those that let a Pod assume a cloud role without static keys, work by projecting a token with a special audience into the Pod. The cloud provider checks the token and issues short-lived cloud credentials. No password is ever stored in the image or the cluster.
7Local Ephemeral Storage: Requests, Limits, and Eviction
Node disks are finite. Kubernetes keeps accounts so that one noisy Pod cannot fill them.
On every node, Kubernetes tracks a resource called ephemeral storage. It covers three things a Pod can use on the node’s local disk: the writable layer of its containers, the logs its containers produce, and the disk-backed emptyDir volumes. You can put a request and a limit on this resource for each container, just as you do for CPU and memory. The request helps the scheduler find a node with enough room. The limit protects the node by letting the kubelet evict a Pod that uses too much.
Ephemeral storage request
The amount the scheduler reserves for the Pod when picking a node. The node must have this much unreserved ephemeral space.
Ephemeral storage limit
The most the Pod may use. If the total use of the Pod goes beyond the combined limits, the kubelet evicts it.
Allocatable ephemeral storage
What is left for Pods after the node reserves space for itself and its system programs.
Hard eviction thresholds
When free space on the node falls below defaults such as ten percent for the main filesystem, the kubelet starts evicting Pods to protect the node.
What gets evicted first
When a node runs low on disk, the kubelet first tries to reclaim space by cleaning unused images and dead containers. If that is not enough, it evicts Pods. It chooses Pods that exceed their requests for the scarce resource first, and among those, lower priority Pods go before higher priority ones. A Pod that stays within its requests is the last to be chosen. The evicted Pod shows a message saying that its ephemeral local storage usage exceeds the limit, or that the node was low on resources.
A hostel has a shared storage room. Each guest is told how many boxes they may keep. When the room is nearly full, the manager first throws out old abandoned boxes. If it is still too full, guests who keep more boxes than they were allowed are asked to leave first. Keeping within your quota is the best protection.
Pods that write large logs or temporary files with no limits are the number one cause of nodes running out of disk. When a Pod is evicted for ephemeral storage, check three places: the container’s writable layer, the log volume, and emptyDir folders. Any of the three may be the culprit.
Production example: protecting shared build nodes
Companies that run continuous integration builds inside Kubernetes often give each build Pod an explicit ephemeral storage request and limit. Without limits, one build that unpacks a huge dependency tree can fill the disk and cause unrelated builds on the same node to be evicted. Limits turn a node-wide outage into a single failed build.
8CSI Ephemeral Inline Volumes: How They Work
A storage driver prepares a folder for your Pod, on request, using options you write directly in the Pod.
A CSI ephemeral volume uses the Container Storage Interface to let a driver prepare a folder for a single Pod. The Pod names the driver and supplies a set of free-form options, called volume attributes, whose meaning is up to the driver. There is no claim, no storage class, and no separate object. The volume exists only in the Pod specification, which is why it is called inline. This feature became generally available in Kubernetes 1.25.
sequenceDiagram
participant U as User
participant API as API Server
participant S as Scheduler
participant K as Kubelet
participant D as CSI node driver
U->>API: Create Pod with an inline CSI volume
API-->>S: Watch event for unscheduled Pod
S->>API: Bind Pod to a node
API-->>K: Watch event for Pod on this node
K->>D: NodePublishVolume with the volume attributes
D->>D: Prepare the folder and mount it
D-->>K: Volume ready
K->>K: Start containers with the volume mounted
Note over K,D: When the Pod ends the kubelet calls NodeUnpublishVolume
The Pod declares the volume
It names the driver and gives attributes. The API server stores it as written, and checks that the driver is allowed to be used this way.
The scheduler places the Pod
It treats the volume as having no special placement needs. It does not reserve any capacity for the volume, because it has no information about it.
The kubelet asks the driver to publish
On the chosen node, the kubelet calls the driver’s publish step, passing the attributes and a target path in the Pod’s volume area.
The driver prepares the folder
What it does depends on the driver: fetch a secret from a vault, generate a certificate, set up a special mount, or allocate local space.
Containers start
With the folder in place, the kubelet starts the containers. If the driver fails, the Pod stays stuck in container creation, with an event describing the failure.
The kubelet unpublishes at the end
When the Pod is removed, the kubelet calls the driver’s unpublish step so it can tear down the mount and clean up its own state.
Imagine telling the hotel concierge, in your booking note, “prepare a safe with my company’s key already inside.” The concierge’s team handles it when you arrive, and clears it when you leave. You wrote the request inline and never filled any other form.
What inline CSI volumes cannot do
Because only the node part of the driver is used, many storage features are unavailable. There is no storage class, no size request in the usual sense, no snapshots, no cloning, no resizing, and no capacity-aware scheduling. They are best seen as a way for a driver to hand a specially prepared folder to a Pod, not as a general purpose disk.
Not every CSI driver supports inline use. A driver must declare support for the ephemeral mode in its CSIDriver object. If you try to use inline mode with a driver that does not declare it, the Pod will not start.
Production example: Secrets Store CSI Driver
The Secrets Store CSI Driver, a Kubernetes SIG project, lets a Pod mount secrets stored in external vaults, such as those from cloud providers or HashiCorp Vault, as files. The Pod references a configuration object that names which secrets to fetch, and the driver retrieves them using the Pod’s identity at start time. The secret is never saved as a Kubernetes Secret unless you ask the driver to sync it.
9The CSIDriver Object: Telling Kubernetes What a Driver Can Do
Every CSI driver describes itself to the cluster through a small object that controls how the kubelet treats it.
Volume lifecycle modes
A list that says whether the driver supports persistent volumes, ephemeral inline volumes, or both. Inline use requires the ephemeral mode.
Pod info on mount
When enabled, the kubelet passes the Pod’s name, namespace, unique ID, and service account name to the driver, plus a flag that says the volume is ephemeral.
Token requests
The driver can ask the kubelet to create service account tokens for chosen audiences and pass them along, so the driver can act as the Pod when talking to outside systems.
Requires republish
The kubelet calls the driver’s publish step again from time to time, which lets drivers refresh content such as rotated secrets or renewed certificates.
File system group policy
Tells the kubelet whether it should adjust file ownership to match the Pod’s group settings or leave it to the driver.
Storage capacity
Tells Kubernetes whether the driver publishes capacity information that the scheduler may use for volumes it provisions.
Why the pod info setting matters
An inline volume has no object of its own to carry identity. Without the Pod’s name, namespace, and service account, a driver would not know whom it is serving. With the pod info setting turned on, the driver receives these details on every publish call and can look up the right secret, issue the right certificate, or enforce the right access rule.
A courier delivering a package needs to know the recipient’s name and address. The CSIDriver settings are the instructions on the delivery slip: whether to ask for ID, whether to come back later with updates, and what details to hand to the courier.
Rotation without restart
Certificates and secrets expire or are rotated. Restarting every Pod for each rotation would be disruptive. With the republish feature, the driver is called again periodically with the same inputs and may update the files in the existing folder. The application reads the new files without a restart, as long as it knows to re-read them. This is how secrets from vaults can stay fresh in long-running Pods.
Production example: cert-manager CSI driver and SPIFFE CSI driver
The cert-manager CSI driver gives each Pod its own freshly generated key pair and certificate, held only in the Pod’s volume, so that private keys are never stored in a Kubernetes Secret. The SPIFFE CSI driver, used with the SPIRE identity system, uses an inline volume to place a socket in the Pod through which the workload gets its cryptographic identity. Both use inline CSI volumes because the data must exist only for the life of one Pod.
10Generic Ephemeral Volumes: A Real Claim That Lives and Dies with the Pod
The most powerful ephemeral type: any storage system that supports dynamic provisioning can supply per-Pod storage.
A generic ephemeral volume lets you put a claim template right inside the Pod. It looks like the template used in a StatefulSet, but the result is tied to the Pod’s life. When the Pod is created, a controller creates a real PersistentVolumeClaim from the template, owned by the Pod. The usual storage machinery then provisions a real volume. When the Pod is deleted, the claim is deleted along with it, and the storage is released according to the rules of its storage class. This feature became generally available in Kubernetes 1.23.
sequenceDiagram
participant U as User
participant API as API Server
participant EC as Ephemeral volume controller
participant S as Scheduler
participant P as CSI provisioner
participant K as Kubelet
U->>API: Create Pod with a claim template
API-->>EC: Watch event for the new Pod
EC->>API: Create PVC owned by the Pod
API-->>S: Pod waits for its claim
S->>API: Choose node and mark claim for that node
API-->>P: Watch event for the claim
P->>P: Create the volume on that node or zone
P->>API: Create PV and bind the claim
API-->>K: Pod can start on its node
K->>K: Attach, mount and start containers
Note over API,EC: When the Pod is deleted the claim is garbage collected
The Pod is created with a template
The template contains the same details as a normal claim: size, access mode, storage class, and optional labels. It sits under the volume in the Pod.
The controller creates the claim
The ephemeral volume controller, part of the controller manager, notices the Pod and creates a claim named after the Pod and the volume. It marks the Pod as the owner.
The scheduler picks a node
With the recommended wait-for-first-consumer binding mode, the claim stays unbound until the scheduler chooses a node, so the storage is created in the right place.
The provisioner creates the storage
The CSI driver’s provisioner creates a real disk or volume, and a PersistentVolume object is made and bound to the claim.
The kubelet attaches and mounts
The volume is attached to the node if needed, mounted, formatted if new, and handed to the containers.
Deletion cascades
When the Pod is deleted, the garbage collector removes the owned claim, and the storage class’s reclaim policy decides whether the underlying volume is deleted or kept.
Think of a car rental booked together with your flight. You tick a box saying “give me a mid-size car with a child seat.” The travel agency makes the booking with the car company for you, tied to your trip. When your trip is cancelled, the car booking is cancelled automatically. The claim template is the tick box, and the owned claim is the booking made on your behalf.
Because the result is an ordinary claim, every storage feature that works with claims is available: any storage class, any driver that supports dynamic provisioning, volume expansion, snapshots as a data source, cloning, topology rules, and capacity-aware scheduling.
11Generic Ephemeral Volumes in Detail: Naming, Ownership, Quotas, and Topology
The fine print that decides whether these volumes behave well in a real cluster.
Naming and conflicts
The claim created for a Pod is named by joining the Pod name and the volume name with a dash. This keeps names predictable, and ensures that each Pod has its own claim. If a claim with that name already exists and is not owned by the Pod, the controller refuses to take it over, and the Pod will not start. This protects other people’s data, but can be confusing if someone created a claim by hand with the same name.
Ownership and cleanup
The claim carries an owner reference pointing to the Pod. The Kubernetes garbage collector deletes objects whose owner is gone, so the claim is removed after the Pod. If a Pod is force-deleted, the claim is still cleaned up later by the garbage collector. The storage class’s reclaim policy matters here: with the common delete policy, the underlying disk is deleted with the claim. With a retain policy, the disk stays behind and must be cleaned manually, which can quietly cost money.
ResourceQuota applies
Because a real claim is created in the Pod’s namespace, quotas on storage size and on the number of claims count it. Teams can be capped as usual.
Pod creation is enough
A user who may create Pods can indirectly create claims through templates, even without permission to create claims directly. Administrators should keep this in mind.
Wait for first consumer
This binding mode delays creating storage until the Pod is placed, so it lands in the right zone and on the right node.
Storage capacity tracking
Drivers can publish how much space each node or zone has left, so the scheduler avoids placing a Pod where its volume cannot be created.
Snapshots and cloning
A template can name a snapshot or another volume as its data source, giving each Pod a private, pre-filled copy of a dataset.
Default storage class
If the template names no class, the cluster’s default class is used. Check that the default suits your Pods.
Why wait-for-first-consumer is almost always right
If a claim is bound immediately, the provisioner has to guess where to create the disk. For local storage or zonal cloud disks, a wrong guess means the disk is in one zone while the Pod is scheduled in another, and the Pod cannot start. Waiting for the scheduler to choose a node first means the disk is created where the Pod will actually run. For ephemeral volumes, which are created together with the Pod, this is the natural choice.
Booking a hotel room before you know which city you will visit is a bad idea. You wait until your flight is fixed, then book near the airport. Wait-for-first-consumer is the rule “do not book the room until the trip destination is known.”
The lifecycle of the claim
stateDiagram-v2
[*] --> Created
Created --> Pending: Claim exists and waits for a node
Pending --> Bound: Provisioner creates the volume
Bound --> InUse: Pod starts and mounts it
InUse --> Terminating: Pod is deleted
Terminating --> Released: Claim removed by garbage collection
Released --> [*]: Volume deleted or retained by policy
Do not edit or delete the owned claim by hand while the Pod is running. The volume under a running Pod can become unusable. Treat the claim as part of the Pod, controlled by it.
Production example: fast local scratch with TopoLVM and cloud disks
Storage projects such as TopoLVM, which provisions local logical volumes on each node and is capacity-aware, document use as generic ephemeral volumes so that each Pod gets a private slice of the node’s fast disks. Teams on cloud platforms use the same mechanism with cloud block storage classes when they need larger scratch space than the node’s boot disk provides.
12Choosing the Right Type: A Decision Guide
Five questions lead to the right volume nearly every time.
flowchart TD
A["What do you need?"] --> B{"Must data outlive the Pod?"}
B -->|"Yes"| P["Use a PersistentVolumeClaim"]
B -->|"No"| C{"Is it configuration, a secret or Pod info?"}
C -->|"Yes"| D["Use configMap, secret, downwardAPI or projected"]
C -->|"No"| E{"Does a special driver prepare it, for example a vault or certificate?"}
E -->|"Yes"| F["Use a CSI inline ephemeral volume"]
E -->|"No"| G{"Need large size, special speed, snapshots or quotas?"}
G -->|"Yes"| H["Use a generic ephemeral volume"]
G -->|"No"| I["Use emptyDir, with a size limit"]
| Feature | emptyDir | CSI inline | Generic ephemeral | PVC |
|---|---|---|---|---|
| Survives Pod deletion | No | No | No | Yes |
| Setup effort | Lowest | Needs a suitable driver | Needs a StorageClass | Separate claim object |
| Start-up speed | Instant | Driver dependent, usually quick | Slower: provision, attach, mount | Fast if already bound |
| Size control | Size limit only | Driver dependent | Exact size in the template | Exact size |
| Scheduler awareness | Via ephemeral storage requests | None for the volume | Capacity tracking possible | Capacity tracking possible |
| Snapshots and cloning | No | No | Yes, as data source | Yes |
| Quota control | Ephemeral storage quota | Little | Claim and size quotas | Claim and size quotas |
| Best for | Scratch and sharing | Vault secrets, certificates | Big, fast, private disks | Data to keep |
Choosing storage is like choosing luggage. A pocket for a few coins, a backpack for a day trip, a hard-shell suitcase with a lock for a long journey, and a storage unit for what you want when you come home. The best choice is the smallest one that does the job.
Image volumes: a newer relative
A newer Kubernetes feature, the image volume, lets a Pod mount the contents of a container image or artifact as a read-only folder. It is useful for delivering large, read-only data such as machine learning model files, separately from the application image. It has been progressing through preview stages in recent releases, so check the status in your version before depending on it. It complements the other types because the data arrives already prepared, with the same caching and distribution as container images.
Mixing is normal. A single Pod can use an emptyDir for temporary files, a projected token for identity, a CSI inline volume for secrets, and a generic ephemeral volume for a large scratch disk, all at once.
13Performance and Scalability
Each type has a different start-up cost and a different effect on the cluster.
emptyDir is nearly free
Creating a directory takes almost no time. Data speed depends on the node disk, or on memory for the memory medium.
CSI inline is driver bound
Start-up time equals whatever the driver does: a vault call can take a second or more, and slow vaults slow every Pod start.
Generic ephemeral costs steps
Create a claim, provision a disk, attach, format, and mount. It can take seconds to minutes depending on the storage system.
More API objects
Every generic ephemeral volume creates a claim and a volume object. At high Pod churn, this increases load on the API server and its database.
Attach limits per node
Nodes can attach only a limited number of block devices. Many Pods with their own disks can hit this ceiling.
Ownership changes
Large recursive permission changes on big volumes slow start-up. Settings can limit this work to when it is truly needed.
Where the time goes for a generic ephemeral volume
Claim creation
Almost instant, since it is just an API object created by a controller.
Scheduling with a pending claim
The scheduler must evaluate storage constraints along with CPU and memory. Capacity tracking helps it avoid bad choices.
Provisioning
The cloud or storage backend creates the disk, which may take several seconds. This is often the biggest part.
Attach and mount
Cloud disks must be attached to the node and then formatted. Local disks skip the attach step and are much faster.
Setting up a pop-up shop in an empty hall is quick if you only need a table (emptyDir). It takes longer if you need a custom-built counter delivered and installed (generic ephemeral). Planning how long each setup step takes tells you which to choose for a job that must start in seconds.
For jobs that start in huge numbers and must begin fast, prefer node-local storage with generic ephemeral volumes, or plain emptyDir. For jobs that are few and heavy, cloud disks are acceptable, because start-up cost is small compared with the work.
Production example: machine learning data staging
Training platforms often stage dataset shards onto fast local NVMe scratch space before a run, since reading from local disk avoids network bottlenecks. Cloud providers offer instance types with built-in local SSDs for exactly this reason, and Kubernetes uses them as ephemeral storage or as local volumes behind a CSI driver.
14High Availability, Failure Scenarios, and Recovery
By definition, ephemeral data is lost with the Pod. The goal is to make the loss harmless and the recovery quick.
| What fails | Effect on ephemeral data | How it recovers |
|---|---|---|
| A container crashes and restarts | emptyDir data stays; container layer data is lost | Container restarts and finds its files in the volume |
| The Pod is deleted or replaced | All its ephemeral volumes are removed | The new Pod starts with fresh volumes; the application must rebuild what it needs |
| The node fails | All data on that node’s ephemeral volumes is lost | The controller creates a replacement Pod on another node |
| Pod evicted for disk pressure | The Pod and its emptyDir data are removed | The controller recreates it; fix limits to avoid repeats |
| CSI node driver is not running | Inline volumes cannot be mounted; the Pod stays in container creation | Restore the driver; the kubelet retries |
| Provisioner fails for a generic volume | The claim stays pending; the Pod waits | Fix the storage class, quota, or backend; provisioning is retried |
| Zone has no capacity | Pod cannot be scheduled with its volume | Use capacity tracking and several zones |
flowchart TD
A["Pod stuck in ContainerCreating or Pending"] --> B{"Is there a FailedMount event?"}
B -->|"Yes"| B1["Check CSI driver Pods on that node, driver name and attributes"]
B -->|"No"| C{"Is the claim Pending?"}
C -->|"Yes"| C1["Check storage class, provisioner logs, quota and zone capacity"]
C -->|"No"| D{"Was the Pod evicted?"}
D -->|"Yes"| D1["Check ephemeral storage limits, logs and emptyDir sizes"]
D -->|"No"| E["Check node health, attach limits and scheduler events"]
If you work at a hotel and a guest’s room has a flood, you move the guest to another room. The old room’s contents are lost, but the guest keeps what they carried. Designing for ephemeral storage means keeping anything precious in your suitcase (a persistent store or an outside system), not in the hotel room.
Designing for loss
Programs that use ephemeral volumes should assume the data can vanish at any moment. Caches should be able to rebuild themselves. Download steps should be repeatable. Results should be written to a permanent store as soon as they are ready, not left in the scratch area until the end. A job that writes ten hours of results to scratch space and uploads at the end loses everything if the Pod is evicted at hour nine.
Storing the only copy of important data in an emptyDir because “the Pod will be fine” is the classic mistake. Pods are replaced all the time during upgrades, node drains, and autoscaling. Anything you cannot afford to lose must be copied out regularly.
15Security of Ephemeral Volumes
Temporary does not mean harmless. Storage decisions are security decisions.
Data on node disks
Disk-backed emptyDir data sits on the node’s disk and can be read by someone with root access to the node. Use memory-backed storage or encrypted disks for sensitive scratch data.
Free choice of CSI driver
Inline CSI volumes let a Pod author name any installed driver and pass any attributes, without any claim or storage class review. A permissive cluster lets users reach drivers that should be restricted.
Indirect claim creation
Generic ephemeral volumes let Pod creators cause claims to be made, which could consume expensive storage if quotas are missing.
Leftover data
Reclaimed disks may hold data from the previous user. Make sure your storage backend clears or encrypts volumes between users.
Controls to put in place
Restrict allowed volume types
Admission policy can limit the volume types a namespace may use. The Pod Security restricted profile already limits the list of permitted types to a small set that includes emptyDir, configMap, secret, projected, claims, and generic ephemeral volumes.
Restrict inline CSI drivers by name
Use a validating admission policy or a policy engine to allow only approved driver names for inline volumes, and to check the attributes they carry.
Set quotas
Apply resource quotas for the number of claims, total requested storage, and ephemeral storage per namespace.
Use encrypted storage classes
For generic ephemeral volumes holding sensitive scratch data, choose a class that encrypts the underlying disk.
Keep secrets in memory
Prefer secret and memory-backed volumes over disk for credentials, and avoid copying secrets into scratch space.
Use read-only mounts and file permissions
Mount volumes read-only where possible, and set group ownership settings so unrelated processes cannot read shared folders.
A hotel that lets guests choose any key-making service in town, with any instructions, is asking for trouble. A careful hotel keeps a list of approved locksmiths. Policy on CSI driver names is that approved list.
The read-only root filesystem setting does not make emptyDir folders read-only. Whatever you mount as writable is writable. Review each writable path, and keep it as small and as specific as you can.
Production example: secrets without a Kubernetes Secret
Security teams that do not want secrets stored in the cluster’s database use inline CSI drivers to deliver them straight from a vault to the Pod’s memory-backed volume. The secret exists only on the node, only during the Pod’s life, and the vault keeps the audit log of who fetched it.
16Monitoring, Logging, and Metrics
Disk problems are slow burns. Good measurement turns surprises into early warnings.
| Signal | What it tells you | Warning sign |
|---|---|---|
| Node ephemeral storage usage versus allocatable | How full the local disk is | Above seventy to eighty percent and rising |
| Pod ephemeral storage usage versus its limit | Whether a Pod is nearing eviction | Close to the limit |
kubelet_volume_stats_used_bytes and capacity | Use of claim-backed volumes, including generic ephemeral ones | Used space close to capacity |
| Pod eviction events | Pods removed for resource pressure | Repeated evictions of the same workload |
| FailedMount and ProvisioningFailed events | Trouble preparing volumes | Any sustained occurrence |
| Number of Pending claims | Provisioning backlog | Claims pending for minutes |
| CSI driver operation latency and errors | Health of the storage driver | Rising latency or failures |
| Node attach count versus attach limit | How close a node is to its block device ceiling | Near the limit |
A fuel gauge warns you long before the engine stops. Disk usage graphs and eviction alerts are the fuel gauge of a node. A driver who only checks the tank when the car stalls has already lost the trip.
Events are the first place to look
When a Pod does not start because of storage, the describe output of the Pod shows events from the kubelet and the controllers. Messages such as a failed mount or a failed provisioning give a precise reason: a wrong driver name, a missing storage class, a quota exceeded, or no capacity in the zone. The exact text of the event usually tells you which team or component to contact.
Alerts that earn their keep
Node disk pressure
Warn before the kubelet’s own eviction thresholds trigger, so humans can react first.
Repeated evictions
Warn when the same workload is evicted several times, which signals wrong limits.
Claims pending too long
Warn when claims stay pending for more than a few minutes.
Driver errors
Warn on a rising error rate for mount or provisioning operations.
Popular monitoring stacks for Kubernetes include ready-made alerts for node filesystem space and for claims running out of capacity. Teams start from these and add alerts specific to their scratch-heavy workloads, such as a threshold on the ratio of ephemeral storage used to limit.
17Advantages, Disadvantages, and Trade-offs
Ephemeral volumes are convenient, and convenience always has limits.
Advantages
- No cleanup: storage disappears with the Pod, so nothing is left behind.
- Simple to declare: everything sits in the Pod specification.
- Fits scaling: every replica gets its own space without sharing or naming problems.
- Safe for sensitive data: memory-backed and driver-managed volumes keep secrets short-lived.
- Flexible: from a blank folder to a fully managed cloud disk.
- Cost friendly: you pay for storage only while it is in use.
Disadvantages
- Data loss by design: anything not copied out is gone.
- Hidden node pressure: disk-backed volumes can fill nodes without good limits.
- Slower start for generic volumes: provisioning adds delay.
- More API objects: generic volumes add claims and volumes to the cluster database.
- Limited features for inline CSI: no snapshots, resizing, or capacity awareness.
- Policy gaps: inline use of drivers can bypass storage class controls.
| Trade-off | Choice | What you gain | What you give up |
|---|---|---|---|
| Disk versus memory emptyDir | Memory | Speed and no disk traces | Costs memory and hits memory limits |
| emptyDir versus generic ephemeral | Generic ephemeral | Size, speed class, quota control | Slower start and more objects |
| Inline CSI versus Kubernetes Secret | Inline CSI | Secrets stay out of the cluster database | Depends on the driver and the vault being available |
| Local disk versus network disk | Local disk | Speed and fast start | Data is tied to the node |
| Strict limits versus no limits | Strict limits | Nodes stay healthy | Pods may be evicted when they overshoot |
| Retain versus delete reclaim policy | Delete | No orphaned disks | No forensic leftovers |
18Design Patterns and Anti-Patterns
A few habits make ephemeral storage reliable. A few others cause late night pages.
Patterns worth copying
Read-only root plus small emptyDir
Lock the container filesystem, and grant tiny writable folders only where the program needs them.
Init container fills, main container reads
An init container downloads or prepares data into an emptyDir, and the main container uses it.
Sidecar sharing
A log shipper or helper reads files written by the main container through a shared emptyDir.
Limits on everything
Size limits on volumes plus ephemeral storage requests and limits on containers.
Copy results out early
Write finished work to permanent storage as soon as each chunk completes.
Wait for first consumer
Use this binding mode for storage classes behind generic ephemeral volumes.
Anti-patterns to avoid
The mistake
Using emptyDir with no size limit and no ephemeral storage limit.
Why it hurts
A single runaway Pod can fill the node’s disk and cause evictions of unrelated Pods.
Do this instead
Set a volume size limit and container requests and limits for ephemeral storage, and alert on node usage.
The mistake
Keeping the only copy of valuable data in an ephemeral volume.
Why it hurts
Evictions, upgrades, and node failures delete it without warning.
Do this instead
Write results to permanent storage regularly and use persistent volumes for data that must survive.
The mistake
Putting large data in a memory-backed emptyDir without raising the memory limit.
Why it hurts
The files count as memory, so the container is killed for exceeding its limit when the folder fills up.
Do this instead
Use disk-backed storage for big data, or size memory limits to include the folder.
The mistake
Allowing any inline CSI driver to be used by any Pod.
Why it hurts
Users can reach drivers and options that were never meant for them, bypassing storage class review.
Do this instead
Admit only an approved list of driver names, and review the attributes they accept.
The mistake
Using a retain reclaim policy behind generic ephemeral volumes.
Why it hurts
Every deleted Pod leaves a disk behind, costing money and cluttering the system.
Do this instead
Use a storage class with the delete policy for ephemeral use, and a separate class for retained data.
The mistake
Mounting a single configuration file by sub-path and expecting live updates.
Why it hurts
Sub-path mounts do not receive updates, so the Pod silently keeps stale settings.
Do this instead
Mount the whole folder and have the program re-read it, or restart Pods on configuration change.
19Best Practices and Common Mistakes
A short checklist to read before you add temporary storage to a Pod.
Choose the simplest type
Start with emptyDir, configMap, or secret, and only move to CSI or generic ephemeral volumes when you need their special abilities.
Always set limits
Put size limits on volumes and ephemeral storage requests and limits on containers, so the scheduler and the kubelet can protect nodes.
Design for loss
Make work repeatable, make caches rebuildable, and write results out early.
Match storage to the workload
Local disks for speed and short start-up, cloud disks for size, memory for small sensitive files.
Use wait-for-first-consumer
For classes behind generic ephemeral volumes, so storage is created where the Pod runs.
Lock down drivers and quotas
Approve driver names, set claim and size quotas, and use encrypted classes for sensitive data.
Monitor disk, events, and claims
Alert on node pressure, repeated evictions, and pending claims.
Test the failure paths
Delete a Pod during a job, evict it, and stop the driver in a test cluster to see what really happens.
Mistakes beginners make
- Thinking emptyDir survives Pod replacement. It survives only container restarts within the same Pod.
- Forgetting that memory-backed folders use memory. The files count toward the container’s memory limit.
- Ignoring container logs and writable layers. They also count as ephemeral storage, and often cause the eviction.
- Creating a claim by hand with the same name as an ephemeral claim. The Pod will refuse to start.
- Using inline CSI with a driver that does not support it. Check the lifecycle modes of the driver first.
- Assuming ConfigMap changes appear instantly. There is a refresh delay, and sub-path mounts never update.
- Leaving the default storage class unchecked. A template with no class will use whatever the default is, which may be slow or expensive.
Before shipping a Pod with temporary storage, ask three questions: what happens if this data vanishes, what is the most this Pod can write, and who or what prepares this folder. If you can answer all three clearly, your design is probably sound.
20Real-World and Industry Examples
The same ideas appear again and again in production clusters.
Argo Workflows and Tekton: pipelines built on shared folders
Pipeline engines run each step as a container, and often pass files between steps using emptyDir volumes in the same Pod. Because the folder is private to the Pod and vanishes afterward, pipelines stay isolated from each other and leave no clutter behind.
Secrets Store CSI Driver: vault secrets as files
Organizations that keep secrets in cloud vaults or HashiCorp Vault use the Secrets Store CSI Driver to deliver them straight into a Pod’s volume. It follows the CSI ephemeral model, so the secret exists only as long as the Pod, and rotation can update files in place.
cert-manager and SPIFFE: identity per Pod
The cert-manager CSI driver and the SPIFFE CSI driver use inline volumes to hand each Pod its own certificate or identity socket. Teams use this to build zero-trust designs in which private keys never leave the node and never sit in a cluster-wide object.
Local storage projects such as TopoLVM
Storage projects built for node-local disks document generic ephemeral volume use, so that each Pod can claim a private slice of fast local storage, with the scheduler aware of how much space remains on each node. This suits caches, build workspaces, and databases that can rebuild from replicas.
Cloud providers: local SSD as ephemeral storage
Major cloud providers offer machines with local SSDs, which Kubernetes can use as ephemeral storage or through a CSI driver. They are popular for heavy scratch workloads because the speed is high and the price is low, with the understanding that the data goes away when the machine does.
In every story, the successful teams treat ephemeral volumes as disposable by design. They set limits, copy out what matters, restrict who can use special drivers, and choose the lightest type that does the job.
21Frequently Asked Questions
Short answers to the questions beginners ask most often.
22Summary and Key Takeaways
You started with a hotel room that is cleaned out at checkout and finished with a clear picture of how Kubernetes handles temporary storage. The simplest tool is emptyDir, a blank folder with the Pod’s life. Configuration and secrets arrive through configMap, secret, and projected volumes. When a driver must prepare the folder, CSI inline volumes step in. When you need a real, sized, quota-controlled disk that still vanishes with the Pod, generic ephemeral volumes create and delete a claim for you. The skill lies in choosing the lightest tool that fits, setting limits, designing for loss, and controlling who can use what.
Key Takeaways
- Ephemeral inline volumes live and die with the Pod. They are declared in the Pod specification and need no separate objects or manual cleanup.
- emptyDir is the everyday choice. It survives container restarts but not Pod removal, can be disk-backed or memory-backed, and should always carry a size limit.
- Ephemeral storage is accounted and enforced. Writable layers, logs, and disk-backed emptyDir folders count toward limits, and the kubelet evicts Pods that overshoot or when the node runs low on disk.
- CSI inline volumes let a driver prepare a Pod-scoped folder, ideal for vault secrets and certificates. They need a driver that supports the ephemeral mode and lack snapshots, resizing, and capacity-aware scheduling.
- Generic ephemeral volumes create a real claim owned by the Pod. They work with any dynamic storage class, support quotas, snapshots, and capacity tracking, and are cleaned up by garbage collection, with start-up cost and extra API objects as the price.
- Design for loss and control access. Copy out important results early, use encrypted or memory-backed storage for sensitive data, restrict inline CSI drivers by name, and set quotas.
- Choose the simplest type that fits. Use the decision path, monitor node disk pressure, evictions, and pending claims, and test the failure paths before relying on temporary storage in production.