Volume Expansion in Kubernetes
How to give a running application more disk space without losing data or rebuilding anything: what really happens between editing a claim and seeing the extra gigabytes, where it can go wrong, and how to make it routine.
Imagine a school locker that was sized for a few books in September. By March it holds books, a sports kit, an art project and a science experiment, and the door will no longer close. The sensible fix is not to throw everything out and buy a new locker. It is to ask the caretaker to make the same locker bigger while your things stay inside. Applications in Kubernetes face the same problem with disk space. A database that started with a small volume grows steadily, and one quiet night the volume fills up and the application stops. Volume expansion is the caretaker’s service: it lets you ask for a larger volume, and the cluster quietly stretches both the underlying disk and the filesystem on top of it, often while the application keeps running. This guide explains every step of that process in plain language.
1Why Volumes Run Out of Space
Disk space is the one resource that almost always grows and almost never shrinks by itself.
Containers are designed to be disposable. When a container restarts, everything inside it vanishes, which is perfect for stateless web servers and terrible for databases, message queues, search indexes and anything else that must remember things. For those workloads, Kubernetes attaches persistent storage: a disk that lives outside the container and survives restarts, rescheduling and even node failures.
When you create that disk, you must choose its size, and you must choose it before you know how the application will behave. Pick too small, and the application runs out of room as data accumulates. Pick far too large, and you pay for empty space every month. Real data growth is rarely predictable. A marketing campaign doubles the number of customers, a new feature writes more logs, an index grows after a schema change, or a backup job leaves large temporary files behind. Sooner or later, almost every persistent volume needs to become bigger.
Organic growth
More users, more orders and more history simply mean more bytes. A volume that was comfortable at launch may be tight a year later.
Sudden bursts
A bulk import, a reindex or a runaway debug log can fill a disk in hours.
Under-estimation
Early sizing is often a guess, made to keep costs low at the start of a project.
Hidden consumers
Write-ahead logs, snapshots, temporary files and lost-and-found data can quietly eat space that nobody planned for.
What happens when a volume fills up
The effects are rarely gentle. Databases may refuse writes or crash into recovery. Message brokers stop accepting messages. Applications that write logs to the volume may hang. Because the failure is often sudden and affects the busiest, most important systems, a full disk is one of the classic causes of night-time pages. Before expansion existed in Kubernetes, the fix usually meant creating a bigger volume, copying all the data across and switching over, which meant downtime and risk.
A town builds a car park for one hundred cars. Years later, it holds three hundred cars every day. The old fix was to build a brand new car park elsewhere, move every car there one by one and demolish the old one. The better fix is to extend the existing car park while cars are still parked in it. Volume expansion is the car park extension.
You raise the requested size on the claim, and Kubernetes together with the storage driver enlarges the real disk and then the filesystem. No copying, and often no restart.
2Storage Building Blocks: PV, PVC, StorageClass and CSI
A short refresher on the four objects that every storage conversation depends on.
Expansion makes more sense once the cast of characters is clear. Kubernetes separates the person who needs storage from the machinery that provides it, and four ideas hold the picture together.
PersistentVolumeClaim
A claim is a request for storage written by an application owner. It says how much space is wanted, what access mode is needed and which class of storage is preferred. You can think of it as a ticket.
PersistentVolume
A volume object represents an actual piece of storage in the cluster, such as a cloud disk. Once a claim is matched with a volume, they are bound to each other.
StorageClass
A class describes a kind of storage, such as fast solid-state disks or cheap standard disks, which driver provides it and what options apply. Classes let volumes be created on demand.
CSI driver
The Container Storage Interface is a standard that lets storage vendors write drivers once and plug them into any Kubernetes cluster. The driver does the real work of creating, attaching and growing disks.
How a claim turns into a disk
With dynamic provisioning, an application owner creates a claim that names a storage class. A component watching the cluster notices the claim, asks the class’s CSI driver to create a disk of the requested size, and creates the matching volume object. The claim and the volume are then bound, and a Pod that mounts the claim sees a directory backed by that disk.
| Object | Who writes it | What it holds | Relevance to expansion |
|---|---|---|---|
| PersistentVolumeClaim | Application owner | Requested size, status size, conditions | The object you edit to ask for more space |
| PersistentVolume | Provisioner or administrator | Actual capacity and driver details | Its capacity is raised after the disk grows |
| StorageClass | Cluster administrator | Driver, parameters and the expansion switch | Must allow expansion |
| CSI driver | Storage vendor | Code that talks to the storage system | Must support resizing |
Booking a hotel room works the same way. The claim is your booking request, the volume is the actual room, the storage class is the room type such as standard or deluxe, and the driver is the hotel staff who prepare the room. When you ask for a bigger room, the staff must be able to provide one, and the hotel must permit upgrades.
A claim carries the size you asked for and, separately, the size the cluster has actually delivered. Expansion is the process of making the second catch up with the first.
3What Volume Expansion Really Is
A request to grow, a pair of resizing steps, and a status that confirms the result.
Volume expansion is the ability to increase the size of an existing persistent volume claim, and therefore the real disk behind it, without recreating anything. The user experience is deliberately simple. You edit the claim and raise the storage amount in its requests. Everything after that is automatic.
Spec and status
Like most Kubernetes objects, a claim has a spec, which describes what you want, and a status, which describes what is real. The requested size lives in the spec. The size the cluster has actually provided lives in the status as capacity. When you raise the spec, the two values differ, and controllers work to bring the status up to match. When they match again, the expansion is done. This is the same declare-and-reconcile pattern that drives the rest of Kubernetes.
A thermostat has a target temperature you set and a current temperature the room actually has. When you raise the target, the heater runs until the room catches up. The claim’s requested size is the target, the status capacity is the room temperature, and the resizing machinery is the heater.
Growing is allowed, shrinking is not
Kubernetes only supports making volumes larger. Attempting to lower the requested size of a bound claim is rejected. There are practical reasons. Most filesystems cannot be shrunk safely while mounted, many cloud disks cannot be made smaller at all, and shrinking could destroy data if the used space exceeded the new size. If you ever need a smaller volume, you must create a new one and move the data, which Chapter 14 covers.
| Change requested | Allowed? | Reason |
|---|---|---|
| Increase the requested size | Yes, if the class allows expansion | The main supported path |
| Keep the same size | Yes, nothing happens | No change to reconcile |
| Decrease the requested size | No for a bound claim | Risk of data loss and lack of support in disks and filesystems |
| Change the storage class of a bound claim | No | The claim is tied to its volume and class |
Because a volume cannot be made smaller, every expansion is a one-way decision. Expanding generously makes incidents less likely, but you will pay for that space until the volume is migrated. Choose each step with both risk and cost in mind.
4Prerequisites: What Must Be True Before You Can Expand
Three switches must all be on, or the request is refused.
Volume expansion is not automatically available for every claim. Three conditions must hold, and knowing them helps you diagnose the common case where an edit is rejected on the spot.
The storage class allows expansion
The class has an explicit setting called allow volume expansion, and it must be true. Without it, an admission check rejects any attempt to raise the claim’s size. Many classes created long ago, or by default in older clusters, have it off.
The CSI driver supports resizing
The driver must advertise that it can expand volumes. Some drivers do this only for the controller side, some also for the node side, and some can do it while the volume is in use. The driver’s documentation lists what it supports.
The claim is bound and the backend can grow
The claim must be bound to a volume, and the storage system itself must be able to enlarge that kind of disk, within its own limits on size and frequency of changes.
The admission check
Kubernetes runs a built-in admission step that examines every update to a claim. If the update raises the requested size and the class does not allow expansion, the update is refused immediately with a clear message. This is a fast, friendly failure, and it is the first thing to check when an expansion request is rejected.
| Symptom | Likely missing prerequisite | What to check |
|---|---|---|
| Edit rejected immediately | Class does not allow expansion | The class setting for volume expansion |
| Edit accepted, but nothing happens | Driver or resizer not running | Resizer component and driver pods |
| Edit accepted, disk grows, filesystem does not | Node-side expansion missing or waiting for a Pod | Claim conditions and driver node support |
| Edit accepted, but backend reports an error | Cloud limits or quotas | Events on the claim and cloud console |
Upgrading a seat on a plane needs three things: the airline must allow upgrades on that ticket, there must be a better seat available, and you must already hold a confirmed ticket. If any one is missing, the upgrade fails, but each failure looks different at the desk.
It is much easier to enable expansion on a storage class before volumes are created from it than to discover the setting is off during an emergency. Cluster administrators often enable it on every class by default.
Switching a class to allow expansion applies to claims already created from it. You do not need to recreate volumes, but make sure the driver behind them really supports resizing.
5The Two Phases: Growing the Disk, Then the Filesystem
The central idea of the whole topic: a bigger disk is not yet a bigger usable space.
A persistent volume has two layers that both have to grow. The lower layer is the disk itself, a block device provided by a cloud or storage system. The upper layer is the filesystem, the organised structure of directories and files written onto that disk. When the disk is made larger, the filesystem on it still thinks it occupies the old, smaller area. It must be told to stretch into the new space. Kubernetes therefore expands volumes in two separate phases.
flowchart TD
A["User raises requested size on the claim"] --> B["Admission check: class allows expansion?"]
B -->|No| X["Request rejected"]
B -->|Yes| C["Controller expansion: storage backend enlarges the disk"]
C --> D["Volume object capacity is raised"]
D --> E{"Does the volume need a filesystem resize?"}
E -->|No| G["Claim capacity updated, done"]
E -->|Yes| F["Claim waits: filesystem resize pending"]
F --> H["Node expansion: filesystem grows on the node"]
H --> G
Controller expansion
Runs in the control plane side of the system. It asks the storage backend to enlarge the underlying disk, then raises the capacity recorded on the volume object. No node is involved.
Node expansion
Runs on the machine where the volume is mounted. The kubelet asks the driver’s node component to grow the filesystem so the extra space becomes usable by the Pod.
Suppose a bookshelf stands in a room and you move the wall back to make the room bigger. The room is larger, but the bookshelf is still the same size until a carpenter extends it into the new space. Growing the disk moves the wall, and growing the filesystem extends the shelf. Both are needed before you can store more books.
Why two phases exist
The two steps need different powers. Enlarging a cloud disk is an operation performed through the provider’s programming interface and needs no access to a particular node. Growing a filesystem, on the other hand, must be done where the volume is attached, because the operating system on that node owns the mount. Splitting the work this way lets each part run in the right place, and it explains why a claim can spend some time in a waiting state between the two.
| Aspect | Controller expansion | Node expansion |
|---|---|---|
| Where it runs | Control plane side or a storage controller | On the node using the volume |
| What it changes | The real disk and the volume capacity | The filesystem inside the disk |
| Needs a Pod using the volume? | Not necessarily | Yes, the volume must be mounted or about to be |
| Typical failure | Cloud limits or driver errors | Filesystem problems or unsupported type |
Seeing a larger disk in the cloud console does not mean the application has more space. Until the filesystem has grown, the claim’s status capacity stays at the old value, and the application still sees the old free space.
6Who Does What: The Components Involved
A small team of programs cooperates to turn an edit into extra space.
Several pieces take part in an expansion. Knowing who is responsible for which step makes troubleshooting far easier, because you can ask which component has not done its job.
sequenceDiagram
participant U as User
participant API as API Server
participant R as External Resizer
participant D as CSI Controller Plugin
participant S as Storage Backend
participant K as Kubelet
participant N as CSI Node Plugin
U->>API: Raise requested size on the claim
API->>API: Admission checks the class allows expansion
R->>API: Notice requested size is larger than capacity
R->>D: Expand this volume
D->>S: Enlarge the disk
S-->>D: Done
D-->>R: New size, node expansion needed
R->>API: Update volume capacity, mark filesystem resize pending
K->>API: Notice pending filesystem resize
K->>N: Expand the filesystem
N-->>K: Done
K->>API: Update claim capacity to the new size
Admission control
Checks the edit at the moment it is made, and refuses size increases on classes that do not allow expansion.
External resizer
A helper that runs alongside the CSI driver. It watches claims, notices when requested size exceeds capacity and asks the driver to expand. It also records progress and errors.
CSI controller plugin
The part of the driver that talks to the storage system to enlarge the disk. It runs in the cluster rather than on every node.
Kubelet and CSI node plugin
On the node, the kubelet notices the pending filesystem resize and the driver’s node component grows the filesystem.
Expand controller
A control plane component that handles expansion for older built-in volume types that predate CSI. New clusters rely mostly on CSI drivers instead.
Storage backend
The real system holding the data, such as a cloud disk service or a storage cluster. It has its own rules about size and timing.
Extending a house involves an architect who approves the plan, a clerk who notices the approved plan, builders who add the new room and a local electrician who connects it to the house wiring. Each does one job, and the extension is not finished until the last one signs off.
Each of these components records what it is doing as events and conditions on the claim. Reading the events in order usually shows which step is stuck.
7The Claim’s Life During Expansion: Conditions and States
How to read what is happening from the claim itself.
While an expansion is underway, the claim records its progress using conditions. A condition is a small piece of status that says something is true or in progress. Reading these conditions is the quickest way to know whether to wait or to investigate.
flowchart LR
A["Steady: requested equals capacity"] -->|"User raises requested size"| B["Resizing: disk is being enlarged"]
B -->|"Disk enlarged, filesystem needs growing"| C["Filesystem resize pending"]
C -->|"Node grows the filesystem"| D["Steady again at the new size"]
B -->|"Backend error"| E["Resize error recorded"]
C -->|"Node error"| E
E -->|"Cause fixed, retry succeeds"| B
| State or condition | What it means | What to do |
|---|---|---|
| Resizing | The backing disk is being enlarged | Wait, and watch events for errors |
| Filesystem resize pending | The disk is larger, the filesystem has not grown yet | Wait for the node step, or ensure a Pod mounts the volume if the driver needs it |
| Resize error | A step failed and will be retried | Read the event message and fix the cause |
| Capacity equals request | Expansion complete | Verify inside the Pod if desired |
The two size fields
Always compare two numbers on the claim: the size requested in the spec and the capacity reported in the status. While they differ, expansion is in progress or stuck. Recent Kubernetes versions also record how much space has been allocated and the status of each resize step, which helps tools and people see exactly how far an expansion has progressed, and supports recovery after a failed attempt.
A parcel tracking page shows stages such as “packed,” “in transit” and “out for delivery.” Nobody panics at “in transit,” but a parcel stuck for days at one stage deserves a phone call. Conditions on a claim work the same way.
Most expansions finish within seconds to a few minutes. A claim that sits at the same stage for much longer, with repeated error events, needs investigation rather than more waiting.
Raising the request again while an earlier expansion is still in progress can confuse both the driver and the backend. Wait for the first expansion to finish, or for a clear error, before changing the size once more.
8Online and Offline Expansion
Whether the application can keep running while its disk grows.
One of the most practical questions is whether a Pod must be stopped. Kubernetes supports expanding a volume while it is in use, which is called online expansion, and this has been a stable feature for several releases. Whether it works for your volume depends on the CSI driver and the filesystem. Some drivers can only complete the filesystem step when the volume is not in use, which is offline expansion, and in that case the filesystem grows the next time a Pod mounts the volume.
flowchart TD
A["Disk has been enlarged"] --> B{"Does the driver support online filesystem growth?"}
B -->|Yes| C["Kubelet grows the filesystem while the Pod keeps running"]
B -->|No| D["Claim shows filesystem resize pending"]
D --> E["Pod is stopped or restarted"]
E --> F["Filesystem grows during the next mount"]
C --> G["Expansion complete"]
F --> G
Online: Advantages
- No downtime for the application
- Resolves a full disk emergency quickly
- No need to schedule a maintenance window
- Fits automated expansion well
Online: Trade-offs
- Depends on driver and filesystem support
- A busy filesystem grows under load, which deserves a little caution
- Problems appear while the application is live
Offline: Advantages
- Growth happens while nothing is writing
- Simple and predictable for older drivers
- Fewer surprises for sensitive workloads
Offline: Trade-offs
- Requires a restart or a gap in service
- Slow to relieve a full disk emergency
- Harder to automate safely
Widening a motorway can sometimes be done one lane at a time while traffic keeps flowing, and sometimes the road must be closed overnight. Online expansion is the first kind, offline expansion is the second, and the right choice depends on the road and the equipment.
Do not guess whether your driver supports online expansion. Try it on a non-critical volume first, and record the result, so that during a real emergency you already know whether a restart is needed.
9Filesystems and Raw Block Volumes
What actually lives on the disk decides how the second phase behaves.
Most volumes in Kubernetes are mounted as filesystems. The two filesystems most often used are ext4 and XFS, and both can be grown while mounted, which is what makes online expansion possible. The node step of an expansion runs the right growing tool for whichever filesystem the volume contains. If the filesystem type is one that cannot be grown, the second phase cannot complete, even though the disk itself became larger.
| Volume content | Can it grow? | Notes |
|---|---|---|
| ext4 | Yes, including while mounted | Very common default, grows reliably |
| XFS | Yes, while mounted | Can only grow, never shrink, which matches Kubernetes behaviour |
| Other filesystem types | Depends on the type and driver | Check driver documentation before relying on it |
| Raw block volume | The disk grows, but there is no filesystem to resize | The application must notice and use the extra space itself |
Raw block mode
A volume can also be given to a Pod as a raw block device, with no filesystem at all. This is used by software that manages the disk directly, such as some databases and storage systems. Expansion enlarges the device, but nobody grows a filesystem, because none exists. The application itself must detect the larger device and extend its own structures. Kubernetes does the first half only.
A filesystem is like a library catalogue laid out over a warehouse. If the warehouse grows, the catalogue must be extended to include the new shelves. A raw block volume is a bare warehouse with no catalogue, and whoever uses it must organise the new space on their own.
Space is more than bytes
A filesystem has two limits: how many bytes of data it can hold and how many files it can track. The second is controlled by a pool of file records called inodes. A volume with millions of tiny files can run out of inodes long before it runs out of bytes. Growing a filesystem generally adds more inodes as well, so expansion helps with both problems, but it is wise to monitor both.
Filesystems keep a small reserve and some space for their own structures. After expansion the usable space is slightly less than the number you requested. Plan for a small margin when choosing sizes.
10Expansion Across Volume Types
Not every kind of storage can be grown, and some do not even mean the same thing by size.
Kubernetes can attach many kinds of storage, and expansion support depends on the type. A rough map of the landscape helps set expectations.
| Storage type | Expansion support | Comment |
|---|---|---|
| Cloud block disks through CSI | Generally yes | Often online, but providers have their own limits |
| Network block storage clusters through CSI | Generally yes | Disk grows, then filesystem is extended on the node |
| Shared filesystems through CSI | Depends on the driver | Size is often a quota rather than a real disk |
| NFS volumes | Not meaningful | The requested size is not enforced by the server |
| Local volumes and host paths | Not handled by Kubernetes itself | Special drivers using logical volume managers can add it |
| Empty directories | Not applicable | They have a size limit setting, but are temporary by design |
Cloud disks
Managed disk services are the most common case. The driver asks the provider to enlarge the disk, and the node then grows the filesystem. Providers may limit how often a disk can be modified and the largest size allowed.
Shared filesystems
For shared filesystem drivers, the size may be a quota on a directory rather than a physical disk. Expansion then means raising the quota, which is quick, and no filesystem growth is involved.
NFS
With plain NFS the requested size is only a label. The server does not enforce it, so raising the number changes nothing real. Space is managed on the NFS server itself.
Local storage
A disk attached to a single node cannot be stretched by Kubernetes. Drivers that carve local volumes from a logical volume manager can provide expansion, because the manager can enlarge a volume if the node still has free space in its pool.
Asking for a bigger storage unit works differently depending on the building. In a modern facility, staff can knock down a wall. In a shared warehouse with only a quota, a manager can simply raise your limit. In a rented shed attached to one house, there may be nothing to expand into.
Every CSI driver documents which expansion features it supports and any limits. Treat that page as part of your storage design, and not as an afterthought.
11Expansion in StatefulSets
The place where expansion stops being a one-line edit and needs a careful routine.
Databases and other stateful systems are usually run as StatefulSets. A StatefulSet creates one Pod per replica and, for each, a separate claim built from a volume claim template. This gives each replica its own persistent disk, which is exactly what a database cluster needs.
The catch is that the volume claim template inside a StatefulSet cannot be changed after creation. If you edit the template to request a larger size, the update is refused. The template only matters when new claims are created, and existing claims are not touched by it. This surprises many teams the first time they try to grow a database.
flowchart TD
A["Need more space for a StatefulSet"] --> B["Raise the requested size on each existing claim one by one"]
B --> C["Wait until every claim has finished expanding"]
C --> D["Delete the StatefulSet object but keep its Pods and claims"]
D --> E["Recreate the StatefulSet with the larger size in its template"]
E --> F["Controller adopts the existing Pods and claims"]
F --> G["Future replicas get the new size"]
Expand each claim
Edit every replica’s claim directly. They are ordinary claims, and the usual expansion process runs for each.
Verify completion
Confirm the status capacity of every claim matches the new request, and that the application sees the extra space.
Remove only the StatefulSet object
Delete the StatefulSet using the orphaning option, which leaves its Pods and claims running and untouched.
Recreate it with the new size
Apply the same definition with the larger size in the template. The new object adopts the Pods that already exist.
Check the next scale-up
When you add replicas later, their new claims will use the larger template size, keeping all replicas consistent.
A restaurant’s menu card says every table gets a standard plate size. If you want bigger plates, changing the menu card does nothing for tables already seated. You must replace the plates at each table, and then reprint the menu so that new tables get the bigger plates too.
Deleting a StatefulSet without the orphaning option removes its Pods. For a database that can mean an outage, so always double-check the deletion mode, and rehearse the whole routine on a test cluster first.
The Kubernetes community has been working on making volume claim templates easier to update. Check the documentation for your version, because the recommended routine may become simpler.
12Quotas, Limits and Cloud Provider Constraints
Even a valid request can be blocked by rules set by administrators or by the storage provider.
An expansion request passes through several layers of control, and any of them may say no. Some belong to Kubernetes and some to the storage system.
Resource quotas
Administrators can limit the total storage a namespace may request. A resize counts toward that total, so an expansion can be refused if it would exceed the quota.
Limit ranges
Rules can set minimum and maximum sizes for a single claim in a namespace. A request above the maximum is rejected.
Maximum disk size
Every disk type has an upper size limit set by the provider, which no setting in Kubernetes can override.
Modification cool-down
Some providers allow only one modification per disk within a waiting period of several hours. A second expansion inside that window is delayed or refused.
Account quotas
The cloud account may have a total storage allowance, and growing a disk consumes it.
Billing
Larger disks cost more, and because they cannot shrink, the extra cost continues until the volume is replaced.
| Limit | Who sets it | Typical sign when hit |
|---|---|---|
| Namespace storage quota | Cluster administrator | Edit refused with a quota message |
| Per-claim maximum | Cluster administrator | Edit refused with a limit message |
| Disk size ceiling | Storage provider | Backend error on the claim |
| Modification waiting period | Storage provider | Expansion stalls with a retry message |
| Account storage allowance | Cloud account | Backend error mentioning limits |
Asking the school for a bigger locker involves the caretaker, who must have lockers of that size, the head of year, who limits how much each student may have, and the budget office, which limits the total for the whole class. Any one of them can say no.
If a provider enforces a waiting period between changes, a team that grows a disk by too small a step may be unable to grow it again for hours, right when the next incident hits. Prefer steps large enough to give real breathing room.
13Failure Modes and Recovery
What can go wrong, how to recognise it, and how to get back on track.
Most expansions succeed without drama. When they do not, the cause usually falls into a short list. Recognising the pattern quickly saves a lot of time.
Stuck at filesystem resize pending
The disk has grown but the claim never reaches the new capacity. Common causes are a driver that needs the volume mounted by a Pod, a Pod that is not running, or a driver without node-side support. Starting or restarting a Pod that uses the claim often completes the step.
Backend refuses to grow the disk
The claim stays in the resizing state with repeated errors. Typical reasons are exceeding the provider’s maximum size, hitting a cool-down period, running out of account allowance or a temporary provider fault. The events on the claim usually quote the provider’s message.
Requested size too large to ever succeed
Someone accidentally requests a huge size that the backend can never provide. Because shrinking is not allowed, the claim could be stuck. Recent Kubernetes versions include a recovery capability that lets the requested size be lowered back to a value that is still not smaller than the current capacity, as long as the failed expansion has not completed. Check whether your version supports it.
Filesystem growth fails on the node
The node step reports an error, such as a damaged filesystem or an unsupported type. This is serious, because the application may be affected. Take a snapshot if possible, inspect the node-side logs, and involve the storage team before retrying.
Read the claim events
The most recent events usually name the component that failed and the reason.
Compare requested size and capacity
See which phase completed and which did not.
Check the resizer and the driver
Look at the logs of the external resizer and the driver’s controller and node components.
Check the backend
Look in the provider’s console or tools for errors, limits and the state of the disk.
Fix, then let it retry
Controllers retry automatically once the cause is removed, so manual edits are rarely needed.
Expansion is routine and usually safe, but it modifies live storage. Taking a snapshot or backup before growing important volumes costs little and gives a way back if something unexpected happens.
14Why You Cannot Shrink, and What to Do Instead
When a volume is far too big, the only route is to build a smaller one and move the data.
Sometimes a volume is simply oversized. Maybe a one-off import filled it, then the data was deleted, and now it is mostly empty and expensive. Since shrinking is not supported, the way to reclaim the space and money is migration: create a new, smaller claim, copy the data across and switch the application over.
flowchart TD
A["Volume is far larger than needed"] --> B["Take a backup or snapshot"]
B --> C["Create a new smaller claim"]
C --> D["Stop writes to the application"]
D --> E["Copy data to the new claim"]
E --> F["Verify the copy"]
F --> G["Point the application at the new claim"]
G --> H["Keep the old volume briefly, then delete it"]
Ways to move the data
The best method depends on the application. For databases, the safest approach is usually to use the database’s own tools, for example by adding a new replica with a smaller volume and removing the old one after replication catches up. For file data, a one-time copy job can transfer files while the application is paused. For some storage systems, restoring from a snapshot into a new claim is the quickest route, provided the driver allows the new claim to be sized as needed.
Migration: Advantages
- Reclaims wasted space and cost
- Can also change the storage class, for example to a cheaper tier
- Opportunity to clean up and verify data
Migration: Trade-offs
- More steps and more risk than an expansion
- May need downtime or a careful cut-over
- Needs temporary extra space during the copy
You cannot shrink a warehouse in place. To pay less rent, you rent a smaller warehouse, carry the goods across and hand back the keys to the old one. It takes effort, so it is wise to avoid getting into this situation by growing in measured steps in the first place.
A large volume that is mostly empty is not unsafe, only costly. If cost is not a pressing concern, leaving it alone avoids the risk of migration.
15GitOps, Helm and the Problem of Drift
When the cluster holds a size that your configuration repository does not know about.
Many teams manage their clusters from configuration stored in a repository, applied by tools such as Helm or by GitOps controllers that continuously compare the repository with the cluster. Volume expansion interacts with this in an awkward way, because the size on the claim in the cluster can become different from the size written in the repository.
Suppose an engineer expands a claim during an incident, directly in the cluster. The repository still says the old, smaller size. The next time the tool applies the configuration, it tries to set the old size on the claim. Since shrinking is not allowed, the update is rejected. Depending on the tool, this shows up as a failed deployment, a permanently out-of-sync application or a noisy alert. The cluster is fine, but the process is broken.
Expand during an incident
A claim is raised directly in the cluster to stop an outage.
Drift appears
The repository still holds the smaller size.
The tool tries to reconcile
It attempts to apply the smaller size and is refused by Kubernetes.
Fix the source of truth
The repository is updated to the new, larger size so that both agree again.
Expand through the repository
Whenever time allows, change the size in the configuration and let the tool apply it, so the record is always correct.
Back-port emergency changes
If you must expand by hand, update the repository straight afterwards as part of the incident follow-up.
Ignore size differences deliberately
Some tools can be told to ignore the size field on claims that are expected to grow. Use this only for volumes managed by automatic expansion.
Mind StatefulSet templates
Remember that templates cannot be edited in place, so the repository must be changed in step with the routine from Chapter 11.
A building’s official floor plan says the office has ten desks, but staff quietly add two more over the winter. The next time an inspector checks the plan, the building appears to be breaking the rules. The fix is not to remove the desks, it is to update the plan so it matches reality.
Since expansion only goes up, a good policy is that the configuration size must always be greater than or equal to the live size. A simple check in the pipeline can catch accidental shrinking requests before they reach the cluster.
16Related Features: Performance Changes and Snapshots
Neighbouring storage features that often come up in the same conversations.
Volume expansion is only one of several ways in which an existing volume can change or be protected. Understanding its neighbours prevents confusion, and helps you pick the right tool for the job.
Volume expansion
Changes how much space a volume has. Only increases are allowed, and the storage class must permit it.
Volume attributes classes
A newer feature that lets certain qualities of a volume, such as its speed or throughput, be changed after creation without changing its size, for drivers that support it. Check the maturity of this feature in your version.
Volume snapshots
A point-in-time copy of a volume. Snapshots are a safety net before expansions and a route for cloning or moving data.
Volume cloning
Creates a new volume that starts with the contents of an existing one. It can be useful for migration to a different size or class.
| Need | Right feature |
|---|---|
| More space on the same volume | Volume expansion |
| More speed on the same volume | Volume attributes class, if the driver supports it |
| A restore point before a risky change | Volume snapshot |
| A copy of a volume for testing | Volume clone or restore from snapshot |
| A smaller volume | Migration to a new claim |
If your locker is full, you ask for a bigger one, which is expansion. If the locker door is slow and sticky, you ask for a better lock, which is a performance change. If you worry about losing your things, you take a photo of everything inside, which is a snapshot.
On many cloud disks, throughput and operations per second depend on the disk size. A bigger disk can be faster as a side effect, but that is a property of the provider, not something Kubernetes promises.
17Monitoring Capacity Before It Becomes an Incident
The best expansion is the one done calmly, days before the disk fills.
The kubelet reports usage for every mounted volume, so you can see how full each one is without logging in to any application. These numbers flow into the monitoring system as metrics, and with them you can build alerts that warn long before a disk fills.
Capacity
The total size of each mounted volume as the node sees it.
Used and available bytes
How much data is stored and how much room remains, which together show the fill level.
Inodes used and free
Tracks the number of files the filesystem can still hold, which can run out before bytes do.
Growth rate
The speed at which usage rises, which lets you predict when the volume will be full.
| Alert | Idea | Typical threshold style |
|---|---|---|
| Warning on high usage | Early notice for planned action | Roughly three-quarters to four-fifths full |
| Critical on very high usage | Act now | Around nine-tenths full |
| Predicted to fill soon | Uses the growth trend rather than a fixed level | Projected to be full within a day or a few days |
| Inodes running low | Catches file-count exhaustion | Similar levels to byte usage |
| Expansion stuck | Claim shows requested size above capacity for too long | Longer than the normal completion time |
Predictive alerts beat fixed thresholds
A fixed threshold treats a slowly growing volume and a rapidly filling one the same way. A volume at eighty percent that grows by one percent a month needs no hurry, while a volume at seventy percent that fills by ten percent an hour is in danger. Alerts based on the growth trend, which estimate the time until full, tell you what you really want to know: how long you have.
A fuel gauge showing a quarter tank means different things on a short commute and on a long desert road. What matters is how fast you are using fuel and how far you must go. Predictive alerts are the trip computer that says “you will run dry in forty minutes.”
Some applications have their own disk safety rules, such as search engines that stop accepting data at certain fill levels. Your alerts should fire before those internal limits are reached.
18Automating Expansion
Letting a controller grow volumes before a human has to notice.
Because expansion is online and quick for many drivers, it is tempting to automate it. Community tools exist that watch the usage metrics and raise the requested size of a claim when it passes a threshold, by a set step, up to a safe maximum. This turns many night-time pages into silent, uneventful growth.
flowchart LR
A["Kubelet reports volume usage"] --> B["Monitoring stores the metrics"]
B --> C["Auto-expansion controller checks fill level"]
C -->|Below threshold| D["Do nothing"]
C -->|Above threshold and below the ceiling| E["Raise the requested size by one step"]
E --> F["Normal two-phase expansion runs"]
F --> A
C -->|At the ceiling| G["Alert a human instead"]
Advantages
- Prevents most full-disk incidents
- Removes repetitive manual work
- Reacts faster than any person can
- Keeps volumes right-sized over time instead of oversized from day one
Disadvantages / Trade-offs
- A runaway process can drive costs up without anyone noticing
- Growth is one-way, so mistakes are permanent
- Cool-down periods at the provider can block rapid repeats
- Drift with GitOps must be handled deliberately
Guardrails for automation
Good automation has limits. Set a maximum size per volume so a bug cannot grow a disk without end. Choose a step that is large enough to respect provider cool-downs but small enough to avoid waste. Alert people whenever an automatic expansion happens, because rapid growth is itself a signal that something is writing more than expected. And make sure the claim’s configuration in the repository is set up to tolerate growth.
An automatic car park barrier lifts to let cars in, but it still has a counter that tells the manager when the lot is nearly full. Automation should open the gate and also keep a human informed, because a lot that keeps filling up deserves a conversation.
If a volume keeps growing quickly, the cause may be a bug, such as logs that never rotate. Automatic expansion can hide the leak while the bill grows. Always investigate unusual growth.
19Security, Access Control and Cost Governance
Growing a disk is a small action with real financial and operational consequences.
Because expansion is as easy as editing one field, it deserves the same care as any other change that affects money and data. Several controls keep it safe.
Who may edit claims
Role-based access control decides who can update claims. Grant it to the teams that own the workload, and watch for broad permissions that would let anyone enlarge disks.
Who may change classes
Turning expansion on or off for a class is a cluster-level decision. Limit it to administrators.
Quotas as cost brakes
Namespace storage quotas stop any single team from growing without bound, and act as a budget guardrail.
Change history
Audit logs record who changed which claim and when, which is valuable when a bill jumps or a size is wrong.
Encryption stays in place
For volumes encrypted by the storage provider, expansion keeps the same encryption keys and settings in normal cases. Confirm this for your driver and provider.
Backups before big changes
Any change to live storage should follow the organisation’s backup rules.
| Risk | Control |
|---|---|
| Uncontrolled storage cost | Namespace quotas, maximum sizes and review of large requests |
| Unauthorised size changes | Tight access rules on claim updates |
| Silent growth from a faulty application | Alerts on growth rate and on automatic expansions |
| Data loss from an interrupted resize | Snapshots or backups before important expansions |
| Unclear who made a change | Audit logging and change records in the repository |
Allowing any department to order extra warehouse space with a single phone call is convenient, but the finance team will want a budget cap, a record of who called and a rule on how much each department may order.
Expose storage use and cost per team on a dashboard. Teams that see their own numbers tend to clean up unused data before asking for more space.
20Best Practices and a Safe Expansion Runbook
Habits and steps that turn expansion into a calm, repeatable task.
Enable expansion on every class
Make it a default when designing classes, so no volume is ever created that cannot grow.
Know your driver
Record whether it supports online expansion, which filesystems it handles and what provider limits apply.
Start modest, grow in sensible steps
Avoid huge initial sizes and also avoid tiny increments that waste cool-down windows.
Alert early and by trend
Use predictive alerts and inode alerts, so planned expansion replaces emergency expansion.
Keep the repository truthful
Record sizes in configuration and keep them equal to or above live sizes.
Snapshot before critical changes
Protect important volumes before expanding them.
Rehearse the StatefulSet routine
Practise the claim-by-claim approach and the orphaning delete on a test cluster.
Guard automation
Use maximum sizes, quotas and notifications whenever expansion is automated.
A runbook for a planned expansion
Confirm the prerequisites
Check that the class allows expansion, the driver supports it and the claim is bound.
Check limits and budget
Look at quotas, provider limits, cool-down timers and cost.
Take a snapshot
Protect the data for important volumes before any change.
Raise the requested size
Apply the change through the normal configuration path, choosing a step with real breathing room.
Watch conditions and events
Follow the claim through resizing and filesystem growth until capacity equals the request.
Verify from the application
Confirm that the application sees the extra free space and behaves normally.
Update records
Make sure the repository, documentation and alerts match the new size.
21Common Mistakes and Anti-patterns
Five habits that regularly turn a simple resize into a long night.
Problem
Storage classes are created without enabling expansion, because nobody expected to need it.
Why It’s Harmful
When a disk fills, the first step of the emergency is to change the class, and existing workflows may not allow that quickly or may lack the permission to do so.
Correct Approach
Enable expansion on every class from the start, and confirm the driver supports it.
Problem
The team assumes that a larger disk in the cloud console means the application has more space.
Why It’s Harmful
The filesystem may not have grown yet, so the application still sees the old free space and keeps failing while people believe the problem is solved.
Correct Approach
Verify that the claim’s status capacity matches the request and that the application sees the new space.
Problem
Engineers expand claims by hand during incidents and never update the configuration repository.
Why It’s Harmful
The next automatic sync tries to apply the old smaller size, is refused, and leaves the application permanently out of sync and noisy.
Correct Approach
Update the repository straight after any manual change, or make all expansions through the repository.
Problem
Volumes are grown by tiny steps every time they fill, such as one extra gigabyte at a time.
Why It’s Harmful
Providers may impose waiting periods between changes, so the next expansion can be blocked just when it is needed, and each change carries some risk and effort.
Correct Approach
Grow in steps that give real breathing room, and use predictive alerts to plan expansions ahead of time.
Problem
Automatic expansion is turned on with no ceiling and no notifications.
Why It’s Harmful
A bug that writes endlessly can grow disks until the budget is gone, and since shrinking is impossible, the cost stays until volumes are migrated.
Correct Approach
Set maximum sizes, use quotas, notify on every automatic change and investigate unusual growth.
| Symptom | Likely cause | First check |
|---|---|---|
| Edit rejected on the spot | Class does not allow expansion | Class setting |
| Claim stuck at filesystem resize pending | No Pod mounting the volume, or no node-side support | Pods using the claim and driver capabilities |
| Repeated backend errors | Provider limits or cool-down | Claim events and provider console |
| GitOps tool reports a failed apply | Repository holds a smaller size than the cluster | Compare sizes in repository and cluster |
| StatefulSet update refused | Claim template cannot be edited | Use the claim-by-claim routine |
22Advantages, Disadvantages and Trade-offs
A balanced view of what expansion gives you and where it stops.
Advantages
- Grows storage in place with no data copying
- Often works online, with no downtime for the application
- Simple for users: one field on one object
- Works the same way across many storage vendors through CSI
- Lets teams start small and grow as real needs appear
- Can be automated safely with proper guardrails
Disadvantages / Trade-offs
- Only growth is possible, never shrinking
- Depends on class settings, driver support and provider limits
- StatefulSet templates need a special routine
- Can create drift with configuration repositories
- Provider cool-down periods can limit repeated expansion
- Some storage types, such as plain NFS or bare local disks, offer no real expansion
How it compares with the alternatives
| Approach | Idea | Strength | Weakness |
|---|---|---|---|
| Oversize from the start | Pick a large volume at creation | Rarely needs changes | Pays for empty space, cannot shrink |
| In-place expansion | Grow the existing volume | No copying, often no downtime | Growth only, depends on driver |
| Migrate to a new volume | Copy data to a differently sized volume | Allows shrinking and class changes | More effort, risk and possible downtime |
| Add replicas or shards | Spread data across more volumes | Scales beyond one disk | Needs application support and rebalancing |
| Offload old data | Archive or delete data to free space | Reduces cost and size together | Needs retention rules and care |
The sensible approach in most teams combines several of these. Start with a reasonable size, enable expansion, alert on trends, grow in measured steps, and review volumes periodically so that oversized ones can be migrated and old data can be archived. Expansion is a tool for handling uncertainty, and not a replacement for knowing how fast your data grows.
23Real-World and Industry Examples
How teams use volume expansion in everyday operations.
Growing databases
A team running a relational database on a StatefulSet starts with modest volumes and raises each replica’s claim as data grows. They follow a written routine for the claim template and verify free space from inside the database after each change.
Search and log platforms
Search clusters store large indexes and are sensitive to disk fill levels, because they stop accepting data when space is low. Teams alert on predicted fill time and expand ahead of those internal thresholds.
Message brokers
Brokers keep backlogs on disk. When a downstream system is slow, backlogs grow quickly. Online expansion buys time to fix the slow consumer without losing messages.
Platform teams with self-service storage
Central platform teams give application teams self-service claims with expansion enabled, protected by namespace quotas and cost dashboards, so teams can grow within a budget without filing tickets.
Automatic expansion for variable workloads
Teams with unpredictable workloads use a controller that grows claims when they pass a threshold, with a maximum size and a notification on every change, turning night-time pages into daytime reviews.
Right-sizing after a one-off event
After a large temporary import left several volumes oversized, one team scheduled a migration to smaller volumes during a quiet weekend, using backups and replica-based copying, and used the opportunity to move data to a cheaper storage class.
Teams that handle expansion well tend to treat it as a routine, documented operation with monitoring, limits and rehearsal, instead of an emergency trick used under pressure.
24Frequently Asked Questions
Short answers to the questions beginners ask most often.
No. Reducing the requested size of a bound claim is not supported. To get a smaller volume, create a new claim of the right size, copy the data across and switch the application to it.
Most often the storage class does not allow volume expansion. An admission check refuses size increases on such classes. Enable the setting on the class, or create a new class that allows it.
Not always. Many drivers support online expansion, so the filesystem grows while the Pod runs. Some drivers complete the filesystem step only when a Pod mounts the volume, which then requires a restart.
The first phase, growing the disk, is done, but the second phase, growing the filesystem, has not finished. Check the claim’s conditions and capacity, and make sure a Pod is mounting the volume if the driver needs one.
The claim template cannot be edited, so expand each existing claim directly, then delete the StatefulSet while leaving its Pods and claims in place, and recreate it with the larger size in the template.
It is designed to be safe and is routine, but it changes live storage. For important data, take a snapshot or backup first, and watch the process until it completes.
The repository probably still holds the old, smaller size, and the tool tries to apply it. Kubernetes refuses to shrink, so the apply fails. Update the repository to match the larger size.
Plain NFS does not enforce the requested size, so changing it has no real effect. Local volumes cannot be grown by Kubernetes itself, although some drivers built on logical volume managers support it.
Yes. Community controllers watch usage metrics and raise the requested size when a threshold is crossed. Use a maximum size, quotas and notifications so that a faulty application cannot grow disks without limit.
If the backend cannot provide it, the claim stays in an error state with retries. Recent Kubernetes versions can recover from failed expansions by letting you lower the request, as long as it stays above the current capacity. Check support in your version and driver.
25Summary and Key Takeaways
Volume expansion lets you give a running application more disk space by changing one number on a claim. Behind that simple edit, Kubernetes and the storage driver work in two phases: first the backing disk is enlarged, then the filesystem on top of it is grown on the node, often while the application keeps running. It requires a storage class that allows expansion, a driver that supports it and a storage backend with room to grow, and it can be limited by quotas and provider rules. It only goes one way, so shrinking means migrating to a new volume. Stateful workloads need a careful routine because their claim templates cannot be edited, and configuration repositories must be kept in step with live sizes. With good monitoring, sensible steps and guarded automation, expansion turns a classic emergency into a routine, quiet operation.
Key Takeaways
- Expansion is a request on the claim — you raise the requested size, and controllers reconcile the real capacity up to that number.
- Two phases must both finish — the backing disk is enlarged first, and the filesystem is grown on the node second. A bigger disk alone gives the application no extra space.
- Three prerequisites apply — the storage class must allow expansion, the CSI driver must support resizing and the backend must be able to grow the disk within its limits.
- Only growth is supported — shrinking requires creating a new smaller volume and migrating data, so each expansion is a one-way decision.
- Online expansion depends on the driver — many drivers grow the filesystem while the Pod runs, while others wait for the next mount, so test your driver before an emergency.
- StatefulSets and GitOps need care — expand each claim directly, recreate the StatefulSet with the orphaning delete, and keep repository sizes equal to or above live sizes.
- Plan, alert and guard — use predictive alerts, snapshots before big changes, quotas and maximum sizes, and investigate unusual growth instead of simply paying for it.