Volume Expansion in Kubernetes

Volume Expansion in Kubernetes

How to give a running application more disk space without losing data or rebuilding anything: what really happens between editing a claim and seeing the extra gigabytes, where it can go wrong, and how to make it routine.

Imagine a school locker that was sized for a few books in September. By March it holds books, a sports kit, an art project and a science experiment, and the door will no longer close. The sensible fix is not to throw everything out and buy a new locker. It is to ask the caretaker to make the same locker bigger while your things stay inside. Applications in Kubernetes face the same problem with disk space. A database that started with a small volume grows steadily, and one quiet night the volume fills up and the application stops. Volume expansion is the caretaker’s service: it lets you ask for a larger volume, and the cluster quietly stretches both the underlying disk and the filesystem on top of it, often while the application keeps running. This guide explains every step of that process in plain language.

1Why Volumes Run Out of Space

Disk space is the one resource that almost always grows and almost never shrinks by itself.

Containers are designed to be disposable. When a container restarts, everything inside it vanishes, which is perfect for stateless web servers and terrible for databases, message queues, search indexes and anything else that must remember things. For those workloads, Kubernetes attaches persistent storage: a disk that lives outside the container and survives restarts, rescheduling and even node failures.

When you create that disk, you must choose its size, and you must choose it before you know how the application will behave. Pick too small, and the application runs out of room as data accumulates. Pick far too large, and you pay for empty space every month. Real data growth is rarely predictable. A marketing campaign doubles the number of customers, a new feature writes more logs, an index grows after a schema change, or a backup job leaves large temporary files behind. Sooner or later, almost every persistent volume needs to become bigger.

Cause 1

Organic growth

More users, more orders and more history simply mean more bytes. A volume that was comfortable at launch may be tight a year later.

Cause 2

Sudden bursts

A bulk import, a reindex or a runaway debug log can fill a disk in hours.

Cause 3

Under-estimation

Early sizing is often a guess, made to keep costs low at the start of a project.

Cause 4

Hidden consumers

Write-ahead logs, snapshots, temporary files and lost-and-found data can quietly eat space that nobody planned for.

What happens when a volume fills up

The effects are rarely gentle. Databases may refuse writes or crash into recovery. Message brokers stop accepting messages. Applications that write logs to the volume may hang. Because the failure is often sudden and affects the busiest, most important systems, a full disk is one of the classic causes of night-time pages. Before expansion existed in Kubernetes, the fix usually meant creating a bigger volume, copying all the data across and switching over, which meant downtime and risk.

Simple Analogy

A town builds a car park for one hundred cars. Years later, it holds three hundred cars every day. The old fix was to build a brand new car park elsewhere, move every car there one by one and demolish the old one. The better fix is to extend the existing car park while cars are still parked in it. Volume expansion is the car park extension.

Grow
expansion only ever makes volumes larger
2
phases: the backing disk first, then the filesystem
0
data copied when a volume is expanded in place
i
The Short Version

You raise the requested size on the claim, and Kubernetes together with the storage driver enlarges the real disk and then the filesystem. No copying, and often no restart.

2Storage Building Blocks: PV, PVC, StorageClass and CSI

A short refresher on the four objects that every storage conversation depends on.

Expansion makes more sense once the cast of characters is clear. Kubernetes separates the person who needs storage from the machinery that provides it, and four ideas hold the picture together.

The request

PersistentVolumeClaim

A claim is a request for storage written by an application owner. It says how much space is wanted, what access mode is needed and which class of storage is preferred. You can think of it as a ticket.

The real thing

PersistentVolume

A volume object represents an actual piece of storage in the cluster, such as a cloud disk. Once a claim is matched with a volume, they are bound to each other.

The menu

StorageClass

A class describes a kind of storage, such as fast solid-state disks or cheap standard disks, which driver provides it and what options apply. Classes let volumes be created on demand.

The plug

CSI driver

The Container Storage Interface is a standard that lets storage vendors write drivers once and plug them into any Kubernetes cluster. The driver does the real work of creating, attaching and growing disks.

How a claim turns into a disk

With dynamic provisioning, an application owner creates a claim that names a storage class. A component watching the cluster notices the claim, asks the class’s CSI driver to create a disk of the requested size, and creates the matching volume object. The claim and the volume are then bound, and a Pod that mounts the claim sees a directory backed by that disk.

ObjectWho writes itWhat it holdsRelevance to expansion
PersistentVolumeClaimApplication ownerRequested size, status size, conditionsThe object you edit to ask for more space
PersistentVolumeProvisioner or administratorActual capacity and driver detailsIts capacity is raised after the disk grows
StorageClassCluster administratorDriver, parameters and the expansion switchMust allow expansion
CSI driverStorage vendorCode that talks to the storage systemMust support resizing
Simple Analogy

Booking a hotel room works the same way. The claim is your booking request, the volume is the actual room, the storage class is the room type such as standard or deluxe, and the driver is the hotel staff who prepare the room. When you ask for a bigger room, the staff must be able to provide one, and the hotel must permit upgrades.

i
Two Sizes to Watch

A claim carries the size you asked for and, separately, the size the cluster has actually delivered. Expansion is the process of making the second catch up with the first.

3What Volume Expansion Really Is

A request to grow, a pair of resizing steps, and a status that confirms the result.

Volume expansion is the ability to increase the size of an existing persistent volume claim, and therefore the real disk behind it, without recreating anything. The user experience is deliberately simple. You edit the claim and raise the storage amount in its requests. Everything after that is automatic.

Spec and status

Like most Kubernetes objects, a claim has a spec, which describes what you want, and a status, which describes what is real. The requested size lives in the spec. The size the cluster has actually provided lives in the status as capacity. When you raise the spec, the two values differ, and controllers work to bring the status up to match. When they match again, the expansion is done. This is the same declare-and-reconcile pattern that drives the rest of Kubernetes.

Simple Analogy

A thermostat has a target temperature you set and a current temperature the room actually has. When you raise the target, the heater runs until the room catches up. The claim’s requested size is the target, the status capacity is the room temperature, and the resizing machinery is the heater.

Growing is allowed, shrinking is not

Kubernetes only supports making volumes larger. Attempting to lower the requested size of a bound claim is rejected. There are practical reasons. Most filesystems cannot be shrunk safely while mounted, many cloud disks cannot be made smaller at all, and shrinking could destroy data if the used space exceeded the new size. If you ever need a smaller volume, you must create a new one and move the data, which Chapter 14 covers.

Change requestedAllowed?Reason
Increase the requested sizeYes, if the class allows expansionThe main supported path
Keep the same sizeYes, nothing happensNo change to reconcile
Decrease the requested sizeNo for a bound claimRisk of data loss and lack of support in disks and filesystems
Change the storage class of a bound claimNoThe claim is tied to its volume and class
!
There Is No Undo

Because a volume cannot be made smaller, every expansion is a one-way decision. Expanding generously makes incidents less likely, but you will pay for that space until the volume is migrated. Choose each step with both risk and cost in mind.

4Prerequisites: What Must Be True Before You Can Expand

Three switches must all be on, or the request is refused.

Volume expansion is not automatically available for every claim. Three conditions must hold, and knowing them helps you diagnose the common case where an edit is rejected on the spot.

1

The storage class allows expansion

The class has an explicit setting called allow volume expansion, and it must be true. Without it, an admission check rejects any attempt to raise the claim’s size. Many classes created long ago, or by default in older clusters, have it off.

2

The CSI driver supports resizing

The driver must advertise that it can expand volumes. Some drivers do this only for the controller side, some also for the node side, and some can do it while the volume is in use. The driver’s documentation lists what it supports.

3

The claim is bound and the backend can grow

The claim must be bound to a volume, and the storage system itself must be able to enlarge that kind of disk, within its own limits on size and frequency of changes.

The admission check

Kubernetes runs a built-in admission step that examines every update to a claim. If the update raises the requested size and the class does not allow expansion, the update is refused immediately with a clear message. This is a fast, friendly failure, and it is the first thing to check when an expansion request is rejected.

SymptomLikely missing prerequisiteWhat to check
Edit rejected immediatelyClass does not allow expansionThe class setting for volume expansion
Edit accepted, but nothing happensDriver or resizer not runningResizer component and driver pods
Edit accepted, disk grows, filesystem does notNode-side expansion missing or waiting for a PodClaim conditions and driver node support
Edit accepted, but backend reports an errorCloud limits or quotasEvents on the claim and cloud console
Simple Analogy

Upgrading a seat on a plane needs three things: the airline must allow upgrades on that ticket, there must be a better seat available, and you must already hold a confirmed ticket. If any one is missing, the upgrade fails, but each failure looks different at the desk.

i
Turn It On Early

It is much easier to enable expansion on a storage class before volumes are created from it than to discover the setting is off during an emergency. Cluster administrators often enable it on every class by default.

!
Existing Volumes Benefit Too

Switching a class to allow expansion applies to claims already created from it. You do not need to recreate volumes, but make sure the driver behind them really supports resizing.

5The Two Phases: Growing the Disk, Then the Filesystem

The central idea of the whole topic: a bigger disk is not yet a bigger usable space.

A persistent volume has two layers that both have to grow. The lower layer is the disk itself, a block device provided by a cloud or storage system. The upper layer is the filesystem, the organised structure of directories and files written onto that disk. When the disk is made larger, the filesystem on it still thinks it occupies the old, smaller area. It must be told to stretch into the new space. Kubernetes therefore expands volumes in two separate phases.

flowchart TD
    A["User raises requested size on the claim"] --> B["Admission check: class allows expansion?"]
    B -->|No| X["Request rejected"]
    B -->|Yes| C["Controller expansion: storage backend enlarges the disk"]
    C --> D["Volume object capacity is raised"]
    D --> E{"Does the volume need a filesystem resize?"}
    E -->|No| G["Claim capacity updated, done"]
    E -->|Yes| F["Claim waits: filesystem resize pending"]
    F --> H["Node expansion: filesystem grows on the node"]
    H --> G
        
FIG 1 — Expansion happens in two phases: the real disk first, then the filesystem on the node
Phase 1

Controller expansion

Runs in the control plane side of the system. It asks the storage backend to enlarge the underlying disk, then raises the capacity recorded on the volume object. No node is involved.

Phase 2

Node expansion

Runs on the machine where the volume is mounted. The kubelet asks the driver’s node component to grow the filesystem so the extra space becomes usable by the Pod.

Simple Analogy

Suppose a bookshelf stands in a room and you move the wall back to make the room bigger. The room is larger, but the bookshelf is still the same size until a carpenter extends it into the new space. Growing the disk moves the wall, and growing the filesystem extends the shelf. Both are needed before you can store more books.

Why two phases exist

The two steps need different powers. Enlarging a cloud disk is an operation performed through the provider’s programming interface and needs no access to a particular node. Growing a filesystem, on the other hand, must be done where the volume is attached, because the operating system on that node owns the mount. Splitting the work this way lets each part run in the right place, and it explains why a claim can spend some time in a waiting state between the two.

AspectController expansionNode expansion
Where it runsControl plane side or a storage controllerOn the node using the volume
What it changesThe real disk and the volume capacityThe filesystem inside the disk
Needs a Pod using the volume?Not necessarilyYes, the volume must be mounted or about to be
Typical failureCloud limits or driver errorsFilesystem problems or unsupported type
!
A Big Disk Is Not Enough

Seeing a larger disk in the cloud console does not mean the application has more space. Until the filesystem has grown, the claim’s status capacity stays at the old value, and the application still sees the old free space.

6Who Does What: The Components Involved

A small team of programs cooperates to turn an edit into extra space.

Several pieces take part in an expansion. Knowing who is responsible for which step makes troubleshooting far easier, because you can ask which component has not done its job.

sequenceDiagram
    participant U as User
    participant API as API Server
    participant R as External Resizer
    participant D as CSI Controller Plugin
    participant S as Storage Backend
    participant K as Kubelet
    participant N as CSI Node Plugin
    U->>API: Raise requested size on the claim
    API->>API: Admission checks the class allows expansion
    R->>API: Notice requested size is larger than capacity
    R->>D: Expand this volume
    D->>S: Enlarge the disk
    S-->>D: Done
    D-->>R: New size, node expansion needed
    R->>API: Update volume capacity, mark filesystem resize pending
    K->>API: Notice pending filesystem resize
    K->>N: Expand the filesystem
    N-->>K: Done
    K->>API: Update claim capacity to the new size
        
FIG 2 — The conversation between components during a typical expansion
Gatekeeper

Admission control

Checks the edit at the moment it is made, and refuses size increases on classes that do not allow expansion.

Watcher

External resizer

A helper that runs alongside the CSI driver. It watches claims, notices when requested size exceeds capacity and asks the driver to expand. It also records progress and errors.

Worker

CSI controller plugin

The part of the driver that talks to the storage system to enlarge the disk. It runs in the cluster rather than on every node.

Local hand

Kubelet and CSI node plugin

On the node, the kubelet notices the pending filesystem resize and the driver’s node component grows the filesystem.

Legacy

Expand controller

A control plane component that handles expansion for older built-in volume types that predate CSI. New clusters rely mostly on CSI drivers instead.

Storage

Storage backend

The real system holding the data, such as a cloud disk service or a storage cluster. It has its own rules about size and timing.

Simple Analogy

Extending a house involves an architect who approves the plan, a clerk who notices the approved plan, builders who add the new room and a local electrician who connects it to the house wiring. Each does one job, and the extension is not finished until the last one signs off.

i
Look at Events

Each of these components records what it is doing as events and conditions on the claim. Reading the events in order usually shows which step is stuck.

7The Claim’s Life During Expansion: Conditions and States

How to read what is happening from the claim itself.

While an expansion is underway, the claim records its progress using conditions. A condition is a small piece of status that says something is true or in progress. Reading these conditions is the quickest way to know whether to wait or to investigate.

flowchart LR
    A["Steady: requested equals capacity"] -->|"User raises requested size"| B["Resizing: disk is being enlarged"]
    B -->|"Disk enlarged, filesystem needs growing"| C["Filesystem resize pending"]
    C -->|"Node grows the filesystem"| D["Steady again at the new size"]
    B -->|"Backend error"| E["Resize error recorded"]
    C -->|"Node error"| E
    E -->|"Cause fixed, retry succeeds"| B
        
FIG 3 — The states a claim passes through while its volume is expanded
State or conditionWhat it meansWhat to do
ResizingThe backing disk is being enlargedWait, and watch events for errors
Filesystem resize pendingThe disk is larger, the filesystem has not grown yetWait for the node step, or ensure a Pod mounts the volume if the driver needs it
Resize errorA step failed and will be retriedRead the event message and fix the cause
Capacity equals requestExpansion completeVerify inside the Pod if desired

The two size fields

Always compare two numbers on the claim: the size requested in the spec and the capacity reported in the status. While they differ, expansion is in progress or stuck. Recent Kubernetes versions also record how much space has been allocated and the status of each resize step, which helps tools and people see exactly how far an expansion has progressed, and supports recovery after a failed attempt.

Simple Analogy

A parcel tracking page shows stages such as “packed,” “in transit” and “out for delivery.” Nobody panics at “in transit,” but a parcel stuck for days at one stage deserves a phone call. Conditions on a claim work the same way.

i
Patience Has Limits

Most expansions finish within seconds to a few minutes. A claim that sits at the same stage for much longer, with repeated error events, needs investigation rather than more waiting.

!
Do Not Edit Again Too Soon

Raising the request again while an earlier expansion is still in progress can confuse both the driver and the backend. Wait for the first expansion to finish, or for a clear error, before changing the size once more.

8Online and Offline Expansion

Whether the application can keep running while its disk grows.

One of the most practical questions is whether a Pod must be stopped. Kubernetes supports expanding a volume while it is in use, which is called online expansion, and this has been a stable feature for several releases. Whether it works for your volume depends on the CSI driver and the filesystem. Some drivers can only complete the filesystem step when the volume is not in use, which is offline expansion, and in that case the filesystem grows the next time a Pod mounts the volume.

flowchart TD
    A["Disk has been enlarged"] --> B{"Does the driver support online filesystem growth?"}
    B -->|Yes| C["Kubelet grows the filesystem while the Pod keeps running"]
    B -->|No| D["Claim shows filesystem resize pending"]
    D --> E["Pod is stopped or restarted"]
    E --> F["Filesystem grows during the next mount"]
    C --> G["Expansion complete"]
    F --> G
        
FIG 4 — Online drivers finish while the application runs, while offline drivers wait for the next mount

Online: Advantages

  • No downtime for the application
  • Resolves a full disk emergency quickly
  • No need to schedule a maintenance window
  • Fits automated expansion well

Online: Trade-offs

  • Depends on driver and filesystem support
  • A busy filesystem grows under load, which deserves a little caution
  • Problems appear while the application is live

Offline: Advantages

  • Growth happens while nothing is writing
  • Simple and predictable for older drivers
  • Fewer surprises for sensitive workloads

Offline: Trade-offs

  • Requires a restart or a gap in service
  • Slow to relieve a full disk emergency
  • Harder to automate safely
Simple Analogy

Widening a motorway can sometimes be done one lane at a time while traffic keeps flowing, and sometimes the road must be closed overnight. Online expansion is the first kind, offline expansion is the second, and the right choice depends on the road and the equipment.

i
Test Your Driver

Do not guess whether your driver supports online expansion. Try it on a non-critical volume first, and record the result, so that during a real emergency you already know whether a restart is needed.

9Filesystems and Raw Block Volumes

What actually lives on the disk decides how the second phase behaves.

Most volumes in Kubernetes are mounted as filesystems. The two filesystems most often used are ext4 and XFS, and both can be grown while mounted, which is what makes online expansion possible. The node step of an expansion runs the right growing tool for whichever filesystem the volume contains. If the filesystem type is one that cannot be grown, the second phase cannot complete, even though the disk itself became larger.

Volume contentCan it grow?Notes
ext4Yes, including while mountedVery common default, grows reliably
XFSYes, while mountedCan only grow, never shrink, which matches Kubernetes behaviour
Other filesystem typesDepends on the type and driverCheck driver documentation before relying on it
Raw block volumeThe disk grows, but there is no filesystem to resizeThe application must notice and use the extra space itself

Raw block mode

A volume can also be given to a Pod as a raw block device, with no filesystem at all. This is used by software that manages the disk directly, such as some databases and storage systems. Expansion enlarges the device, but nobody grows a filesystem, because none exists. The application itself must detect the larger device and extend its own structures. Kubernetes does the first half only.

Simple Analogy

A filesystem is like a library catalogue laid out over a warehouse. If the warehouse grows, the catalogue must be extended to include the new shelves. A raw block volume is a bare warehouse with no catalogue, and whoever uses it must organise the new space on their own.

Space is more than bytes

A filesystem has two limits: how many bytes of data it can hold and how many files it can track. The second is controlled by a pool of file records called inodes. A volume with millions of tiny files can run out of inodes long before it runs out of bytes. Growing a filesystem generally adds more inodes as well, so expansion helps with both problems, but it is wise to monitor both.

!
Free Space Is Not Always Usable

Filesystems keep a small reserve and some space for their own structures. After expansion the usable space is slightly less than the number you requested. Plan for a small margin when choosing sizes.

10Expansion Across Volume Types

Not every kind of storage can be grown, and some do not even mean the same thing by size.

Kubernetes can attach many kinds of storage, and expansion support depends on the type. A rough map of the landscape helps set expectations.

Storage typeExpansion supportComment
Cloud block disks through CSIGenerally yesOften online, but providers have their own limits
Network block storage clusters through CSIGenerally yesDisk grows, then filesystem is extended on the node
Shared filesystems through CSIDepends on the driverSize is often a quota rather than a real disk
NFS volumesNot meaningfulThe requested size is not enforced by the server
Local volumes and host pathsNot handled by Kubernetes itselfSpecial drivers using logical volume managers can add it
Empty directoriesNot applicableThey have a size limit setting, but are temporary by design

Cloud disks

Managed disk services are the most common case. The driver asks the provider to enlarge the disk, and the node then grows the filesystem. Providers may limit how often a disk can be modified and the largest size allowed.

Shared filesystems

For shared filesystem drivers, the size may be a quota on a directory rather than a physical disk. Expansion then means raising the quota, which is quick, and no filesystem growth is involved.

NFS

With plain NFS the requested size is only a label. The server does not enforce it, so raising the number changes nothing real. Space is managed on the NFS server itself.

Local storage

A disk attached to a single node cannot be stretched by Kubernetes. Drivers that carve local volumes from a logical volume manager can provide expansion, because the manager can enlarge a volume if the node still has free space in its pool.

Simple Analogy

Asking for a bigger storage unit works differently depending on the building. In a modern facility, staff can knock down a wall. In a shared warehouse with only a quota, a manager can simply raise your limit. In a rented shed attached to one house, there may be nothing to expand into.

i
Read the Driver Guide

Every CSI driver documents which expansion features it supports and any limits. Treat that page as part of your storage design, and not as an afterthought.

11Expansion in StatefulSets

The place where expansion stops being a one-line edit and needs a careful routine.

Databases and other stateful systems are usually run as StatefulSets. A StatefulSet creates one Pod per replica and, for each, a separate claim built from a volume claim template. This gives each replica its own persistent disk, which is exactly what a database cluster needs.

The catch is that the volume claim template inside a StatefulSet cannot be changed after creation. If you edit the template to request a larger size, the update is refused. The template only matters when new claims are created, and existing claims are not touched by it. This surprises many teams the first time they try to grow a database.

flowchart TD
    A["Need more space for a StatefulSet"] --> B["Raise the requested size on each existing claim one by one"]
    B --> C["Wait until every claim has finished expanding"]
    C --> D["Delete the StatefulSet object but keep its Pods and claims"]
    D --> E["Recreate the StatefulSet with the larger size in its template"]
    E --> F["Controller adopts the existing Pods and claims"]
    F --> G["Future replicas get the new size"]
        
FIG 5 — The common routine: expand the claims, then recreate the StatefulSet definition without disturbing Pods
1

Expand each claim

Edit every replica’s claim directly. They are ordinary claims, and the usual expansion process runs for each.

2

Verify completion

Confirm the status capacity of every claim matches the new request, and that the application sees the extra space.

3

Remove only the StatefulSet object

Delete the StatefulSet using the orphaning option, which leaves its Pods and claims running and untouched.

4

Recreate it with the new size

Apply the same definition with the larger size in the template. The new object adopts the Pods that already exist.

5

Check the next scale-up

When you add replicas later, their new claims will use the larger template size, keeping all replicas consistent.

Simple Analogy

A restaurant’s menu card says every table gets a standard plate size. If you want bigger plates, changing the menu card does nothing for tables already seated. You must replace the plates at each table, and then reprint the menu so that new tables get the bigger plates too.

!
Use the Orphan Option Carefully

Deleting a StatefulSet without the orphaning option removes its Pods. For a database that can mean an outage, so always double-check the deletion mode, and rehearse the whole routine on a test cluster first.

i
Improvements Are Coming

The Kubernetes community has been working on making volume claim templates easier to update. Check the documentation for your version, because the recommended routine may become simpler.

12Quotas, Limits and Cloud Provider Constraints

Even a valid request can be blocked by rules set by administrators or by the storage provider.

An expansion request passes through several layers of control, and any of them may say no. Some belong to Kubernetes and some to the storage system.

Kubernetes

Resource quotas

Administrators can limit the total storage a namespace may request. A resize counts toward that total, so an expansion can be refused if it would exceed the quota.

Kubernetes

Limit ranges

Rules can set minimum and maximum sizes for a single claim in a namespace. A request above the maximum is rejected.

Provider

Maximum disk size

Every disk type has an upper size limit set by the provider, which no setting in Kubernetes can override.

Provider

Modification cool-down

Some providers allow only one modification per disk within a waiting period of several hours. A second expansion inside that window is delayed or refused.

Provider

Account quotas

The cloud account may have a total storage allowance, and growing a disk consumes it.

Cost

Billing

Larger disks cost more, and because they cannot shrink, the extra cost continues until the volume is replaced.

LimitWho sets itTypical sign when hit
Namespace storage quotaCluster administratorEdit refused with a quota message
Per-claim maximumCluster administratorEdit refused with a limit message
Disk size ceilingStorage providerBackend error on the claim
Modification waiting periodStorage providerExpansion stalls with a retry message
Account storage allowanceCloud accountBackend error mentioning limits
Simple Analogy

Asking the school for a bigger locker involves the caretaker, who must have lockers of that size, the head of year, who limits how much each student may have, and the budget office, which limits the total for the whole class. Any one of them can say no.

!
The Cool-down Trap

If a provider enforces a waiting period between changes, a team that grows a disk by too small a step may be unable to grow it again for hours, right when the next incident hits. Prefer steps large enough to give real breathing room.

13Failure Modes and Recovery

What can go wrong, how to recognise it, and how to get back on track.

Most expansions succeed without drama. When they do not, the cause usually falls into a short list. Recognising the pattern quickly saves a lot of time.

Stuck at filesystem resize pending

The disk has grown but the claim never reaches the new capacity. Common causes are a driver that needs the volume mounted by a Pod, a Pod that is not running, or a driver without node-side support. Starting or restarting a Pod that uses the claim often completes the step.

Backend refuses to grow the disk

The claim stays in the resizing state with repeated errors. Typical reasons are exceeding the provider’s maximum size, hitting a cool-down period, running out of account allowance or a temporary provider fault. The events on the claim usually quote the provider’s message.

Requested size too large to ever succeed

Someone accidentally requests a huge size that the backend can never provide. Because shrinking is not allowed, the claim could be stuck. Recent Kubernetes versions include a recovery capability that lets the requested size be lowered back to a value that is still not smaller than the current capacity, as long as the failed expansion has not completed. Check whether your version supports it.

Filesystem growth fails on the node

The node step reports an error, such as a damaged filesystem or an unsupported type. This is serious, because the application may be affected. Take a snapshot if possible, inspect the node-side logs, and involve the storage team before retrying.

1

Read the claim events

The most recent events usually name the component that failed and the reason.

2

Compare requested size and capacity

See which phase completed and which did not.

3

Check the resizer and the driver

Look at the logs of the external resizer and the driver’s controller and node components.

4

Check the backend

Look in the provider’s console or tools for errors, limits and the state of the disk.

5

Fix, then let it retry

Controllers retry automatically once the cause is removed, so manual edits are rarely needed.

!
Snapshot First

Expansion is routine and usually safe, but it modifies live storage. Taking a snapshot or backup before growing important volumes costs little and gives a way back if something unexpected happens.

14Why You Cannot Shrink, and What to Do Instead

When a volume is far too big, the only route is to build a smaller one and move the data.

Sometimes a volume is simply oversized. Maybe a one-off import filled it, then the data was deleted, and now it is mostly empty and expensive. Since shrinking is not supported, the way to reclaim the space and money is migration: create a new, smaller claim, copy the data across and switch the application over.

flowchart TD
    A["Volume is far larger than needed"] --> B["Take a backup or snapshot"]
    B --> C["Create a new smaller claim"]
    C --> D["Stop writes to the application"]
    D --> E["Copy data to the new claim"]
    E --> F["Verify the copy"]
    F --> G["Point the application at the new claim"]
    G --> H["Keep the old volume briefly, then delete it"]
        
FIG 6 — Shrinking is done by migrating to a new, smaller volume

Ways to move the data

The best method depends on the application. For databases, the safest approach is usually to use the database’s own tools, for example by adding a new replica with a smaller volume and removing the old one after replication catches up. For file data, a one-time copy job can transfer files while the application is paused. For some storage systems, restoring from a snapshot into a new claim is the quickest route, provided the driver allows the new claim to be sized as needed.

Migration: Advantages

  • Reclaims wasted space and cost
  • Can also change the storage class, for example to a cheaper tier
  • Opportunity to clean up and verify data

Migration: Trade-offs

  • More steps and more risk than an expansion
  • May need downtime or a careful cut-over
  • Needs temporary extra space during the copy
Simple Analogy

You cannot shrink a warehouse in place. To pay less rent, you rent a smaller warehouse, carry the goods across and hand back the keys to the old one. It takes effort, so it is wise to avoid getting into this situation by growing in measured steps in the first place.

i
Free Space Inside Is Still Free

A large volume that is mostly empty is not unsafe, only costly. If cost is not a pressing concern, leaving it alone avoids the risk of migration.

15GitOps, Helm and the Problem of Drift

When the cluster holds a size that your configuration repository does not know about.

Many teams manage their clusters from configuration stored in a repository, applied by tools such as Helm or by GitOps controllers that continuously compare the repository with the cluster. Volume expansion interacts with this in an awkward way, because the size on the claim in the cluster can become different from the size written in the repository.

Suppose an engineer expands a claim during an incident, directly in the cluster. The repository still says the old, smaller size. The next time the tool applies the configuration, it tries to set the old size on the claim. Since shrinking is not allowed, the update is rejected. Depending on the tool, this shows up as a failed deployment, a permanently out-of-sync application or a noisy alert. The cluster is fine, but the process is broken.

1

Expand during an incident

A claim is raised directly in the cluster to stop an outage.

2

Drift appears

The repository still holds the smaller size.

3

The tool tries to reconcile

It attempts to apply the smaller size and is refused by Kubernetes.

4

Fix the source of truth

The repository is updated to the new, larger size so that both agree again.

Habit 1

Expand through the repository

Whenever time allows, change the size in the configuration and let the tool apply it, so the record is always correct.

Habit 2

Back-port emergency changes

If you must expand by hand, update the repository straight afterwards as part of the incident follow-up.

Habit 3

Ignore size differences deliberately

Some tools can be told to ignore the size field on claims that are expected to grow. Use this only for volumes managed by automatic expansion.

Habit 4

Mind StatefulSet templates

Remember that templates cannot be edited in place, so the repository must be changed in step with the routine from Chapter 11.

Simple Analogy

A building’s official floor plan says the office has ten desks, but staff quietly add two more over the winter. The next time an inspector checks the plan, the building appears to be breaking the rules. The fix is not to remove the desks, it is to update the plan so it matches reality.

i
Make the Direction One-Way

Since expansion only goes up, a good policy is that the configuration size must always be greater than or equal to the live size. A simple check in the pipeline can catch accidental shrinking requests before they reach the cluster.

16Related Features: Performance Changes and Snapshots

Neighbouring storage features that often come up in the same conversations.

Volume expansion is only one of several ways in which an existing volume can change or be protected. Understanding its neighbours prevents confusion, and helps you pick the right tool for the job.

Size

Volume expansion

Changes how much space a volume has. Only increases are allowed, and the storage class must permit it.

Performance

Volume attributes classes

A newer feature that lets certain qualities of a volume, such as its speed or throughput, be changed after creation without changing its size, for drivers that support it. Check the maturity of this feature in your version.

Protection

Volume snapshots

A point-in-time copy of a volume. Snapshots are a safety net before expansions and a route for cloning or moving data.

Copy

Volume cloning

Creates a new volume that starts with the contents of an existing one. It can be useful for migration to a different size or class.

NeedRight feature
More space on the same volumeVolume expansion
More speed on the same volumeVolume attributes class, if the driver supports it
A restore point before a risky changeVolume snapshot
A copy of a volume for testingVolume clone or restore from snapshot
A smaller volumeMigration to a new claim
Simple Analogy

If your locker is full, you ask for a bigger one, which is expansion. If the locker door is slow and sticky, you ask for a better lock, which is a performance change. If you worry about losing your things, you take a photo of everything inside, which is a snapshot.

!
Do Not Confuse Size and Speed

On many cloud disks, throughput and operations per second depend on the disk size. A bigger disk can be faster as a side effect, but that is a property of the provider, not something Kubernetes promises.

17Monitoring Capacity Before It Becomes an Incident

The best expansion is the one done calmly, days before the disk fills.

The kubelet reports usage for every mounted volume, so you can see how full each one is without logging in to any application. These numbers flow into the monitoring system as metrics, and with them you can build alerts that warn long before a disk fills.

Metric

Capacity

The total size of each mounted volume as the node sees it.

Metric

Used and available bytes

How much data is stored and how much room remains, which together show the fill level.

Metric

Inodes used and free

Tracks the number of files the filesystem can still hold, which can run out before bytes do.

Trend

Growth rate

The speed at which usage rises, which lets you predict when the volume will be full.

AlertIdeaTypical threshold style
Warning on high usageEarly notice for planned actionRoughly three-quarters to four-fifths full
Critical on very high usageAct nowAround nine-tenths full
Predicted to fill soonUses the growth trend rather than a fixed levelProjected to be full within a day or a few days
Inodes running lowCatches file-count exhaustionSimilar levels to byte usage
Expansion stuckClaim shows requested size above capacity for too longLonger than the normal completion time

Predictive alerts beat fixed thresholds

A fixed threshold treats a slowly growing volume and a rapidly filling one the same way. A volume at eighty percent that grows by one percent a month needs no hurry, while a volume at seventy percent that fills by ten percent an hour is in danger. Alerts based on the growth trend, which estimate the time until full, tell you what you really want to know: how long you have.

Simple Analogy

A fuel gauge showing a quarter tank means different things on a short commute and on a long desert road. What matters is how fast you are using fuel and how far you must go. Predictive alerts are the trip computer that says “you will run dry in forty minutes.”

i
Watch the Application Too

Some applications have their own disk safety rules, such as search engines that stop accepting data at certain fill levels. Your alerts should fire before those internal limits are reached.

18Automating Expansion

Letting a controller grow volumes before a human has to notice.

Because expansion is online and quick for many drivers, it is tempting to automate it. Community tools exist that watch the usage metrics and raise the requested size of a claim when it passes a threshold, by a set step, up to a safe maximum. This turns many night-time pages into silent, uneventful growth.

flowchart LR
    A["Kubelet reports volume usage"] --> B["Monitoring stores the metrics"]
    B --> C["Auto-expansion controller checks fill level"]
    C -->|Below threshold| D["Do nothing"]
    C -->|Above threshold and below the ceiling| E["Raise the requested size by one step"]
    E --> F["Normal two-phase expansion runs"]
    F --> A
    C -->|At the ceiling| G["Alert a human instead"]
        
FIG 7 — A feedback loop: measure, decide, expand within limits, and alert at the ceiling

Advantages

  • Prevents most full-disk incidents
  • Removes repetitive manual work
  • Reacts faster than any person can
  • Keeps volumes right-sized over time instead of oversized from day one

Disadvantages / Trade-offs

  • A runaway process can drive costs up without anyone noticing
  • Growth is one-way, so mistakes are permanent
  • Cool-down periods at the provider can block rapid repeats
  • Drift with GitOps must be handled deliberately

Guardrails for automation

Good automation has limits. Set a maximum size per volume so a bug cannot grow a disk without end. Choose a step that is large enough to respect provider cool-downs but small enough to avoid waste. Alert people whenever an automatic expansion happens, because rapid growth is itself a signal that something is writing more than expected. And make sure the claim’s configuration in the repository is set up to tolerate growth.

Simple Analogy

An automatic car park barrier lifts to let cars in, but it still has a counter that tells the manager when the lot is nearly full. Automation should open the gate and also keep a human informed, because a lot that keeps filling up deserves a conversation.

!
Do Not Hide a Leak

If a volume keeps growing quickly, the cause may be a bug, such as logs that never rotate. Automatic expansion can hide the leak while the bill grows. Always investigate unusual growth.

19Security, Access Control and Cost Governance

Growing a disk is a small action with real financial and operational consequences.

Because expansion is as easy as editing one field, it deserves the same care as any other change that affects money and data. Several controls keep it safe.

Access

Who may edit claims

Role-based access control decides who can update claims. Grant it to the teams that own the workload, and watch for broad permissions that would let anyone enlarge disks.

Access

Who may change classes

Turning expansion on or off for a class is a cluster-level decision. Limit it to administrators.

Budget

Quotas as cost brakes

Namespace storage quotas stop any single team from growing without bound, and act as a budget guardrail.

Audit

Change history

Audit logs record who changed which claim and when, which is valuable when a bill jumps or a size is wrong.

Data

Encryption stays in place

For volumes encrypted by the storage provider, expansion keeps the same encryption keys and settings in normal cases. Confirm this for your driver and provider.

Safety

Backups before big changes

Any change to live storage should follow the organisation’s backup rules.

RiskControl
Uncontrolled storage costNamespace quotas, maximum sizes and review of large requests
Unauthorised size changesTight access rules on claim updates
Silent growth from a faulty applicationAlerts on growth rate and on automatic expansions
Data loss from an interrupted resizeSnapshots or backups before important expansions
Unclear who made a changeAudit logging and change records in the repository
Simple Analogy

Allowing any department to order extra warehouse space with a single phone call is convenient, but the finance team will want a budget cap, a record of who called and a rule on how much each department may order.

i
Make Cost Visible

Expose storage use and cost per team on a dashboard. Teams that see their own numbers tend to clean up unused data before asking for more space.

20Best Practices and a Safe Expansion Runbook

Habits and steps that turn expansion into a calm, repeatable task.

Practice 1

Enable expansion on every class

Make it a default when designing classes, so no volume is ever created that cannot grow.

Practice 2

Know your driver

Record whether it supports online expansion, which filesystems it handles and what provider limits apply.

Practice 3

Start modest, grow in sensible steps

Avoid huge initial sizes and also avoid tiny increments that waste cool-down windows.

Practice 4

Alert early and by trend

Use predictive alerts and inode alerts, so planned expansion replaces emergency expansion.

Practice 5

Keep the repository truthful

Record sizes in configuration and keep them equal to or above live sizes.

Practice 6

Snapshot before critical changes

Protect important volumes before expanding them.

Practice 7

Rehearse the StatefulSet routine

Practise the claim-by-claim approach and the orphaning delete on a test cluster.

Practice 8

Guard automation

Use maximum sizes, quotas and notifications whenever expansion is automated.

A runbook for a planned expansion

1

Confirm the prerequisites

Check that the class allows expansion, the driver supports it and the claim is bound.

2

Check limits and budget

Look at quotas, provider limits, cool-down timers and cost.

3

Take a snapshot

Protect the data for important volumes before any change.

4

Raise the requested size

Apply the change through the normal configuration path, choosing a step with real breathing room.

5

Watch conditions and events

Follow the claim through resizing and filesystem growth until capacity equals the request.

6

Verify from the application

Confirm that the application sees the extra free space and behaves normally.

7

Update records

Make sure the repository, documentation and alerts match the new size.

21Common Mistakes and Anti-patterns

Five habits that regularly turn a simple resize into a long night.

ANTI-PATTERN-01 Avoid
Problem

Storage classes are created without enabling expansion, because nobody expected to need it.

Why It’s Harmful

When a disk fills, the first step of the emergency is to change the class, and existing workflows may not allow that quickly or may lack the permission to do so.

Correct Approach

Enable expansion on every class from the start, and confirm the driver supports it.

ANTI-PATTERN-02 Avoid
Problem

The team assumes that a larger disk in the cloud console means the application has more space.

Why It’s Harmful

The filesystem may not have grown yet, so the application still sees the old free space and keeps failing while people believe the problem is solved.

Correct Approach

Verify that the claim’s status capacity matches the request and that the application sees the new space.

ANTI-PATTERN-03 Avoid
Problem

Engineers expand claims by hand during incidents and never update the configuration repository.

Why It’s Harmful

The next automatic sync tries to apply the old smaller size, is refused, and leaves the application permanently out of sync and noisy.

Correct Approach

Update the repository straight after any manual change, or make all expansions through the repository.

ANTI-PATTERN-04 Avoid
Problem

Volumes are grown by tiny steps every time they fill, such as one extra gigabyte at a time.

Why It’s Harmful

Providers may impose waiting periods between changes, so the next expansion can be blocked just when it is needed, and each change carries some risk and effort.

Correct Approach

Grow in steps that give real breathing room, and use predictive alerts to plan expansions ahead of time.

ANTI-PATTERN-05 Avoid
Problem

Automatic expansion is turned on with no ceiling and no notifications.

Why It’s Harmful

A bug that writes endlessly can grow disks until the budget is gone, and since shrinking is impossible, the cost stays until volumes are migrated.

Correct Approach

Set maximum sizes, use quotas, notify on every automatic change and investigate unusual growth.

SymptomLikely causeFirst check
Edit rejected on the spotClass does not allow expansionClass setting
Claim stuck at filesystem resize pendingNo Pod mounting the volume, or no node-side supportPods using the claim and driver capabilities
Repeated backend errorsProvider limits or cool-downClaim events and provider console
GitOps tool reports a failed applyRepository holds a smaller size than the clusterCompare sizes in repository and cluster
StatefulSet update refusedClaim template cannot be editedUse the claim-by-claim routine

22Advantages, Disadvantages and Trade-offs

A balanced view of what expansion gives you and where it stops.

Advantages

  • Grows storage in place with no data copying
  • Often works online, with no downtime for the application
  • Simple for users: one field on one object
  • Works the same way across many storage vendors through CSI
  • Lets teams start small and grow as real needs appear
  • Can be automated safely with proper guardrails

Disadvantages / Trade-offs

  • Only growth is possible, never shrinking
  • Depends on class settings, driver support and provider limits
  • StatefulSet templates need a special routine
  • Can create drift with configuration repositories
  • Provider cool-down periods can limit repeated expansion
  • Some storage types, such as plain NFS or bare local disks, offer no real expansion

How it compares with the alternatives

ApproachIdeaStrengthWeakness
Oversize from the startPick a large volume at creationRarely needs changesPays for empty space, cannot shrink
In-place expansionGrow the existing volumeNo copying, often no downtimeGrowth only, depends on driver
Migrate to a new volumeCopy data to a differently sized volumeAllows shrinking and class changesMore effort, risk and possible downtime
Add replicas or shardsSpread data across more volumesScales beyond one diskNeeds application support and rebalancing
Offload old dataArchive or delete data to free spaceReduces cost and size togetherNeeds retention rules and care

The sensible approach in most teams combines several of these. Start with a reasonable size, enable expansion, alert on trends, grow in measured steps, and review volumes periodically so that oversized ones can be migrated and old data can be archived. Expansion is a tool for handling uncertainty, and not a replacement for knowing how fast your data grows.

“Expansion lets you be wrong about the size of a disk once, cheaply. It does not let you stop paying attention.”

23Real-World and Industry Examples

How teams use volume expansion in everyday operations.

Growing databases

A team running a relational database on a StatefulSet starts with modest volumes and raises each replica’s claim as data grows. They follow a written routine for the claim template and verify free space from inside the database after each change.

Search and log platforms

Search clusters store large indexes and are sensitive to disk fill levels, because they stop accepting data when space is low. Teams alert on predicted fill time and expand ahead of those internal thresholds.

Message brokers

Brokers keep backlogs on disk. When a downstream system is slow, backlogs grow quickly. Online expansion buys time to fix the slow consumer without losing messages.

Platform teams with self-service storage

Central platform teams give application teams self-service claims with expansion enabled, protected by namespace quotas and cost dashboards, so teams can grow within a budget without filing tickets.

Automatic expansion for variable workloads

Teams with unpredictable workloads use a controller that grows claims when they pass a threshold, with a maximum size and a notification on every change, turning night-time pages into daytime reviews.

Right-sizing after a one-off event

After a large temporary import left several volumes oversized, one team scheduled a migration to smaller volumes during a quiet weekend, using backups and replica-based copying, and used the opportunity to move data to a cheaper storage class.

i
Common Thread

Teams that handle expansion well tend to treat it as a routine, documented operation with monitoring, limits and rehearsal, instead of an emergency trick used under pressure.

24Frequently Asked Questions

Short answers to the questions beginners ask most often.

Q1Can I make a volume smaller in Kubernetes?

No. Reducing the requested size of a bound claim is not supported. To get a smaller volume, create a new claim of the right size, copy the data across and switch the application to it.

Q2Why was my edit to the claim size rejected immediately?

Most often the storage class does not allow volume expansion. An admission check refuses size increases on such classes. Enable the setting on the class, or create a new class that allows it.

Q3Do I need to restart my Pod to expand a volume?

Not always. Many drivers support online expansion, so the filesystem grows while the Pod runs. Some drivers complete the filesystem step only when a Pod mounts the volume, which then requires a restart.

Q4The cloud disk is bigger, but my application still sees the old size. Why?

The first phase, growing the disk, is done, but the second phase, growing the filesystem, has not finished. Check the claim’s conditions and capacity, and make sure a Pod is mounting the volume if the driver needs one.

Q5How do I expand volumes created by a StatefulSet?

The claim template cannot be edited, so expand each existing claim directly, then delete the StatefulSet while leaving its Pods and claims in place, and recreate it with the larger size in the template.

Q6Is expanding a volume safe for my data?

It is designed to be safe and is routine, but it changes live storage. For important data, take a snapshot or backup first, and watch the process until it completes.

Q7Why does my GitOps tool complain after someone expanded a claim?

The repository probably still holds the old, smaller size, and the tool tries to apply it. Kubernetes refuses to shrink, so the apply fails. Update the repository to match the larger size.

Q8Can I expand NFS or local volumes?

Plain NFS does not enforce the requested size, so changing it has no real effect. Local volumes cannot be grown by Kubernetes itself, although some drivers built on logical volume managers support it.

Q9Can expansion be automated?

Yes. Community controllers watch usage metrics and raise the requested size when a threshold is crossed. Use a maximum size, quotas and notifications so that a faulty application cannot grow disks without limit.

Q10What if I request a size that is far too large?

If the backend cannot provide it, the claim stays in an error state with retries. Recent Kubernetes versions can recover from failed expansions by letting you lower the request, as long as it stays above the current capacity. Check support in your version and driver.

25Summary and Key Takeaways

Volume expansion lets you give a running application more disk space by changing one number on a claim. Behind that simple edit, Kubernetes and the storage driver work in two phases: first the backing disk is enlarged, then the filesystem on top of it is grown on the node, often while the application keeps running. It requires a storage class that allows expansion, a driver that supports it and a storage backend with room to grow, and it can be limited by quotas and provider rules. It only goes one way, so shrinking means migrating to a new volume. Stateful workloads need a careful routine because their claim templates cannot be edited, and configuration repositories must be kept in step with live sizes. With good monitoring, sensible steps and guarded automation, expansion turns a classic emergency into a routine, quiet operation.

Key Takeaways

  • Expansion is a request on the claim — you raise the requested size, and controllers reconcile the real capacity up to that number.
  • Two phases must both finish — the backing disk is enlarged first, and the filesystem is grown on the node second. A bigger disk alone gives the application no extra space.
  • Three prerequisites apply — the storage class must allow expansion, the CSI driver must support resizing and the backend must be able to grow the disk within its limits.
  • Only growth is supported — shrinking requires creating a new smaller volume and migrating data, so each expansion is a one-way decision.
  • Online expansion depends on the driver — many drivers grow the filesystem while the Pod runs, while others wait for the next mount, so test your driver before an emergency.
  • StatefulSets and GitOps need care — expand each claim directly, recreate the StatefulSet with the orphaning delete, and keep repository sizes equal to or above live sizes.
  • Plan, alert and guard — use predictive alerts, snapshots before big changes, quotas and maximum sizes, and investigate unusual growth instead of simply paying for it.