Local SSD in Google Cloud: Fast, Disposable Storage Built Into the Server
A deep, plain-English guide to Local SSD: how it differs from network disks, how fast it really is, when your data survives and when it vanishes, and the design patterns that make ephemeral storage safe in production.
Think about the desk you work at. Your notebook sits right beside your hand, so grabbing a note takes a second. Your filing cabinet is down the hall, safer and bigger, but every trip costs time. Local SSD is the notebook on your desk. It is storage bolted directly onto the server that runs your virtual machine, so it is extremely fast. But like a notebook left on a desk you are about to vacate, it does not travel with you, and if the desk is cleared, the notes are gone.
1What Local SSD Is and Why It Exists
Local SSD is high-speed solid state storage that is physically attached to the host server running your Compute Engine virtual machine.
Most disks in Google Cloud are network disks. When your virtual machine reads or writes data on a Persistent Disk or Hyperdisk volume, the request travels across Google’s data center network to a storage system that keeps the data safe and replicated. That design gives you durability and flexibility, but every request pays a small cost for crossing the network. For most applications that cost is perfectly fine.
Some workloads cannot accept even that small cost. A database serving millions of lookups per second, a cache that must answer in microseconds, or a data processing job that writes huge temporary files all benefit from storage that sits on the same machine. Local SSD fills that role. Because the drive is attached to the host, requests skip the network entirely, giving very low latency and very high throughput for the price.
The trade-off is the heart of this entire topic. Local SSD is ephemeral. It is not replicated, it is not backed up, and it does not follow your VM if the VM is stopped or the host fails. It is a performance tool, not a safe place to keep the only copy of anything important.
A chef keeps frequently used knives and cutting boards on the counter, within arm’s reach. The walk-in freezer holds the real inventory but takes a trip to reach. Local SSD is the counter: fast, convenient, and cleared at the end of every shift. Persistent Disk is the freezer: slower to reach, but it keeps everything safe overnight.
Who uses it and why
Teams reach for Local SSD when they need speed more than they need single-copy durability. Typical cases include caches, scratch space for data processing, hot tiers of replicated databases, staging areas for machine learning training data, and temporary files created during video encoding. In each case, the data is either rebuildable from elsewhere or is already copied to other machines by the application itself.
Local SSD is not a special kind of Persistent Disk. It is a different category of storage with different physics, different rules, and different billing. Treat it as its own tool.
2Core Concepts You Must Know Cold
A small set of terms explains almost every decision you will make about Local SSD.
Each term below is defined by what it is, why it exists, and where you will meet it. Spend a moment on each card, because mixing them up is the cause of most Local SSD surprises.
Local SSD disk
A solid state drive attached to the host server. On most machine types each disk has a fixed size, and a VM can have several.
Ephemeral storage
Storage whose contents are not guaranteed to outlive the VM or its host. The opposite of durable storage.
Persistent Disk and Hyperdisk
Network-attached block storage that is replicated, survives VM stops, can be snapshotted and can move between VMs.
NVMe interface
A modern, low-latency protocol for talking to flash drives. It allows many parallel queues and gives the best Local SSD performance.
SCSI interface
An older protocol that is widely supported but offers lower performance than NVMe for the same hardware.
IOPS
Input and output operations per second, the count of small reads and writes a drive handles each second.
Throughput
The volume of data moved per second, usually in megabytes or mebibytes per second. It matters for large sequential transfers.
Live migration
The process of moving a running VM to another host during maintenance without stopping it. Local SSD data can ride along through this event.
IOPS is how many separate errands a courier completes per hour. Throughput is how heavy a load the courier carries per hour. A courier delivering tiny envelopes is limited by errands, while a courier moving furniture is limited by weight. Different workloads care about different limits.
Block device, not a file system
A new Local SSD is a raw block device. It has no files, no folders, and no file system until you create one. This surprises beginners who expect a ready-to-use folder. You, or your startup scripts, must format the device, mount it, and handle the case where it is empty after certain events.
Many people assume Local SSD behaves like a USB drive that keeps its data until erased. It does not. The data lifetime is tied to the VM and its host, and several ordinary events erase it.
3Architecture: Host-Attached Versus Network-Attached Storage
The speed and the fragility of Local SSD both come from the same fact: it lives inside the server.
To understand Local SSD you need a clear picture of where the bytes live. A Compute Engine virtual machine runs on a physical host, and that host has its own processors, memory, and, for Local SSD, flash drives physically installed inside or directly attached. When your VM talks to a Local SSD, the request travels over the host’s internal connection, a path of a few inches of wire and silicon.
When your VM talks to a Persistent Disk or Hyperdisk volume, the request leaves the host, crosses the data center network, and reaches a separate storage service that stores multiple copies of your data. This separation is what gives network disks their durability and flexibility. It is also what adds latency.
flowchart TD
VM["Your Virtual Machine"] --> HOST["Physical Host Server"]
HOST --> LSSD["Local SSD: attached to the host"]
HOST --> NET["Data Center Network"]
NET --> PD["Persistent Disk or Hyperdisk Storage Service"]
PD --> R1["Replica 1"]
PD --> R2["Replica 2"]
PD --> R3["Replica 3"]
Why this explains every trade-off
Benefits of Host Attachment
- Very low latency because no network hop is involved
- Very high IOPS and throughput for the cost
- Performance not shared with other network storage traffic
- Predictable behavior for cache and scratch workloads
Costs of Host Attachment
- No built-in replication or backup of the data
- Data tied to one host and one VM lifetime
- Cannot be detached and attached to another VM
- No snapshots or images from the device
- Must be chosen when the VM is created
Keeping a document in your pocket means you can read it instantly, but if you lose the jacket, the document is gone. Keeping it in a bank safe deposit box means a trip across town, but it survives fire and theft. Host-attached versus network-attached storage is the pocket versus the safe.
What Local SSD cannot be
Local SSD cannot be used as a boot disk, because the operating system image and the VM’s identity must survive stops and restarts. It cannot be shared between VMs, because each device is wired to one host. It cannot be resized after creation, and it cannot be snapshotted by Compute Engine’s snapshot feature. These restrictions all follow from the same fact: the drive belongs to the host, not to the cloud storage system.
4The Data Lifecycle: When Data Survives and When It Vanishes
The most important question to answer before using Local SSD is which events preserve your data and which erase it.
Local SSD data persists through some events and is lost in others. Google’s documentation lists the exact rules, and they can vary by machine series and by newer features, so always verify against the current documentation. The general shape is stable and worth memorizing.
| Event | Typical effect on Local SSD data | Why |
|---|---|---|
| Guest operating system reboot | Data is preserved | The VM stays on the same host with the same devices |
| Live migration for host maintenance | Data is preserved | Compute Engine copies the data to the new host along with the VM |
| VM stop or shutdown | Data is generally lost | The VM gives up its hold on the host’s drives |
| VM deletion | Data is lost | The devices are released and securely erased |
| Spot VM preemption | Data is lost | The VM is stopped by Google |
| Host failure | Data is generally lost | The host disappears and the VM restarts elsewhere |
sequenceDiagram
participant A as Application
participant V as VM with Local SSD
participant H as Host Server
participant G as Google Maintenance
A->>V: Write data to Local SSD
V->>H: Data stored on host drive
G->>H: Planned maintenance needed
H->>V: Live migrate VM to new host
Note over V,H: Data copied and preserved
G->>H: Hardware failure strikes
H--xV: Host lost, drive lost
Note over V,H: VM restarts elsewhere with empty Local SSD
Planned versus unplanned
The difference between a planned event and an unplanned one is the key insight. When Google plans to update a host, it can live migrate your VM and carry the Local SSD contents along. When something breaks without warning, there is no chance to copy anything. A design that survives only the planned events is not safe, because unplanned failures eventually happen to every fleet.
Newer preservation features
Google has been expanding what happens to Local SSD data around certain operations on some machine series, such as suspend and resume behavior. Features like this are specific to certain series and change over time. Do not build a design that depends on them without confirming support for your exact machine type in the current documentation.
Testing only with reboots and concluding that data is safe. Reboots preserve data, but stops, preemptions and host failures do not. Test the failure cases deliberately, because those are the ones that hurt.
Write a one-line data policy for every Local SSD user in your team: “This data can be rebuilt from X within Y minutes.” If nobody can finish that sentence, the data does not belong on Local SSD.
5Performance Anatomy: IOPS, Throughput and Latency
Local SSD is fast, but how fast depends on the interface, the number of disks and the size of the VM.
Three numbers describe storage performance. Latency is how long a single request takes. IOPS is how many requests complete each second. Throughput is how much data moves each second. Local SSD gives very good numbers on all three because there is no network between the VM and the drive.
NVMe versus SCSI
Local SSD can be attached using NVMe or SCSI. NVMe was designed for flash. It supports many parallel queues, so multiple processor cores can issue requests at the same time without waiting on each other. SCSI is an older protocol designed in the era of spinning disks, and it handles flash more slowly. When your machine type and operating system support NVMe, it is usually the better choice for performance. Google’s documentation publishes representative numbers per disk and for a fully populated VM, with NVMe delivering substantially higher read IOPS than SCSI at the same capacity. Treat the published figures as the authority, because they differ by machine series.
| Aspect | NVMe | SCSI |
|---|---|---|
| Design origin | Built for flash storage | Inherited from the spinning disk era |
| Parallel queues | Many, per processor core | Fewer, more limited |
| Typical performance | Highest available for Local SSD | Lower for the same hardware |
| Best for | Databases, caches, intense random I/O | Older images or tools that need SCSI |
Performance grows with disk count and vCPUs
Each Local SSD contributes its own share of performance, so adding disks adds capacity and speed together. Combining several disks into a striped volume lets one file system use all of them at once. But there is a catch: the VM must have enough virtual CPUs to drive the disks. A tiny VM cannot generate enough parallel requests to saturate many drives, and Google’s documentation notes that achievable performance depends on the vCPU count. Choosing a very small VM with many disks wastes money.
Interface
NVMe usually outperforms SCSI by a wide margin on the same drives.
Number of disks
More disks provide more total capacity and total throughput when combined.
vCPU count
The VM needs enough processors to issue requests fast enough to use the drives.
Queue depth and parallelism
Flash shines when many requests are in flight at once. A single sequential thread leaves speed unused.
Block size and access pattern
Small random requests are limited by IOPS, large sequential ones by throughput.
A multi-lane highway only moves lots of cars if there are lots of cars to drive on it. A single driver on an eight-lane road uses one lane. NVMe is the wide highway, and parallel requests from many processor cores are the cars.
Benchmark with your real access pattern, not just a synthetic test. A database doing small random reads behaves very differently from a job writing large sequential files, and Local SSD shines differently in each.
6Capacity, Machine Types and Hard Limits
You cannot choose any size you like, and the rules differ across machine families.
On many machine types, each Local SSD disk has a fixed size of 375 gibibytes, and you attach a number of disks at VM creation. The maximum number of disks per VM depends on the machine series and often on the number of vCPUs, so a larger VM can usually hold more disks than a smaller one. Some newer or specialized machine series work differently, offering variants where a set amount of Local SSD is bundled into the machine type, sometimes in larger sizes. Always consult the machine family documentation for the exact options, because these details change as new series appear.
Decide whether you need Local SSD at creation time
Local SSD can be added only when the VM is created. You cannot attach it to an existing VM later, so plan before launch.
Check machine series support
Not every machine type supports Local SSD, and some support only certain disk counts. Confirm your chosen series.
Pick the disk count
Choose how many disks to attach. Fewer disks may not give enough capacity or performance, while more cost more.
Choose the interface
Select NVMe where possible for best performance, and confirm that your operating system image supports it.
Confirm zonal availability and quota
Local SSD capacity is counted against regional quotas and is not offered identically in all zones.
Hard limits worth memorizing
| Constraint | Meaning | Consequence for design |
|---|---|---|
| Fixed at VM creation | Cannot add or remove disks later | Decide capacity up front or recreate the VM |
| Fixed disk size on most types | Capacity comes in whole disk units | Size by rounding up to whole disks |
| Maximum per VM | Limited by machine series and vCPUs | Larger datasets may need more VMs |
| Not bootable | Cannot hold the operating system | Always use a persistent boot disk |
| Not shareable | Belongs to one VM | Share data through the application or network storage |
| No snapshots | Compute Engine snapshots do not cover it | Copy data out through the application when you need backups |
Buying Local SSD is like choosing the number of lockers when you rent a gym membership. You choose once, at signup, you get fixed-size lockers, and you cannot add more mid-month without signing up again.
Launching a VM without Local SSD and then discovering a month later that the application needs it. The only fix is to create a new VM, which can mean a migration project. Decide early.
7From Raw Device to Usable Storage
A fresh Local SSD is an empty block device, and getting it ready is your job.
When a VM with Local SSD first starts, the operating system sees one raw device per disk. There is no file system and no mount point. Before an application can use the space, someone has to prepare it. In production, this preparation is automated so that a replacement VM comes up ready without human help.
Discover the devices
The startup process finds the Local SSD devices. Because device names can differ by interface, discovery should use stable identifiers rather than assuming a fixed name.
Combine disks if needed
If you have several disks and want one large fast volume, combine them with a striping layer such as software RAID or a logical volume manager. Striping spreads data across all drives for combined speed.
Create a file system
Choose a file system suited to the workload. Common Linux choices include ext4 and XFS, both well tested on flash drives.
Mount with sensible options
Mount the file system at a known path. Consider options that enable TRIM support so the drive can keep its write performance over time.
Make the process repeatable
Place the steps in a startup script or configuration management so every new or restarted VM prepares its Local SSD the same way, and handles the case where the drive is empty.
Striping and the redundancy question
Striping several disks into one volume multiplies speed and capacity but also multiplies risk in a specific way: if any one disk is lost, the whole striped volume is lost. For Local SSD this matters less than for ordinary servers, because the data is already considered disposable and the failure of any disk is usually tied to the loss of the whole VM or host anyway. Mirroring would waste half your capacity and does not protect against host loss. The sound strategy is to stripe for speed and protect data by replicating across machines at the application level.
Handling an empty drive on restart
Because data can disappear while the VM comes back, your startup process must distinguish between “this drive has my data” and “this drive is empty and needs formatting.” Reformatting a drive that actually holds data would destroy it, while skipping formatting on an empty device leaves the application with nothing to write to. Robust startup logic checks the device before acting and records what it did.
A hotel housekeeper checks each room before preparing it. If a guest’s luggage is still inside, the room is left alone. If the room is empty, it is cleaned and made up. A good startup script does the same check on every Local SSD.
Prefer mount entries that tolerate a missing or empty device without blocking boot. A VM that hangs at startup because a Local SSD was empty is a self-inflicted outage.
8Designing for Ephemeral Storage: Replication and Recovery
Using Local SSD safely means moving durability out of the disk and into the architecture.
Since Local SSD offers no durability of its own, every safe design adds durability somewhere else. There are three main approaches, and most real systems combine them.
Replicate across machines
The application keeps copies on several VMs, ideally in different zones. Losing one VM’s Local SSD loses one copy, not the data.
Treat it as a cache
The real data lives in durable storage, and Local SSD holds a fast copy that can be rebuilt. Loss costs time, not data.
Checkpoint to durable storage
Long computations write progress periodically to Cloud Storage or a network disk, so a lost VM resumes from the last checkpoint.
Write through to durable storage
Each important write goes to both Local SSD and a durable store. Reads use the fast copy, and recovery uses the durable one.
flowchart LR
APP["Application"] --> W{"Write path"}
W --> L["Local SSD: fast working copy"]
W --> D["Durable store: Cloud Storage or network disk"]
APP --> R{"Read path"}
R --> L
L -. "Lost or empty" .-> REB["Rebuild from durable store"]
D --> REB
REB --> L
Know your recovery time
Every Local SSD design has a recovery time: how long it takes to return to a healthy state after a VM loses its data. Rebuilding a few gigabytes of cache may take seconds. Resynchronizing a replica of several terabytes from its peers may take hours, during which the cluster has less redundancy. Calculate this number, not just the happy-path performance, because it determines how much risk you actually carry.
| Data size per VM | Rebuild source | Rough concern |
|---|---|---|
| Small cache | Origin database | Brief slowdown while the cache refills |
| Medium working set | Cloud Storage | Minutes of warm-up, depends on bandwidth |
| Large replica | Peer nodes in the cluster | Hours of resync, reduced redundancy meanwhile |
| Computation state | Last checkpoint | Lost progress equals time since last checkpoint |
Putting all replicas of a database in one zone. Replication across machines only protects you if a single failure cannot remove every copy at once.
9Local SSD Compared with Other Storage Options
Choosing storage is about matching speed, durability and flexibility to the job.
Google Cloud offers several storage categories, and Local SSD is only one of them. The right choice follows from how your application treats data, not from which option sounds fastest.
| Option | Location | Durability | Best for |
|---|---|---|---|
| Local SSD | Attached to the host | Ephemeral, no replication | Caches, scratch space, hot tiers of replicated systems |
| Persistent Disk | Network-attached | Replicated, survives VM stops | Boot disks and general purpose block storage |
| Hyperdisk | Network-attached | Replicated, tunable performance | Demanding databases needing configurable IOPS and throughput |
| Cloud Storage | Object storage service | Highly durable | Large files, backups, data lakes |
| Filestore or parallel file services | Network file systems | Managed and durable | Shared file access across many VMs |
| Memory | Inside the VM | Lost on any restart | Fastest possible cache for small hot data |
Where Local SSD sits in the speed ladder
Memory is faster than Local SSD, and Local SSD is generally faster than network disks in latency terms. But memory is small and expensive per gigabyte, while Local SSD offers far more capacity at a much lower price per gigabyte than memory. That makes Local SSD the natural overflow tier: bigger than memory, faster than network disks, cheaper per gigabyte than either memory or high-performance network volumes for the speed it gives.
Choose Local SSD When
- You need very low latency and high IOPS or throughput
- Data can be rebuilt or is replicated elsewhere
- The data is temporary scratch or cache
- You want strong performance per dollar
Avoid Local SSD When
- It would hold the only copy of important data
- You need snapshots or easy backups
- You must move the disk between VMs
- You need to resize storage without recreating the VM
Memory is the notes in your head, Local SSD is the notebook on your desk, Persistent Disk is the filing cabinet in the next room, and Cloud Storage is the archive warehouse across town. You use each at the speed you need and for the importance of what you store.
Many strong designs use two tiers at once: a durable network disk for the system of record, and Local SSD for the hot working set. Combining tiers gives both safety and speed.
10Databases and Stateful Data Systems
Distributed databases are the classic home of Local SSD, because they bring their own replication.
Systems such as distributed wide-column stores, search engines, analytical databases and key-value stores are designed to run on many machines and to keep several copies of each piece of data. They tolerate the loss of a machine by design. That makes them a natural match for ephemeral, very fast storage: each node uses Local SSD for speed, and the cluster itself provides durability.
How the cluster protects the data
In a typical replicated database, each record is stored on several nodes, often placed in different zones. When a node loses its Local SSD, the database notices that some data now has fewer copies than required and begins copying data from healthy nodes to restore the target number. The node returns to service once its share is rebuilt. The application sees nothing, as long as enough replicas survive.
Replication factor of at least three
Enough copies that losing one or two nodes does not lose data or block writes.
Spread across zones
Place replicas in different zones so a single zonal event cannot remove every copy.
Limit simultaneous node loss
Roll maintenance and updates slowly so the cluster is never missing too many replicas at once.
Separate commit logs if required
Some designs keep write-ahead logs on durable disks and data files on Local SSD. Follow the vendor’s guidance.
Maintain external backups
Regularly export data to Cloud Storage so a cluster-wide failure or human mistake can still be recovered from.
Single-node relational databases
A single relational database server that holds the only copy of important data is a poor fit for Local SSD, unless a replica or streaming backup protects the data. Some teams do run a primary on Local SSD for speed, with synchronous or asynchronous replicas on separate VMs and continuous archiving of the transaction log to durable storage. This can work, but it demands rigorous testing of the failover and recovery procedure, and it accepts a larger operational burden than simply using a fast network disk.
A team of librarians, each holding a copy of every important book, can lose one librarian’s shelf without losing any book, because the others still have it. A replicated database is that team of librarians, and Local SSD is each librarian’s fast personal shelf.
Letting automation restart or update all nodes of a database cluster at the same time. Even with replication, simultaneous loss of every node’s Local SSD erases the entire dataset.
11Analytics, Caching and Scratch Workloads
Whenever a job creates large temporary data, Local SSD is a natural fit.
Data processing frameworks often write intermediate results to disk, for example when sorting or when moving data between stages of a computation. These intermediate files are needed only until the job completes, and they can be recreated if lost by rerunning part of the job. That description matches Local SSD perfectly: huge volumes, short lives, and a need for speed.
Typical scratch uses
Shuffle and spill space for data processing
When a distributed job redistributes data between stages, it writes and reads large temporary files. Fast local storage shortens these steps, which are often the slowest part of the job.
Sorting and merging large datasets
Sorting data too big for memory requires writing sorted chunks to disk and merging them. Local SSD makes this far faster than using network disks.
Application and web caches
Content caches, thumbnail caches and compiled-asset caches benefit from fast local reads, and any lost entry is simply fetched again from the origin.
Build and continuous integration workspaces
Compiling large software projects creates many small temporary files. Local SSD speeds up builds, and the workspace is disposable by nature.
Staging input data for repeated reads
When a job reads the same dataset many times, copying it once from Cloud Storage to Local SSD makes every later pass fast, and the copy can be fetched again if lost.
The cache warm-up cost
A cache on Local SSD starts empty after any data-loss event. During the warm-up period, requests fall through to the origin, which can briefly overload it. If many cache nodes restart at the same time, the origin may see a sudden flood. Plan for this by staggering restarts, rate limiting fallbacks to the origin, and, for important caches, pre-loading hot data when a node starts.
A shop that stocks its front shelf from a warehouse is fine when one shelf runs empty. If every shelf in every branch empties at once, the warehouse gets overwhelmed with refill requests. Staggering the refills keeps the warehouse calm.
Measure the cache hit rate and the origin load after a simulated cold start. If the origin cannot survive that moment, the cache design needs rate limiting or warm-up logic before it goes to production.
12AI, Machine Learning, Media and High-Performance Computing
Data-hungry workloads often starve on slow storage, and Local SSD feeds them quickly.
Machine learning training reads huge datasets repeatedly. If the storage cannot deliver data fast enough, expensive accelerators sit idle waiting for the next batch. By staging the dataset onto Local SSD, training jobs read at local speeds while the permanent copy remains safely in Cloud Storage. The same idea helps media transcoding, scientific simulation, genomics pipelines and other jobs that process large files.
Keep the master data in durable storage
The authoritative dataset lives in Cloud Storage or another durable service, never only on Local SSD.
Stage a working copy at job start
The job copies the portion it needs to Local SSD before computation begins, ideally in parallel to use available bandwidth.
Compute against local data
Training or processing reads from the fast local copy, keeping accelerators and processors busy.
Checkpoint results to durable storage
Model checkpoints, final outputs and important intermediate results are written back to durable storage on a regular schedule.
Discard the scratch
When the job ends, the Local SSD contents are no longer needed and disappear with the VM, with no cleanup required.
Why staging beats streaming for repeated reads
Streaming data from object storage is convenient and scales well, but each pass over the data pays network and request overhead. If a job makes many passes, copying once to Local SSD and then reading locally can be much faster overall. If a job reads each item only once, streaming may be simpler and just as fast. Measure both approaches for your workload.
Accelerator machines and bundled storage
Some accelerator-equipped and high-performance machine series include Local SSD as part of the machine configuration, because their workloads are so data-hungry. If you use such a series, check what storage is included, how it is presented to the operating system, and how its data behaves when the VM stops, since these details depend on the series.
A baker who keeps flour in a warehouse across town will waste time if every loaf needs a trip. A smart baker carries a day’s flour to the kitchen in the morning and returns finished bread to the warehouse in the evening. Staging data on Local SSD is the same habit.
Writing final results only to Local SSD “until the end of the job.” If the VM is lost or preempted at ninety percent complete, all the output vanishes. Write results to durable storage as you go.
13Containers and Google Kubernetes Engine
Kubernetes treats Local SSD as node-local storage, which shapes how pods may use it.
In Google Kubernetes Engine, you can attach Local SSD to the nodes in a node pool. Pods can then use that fast storage in one of two broad ways. The first is as ephemeral storage, where the space backs temporary directories and container scratch areas. The second is as raw block storage that a workload or an add-on manages directly. Which mechanisms are available and how they are configured evolves with GKE releases, so follow the current documentation for the exact options.
Ephemeral storage backing
The node’s Local SSD provides the space for temporary volumes and container writable layers. Simple, and good for caches and scratch data.
Raw block devices
The disks are exposed to a workload as block devices it formats and manages itself. Suited to databases that want direct control.
Node-local persistent volumes
A local volume provisioner can present each disk as a volume bound to one node. The pod is scheduled where its data lives.
The scheduling consequence
Data on a node’s Local SSD exists only on that node. If a pod using node-local data is rescheduled to a different node, it will not find its data. That is acceptable for caches and scratch space, but it must be handled explicitly for stateful workloads. Teams solve it by pairing node-local volumes with application-level replication, by pinning pods to nodes, and by accepting that node loss means rebuilding from peers.
flowchart LR
P["Pod using node-local volume"] --> N1["Node A with Local SSD"]
N1 --> D1["Data lives only on Node A"]
N1 -. "Node lost or replaced" .-> X["Data gone"]
X --> RB["Application rebuilds from replicas"]
P2["Pod rescheduled"] --> N2["Node B with its own Local SSD"]
RB --> N2
Node upgrades and autoscaling
Cluster operations that replace nodes, such as upgrades and automatic repairs, create new nodes with empty Local SSD. Cluster autoscaling adds nodes with empty drives and removes nodes along with their data. Any design that keeps important state on node-local storage must tolerate these routine events, using surge settings, careful ordering and replication so that the cluster never loses too many replicas at once.
Enabling aggressive automatic node upgrades for a node pool that stores unreplicated state on Local SSD. Every upgrade then silently erases data. Match the upgrade strategy to the data’s durability model.
14Spot VMs, Instance Groups and Automation
Automation platforms recreate VMs freely, so Local SSD design must assume constant replacement.
Spot VMs and Local SSD
Spot VMs offer large discounts in exchange for the possibility of being stopped by Google when capacity is needed. Local SSD can be used with Spot VMs, and it is sometimes discounted as well. The combination suits fault-tolerant batch work well, because the job is already built to survive losing machines. Since preemption erases Local SSD data, treat Spot plus Local SSD as strictly scratch storage with checkpoints in durable storage.
Managed instance groups
A managed instance group creates VMs from an instance template, and the template can include Local SSD. When the group heals a failed VM or scales out, the new VM arrives with a new, empty Local SSD. Even stateful instance group features, which preserve certain disks and settings across recreation, do not carry the contents of a Local SSD with them. The group is therefore a good fit for stateless caches and scratch-heavy workers, and a risky fit for anything that needs its Local SSD data to survive recreation.
| Scenario | Local SSD data after the event | Design response |
|---|---|---|
| Instance group auto-heals a VM | New VM, empty drive | Rebuild from durable source or peers |
| Autoscaler adds a VM | New VM, empty drive | Warm up before sending full traffic |
| Autoscaler removes a VM | Drive erased with VM | Drain and replicate or flush before removal |
| Spot VM preempted | Lost | Checkpoint to durable storage |
| Rolling update replaces VMs | New VMs with empty drives | Update in small batches, wait for resync |
Infrastructure as code and startup automation
Because VMs with Local SSD are replaced regularly, the storage preparation must be automated and idempotent. Idempotent means that running the process twice has the same result as running it once. A good startup process detects an already prepared device and leaves it alone, formats a blank device, and reports status through logs so operators can see what happened.
Imagine a restaurant that replaces its kitchen staff every few hours. The head chef needs a written routine so each new cook sets up their station the same way, no matter what state the station was left in. Startup automation is that written routine.
Use health checks that report a VM as ready only after its Local SSD is prepared and warmed up. This prevents the load balancer from sending traffic to a VM whose cache or data is still empty.
15Security and Encryption
Local SSD is encrypted at rest by default, but ephemeral storage still deserves careful security thinking.
Compute Engine encrypts data on Local SSD automatically, without any action from you. The encryption is managed by Google, and customer-managed encryption keys are generally not available for Local SSD the way they are for Persistent Disk, so teams with strict key-control requirements must check whether Local SSD satisfies their policy. When a VM is deleted or its Local SSD is released, the data is made unreadable and the drive is prepared for reuse, so another customer cannot read what you stored.
Encryption at rest
Applied automatically with Google-managed keys. No setup, but also no customer key control for Local SSD in most cases.
Secure erasure on release
Drives are cleared when a VM releases them, protecting your data from the next tenant.
Application-level encryption
For stronger control, encrypt sensitive data inside the application with keys you manage in Cloud KMS.
Access control inside the VM
File permissions and service accounts decide who in the VM can read the data. Treat the mount point like any sensitive directory.
Sensitive data and ephemeral storage
There is a useful security side effect of ephemeral storage: data that disappears with the VM leaves no long-lived copy to leak. Scratch files containing sensitive intermediate data are automatically destroyed when the job ends. That said, you should not rely on disappearance as a deletion policy. If regulations require provable deletion at a specific time, design and document an explicit process.
Compliance questions to ask early
| Question | Why it matters |
|---|---|
| Do policies require customer-managed keys for all storage? | Local SSD may not support them, so you may need application-level encryption |
| Must data be recoverable after a failure? | Local SSD alone cannot satisfy this, so replication or backup is required |
| Are there retention or deletion requirements? | Ephemeral data needs a documented lifecycle rather than assumptions |
| Does the data type allow storage on shared hardware? | Some policies need dedicated hardware options in addition to encryption |
A whiteboard wiped clean at the end of every meeting leaves no trace for the next group, which is good for privacy. But a company cannot claim it “securely archived” the meeting notes just because the whiteboard was wiped. Wiping and archiving are different promises.
Assuming that because Local SSD is temporary, it needs no protection. While the VM runs, the data is fully readable by anyone with access to the VM, so permissions and service account scope still matter.
16The Cost Model
Local SSD is priced by capacity and tends to give excellent performance per dollar, but it is not free of traps.
Local SSD is billed by the amount of capacity provisioned, measured in gibibytes over time, as part of the VM’s resources. Compared with network disks, you do not buy separate performance tiers or provisioned IOPS: performance comes with the hardware. Spot VMs can receive discounted Local SSD pricing, and committed use arrangements may reduce the cost of steady, long-term usage. Prices and discount programs change, so check the official pricing page for current numbers and conditions.
Performance per dollar
For workloads that need very high IOPS or throughput, Local SSD is often far cheaper than buying equivalent performance from network volumes, because you pay for capacity and receive speed as a built-in bonus. The comparison reverses when you need little speed or when the data must be durable, since you would then pay for replication and backup elsewhere on top of the Local SSD.
Hidden cost drivers
Oversized VMs
Some disk counts are only available with larger VM sizes, which can raise compute cost more than the storage itself.
Redundant copies
Replication across machines multiplies the Local SSD you need. Three replicas mean three times the disks.
Durable backing stores
Backups, checkpoints and origin databases add their own storage and network charges.
Idle VMs holding disks
A running VM with Local SSD costs money even when idle. Remove unused machines promptly.
Rebuild traffic
Resynchronizing a replaced node moves a lot of data across the network, which can add transfer cost across zones.
Renting a fast sports car by the day can be cheaper per mile than hiring a chauffeur service for the same speed. But if you must also pay for a backup car in every city in case the first breaks down, the total cost rises. Replication is the backup car.
Compare total cost of the full design, including replicas, checkpoints and rebuild traffic, against a design using high-performance network volumes. Compare designs, not price tags per gigabyte.
17Monitoring and Operations
Ephemeral storage needs watching, because failures arrive silently as empty drives.
Operating Local SSD well means observing three layers: the health of the storage devices, the behavior of the application on top of them, and the state of the cluster that protects the data. Because the storage hides no durability machinery, the monitoring must watch for the symptoms of data loss and recovery.
Device utilization and latency
Guest-level metrics showing how busy the drives are and how long requests take. Collected from inside the VM with a monitoring agent.
Free space
Filling a Local SSD to the brim slows writes and can crash applications. Alert well before full.
Replication and rebuild status
For clustered systems, the number of under-replicated shards and the rebuild progress are the real durability indicators.
Cache hit rate and origin load
A sudden drop in hit rate often means a node lost its data and is refilling.
VM lifecycle events
Stops, restarts, preemptions and live migrations recorded in logs explain why data appeared or disappeared.
Alerts that matter
Alert on rising numbers of under-replicated data, because that is the moment your cluster is one failure away from losing data. Alert on sudden cache hit rate drops and on origin load spikes. Alert on repeated VM restarts, which may indicate a startup script problem reformatting drives. Alert on disk space above a safe threshold, and on rebuild operations that run unusually long.
Game days and failure drills
The best way to trust a Local SSD design is to break it on purpose. Stop a VM, delete one, simulate a preemption, and watch how the system recovers. Record how long it takes and whether any user-visible errors appear. Do this in a safe environment first, then regularly in production-like conditions, because systems drift and assumptions go stale.
A fire drill is uncomfortable but it is how you learn that an exit door is locked before there is a fire. A failure drill is the same: discover the weak point on a quiet afternoon, not in the middle of an incident.
Monitoring only CPU and memory. Storage-related trouble then shows up as slow responses or missing data with no clear cause. Add storage-specific and replication-specific signals from the start.
18Design Patterns and Anti-Patterns
A few reliable patterns make Local SSD safe, and a few predictable traps make it dangerous.
Patterns that work well
Replicated hot tier
A distributed database keeps its working data on Local SSD in every node, with several replicas spread across zones. The cluster rebuilds any lost node from its peers, and external backups protect against catastrophic loss.
Read-through cache
Local SSD holds a large cache in front of a durable origin. A miss fetches from the origin and fills the cache, and a lost cache costs only warm-up time.
Stage, compute, checkpoint
Batch jobs copy input from durable storage to Local SSD, compute against the fast copy, and write progress and results back to durable storage at intervals.
Two-tier storage on one VM
A durable network disk holds the system of record or the transaction log, while Local SSD holds indexes and hot data files that can be rebuilt from it.
Disposable worker fleet
A fleet of identical workers uses Local SSD for scratch and is freely replaced by automation. Nothing important lives on any individual worker.
Anti-patterns to avoid
Problem
The only copy of important data is stored on Local SSD with no replication or backup.
Why It’s Harmful
A VM stop, preemption or host failure permanently destroys the data, and Compute Engine cannot recover it.
Correct Approach
Keep an authoritative copy in durable storage, or replicate across machines and zones, and treat Local SSD as a performance layer only.
Problem
All replicas of a clustered system run in one zone, or are restarted together by automation.
Why It’s Harmful
One zonal problem or one careless rollout removes every copy at once, defeating the replication.
Correct Approach
Spread replicas across zones, and roll updates in small batches, waiting for resynchronization between batches.
Problem
The startup script blindly formats the Local SSD every time the VM boots.
Why It’s Harmful
After a reboot or live migration, where data is meant to survive, the script wipes it, turning a harmless event into data loss.
Correct Approach
Detect whether the device already holds a file system and data. Format only blank devices, and log every decision.
Problem
A tiny VM is given many Local SSD disks, or a huge VM is chosen only to unlock disk counts that are never used.
Why It’s Harmful
A small VM cannot drive the disks at full speed, and a needlessly large VM wastes compute money.
Correct Approach
Size the VM and disk count together, benchmark the real workload, and choose the cheapest combination that meets the target.
19Best Practices and Common Mistakes
A short checklist separates calm Local SSD operations from painful ones.
Best practices
State the data policy
For every dataset, write down how it is rebuilt and how long that takes before placing it on Local SSD.
Keep a durable system of record
Authoritative data lives in a replicated or durable service, with Local SSD as a fast layer.
Prefer NVMe when possible
It gives the best performance, provided your image and tools support it.
Automate preparation
Use idempotent startup logic that formats blank devices only and records what it did.
Spread replicas across zones
Let no single failure remove every copy of a dataset.
Roll changes slowly
Update and replace VMs in small batches, waiting for resync before continuing.
Benchmark realistically
Test with the real access pattern, the intended VM size and the full disk count.
Drill the failures
Stop, delete and preempt VMs on purpose to verify recovery time and correctness.
Common mistakes
Believing a reboot test proves durability. Reboots preserve data, but stops, preemptions and host failures do not. Test those cases.
Forgetting that Local SSD must be chosen at VM creation. Retrofitting later means building and migrating to new VMs.
Expecting snapshots or backups to cover Local SSD automatically. They do not. Build your own export or replication path.
Ignoring cache warm-up and rebuild load. A cold cache can flood the origin, and a resync can saturate the network.
Letting automation treat storage-bearing nodes as interchangeable. Upgrades, repairs and scale-in events erase node-local data and must be coordinated with replication.
After any data-loss incident, add the cause to a team runbook and a game-day scenario, so every lesson becomes a rehearsed drill rather than a memory.
20Real-World and Industry Scenarios
Concrete stories show how the ideas combine under real pressure.
Ad-tech lookup service
A company serving real-time bidding needs sub-millisecond lookups across a very large key-value dataset. It runs a replicated key-value store with data on Local SSD in every node, replicas in three zones, and a nightly export to Cloud Storage. Node failures trigger automatic resync, and users never notice.
Search platform hot tier
A search team keeps recent, frequently queried indexes on Local SSD nodes for speed, and older indexes on cheaper network-backed nodes. Indexes are replicated, and snapshots are written regularly to object storage for disaster recovery.
Genomics pipeline
A research lab processes large sequencing files. Each job copies the input to Local SSD, runs several passes of analysis, and uploads results and intermediate checkpoints to durable storage. Jobs run on discounted Spot VMs, and interrupted jobs resume from their latest checkpoint.
Video transcoding farm
A media company converts uploaded videos to many formats. Workers pull source files into Local SSD, encode at high speed, and push finished files to object storage. The fleet scales up and down with the queue, and nothing important lives on any worker.
Continuous integration build farm
An engineering organization runs thousands of builds each day. Build agents use Local SSD for source checkouts and compiler caches, cutting build times. Agents are replaced often, and artifacts are uploaded to a durable repository.
Machine learning training cluster
A team trains image models on a large dataset. The dataset is staged from Cloud Storage to Local SSD at job start so accelerators are fed quickly, and model checkpoints are written back to Cloud Storage every few minutes.
In every story, the permanent copy of the data lives somewhere durable or is replicated across machines, and Local SSD simply makes the working set fast.
21Frequently Asked Questions
Short answers to the questions engineers ask most when evaluating Local SSD.
Yes, a guest operating system reboot normally preserves the data, as does live migration during host maintenance. Stops, deletions, preemptions and host failures generally do not preserve it, so verify the current rules for your machine series.
No. Local SSD is chosen when the VM is created. To get it later, create a new VM with Local SSD and move your workload to it.
No. Compute Engine snapshots and machine images do not capture Local SSD contents. Copy important data out through your application, to Cloud Storage or to a durable disk.
Generally it offers lower latency and very high IOPS and throughput because it is attached to the host, but high-end network volumes also offer strong performance with durability. Compare against published figures for your exact machine and volume types.
No. The boot disk must be a persistent volume, because the operating system must survive stops and restarts.
Often yes, for a single larger and faster volume, since the data is disposable and durability comes from elsewhere. Remember that losing any one disk loses the whole striped volume, so rely on replication or rebuild at the application level.
Yes, it is encrypted at rest automatically with Google-managed keys. Customer-managed encryption keys are generally not supported for Local SSD, so use application-level encryption if you need your own key control.
Yes, but data is lost on preemption and on recreation of a VM by the group. Use it for scratch, caches and replicated systems, with checkpoints in durable storage.
Common causes are using SCSI instead of NVMe, too few vCPUs to drive the disks, too little parallelism in the application, a nearly full drive, or a workload pattern that differs from the benchmark. Test with your real access pattern.
Ask what happens if every byte on it disappears right now. If the answer is a short, planned rebuild, Local SSD is a good fit. If the answer is a crisis, put the data somewhere durable.
22Summary and Key Takeaways
Local SSD offers a clear bargain: storage physically attached to the host gives very low latency and very high throughput at a good price, in exchange for giving up durability. Its data lives and dies with the VM and its host. It survives reboots and live migrations, but not stops, deletions, preemptions or host failures, and it cannot be snapshotted, shared, resized or used as a boot disk.
The skill in using it is architectural. Place the authoritative copy of your data in durable storage or replicate it across machines and zones, automate the preparation of each fresh drive, plan for cache warm-up and rebuild traffic, and test failures on purpose. Used for caches, scratch space, hot tiers of replicated databases and staged training data, Local SSD makes systems dramatically faster. Used as the only home of precious data, it eventually causes an outage.
Key Takeaways
- Fast because it is local — the drive sits on the host, so there is no network hop, which yields low latency and high IOPS and throughput.
- Ephemeral by design — data survives reboots and live migration but is generally lost on stop, delete, preemption and host failure, so never keep the only copy there.
- Fixed and limited — chosen at VM creation, fixed disk sizes on most machine types, not bootable, not shareable and not covered by snapshots.
- Performance needs the right setup — prefer NVMe, provide enough vCPUs, generate parallel requests, and combine disks with striping when you need one large fast volume.
- Durability belongs in the architecture — replicate across zones, treat Local SSD as a rebuildable cache, or checkpoint to durable storage, and know your recovery time.
- Automate and roll carefully — use idempotent startup logic, avoid reformatting drives that hold data, and update nodes in small batches in clusters and Kubernetes.
- Judge cost by the whole design — count replicas, backups, rebuild traffic and VM sizing, and compare complete designs against network-volume alternatives.