Cloud Spanner in Google Cloud

Cloud Spanner in Google Cloud

A plain-English guide to Google's globally distributed relational database. Learn how it keeps data correct across continents, how it scales without sharding headaches, how to design for it, and how to avoid paying for more than you need.

Picture a bank with branches in Mumbai, London and New York. A customer in Mumbai sends money to a friend in New York. Every branch must agree, at every moment, on exactly how much money each person has. If any branch shows an old balance, someone could spend the same money twice. For decades, running such a bank on a normal database meant choosing between two painful options: keep everything in one place and accept that distant customers wait, or spread the data around and accept that branches might briefly disagree.

Cloud Spanner is Google’s answer to that dilemma. It is a database that spreads your data across many machines, and even many continents, while still behaving like one single, correct, always-consistent database. You talk to it with ordinary SQL, the language most developers already know. Behind the scenes, Google handles copying, splitting, failing over and scaling. In this guide you will learn what Cloud Spanner is, how it works inside, how to design tables for it, how to keep it fast and safe, and what it costs.

Every new word is explained the first time it appears. You do not need to know anything about distributed systems before you begin. By the end, you will be able to explain Spanner to a teammate, decide when it is the right tool, and avoid the design mistakes that trip up most newcomers.

AWhat Is Cloud Spanner?

Start with the big picture: what Spanner is, and why people call it special.

Cloud Spanner is a fully managed, relational database service from Google Cloud. A relational database stores information in tables made of rows and columns, and lets you connect tables together, for example linking customers to their orders. It supports SQL, the standard language for asking questions of such databases, and it supports transactions, which are groups of changes that either all succeed or all fail.

What makes Spanner unusual is that it is also horizontally scalable and globally distributed. Horizontal scaling means you handle more load by adding more machines, rather than buying one bigger machine. Global distribution means copies of your data can live in different regions of the world. Most databases give you one or the other: either strong relational guarantees on a single machine, or easy scaling with weaker guarantees. Spanner aims to give you both together.

Analogy

Think of a very large library with branches in many cities. A normal database is one giant library building: everything is correct, but everyone must travel there. A typical scalable database is many small libraries that sometimes disagree about which books they hold. Spanner is a worldwide library network with a perfect shared catalogue, where every branch always knows exactly which book is checked out, no matter where it was borrowed.

Where it came from

Spanner began inside Google. Google needed a database for its advertising business that could grow enormously, survive the loss of entire data centres, and still keep money-related data perfectly correct. Existing systems could not do all three. Google’s engineers built Spanner to meet that need, described it in a well-known research paper in 2012, and later offered it to the public as Cloud Spanner. The fact that it was born to run some of Google’s most important systems is a big reason why many companies trust it.

SQL
the familiar query language
ACID
transactions that stay correct
Global
scales across machines and regions

What you give and what you get

You create an instance, choose where it lives and how much computing power it gets, create a database inside it, and define your tables. In return, Google takes care of replicating every piece of data to several places, splitting large tables across machines, moving load around, patching software, taking backups on request and replacing failed hardware. You never log in to a database server, because there is no server for you to manage.

i
Good to know

Spanner offers two flavours of SQL: Google’s own GoogleSQL dialect, and a PostgreSQL-compatible dialect. This lets teams that already use PostgreSQL carry over many of their skills and tools. Check the official documentation for exactly which PostgreSQL features are supported.

BThe Problem Spanner Solves

To understand any tool, first feel the pain it was built to remove.

Imagine you start a small online shop with a standard relational database on one machine. It works beautifully. Then the shop becomes popular. Orders flood in, the machine slows down, and you buy a bigger machine. This approach is called vertical scaling, and it works until you reach the biggest machine money can buy. After that, there is nowhere to go.

The usual next step is sharding, which means splitting your data into pieces and putting each piece on a different database machine. For example, customers whose names start with A to M go on one machine, and N to Z on another. Sharding works, but it brings serious pain. Your application must know which shard holds which data. A change that touches two shards cannot easily be made safely. Reports that span all shards need special tools. Adding a new shard means moving data around while the shop is running. Many companies have spent years of engineering effort building and maintaining their own sharding systems.

Analogy

Sharding by hand is like a school that runs out of space and splits students across several buildings. Now every teacher must remember which building each student sits in, and a school-wide exam needs messengers running between buildings. Spanner is a school that quietly adds classrooms as needed, while the head office keeps one perfect register that every teacher sees.

The classic trade-off

Some modern databases, often called NoSQL databases, solved the scaling problem by giving up some features. They spread data across many machines easily, but they usually drop joins, flexible SQL queries or strong guarantees about transactions. Teams then had to write that missing logic inside their applications, which is hard and error-prone. For data like game scores or page views, that is often acceptable. For money, inventory and bookings, it is dangerous.

Scale

One machine is not enough

A single database server has a ceiling. Growth beyond that ceiling needs a different design.

Sharding

Manual sharding is painful

Splitting data yourself burdens the application, complicates changes and makes cross-shard work risky.

Correctness

NoSQL gives up guarantees

Weaker consistency and missing transactions push complexity and bugs into your code.

Availability

Disasters are real

A single data centre can fail. Serious systems need copies in more than one place, always in agreement.

Spanner removes the false choice. It lets you keep SQL, joins and strong transactions while scaling out across machines automatically, and it keeps replicas in several places that always agree. You pay for this with a different cost model and a design style that you must learn, which later chapters explain.

?
The trade-off

Spanner is not the cheapest or simplest option for a small app. Its strengths appear when you need very high scale, very high availability, or strong consistency across regions. A later chapter helps you decide whether you really need it.

CCore Concepts You Must Know

A handful of words explain almost everything in Spanner. Learn them well and the rest becomes easy.

Instance

An instance is the basic unit you create and pay for. It defines where your data lives, called the instance configuration, and how much computing power you have, called the compute capacity. A single instance can hold many databases. Think of an instance as a rented office floor, with databases as the separate companies working on it.

Database, table, row and column

A database lives inside an instance and holds your tables. A table is a grid of information. Each row is one record, such as one customer, and each column is one fact about it, such as an email address. Every table must have a primary key, which is a column, or a group of columns, that uniquely identifies each row. In Spanner, the primary key matters far more than in a single-machine database, because it also decides where the row is stored.

Analogy

A primary key is like a house address. It identifies exactly one house, and it also tells the postman where in the city to go. In Spanner, rows with nearby keys are stored near each other, just as houses with nearby numbers are on the same street.

Node and processing unit

Compute capacity is measured in nodes or in smaller pieces called processing units. A node is a bundle of computing and serving capacity. One node equals one thousand processing units, so you can start small with a few hundred processing units and grow to many nodes. More capacity means more reads, writes and storage that the instance can handle.

Replica and leader

Spanner keeps several copies of your data, called replicas, in different zones or regions. For each piece of data, one replica acts as the leader. The leader coordinates writes and makes sure all copies agree. If the leader’s location fails, another replica takes over automatically.

Split

Spanner automatically cuts each table into pieces called splits, based on ranges of the primary key. Different splits can live on different machines. When one split gets too big or too busy, Spanner divides it into two and may move one part to another machine. This is how it scales without you sharding by hand.

Unit

Instance

What you create and pay for. Defines location and compute capacity.

Storage

Database and tables

Where your rows live. Each table has a primary key that also controls placement.

Power

Nodes and processing units

The measure of compute capacity. One node is one thousand processing units.

Safety

Replicas

Multiple copies in different places so no single failure loses data.

Order

Leader

The replica that coordinates writes for a piece of data.

Scale

Splits

Key ranges that Spanner divides and moves automatically as data and load grow.

Transactions and ACID

A transaction groups several steps into one all-or-nothing action. Transferring money involves subtracting from one account and adding to another. If the machine dies between the two steps, you must not lose the money. Databases promise ACID behaviour to guard against this. The letters stand for atomic, meaning all-or-nothing; consistent, meaning rules stay true; isolated, meaning simultaneous transactions do not trample each other; and durable, meaning once confirmed, the result is never lost. Spanner offers full ACID transactions even across machines and regions.

i
Remember this chain

Instance holds databases, databases hold tables, tables are cut into splits, splits are copied as replicas, and each split has one leader. Almost every Spanner idea fits somewhere on this chain.

DInstance Configurations and Editions

The first big decision is where your data lives and how many copies you keep. This choice sets your availability, speed and price.

When you create an instance, you pick an instance configuration. This is a template that says where your replicas will be placed. The three main families are regional, dual-region and multi-region. Names and options evolve, so confirm the current list in the official documentation before you decide.

Regional

Regional configuration

Replicas sit in several zones inside one region, such as Mumbai. It protects against a single zone failure, offers low latency for nearby users and costs the least of the three.

Dual-region

Dual-region configuration

Replicas sit in two regions within one country or area. It adds protection against the loss of a whole region while keeping data inside a chosen area, which helps with data residency rules.

Multi-region

Multi-region configuration

Replicas sit across several regions, often on one continent or across continents. It gives the highest availability and lets users in several places read nearby, at a higher price and with slower writes.

Analogy

A regional configuration is keeping copies of a contract in several offices of one building. A dual-region setup is keeping copies in two different cities. A multi-region setup is keeping copies in cities on different continents. More distance gives stronger protection from disasters, but it takes longer to send a message between the offices.

Types of replica

Not every copy does the same job. Spanner uses three kinds of replicas, and understanding them explains how the configurations differ.

Replica typeStores dataVotes on writesCan become leaderPurpose
Read-writeYesYesYesThe main copies that serve reads and writes
Read-onlyYesNoNoServes fast reads near users without slowing writes
WitnessNo, only the logYesNoHelps form a majority cheaply in multi-region setups

The leader region

In configurations with more than one region, writes are coordinated by the leader, which lives in a chosen leader region. Writes from far away must travel to the leader and back, so placing the leader near most of your writers reduces delay. You can set a preferred leader region for a database, so the leaders sit where your writing traffic is heaviest.

Editions

Spanner is also sold in editions, such as Standard, Enterprise and Enterprise Plus. Each edition includes a different set of features, with the higher editions adding capabilities such as more advanced data-handling options, extra availability choices and stronger operational features. The exact feature split changes over time, so compare editions on the official pricing and feature pages before you choose. A sensible approach is to start with the lowest edition that has everything your application needs.

!
Watch out

You cannot freely move an existing instance between every configuration. Changing from one configuration to another has rules and may need a special process. Think carefully about location and availability at the start, as changing later is harder than choosing well now.

EArchitecture and Components

Let us look at the main pieces that sit between your application and the stored rows.

When your program sends a query, a chain of components handles it. Knowing this chain helps you understand why some designs are fast and others are slow. The diagram below shows the parts in a typical regional instance.

flowchart LR
  APP["Your Application"] --> CL["Spanner Client Library"]
  CL --> FE["Spanner API Front End"]
  FE --> SRV["Spanner Servers: Compute Nodes"]
  SRV --> SP1["Split A: rows 1 to 1000"]
  SRV --> SP2["Split B: rows 1001 to 2000"]
  SRV --> SP3["Split C: rows 2001 to 3000"]
  SP1 --> Z1[("Replica in Zone 1")]
  SP1 --> Z2[("Replica in Zone 2")]
  SP1 --> Z3[("Replica in Zone 3")]
  Z1 --> FS[("Distributed File System")]
  Z2 --> FS
  Z3 --> FS
    
Figure 1: The path from an application through Spanner’s servers, splits and replicas down to durable storage.

Walking through the diagram

Your application talks to Spanner through a client library, a ready-made piece of code for languages such as Java, Python, Go and others. The library manages connections for you, using a pool of sessions, which are reusable conversations with the database. Requests go to the Spanner front end, which checks who you are and routes the request.

The request then reaches the Spanner servers, which are the compute nodes you pay for. These servers are responsible for the splits. Each split is a range of rows, and each split is copied to several replicas in different zones. The replicas keep their data on Google’s distributed file system, which itself stores data redundantly. This separation between compute and storage is an important idea. It means Spanner can add or remove compute nodes quickly without copying huge amounts of data, because the data stays in the shared storage layer.

Analogy

Imagine a large warehouse with many counters. The counters are the compute nodes, where customers are served. The shelves are the splits, each holding a range of products. The warehouse floor with all the stock is the distributed file system. If the shop gets busier, you open more counters without moving any stock, because every counter can reach the shelves.

Key components in plain words

Client

Client library

Handles sessions, retries and the protocol, so your code talks in simple calls.

Front end

API front end

Authenticates requests and sends them to the right servers.

Compute

Spanner servers

The nodes that run queries and coordinate transactions. Their number sets your capacity.

Data

Splits

Ranges of rows that move and divide automatically as load and size change.

Copies

Replicas

Redundant copies across zones or regions that always agree.

Storage

Distributed file system

The durable layer underneath, which stores the bytes with its own redundancy.

One last idea is worth noting. Because your data is split by key ranges and each range is a unit of movement, the way you choose primary keys shapes how evenly the work spreads across nodes. This single design choice is the biggest influence on whether Spanner feels fast or slow, and a whole later chapter is devoted to it.

FInternal Working: The Life of a Write

Follow one money transfer from your application to safe, agreed storage.

Let us trace what happens when your application saves a change, such as adding a new order. This walk-through shows where the time is spent and why Spanner can promise that data is never lost.

sequenceDiagram
  participant A as Application
  participant L as Split Leader
  participant R1 as Replica 2
  participant R2 as Replica 3
  A->>L: Begin transaction and write order row
  L->>L: Take locks and pick a commit timestamp
  L->>R1: Send change to replicate
  L->>R2: Send change to replicate
  R1-->>L: Acknowledge
  R2-->>L: Acknowledge
  L->>L: Wait out clock uncertainty
  L-->>A: Commit confirmed
    
Figure 2: A single-split write. The leader replicates to a majority and waits briefly before confirming.

Step by step

1

The client starts a transaction

The application sends its change. The client library picks a session and routes the request toward the leader of the split that owns the row.

2

The leader takes locks

The leader locks the affected rows so no other transaction changes them at the same time. This is what gives isolation.

3

The leader chooses a timestamp

Every committed write receives a commit timestamp, a precise point in time. These timestamps put all changes into one agreed order.

4

The change is replicated

The leader sends the change to the other replicas. A majority of replicas must record it before it counts as committed. This is how copies stay in agreement even if one fails.

5

A short wait for clock uncertainty

Before confirming, the leader waits a few milliseconds to be sure its timestamp is truly in the past everywhere. The next chapter explains why.

6

Commit is confirmed

The application receives success. Locks are released and the change becomes visible to later readers.

Majority agreement and Paxos

Spanner uses a well-known method called Paxos to make replicas agree. You do not need to know its details. The key idea is that a change is committed once a majority of the replicas for that split have stored it. With three replicas, two must agree. If one replica fails or is unreachable, the other two still form a majority, and the system keeps working without losing any confirmed data.

Analogy

Think of a committee of three people deciding on a rule. The rule passes when two members sign. If one member is on holiday, the other two can still pass it, and later the third member reads the minutes and catches up. Nobody can claim the rule failed, because two signatures are already on record.

When a write touches more than one split

Sometimes one transaction changes rows that live in different splits, perhaps on different machines or even in different regions. Spanner then uses two-phase commit. In the first phase, the leader of each split promises that it is ready to commit. In the second phase, a coordinator tells everyone to commit together. Because each leader’s promise is itself replicated, the process survives failures. Such transactions cost more time than single-split ones, which is why good designs keep related rows together, as explained in the data model chapter.

!
Cost of distance

Every write must reach a majority of replicas. In a multi-region configuration, that may mean crossing oceans, so write latency is higher than in a regional setup. Reads can often be served nearby, but writes always pay the price of agreement.

GTrueTime and External Consistency

The secret ingredient that lets Spanner order events correctly across the whole planet.

Imagine two people on opposite sides of the world, each saving a change to the same database at nearly the same moment. Which change happened first? To answer, the database needs a clock that everyone can trust. But ordinary computer clocks drift, and two machines never agree perfectly on the time. This is one of the hardest problems in distributed systems.

Google’s solution is TrueTime. Google fits its data centres with very accurate clocks: atomic clocks and GPS receivers. Instead of pretending that time is exact, TrueTime tells software the truth: “the time is now somewhere between this moment and that moment.” It returns a small interval of uncertainty, typically a few milliseconds wide, rather than a single point.

Analogy

Imagine a race where the timekeepers are not sure of the exact finishing second, but they know it was between 10:00:00.003 and 10:00:00.007. To announce results fairly, they wait until the later time has surely passed before declaring the race over. TrueTime does exactly this with every commit.

Commit wait

When Spanner commits a write, it picks a commit timestamp and then waits until that timestamp is definitely in the past according to the uncertainty interval. This pause is called commit wait. It usually lasts only a few milliseconds. In return, Spanner gets a powerful promise: if transaction A finishes before transaction B starts, then A’s timestamp is always smaller than B’s, anywhere in the world.

External consistency

This promise has a name: external consistency, sometimes described as the strongest form of consistency, close to what developers call linearizability for transactions. In plain words, the database behaves as if every transaction happened one at a time, in an order that matches real-world time. If you see a change confirmed to a user, any later read anywhere will see it. Applications do not need to reason about stale copies or surprising orderings.

Clocks

Atomic clocks and GPS

Hardware in Google data centres keeps time very precisely and corrects drift.

Interval

Time as a range

TrueTime reports earliest and latest possible times, admitting its own uncertainty honestly.

Wait

Commit wait

A short pause before confirming, so the timestamp is surely in the past everywhere.

Promise

External consistency

Transactions are ordered the same way real time orders them, across the whole world.

Why this matters to you

Without such ordering, applications need workarounds. A shopper might buy the last item in one country while another shopper sees it in stock in another. A bank might show an old balance after a deposit. With external consistency, these surprises do not occur. You can write application logic in the simple way you would on a single database, and Spanner worries about the planet.

i
The CAP theorem, briefly

The famous CAP theorem says that when a network splits, a distributed system must choose between staying consistent and staying available. Spanner chooses consistency, but it runs on Google’s private, highly reliable network and uses redundancy so well that real-world outages that hurt availability are extremely rare. Google describes this as effectively giving you both, even though in theory a choice is always being made.

Strong reads and stale reads

By default, reads in Spanner are strong, meaning they see all committed changes up to the moment of the read. Strong reads may need to check with the leader, which can add delay when you are far away. If your application can accept slightly old data, for example a product listing that is a few seconds behind, you can ask for a stale read. A stale read says “give me the data as it was ten seconds ago” and can be served by the nearest replica without contacting the leader. This trade of freshness for speed is a favourite tuning tool, and a later chapter returns to it.

HThe Data Model: Keys, Interleaving and Indexes

This is the most important design chapter. Good keys make Spanner fly, and bad keys make it crawl.

In an ordinary database on one machine, the primary key is mostly about identity. In Spanner, the primary key also controls physical placement. Rows are stored in sorted order of their primary key, and Spanner divides that sorted order into splits. Rows with similar keys sit together, and rows with distant keys can live on different machines.

Analogy

Imagine a huge bookshelf sorted by title, cut into sections with a librarian for each section. If all new books arrive with titles starting with Z, one librarian is flooded while the others sit idle. The shelf is fine; the sorting rule caused the traffic jam. The primary key is that sorting rule.

The hotspot problem

A hotspot is a part of the database that receives far more traffic than the rest. It is the number one performance mistake in Spanner. It usually happens when keys always increase, such as an auto-counting number or a current timestamp placed first in the key. Every new row lands at the very end of the sorted order, so one split and one server receive all the writes. Adding more nodes does not help, because only one node owns the end of the table.

flowchart TD
  K{"Does the first part of the key always increase over time?"}
  K -- "Yes" --> BAD["Hotspot risk: all new rows hit one split"]
  K -- "No" --> GOOD["Writes spread across many splits"]
  BAD --> F1["Use a random UUID as the key"]
  BAD --> F2["Use a bit-reversed sequence"]
  BAD --> F3["Add a hash or shard prefix"]
  BAD --> F4["Put a well-spread column such as user ID first"]
  F1 --> GOOD
  F2 --> GOOD
  F3 --> GOOD
  F4 --> GOOD
    
Figure 3: Increasing keys create hotspots, and four common ways to avoid them.

Ways to choose a good key

Random

UUID keys

A universally unique identifier is random, so new rows scatter across the whole key range. Spanner can generate them for you.

Sequence

Bit-reversed sequences

Spanner can produce numbers whose bits are reversed, so values look like a counter but spread evenly instead of climbing in order.

Prefix

Hash or shard prefix

Add a small computed value in front of an ordered key so rows fan out. Queries must then account for the prefix.

Order

Put the spread column first

For a table of events, use user identifier first and timestamp second. Each user’s events stay together, and different users spread across splits.

!
Never do this

Do not use a plain auto-increment number or a timestamp as the first part of a primary key on a table that receives heavy writes. It feels natural from other databases, and it is the classic Spanner mistake. Indexes on timestamp columns can cause the same trouble, so check them as well.

Interleaved tables: keeping family together

Many data sets have parent and child relationships, such as a customer and their orders, or an order and its line items. In Spanner, you can declare a child table as interleaved in its parent. This tells Spanner to store each child row physically next to its parent row, in the same split. When you read a customer and all their orders, the data comes from one place, and when you change both in one transaction, it commits quickly without needing a multi-machine agreement.

Analogy

Interleaving is like storing a family’s documents together in one folder, instead of filing birth certificates, school records and tax papers in three different buildings. Fetching the whole family takes one trip.

Interleaving can also delete children automatically when a parent is deleted, if you choose the cascade option. The child table’s primary key begins with the parent’s key, followed by its own column. Use interleaving when you usually read parent and children together, and when the number of children per parent is reasonable. Avoid it when children are accessed independently of their parents, or when one parent could have an enormous number of children.

Foreign keys

Spanner also supports ordinary foreign keys, which are rules that link one table to another and keep references valid. Unlike interleaving, foreign keys do not place rows together. Use them when relationships are not strictly parent-and-child, or when you want the rule enforced without co-locating data. Interleaving is a performance tool, and foreign keys are a correctness tool, and you can use both.

Secondary indexes

A secondary index lets you find rows quickly by a column other than the primary key, such as searching customers by email. In Spanner, an index is stored like its own small table, sorted by the indexed columns. You can add extra columns to an index with a storing clause, so that a query can be answered entirely from the index without visiting the main table. Every index adds work to each write and uses extra storage, so create only the indexes your queries truly need.

Design choiceHelps withCosts you
Interleaved child tableFast parent and child reads, cheap joint writesLess flexibility for independent access
Foreign keyData correctness across tablesNo co-location benefit, some extra write checks
Secondary indexFast lookups by other columnsExtra storage and slower writes
Storing columns in an indexQueries answered from the index aloneLarger indexes

Changing the schema

Spanner lets you change the schema while the database is running. Adding a column, creating an index or changing certain settings are done as online schema changes, which run in the background without blocking reads and writes. Large changes, such as building an index over a billion rows, can take a while, and you can monitor their progress. Plan changes carefully, release in small steps, and make changes backward compatible so old and new versions of your application can both work during a release.

i
Design rule

Choose keys so that writes spread out, and shape tables so that data you read together is stored together. If you remember only two sentences about Spanner design, remember those two.

ITransactions, Locking and Read Types

How Spanner keeps simultaneous users from stepping on each other, and how to choose the right kind of read.

Spanner offers different transaction styles for different needs. Choosing the right one is both a correctness and a cost decision, because stronger styles use more resources.

Read-write transactions

A read-write transaction is used whenever you change data. It uses locks, which are temporary reservations on rows. While you hold a lock on a row, others who want to change it must wait. By default, Spanner provides the strictest level of isolation, called serializable, which means the result is as if all transactions ran one after another. Some versions also offer a gentler isolation option for workloads that prefer fewer conflicts, so check the current documentation for what is available.

Analogy

A lock is like the occupied sign on a shop’s fitting room. While one customer is inside, the next customer waits outside. The sign prevents two people from walking in at once and keeps everyone’s experience orderly.

Deadlocks and aborts

A deadlock happens when two transactions each hold something the other needs, and both wait forever. Spanner prevents this by choosing one transaction to abort, which means cancelling it. The aborted transaction must be tried again. This is normal and expected, not an error in your system. The official client libraries include helper functions that automatically retry aborted transactions, so write your transaction code inside those helpers and make sure it is safe to run more than once.

Read-only transactions and snapshot reads

A read-only transaction sees one consistent picture of the database at a single moment, even if you run many queries. It takes no locks and does not block writers, so it is cheaper and faster than a read-write transaction. Whenever your code only reads, use read-only mode. This is especially useful for reports that must show matching totals across several queries.

Stale reads

Strong

Strong read

Sees all changes committed before the read. Default and safest, but may need to contact the leader.

Exact

Exact-staleness read

Reads data exactly as it was a set time ago, for example fifteen seconds. Consistent and often cheaper.

Bounded

Bounded-staleness read

Reads data no older than a limit, letting Spanner pick the freshest copy the nearest replica can serve.

Timestamp

Read at a timestamp

Reads data as of a chosen moment. Useful for audits and for debugging what a system looked like earlier.

Stale reads can be served by any replica that is up to date enough, including read-only replicas near the user. This can cut read latency dramatically for global applications. Use them for content such as catalogues, profiles and dashboards, and keep strong reads for things like account balances and stock checks at purchase time.

Writing efficiently

Spanner gives you two ways to change data. Mutations are direct instructions such as “insert this row” or “update that row”, sent together in one commit. DML, short for data manipulation language, uses SQL statements such as insert, update and delete. Both are transactional. Mutations are often more efficient for plain inserts and updates, while DML shines when the change depends on a query. For huge changes, such as deleting millions of old rows, partitioned DML splits the work into many independent pieces that run in parallel, trading atomic all-or-nothing behaviour for speed and scale.

1

Keep transactions short

Long transactions hold locks longer, which blocks others and raises the chance of aborts.

2

Never wait for a human inside one

Do not open a transaction and then wait for a user to click a button. Gather input first, then run the transaction quickly.

3

Read only what you need

Reading fewer rows locks fewer rows. Select specific columns and use precise conditions.

4

Make retries safe

Because aborted transactions are retried, avoid side effects such as sending emails inside the transaction body.

5

Use read-only mode for reads

Choose read-only transactions or single reads whenever nothing needs to change.

!
Hidden side effects

If your transaction code sends an email or charges a card and then the transaction is retried, the action may happen twice. Do external actions after the commit succeeds, or make them safe to repeat.

JScaling: Nodes, Splits and Autoscaling

Scaling is Spanner’s headline feature. Here is what happens when you add capacity and when load grows.

To give your database more power, you increase the compute capacity of the instance, adding nodes or processing units. This can be done while the database is serving traffic, with no downtime. Spanner then spreads splits across the larger set of servers, so each server handles a smaller share. Storage capacity grows with the number of nodes as well, since each node supports a certain amount of data, so check current limits when planning large databases.

flowchart TD
  A["Split S receives heavy traffic or grows large"] --> B["Spanner detects load or size above threshold"]
  B --> C["Split S is divided into S1 and S2 at a key boundary"]
  C --> D["S2 is moved to a less busy server"]
  D --> E["Load is shared across more servers"]
  E --> F{"Is load still too high overall?"}
  F -- "Yes" --> G["Add nodes or processing units"]
  G --> B
  F -- "No" --> H["System stays balanced"]
    
Figure 4: How Spanner splits and rebalances data automatically, and where you add capacity.

How splitting works

Spanner watches each split for size and for traffic. When a split grows too big, or becomes too busy, Spanner cuts it in two at a key boundary and may move one half elsewhere. This is called load-based splitting. It happens automatically, and you do not have to plan it. However, splitting works only if the work is spread over many key ranges. If all traffic hits one key, no split can help, since a single row cannot be cut in half. That is why single-row hotspots, such as one global counter updated by everyone, are an anti-pattern.

Analogy

Splitting is like a restaurant opening a second counter when a queue forms at the first one. It works when customers want different dishes. It fails when everybody wants the same single plate, because a plate cannot be divided among two chefs.

Choosing capacity

The main measure to watch is CPU utilisation of your Spanner servers. Google’s guidance has recommended keeping high-priority CPU below roughly sixty-five percent for regional instances and around forty-five percent for multi-region instances, leaving headroom for traffic spikes and for failover. These numbers may change, so confirm them in the documentation. If utilisation stays above the guidance, add capacity. If it stays far below, you may be overpaying and can reduce capacity.

Start

Start small

You can begin with a few hundred processing units for development and small workloads, and grow when needed.

Headroom

Keep spare capacity

Do not run hot. A failure or a sudden spike needs spare room, or latency will climb.

Grow

Scale before events

Raise capacity before a big sale or launch. New capacity takes a little time to be fully used as load balances.

Shrink

Scale down gently

Reduce capacity in steps and watch latency, since too sharp a cut can overload the remaining servers.

Autoscaling

Spanner offers an autoscaler option, which adjusts compute capacity automatically between a minimum and a maximum you choose, based on measured load. This saves money during quiet hours and protects performance during busy ones. You can set targets for CPU use and for storage use. Autoscaling helps most when traffic follows daily or weekly cycles. For sudden, extreme spikes, combine it with scheduled scaling before known events, because reactive scaling needs a little time to respond.

i
Scaling reads without scaling writes

If most of your traffic is reading, consider read-only replicas in multi-region setups and stale reads. They add read capacity close to users without making writes slower.

KQuerying Spanner: SQL, the Optimizer and Beyond

You use familiar SQL, but a few Spanner habits make queries faster and cheaper.

Spanner understands standard SQL features such as selecting, joining, filtering, grouping and sorting. It has a query optimizer, a built-in planner that decides how to run each query, for example which index to use and in what order to join tables. You can ask Spanner to show the query plan, which is the optimizer’s step-by-step approach, and read it to find expensive steps. In rare cases, you can guide the optimizer with hints, but it is wise to try better indexes and cleaner queries first.

Analogy

The query optimizer is like a satellite navigation app. You say where you want to go, and it chooses the route. Usually its choice is excellent, but now and then you look at the map yourself and notice a faster road. A query plan is that map.

Two dialects

Spanner supports GoogleSQL, the dialect shared with other Google data products such as BigQuery, and a PostgreSQL dialect for teams that prefer PostgreSQL syntax. You choose the dialect when creating the database. Both run on the same engine with the same guarantees. Not every PostgreSQL feature is available, so test your queries early.

Habits for fast queries

  • Use query parameters. Send values separately from the SQL text. This helps Spanner reuse plans and also protects against injection attacks.
  • Filter on key columns. Conditions on the primary key, or on an index, let Spanner read only a small range instead of scanning the whole table.
  • Select only needed columns. Fetching fewer columns moves less data and may allow an index-only answer.
  • Prefer direct reads for simple lookups. The read API fetches rows by key without the work of parsing and planning a query.
  • Keep statistics fresh. Spanner collects statistics about your data to help the optimizer. Large changes may call for a refresh.
  • Avoid full table scans on large tables. Scans are fine for small tables and analytics, but they are costly for busy transactional workloads.

Analytics without hurting the main workload

Large analytical queries, such as summing a year of sales, can consume a lot of compute and slow down your real-time traffic. Data Boost is an option that runs such queries on separate, on-demand compute, so they do not take capacity from your main nodes. It is billed separately for the resources used. Another approach is to connect BigQuery to Spanner so analysts can query live data through federated queries, or to stream changes into a warehouse for heavy analysis.

Change streams

A change stream records every insert, update and delete in chosen tables, so other systems can follow along. You can send these changes to a messaging system, a search index or a data warehouse. This lets you keep other systems in sync without writing triggers or polling, and it supports patterns such as event-driven architectures and audit trails.

Other modern features

Search

Full-text search

Search text columns by words and relevance without a separate search server, in versions that support it.

Vectors

Vector search

Store and compare embeddings, the numbers used by AI models, to find similar items such as related products or documents.

Graph

Graph queries

Ask relationship questions such as friends of friends, using graph features that sit on top of the same tables.

Events

Change streams

Follow every data change to feed other systems in near real time.

?
Check availability

Newer features such as search, vectors and graph are added over time and may depend on the edition and dialect you use. Confirm what is supported in the current documentation before you design around a feature.

LBackup, Point-in-Time Recovery and Restore

Replication keeps you safe from hardware failure. Backups keep you safe from human error.

Spanner’s replicas protect against failures of machines, zones and regions. But replicas faithfully copy mistakes too. If someone drops a table, or an application bug writes wrong values, every replica agrees on the damage. To recover from such mistakes, you need backups and the ability to look back in time.

Analogy

Replicas are like having the same notebook photocopied into three offices. If one office burns down, the others still have it. But if you scribble over a page, all three copies show the scribble as soon as they are updated. A backup is a dated photograph of the notebook, which you can go back to.

Three ways to protect and recover

Backups

Backups and schedules

Create a backup on demand or on a schedule. Backups are consistent snapshots of a database at one moment, kept for a retention period you choose. Newer options include incremental backups that store only changes.

PITR

Point-in-time recovery

Spanner keeps older versions of data for a set period, which can be extended up to a limit such as seven days. You can read data as of a past moment, or restore a database to that moment.

Copy

Cross-region copies

Copy a backup to another instance or region, so a regional disaster or a misconfigured instance cannot take your only backup with it.

Export

Export and import

Export data to files in Cloud Storage and import it back, which is useful for archiving, migration and sharing with other systems.

The recovery goals

Two numbers define every recovery plan. The recovery point objective, or RPO, is how much recent data you can afford to lose. The recovery time objective, or RTO, is how long you can afford to be down. Point-in-time recovery gives a very small RPO for mistakes, because you can choose a moment just before the problem. Restores create a new database, and large restores take time, so measure how long yours takes and compare it with your RTO.

How a restore works

1

Decide the target moment

Find out when the bad change happened using audit logs or application logs, and pick a time just before it.

2

Restore into a new database

Spanner restores into a new database rather than overwriting the existing one, so you can inspect it safely.

3

Compare and repair

If only a few rows were damaged, copy the good rows back into the live database. If much is damaged, switch your application to the restored copy.

4

Retire the temporary copy

Once recovery is confirmed, delete the temporary database to stop paying for it.

!
Untested backups fail

Practise restoring at least every quarter in a test project. Time it, verify the data and update your guide. A backup you have never restored is only a hope.

Protecting against accidental deletion

Spanner offers a deletion protection setting for databases, which blocks accidental deletion until the setting is turned off. Combine it with strict permissions, so only a few people can delete databases or backups, and with audit logging, so any such action is recorded.

MAvailability, Replication and Failure Scenarios

What Spanner does when parts of the world go dark.

Spanner is designed so that failures are routine events, not emergencies. Google publishes availability targets, called a service level agreement, that are among the highest in the industry, commonly described as four nines for regional instances and five nines for multi-region instances. Those figures are targets and promises with conditions, so read the current agreement to understand exactly what is covered.

Analogy

Think of an aeroplane with several engines. The pilots do not panic when one engine fails, because the others carry the load and the plane lands safely. Spanner treats a failed server, zone or region the same way: the remaining replicas continue while repairs happen quietly.

flowchart TD
  subgraph R1["Region 1: leader region"]
    L["Leader replica"]
    RW1["Read-write replica"]
  end
  subgraph R2["Region 2"]
    RW2["Read-write replica"]
  end
  subgraph R3["Region 3"]
    W["Witness replica"]
  end
  L |"Replicate"| RW2
  L |"Replicate"| W
  L -. "Region 1 fails" .-> X["Majority still alive in Regions 2 and 3"]
  X --> N["New leader elected in Region 2"]
  N --> S["Service continues, clients retry briefly"]
    
Figure 5: A multi-region configuration surviving the loss of its leader region.

What happens in each failure

What failsWhat Spanner doesWhat you do
A single server or nodeOther servers take over its splits within momentsNothing. Client libraries retry automatically
A whole zoneReplicas in other zones keep a majority and continueNothing in a regional or larger configuration
The leader replica’s locationA new leader is elected from the remaining replicasExpect a short blip of higher latency or retries
A whole region (multi-region)Majority in other regions keeps serving, with a new leaderNothing, but writes may be slower from far away
A whole region (single-region setup)Data is unavailable until the region returnsRestore from a cross-region backup if needed
A bad application changeReplication copies the damageUse point-in-time recovery or backups

Why majority matters

The rule of majority explains most of Spanner’s behaviour. With three voting replicas, two are enough to continue. With five, three are enough. When failures take away the majority, the affected data becomes unavailable rather than risking two sides making conflicting changes. This is the consistency-first choice described earlier. Configurations are designed so that ordinary failures never remove a majority.

Designing your client to be resilient

Even with perfect server design, applications can fail badly if they do not handle brief interruptions. The client libraries already retry many transient errors, but you must also set sensible timeouts, handle aborted transactions, and avoid piling retries on top of a struggling system. A technique called exponential backoff, which waits a little longer after each failed attempt, prevents a flood of retries from deepening a problem.

i
Practise failure

Do game days where you simulate trouble, such as heavy load or a temporary loss of connectivity, in a test environment. Watch how your application behaves. The goal is to find weak spots before a real incident does.

NSecurity and Compliance

Google secures the infrastructure. You decide who may see and change your data.

Cloud security follows a shared responsibility model. Google protects the data centres, the hardware, the network and the Spanner service itself. You are responsible for who can reach your databases, which permissions they hold, how you protect keys, and what data you choose to store. Both halves must be done well.

Analogy

A bank building has guards, vault doors and cameras, which the bank provides. But the customer still decides who gets a copy of their locker key. If you hand your key to a stranger, even the best vault cannot save you.

Identity and access

Identity and Access Management, called IAM, decides which people and programs can do what. A role is a bundle of permissions. You can grant roles at the project, instance or database level. Following the principle of least privilege, give each person or program only the roles they need. An application that only reads data should not hold a role that can delete tables. Create separate service accounts for each application, instead of sharing one powerful identity.

For finer control inside a database, Spanner offers fine-grained access control. You create database roles, grant them access to specific tables, columns or views, and then let users or services use those roles. This lets you hide sensitive columns, such as national identification numbers, from people who do not need them.

Encryption

At rest

Default encryption

All data is encrypted when stored, with keys managed by Google. You do not need to switch it on.

In transit

Encrypted connections

Traffic between your application and Spanner uses secure connections.

CMEK

Customer-managed keys

You hold the encryption keys in Cloud Key Management Service. You can rotate or disable them, giving you control and an audit trail.

Residency

Data location

Choosing a regional or dual-region configuration keeps data in a chosen area, which helps with data residency laws.

Network protection

You can limit how traffic reaches Spanner by using private connectivity, so that applications in your private network connect without crossing the public internet. VPC Service Controls create a protective boundary around Google Cloud services, which helps prevent data from being copied out of your organisation, even by someone with valid credentials. For highly sensitive systems, combine private connectivity, service controls and strict identity rules.

Auditing and data lifecycle

  • Audit logs. Cloud Audit Logs record administrative actions, and can record data access too, if you enable it. Keep them long enough for your compliance rules.
  • Row deletion policies. Spanner can delete rows automatically after a time you set, such as removing sessions after thirty days. This reduces both storage cost and the amount of personal data you keep.
  • Separation of environments. Keep development, testing and production in different projects, with different permissions.
  • Sensitive data hygiene. Do not put secrets or unneeded personal data in the database. Collect only what you must keep.
!
Key loss

If you use customer-managed keys and the key is disabled or destroyed, your data becomes unreadable until it is restored, and destroyed keys cannot be recovered. Treat key management as a serious process with owners, reviews and backups of the procedure.

Compliance

Google Cloud publishes compliance reports and certifications for many standards, and Spanner is covered by many of them. Certifications apply to the service, but your own configuration must also meet the rules. Check the current compliance documentation for the standards that matter to your industry, and ask your compliance team to review your design before launch.

OMonitoring, Logging and Metrics

A slow database rarely announces itself. You must watch the right signals to notice early.

Spanner sends metrics to Cloud Monitoring, where you can build dashboards and alerts. It also provides its own built-in tools designed specifically for finding database problems. Learning these tools is one of the most valuable skills for a Spanner user.

Analogy

Think of a hospital patient monitor. It shows heart rate, blood pressure and oxygen on one screen, and nurses learn what the combination means. Good database monitoring is the same: several signals read together tell a story that no single number can.

The signals worth watching

CPU
how busy the servers are
Latency
how long requests take
Storage
how full the database is
Load

CPU utilisation

The best sign of whether you need more capacity. Watch the high-priority CPU figure against the recommended limits.

Speed

Request latency

Track the median and the slowest few percent. A rising slow tail often warns of trouble before users complain.

Space

Storage use

Watch growth over time. Storage limits depend on node count, so plan capacity before you hit a wall.

Errors

Aborts and errors

A rising rate of aborted transactions points to lock contention. Error codes show timeouts and resource limits.

Clients

Sessions and connections

Too few sessions cause waiting in the client. Too many waste resources.

Backup

Backup and schedule health

Confirm that scheduled backups succeed and that the newest backup is recent enough.

Spanner’s built-in diagnostic tools

1

Key Visualizer

A heat map showing how traffic is spread across your key ranges over time. Hot, bright stripes reveal hotspots. This is the first tool to open when performance is uneven.

2

Query insights

Shows which queries use the most CPU, take the longest and run most often. It helps you find the single bad query behind a slowdown.

3

Transaction and lock insights

Reveals which transactions wait on locks, which rows are contested and how often transactions abort.

4

Built-in statistics tables

Special system tables that you can query with SQL to see request counts, latency and top consumers over recent time windows.

5

Tracing

Client libraries can emit traces, so you can follow one request from your application through Spanner and see where time is spent.

Alerts that help

Good alerts describe problems that users feel: CPU above the recommended limit for several minutes, latency far above normal, storage nearing its limit, backups failing, or a sudden jump in errors. Weak alerts fire on brief spikes and train people to ignore the pager. Link each alert to a short guide that says what to check first and how to add capacity or find the offending query.

i
Load test, then look

After every load test, open Key Visualizer and Query insights. Testing tells you how fast the system was, and these tools tell you why it was or was not fast, which is where improvement begins.

PPerformance Tuning

Small design and code habits that make Spanner feel fast and keep the bill low.

Performance in Spanner comes from three places: how you design keys and tables, how your application talks to the database, and how much capacity you provide. Money cannot fix a design problem, but a good design uses money far more efficiently.

Habits that pay off

Keys

Spread writes evenly

Avoid increasing keys, use well-spread values and check the key heat map after testing.

Together

Co-locate related rows

Interleave children with parents so common reads and writes stay within one split.

Place

Run clients near the leader

Place application servers in the same region as the leader to cut round-trip time on writes.

Stale

Use stale reads where possible

Reads that tolerate a few seconds of age can be served by nearby replicas, with lower latency and less load on the leader.

Batch

Batch writes sensibly

Group many row changes into one commit, but keep batches moderate, because huge commits take long and hold locks.

Index

Index for your real queries

Create indexes that match your frequent filters and sorts, and remove unused ones to speed writes.

Analogy

Tuning Spanner is like arranging a busy supermarket. Put popular items near the entrance on many shelves rather than piled in one aisle, keep related products together, and open extra checkouts only when the queue still builds after the layout is right.

Client-side matters

Many slowdowns are caused by the application, not by Spanner. Create the database client once and share it across your whole application, instead of creating a new client for each request, because creating clients is slow and wasteful. Size the session pool to match your concurrency. Set timeouts and retry policies on purpose. Be careful with object-relational mappers, which are tools that generate SQL for you, as they may produce many small queries or inefficient joins. Review the SQL they create, and replace the worst offenders with hand-written queries.

The measure-change-measure loop

1

Build a realistic test

Use realistic data volumes and traffic patterns, including your busiest hour. Tiny data sets hide design problems.

2

Run and record

Record latency, CPU and errors, and open the heat map and query insights.

3

Change one thing

Fix the biggest problem you found, such as one hot key range or one expensive query.

4

Test again

Repeat the same test and compare. If the result improved, keep the change. If not, undo it.

Warm-up and ramping traffic

A brand-new or newly resized database needs a little time for Spanner to learn where load is and to split and move data accordingly. Jumping instantly from no traffic to full traffic can cause short-lived slowness. When you can, ramp traffic up over several minutes or hours before a big launch, and test with the same ramp pattern.

!
Do not guess

Without measurement, tuning turns into folklore. Many teams spend days on unhelpful changes before a five-minute look at the key heat map reveals the real problem.

QCost and Billing

Spanner is a premium service. Knowing what you pay for lets you spend wisely.

Spanner billing has several parts. Prices change and vary by region, edition and configuration, so always check the official pricing page and use the pricing calculator before planning a budget. The parts below describe how the bill is built, rather than quoting numbers.

Compute

Nodes or processing units

Charged for each hour that capacity is provisioned, whether or not you use it. This is usually the largest part of the bill.

Storage

Database storage

Charged per gigabyte stored each month. Indexes and older versions kept for recovery count too.

Backup

Backup storage

Charged per gigabyte for backups kept. More backups and longer retention mean a higher charge.

Network

Network traffic

Charges can apply to data leaving a region or reaching the internet. Keep applications near the database.

Analytics

Data Boost

Billed by the compute used by those analytical queries, separately from your main capacity.

Discount

Committed use discounts

Promising to use a certain amount of compute for a year or longer can lower the price for steady workloads.

Analogy

Provisioned compute is like renting a number of taxis by the hour. You pay for each taxi whether it carries passengers or waits at the stand. The smart operator keeps enough taxis for the busy hours and returns extras when the streets are quiet.

What drives the bill up

  • Over-provisioned compute. Running at ten percent CPU all day means paying for much more than you use. Right-size, or use autoscaling.
  • Multi-region where regional would do. Multi-region costs substantially more. Use it only where the availability or global read needs justify it.
  • Too many indexes. Each index uses storage and slows writes, increasing both storage and compute needs.
  • Long recovery retention. Keeping many days of older versions grows storage. Choose the shortest period that meets your recovery needs.
  • Forgotten instances. Test and development instances left running quietly bill every hour.
  • Heavy analytics on main nodes. Running big reports on production capacity forces you to buy more nodes than the transactions need.

A thought experiment

Consider two systems. The first is a small internal booking tool for a few hundred employees, with a few queries per second. A tiny instance of a few hundred processing units, in a single region, could serve it, and Spanner’s ability to start small makes this possible. The second is a payment ledger for millions of customers across several countries, with thousands of writes per second and a strict need for correct, always-available data. It needs many nodes, a multi-region configuration and careful design. Both systems can use Spanner, but their bills differ by large factors, and both are reasonable. The wise habit is to ask, for each system, whether the guarantees it pays for are guarantees it truly needs.

i
Practical controls

Set budget alerts, label instances by team and purpose, review utilisation monthly, delete unused test instances, and use autoscaling for workloads with daily cycles. Ask whether a smaller edition or configuration could do the same job.

RAdvantages, Disadvantages and Trade-offs

An honest look at what you gain and what you give up.

Advantages

  • SQL, joins and full ACID transactions at very large scale.
  • Strong, external consistency, even across regions.
  • Automatic splitting, rebalancing and replication.
  • Online scaling and online schema changes, with no downtime.
  • Very high availability targets, including multi-region.
  • Fully managed: no servers, patches or manual failovers.
  • Point-in-time recovery, backups and change streams built in.
  • Starts small and grows with the business.

Disadvantages

  • More expensive than simpler databases for small workloads.
  • Requires learning key design to avoid hotspots.
  • Write latency grows with distance in multi-region setups.
  • Not every feature of PostgreSQL or MySQL is available.
  • Moving an existing application needs schema and code changes.
  • Not designed to be a data warehouse for heavy analytics.
  • Tied to Google Cloud, so leaving takes real effort.

The big trade-offs in plain words

Consistency against latency. Spanner chooses to always be correct, and that means writes must reach a majority of replicas. In one region this costs a few milliseconds. Across continents, it costs tens or hundreds of milliseconds. You trade some speed for the peace of mind that comes from never seeing wrong data.

Power against price. The features that make Spanner special, such as global consistency and automatic scaling, require provisioned capacity and a premium service. If your data fits comfortably on a single machine and a single region, a simpler database is cheaper and perfectly adequate.

Freedom against responsibility. You never manage servers, but you must learn to design keys and transactions for a distributed system. Habits from single-machine databases, such as auto-increment keys, can hurt here.

Analogy

Spanner is like a high-speed railway network. It carries enormous numbers of passengers reliably across great distances, and it is a marvel when you need that. But for a short trip across town, a bicycle is cheaper, simpler and just as quick.

Choose Spanner for the guarantees you truly need, not for the scale you hope to have one day.

SSpanner vs Other Google Cloud Databases

Google Cloud offers many databases. Knowing the neighbours helps you pick the right home for each workload.

Relational

Cloud SQL

Managed MySQL, PostgreSQL and SQL Server. Simple and affordable for traditional applications that fit on one primary machine, with read replicas for extra reads.

Relational

AlloyDB

A PostgreSQL-compatible database designed for high performance and demanding transactional and analytical workloads, scaling mainly within a region.

Relational

Cloud Spanner

Globally distributed relational database with horizontal scaling and strong consistency across regions.

NoSQL

Bigtable

A wide-column database for enormous throughput and low latency on simple access patterns, such as time series and large key lookups. No full SQL joins.

NoSQL

Firestore

A serverless document database popular for mobile and web apps, with easy scaling and real-time features.

Warehouse

BigQuery

A serverless data warehouse for large analytical queries over huge data sets, not for high-rate transactions.

flowchart TD
  Q1{"Mainly large-scale analytics and reports?"}
  Q1 -- "Yes" --> BQ["BigQuery"]
  Q1 -- "No" --> Q2{"Need relational data with transactions?"}
  Q2 -- "No" --> Q3{"Huge key-based throughput or time series?"}
  Q3 -- "Yes" --> BT["Bigtable"]
  Q3 -- "No" --> FS["Firestore"]
  Q2 -- "Yes" --> Q4{"Need to scale writes beyond one machine, or span regions with strong consistency?"}
  Q4 -- "Yes" --> SP["Cloud Spanner"]
  Q4 -- "No" --> Q5{"Need top PostgreSQL performance inside one region?"}
  Q5 -- "Yes" --> AL["AlloyDB"]
  Q5 -- "No" --> CS["Cloud SQL"]
    
Figure 6: A simple path for choosing a database on Google Cloud.

Comparing the relational options

TopicCloud SQLAlloyDBCloud Spanner
Engine familyMySQL, PostgreSQL, SQL ServerPostgreSQL compatibleOwn engine, GoogleSQL and PostgreSQL dialects
Write scalingOne primary, scale upOne primary, scale upScale out across many nodes
Multi-region strong consistencyNo, replicas lagNo, replicas lagYes, built in
Compatibility with existing appsHighestHigh for PostgreSQLNeeds adaptation
Relative cost for small workloadsLowestMediumHighest, though small instances exist
Best forStandard applicationsDemanding PostgreSQL workloadsGlobal, very large or always-on systems

The most common beginner mistake is reaching for Spanner by default because it sounds impressive. If your application is a typical web app with a single region and modest traffic, Cloud SQL or AlloyDB will probably be simpler and cheaper. Another common mistake is the reverse: forcing a single-machine database to cope with global users and massive write rates, and then spending years building sharding and failover by hand. Match the tool to the real need.

Analogy

Choosing a database is like choosing a vehicle. A bicycle (Cloud SQL) suits short, simple trips. A powerful car (AlloyDB) handles demanding roads. A cargo aeroplane (Spanner) crosses oceans with enormous loads. Using an aeroplane to buy groceries is silly, and using a bicycle to cross an ocean is impossible.

i
Using several together

Large systems often combine databases: Spanner for the core ledger, Bigtable for telemetry, BigQuery for analytics and a cache for hot reads. Each does what it does best, and change streams or data pipelines connect them.

TDesign Patterns That Fit Spanner

A pattern is a proven recipe for a common problem. These six work especially well on Spanner.

1

Tenant-first keys

For software that serves many customers, put the tenant identifier first in every key. One tenant’s data stays together, tenants spread across splits, and queries that filter by tenant are efficient.

2

Interleaved hierarchies

Model natural parent and child structures, such as account, order and line items, as interleaved tables, so common transactions touch only one split.

3

Sharded counters

Instead of one row that everyone increments, use many counter rows and add them up when reading. Updates spread out, and the hotspot disappears.

4

Outbox with change streams

Write a business change and an event record in the same transaction, then use a change stream to publish events. This avoids the danger of updating the database and a message system separately.

5

Stale reads for global audiences

Serve catalogues, profiles and dashboards from nearby replicas with bounded staleness, and keep strong reads for money and stock decisions.

6

Time-based expiry

Use row deletion policies to remove old sessions, logs and temporary records automatically, keeping storage and cost under control.

Analogy

A sharded counter is like many ticket booths at a stadium gate, each counting its own visitors. The total is found by adding up all the booths. A single booth with one huge queue would be the old way, and everybody would wait.

The outbox pattern in more detail

Suppose a customer places an order, and you must also tell the shipping system. If you save the order and then separately send a message, a crash between the two steps leaves the systems in disagreement. With the outbox pattern, you write the order and an “order placed” record in one transaction. A change stream watches the outbox and publishes each new record. Either both writes happen or neither does, and the message is eventually delivered. Consumers must tolerate seeing the same message twice, so make them safe to repeat.

Multi-tenant isolation

When one database serves many customers, you can keep tenant data separate by key design, by using different databases per large tenant, or by using fine-grained access control so each tenant’s service can only see its own rows. The best mix depends on how different your tenants are in size and in compliance needs. A very large tenant may deserve its own database or even its own instance, while thousands of small tenants share one.

UAnti-patterns to Avoid

An anti-pattern is a habit that looks sensible at first and causes pain later.

AP-01 · INCREASING PRIMARY KEYSAvoid
What it looks like

A table uses an auto-increment number or a timestamp as the first part of the primary key.

Why it hurts

All new rows land on the last split, so one server does all the writing. Adding nodes does not help, and latency climbs under load.

Better approach

Use random identifiers, bit-reversed sequences, hash prefixes or a well-spread leading column, and verify with the key heat map.

AP-02 · THE GLOBAL COUNTER ROWAvoid
What it looks like

One row holds a total, such as the number of orders today, and every transaction updates it.

Why it hurts

Every transaction fights for the same lock. Throughput collapses to one update at a time, and aborts multiply.

Better approach

Use sharded counters, or compute totals from the underlying rows when needed.

AP-03 · HUGE, LONG TRANSACTIONSAvoid
What it looks like

A single transaction reads and updates millions of rows, or stays open while waiting for a user or another service.

Why it hurts

It holds locks for a long time, blocks other work and is likely to abort. Limits on commit size may also reject it.

Better approach

Break big jobs into smaller batches, use partitioned DML for bulk changes, and keep transactions short and focused.

AP-04 · TREATING SPANNER AS A WAREHOUSEAvoid
What it looks like

Analysts run huge scans and heavy aggregations directly on the production nodes.

Why it hurts

Analytical queries steal capacity from real-time traffic, forcing you to buy more nodes, and customers feel the slowdown.

Better approach

Use Data Boost, federated queries from BigQuery, or stream data to a warehouse with change streams.

AP-05 · INDEX EVERYTHINGAvoid
What it looks like

A team adds an index for every column “just in case”.

Why it hurts

Each index adds storage and slows every write. Some indexes, especially on timestamps, create their own hotspots.

Better approach

Create indexes for known, frequent queries, check query plans, and remove indexes that are never used.

AP-06 · IGNORING RETRIESAvoid
What it looks like

Application code treats an aborted transaction as a fatal error, or retries without a pause.

Why it hurts

Aborts are normal in a system with locks. Failing on them loses user actions, while instant retries can overload the database.

Better approach

Use the client library’s retry helpers, add backoff, and make transaction code safe to run more than once.

VBest Practices and Common Mistakes

A checklist to read before your first production launch.

Best practices

Design

Design keys first

Choose primary keys that spread writes and keep related rows together before you write a line of application code.

Place

Choose the configuration carefully

Match regional, dual-region or multi-region to real availability and residency needs, and put the leader near writers.

Test

Load test with real data

Use realistic volumes and traffic shapes, then check the key heat map and query insights.

Headroom

Keep CPU headroom

Stay under the recommended utilisation so failures and spikes do not hurt users.

Retry

Handle aborts and timeouts

Use client library retry helpers, backoff, and idempotent transaction code.

Protect

Back up and practise restores

Schedule backups, set a point-in-time window, copy backups across regions and test restores every quarter.

Secure

Least privilege

Separate service accounts, narrow roles, fine-grained access control and audit logs.

Cost

Right-size and autoscale

Review utilisation monthly, use autoscaling for cyclical load and delete unused instances.

Common mistakes

Copying a schema from another database unchanged

Auto-increment keys and single-row counters work on one machine but cause hotspots here. Redesign keys before migrating.

Testing with a tiny data set

Everything feels fast with a thousand rows. Hotspots and slow queries show up only with realistic volumes.

Running too hot

Operating near full CPU leaves no room for failover or spikes, and latency climbs steeply near the limit.

Creating a new client for every request

Clients and session pools are meant to be long-lived and shared. Recreating them wastes time and resources.

Choosing multi-region by default

It costs more and slows writes. Use it when you truly need its availability or global reach.

Relying on replication as a backup

Replicas copy mistakes too. Backups and point-in-time recovery are what protect against human error.

Ignoring the leader location

Application servers far from the leader region add avoidable delay to every write.

i
Habit

Write a short launch checklist covering keys, configuration, capacity, alerts, backups, restore tests, permissions and retry behaviour. Review it before every major release, like a pilot checking the list before every flight.

WReal-World and Industry Examples

Where Spanner tends to be used, and why it fits.

Detailed architectures of private companies change often and are rarely published in full, so the examples below describe well-known origins and common industry patterns rather than guaranteed facts about any one named customer. Always verify specific claims with the company’s own case studies.

Google’s own advertising systems

The original Spanner research paper describes how Google’s advertising backend, a system called F1, moved from a manually sharded MySQL setup to Spanner. The old arrangement needed painful, risky operations whenever more capacity was required. Spanner let the team scale and rebalance without those procedures, while keeping the strict correctness that billing data requires. This is the textbook example of the problem Spanner was built to solve.

Financial ledgers and payments

Banks, payment companies and fintech start-ups need every balance to be exactly right, at every moment, and across several locations. Strong transactions and external consistency make Spanner a natural fit for ledgers, transfers and order books, with multi-region configurations offering resilience against the loss of an entire region. Strict audit and data-residency needs can be met by choosing locations and enabling audit logs.

Retail inventory and order management

A retailer with online stores, physical shops and warehouses in many countries needs one trustworthy view of stock. If two customers buy the last item at once, exactly one must succeed. Spanner’s transactions prevent overselling, while its scale absorbs enormous sale-day traffic, and its stale reads keep browse pages quick for shoppers far from the leader region.

Online games and player profiles

Game studios with millions of players need profiles, inventories and purchases to be correct, because a lost purchase quickly becomes an angry customer. Games also face huge, sudden spikes at launches and events. Online scaling, tenant-first or player-first keys and strong transactions suit this pattern well.

Telecom, logistics and identity platforms

Systems that track subscribers, shipments or user identities across regions need huge numbers of small, correct updates and lookups. They benefit from high availability, easy horizontal scaling and a single global view, rather than many regional databases that must be reconciled.

Software-as-a-service platforms

Companies that offer software to thousands of business customers often use tenant-first keys, so each customer’s data is neatly grouped while the whole platform scales as one database. Online schema changes let them release new features without scheduled downtime, which their customers expect.

?
The common thread

In each example, the organisation needs several things at once: relational correctness, scale beyond one machine, high availability and often a presence in several regions. When only one or two of those apply, a simpler database is usually the better choice.

XMigrating to Spanner and Keeping Current

Moving an existing system to Spanner is a project, not a button. Here is a calm path.

Applications built for MySQL, PostgreSQL or other databases cannot simply be pointed at Spanner. Differences in keys, features and behaviour need attention. The good news is that Google provides tools and guidance, and that a careful migration can be done with little or no downtime.

Analogy

Migrating a database is like moving a busy restaurant to a new building without closing. You prepare the new kitchen, test every recipe, copy over the pantry, run both for a while, and switch customers over only when you are sure the new place works.

A step-by-step path

1

Assess

Inventory the schema, queries, features and traffic of the current system. Spanner’s migration tool can analyse a source database and report which parts convert easily and which need redesign.

2

Redesign the schema

Replace auto-increment keys, plan interleaving, choose indexes and decide how to handle features that Spanner does not offer, such as some triggers or stored code.

3

Adapt the application

Switch to Spanner client libraries or compatible drivers, add transaction retries and review generated SQL.

4

Load existing data

Copy the bulk of the data using the migration tooling or data pipelines, such as Dataflow, spreading writes to avoid hotspots during the load.

5

Keep in sync

Stream ongoing changes from the old database to Spanner with change data capture tools, so both stay up to date during testing.

6

Validate

Compare row counts, checksums and query results between old and new systems. Run load tests with realistic traffic.

7

Cut over with a way back

Switch traffic during a quiet period. Keep the old system ready, and consider replicating changes backward for a while so you can roll back safely.

Compatibility tools

Teams using PostgreSQL can take advantage of the PostgreSQL dialect and of a compatibility layer that lets many standard PostgreSQL drivers and tools connect to Spanner. This reduces code changes, but it does not remove the need to redesign keys and to test thoroughly. Always read the current list of supported and unsupported features before committing to a migration plan.

Keeping current

Spanner evolves quickly. New editions, features, limits and prices appear regularly. Make it a habit to read the release notes a few times a year, review your instance configurations and editions, and re-check recommendations such as CPU targets when you upgrade or change capacity. Subscribe to service announcements so you hear about changes before they affect you.

i
Start with a pilot

Migrate a small, low-risk part of your system first. You will learn how Spanner behaves with your data and team, and the lessons will make the larger migration smoother.

YFrequently Asked Questions

Quick answers to the questions beginners ask most often.

Q1Is Spanner a SQL database or a NoSQL database?
It is a relational SQL database with full transactions, but it scales horizontally like many NoSQL systems. It is often called a NewSQL database for this reason.
Q2What is the difference between a node and a processing unit?
A node is a unit of compute capacity, and a processing unit is one thousandth of a node. Processing units let you start with a small instance and grow in fine steps.
Q3Why are auto-increment keys discouraged?
Increasing keys send all new rows to the end of the table, so one server handles all the writes. Random or well-spread keys let many servers share the work.
Q4What does TrueTime do?
It gives Spanner a very accurate view of time, expressed as a small interval. Spanner uses it to order transactions correctly across the world and to promise external consistency.
Q5Is my data safe if a whole data centre fails?
Yes, within the limits of the configuration you chose. Replicas in other zones or regions hold copies and keep a majority, so no confirmed data is lost, and service continues with a brief pause at most.
Q6Do replicas replace backups?
No. Replicas protect against hardware and location failures, but they copy mistakes too. Backups and point-in-time recovery protect against deletion, corruption and bugs.
Q7When should I use a stale read?
Use it when slightly old data is acceptable, such as product listings or dashboards. It can be served by nearby replicas, with lower latency. Use strong reads for balances and stock checks.
Q8Why do some of my transactions abort?
Aborts are a normal way for Spanner to resolve lock conflicts and prevent deadlocks. Use the client library’s retry helpers, keep transactions short and make them safe to repeat.
Q9Can Spanner replace my data warehouse?
Not usually. It is built for transactions. For heavy analytics, use Data Boost, federated queries from BigQuery, or stream data into a warehouse.
Q10Can I change my instance configuration later?
Some changes are supported with specific processes, but not all moves are simple. Plan the configuration carefully at the start and check the current documentation before relying on changing it.
Q11How big can my database get?
Storage capacity grows with the amount of compute you provision, and each node supports a certain amount of data. Check the current limits for your configuration and plan capacity ahead of growth.
Q12Is Spanner too expensive for small projects?
It can be more costly than simpler databases, but small instances and trial options exist. If you do not need global scale, very high availability or strong multi-region consistency, a simpler database is probably the better value.
Q13How do I find the slow query?
Open query insights to see which queries use the most CPU or take longest, then view the query plan to find the expensive step. Add an index or rewrite the query, and measure again.
Q14How do I avoid hotspots in a time-series table?
Do not put the timestamp first. Put a well-spread identifier first, such as device or user, and the timestamp second. Check the key heat map after load testing.
Q15How do I avoid a surprise bill?
Right-size compute, use autoscaling for cyclical load, choose the smallest configuration that meets your needs, limit indexes and retention, delete unused instances and set budget alerts.

ZSummary and Key Takeaways

If you remember only one page of this guide, make it this one.

Cloud Spanner is a fully managed relational database that scales across machines and regions while keeping transactions strictly correct. It cuts tables into splits, copies each split to several replicas, agrees on every write by majority vote, and orders events across the planet using accurate clocks. You gain SQL, scale, availability and consistency together, and you pay with a premium price and a design style that takes care. Success comes from good primary keys, short transactions, headroom in capacity, tested backups, careful security and honest thinking about whether you need what Spanner offers.

Key Takeaways

  • Spanner gives you scale and correctness together: SQL, ACID transactions and external consistency, with automatic splitting and replication across zones and regions.
  • Primary keys decide performance: avoid increasing keys, spread writes with random or well-spread values, and interleave related tables to keep common work in one place.
  • Writes need a majority: regional setups are fast, and multi-region setups trade some write latency for stronger availability and global reach.
  • Use the right read and transaction type: read-only transactions and stale reads cut cost and latency, while short, retry-safe read-write transactions keep conflicts low.
  • Protect and observe: replicas are not backups, so schedule backups, use point-in-time recovery, practise restores, and watch CPU, latency, the key heat map and query insights.
  • Control cost and access: right-size compute, use autoscaling, keep indexes and retention lean, and apply least privilege, audit logs and the right encryption options.
  • Choose it for the guarantees you need: for modest single-region applications Cloud SQL or AlloyDB may serve better, and the current documentation should be checked before every major design.