Cross-Cluster Replication in Elasticsearch

Cross-Cluster Replication in Elasticsearch

How Elasticsearch keeps live copies of your data in a second cluster, in another building, city or continent, so your search stays close to your users and survives the worst day.

Picture a company that runs its online shop from a data center in Frankfurt. Every product, order and customer search lives in one Elasticsearch cluster there. On an ordinary day, everything is fine. Then a power failure, a cut cable or a mistake by a contractor takes the whole building offline. The shop goes dark, customers cannot search, and every minute of silence costs money and trust.

Now picture a second company with customers in Tokyo, London and New York, all searching one cluster in Virginia. The Tokyo customer waits far longer for each search than the New York customer, simply because the data travels half the planet. Nothing is broken, yet the experience is unequal and slow.

Both problems have the same cure: keep a live copy of the data somewhere else. Elasticsearch has a built-in feature for exactly this, called Cross-Cluster Replication, or CCR. This guide explains CCR from zero. Every new word is explained the first time it appears, and each idea builds on the last. By the end you will know what CCR is, how it works inside, how to plan for it, what it costs, and which traps to avoid.

1What Is Cross-Cluster Replication?

We start with the words in the name, because each one carries part of the meaning.

Cluster, Replication and Cross

A cluster is a group of Elasticsearch servers, called nodes, that work together and share the job of storing and searching data. Replication means keeping copies of the same data in more than one place. Cross-cluster means the copies live in different clusters, not just on different servers of the same cluster.

Put together, Cross-Cluster Replication is an Elasticsearch feature that automatically copies indices from one cluster to another and keeps the copy up to date as the original changes. An index is the basic container for documents in Elasticsearch, a bit like a table in a database. So CCR copies whole indices, continuously, from a source cluster to a target cluster.

Everyday Analogy

Think of a national newspaper. The head office in the capital writes and edits every story. Instead of making readers in every city wait for trucks, the paper sends each finished page electronically to printing presses in other cities. Local readers get the same paper, printed nearby, only a few moments after the head office finished it. If the head office loses power, the regional presses still hold the latest pages. CCR is the electronic link, and the regional presses are the follower clusters.

Leader and Follower

CCR uses two simple roles. The index that gets changed by applications is the leader index, and it lives in the leader cluster. The copy is the follower index, and it lives in the follower cluster. The follower watches the leader and copies every change. The follower never changes the leader.

i
Definition

Cross-Cluster Replication (CCR) is an Elasticsearch feature in which a follower index in one cluster continuously pulls every change from a leader index in another cluster, so that the follower stays a near-real-time, read-only copy of the leader.

Three Words to Remember

  • Asynchronous: the leader does not wait for the follower. It saves the data locally, tells the application “done”, and the follower catches up a moment later.
  • Pull-based: the follower asks the leader for new changes. The leader does not push anything out.
  • Read-only copy: applications can search the follower, but they cannot write to it while it is following.
2
roles: a leader that accepts writes and a follower that copies them
3
main uses: disaster recovery, nearby reads, central reporting
Seconds
typical delay between a change on the leader and its arrival on the follower, when the network is healthy

What CCR Is Not

CCR is not a backup. If someone deletes an index by mistake on the leader, that deletion is copied to the follower too, because the follower faithfully copies everything. CCR is also not a way to write to two clusters at once for the same index. And it is not the same as searching several clusters together, which is a different feature called cross-cluster search. We will compare all of these later.

2Why Copies Inside One Cluster Are Not Enough

Elasticsearch already keeps copies of your data. To see why we need CCR as well, we must understand what those copies protect against and what they cannot.

Shards and Replicas Inside a Cluster

An index is split into pieces called shards, and shards are spread across the nodes of a cluster. Each shard normally has one or more replica shards, which are extra copies placed on different nodes. If one node crashes, a replica on another node takes over, and no data is lost. This is Elasticsearch’s everyday safety net, and it works very well for single-machine failures.

Where the Safety Net Ends

All those replicas still live inside one cluster, usually in one data center or one cloud region. That means they share the same fate in several situations:

  • Site failure: a fire, flood, power cut or network outage at the data center takes every node down together.
  • Regional cloud outage: a problem in one cloud region can affect all zones or the services they rely on.
  • Distance: users far from the cluster always pay the price of long network trips, no matter how many replicas exist.
  • Rules about location: some laws and contracts require data to stay in a certain country or to be served from inside it.
  • Growth of many small clusters: an organization with dozens of separate clusters at branches or factories has no built-in way to see all their data in one place.
Everyday Analogy

Keeping three copies of an important document in three drawers of the same office protects you if one drawer jams. It does not help if the whole office burns down. To be truly safe, you also keep a copy in another building. Replicas are the extra drawers, and CCR is the second building.

Why Not Just Stretch One Cluster Across Regions?

A natural idea is to place some nodes of one big cluster in Frankfurt and others in Virginia. Elasticsearch is not designed for this. Nodes in one cluster talk to each other constantly and expect fast, steady links. Across continents the delay is high and the network can wobble. A cluster spread this way can become slow, unstable, or unable to agree on who is in charge when the link between regions breaks. Elastic’s guidance is to keep a cluster within one low-latency location and to use CCR to connect separate clusters.

!
Watch Out

A stretched cluster looks cheap and simple on a diagram, but it turns every network hiccup between regions into a cluster-wide problem. Two independent clusters joined by CCR fail separately, which is what you want in a disaster.

Replicas protect you from a broken server. Cross-cluster replication protects you from a broken place.

3Core Concepts You Need First

A few terms appear in every CCR discussion. Learn them here and the later chapters will feel easy.

Word 1

Remote Cluster

A saved connection from one cluster to another, with a short nickname. The follower cluster keeps a remote cluster entry that points to the leader.

Word 2

Leader Index

The original index that applications write to. Every change on it gets a number that tells the order of events.

Word 3

Follower Index

The read-only copy in the follower cluster. It keeps asking the leader for changes and applying them.

Word 4

Sequence Number

A counter that Elasticsearch attaches to every change on a shard. It lets a follower know exactly which changes it has and which it still needs.

Word 5

Global Checkpoint

The highest sequence number that is safely stored on all in-sync copies of a shard. It marks how far a shard can be trusted.

Word 6

Soft Deletes

A setting that makes a shard remember recent changes, including deletions, for a while, so followers can read the history they missed.

Word 7

Auto-Follow Pattern

A rule that says “whenever a new leader index with a name like this appears, create a follower for it automatically.”

Word 8

Replication Lag

The gap between the leader’s latest change and the follower’s latest applied change. Small lag means a fresh copy.

Two Clusters, One Direction

flowchart LR
    APP["Applications writing data"] --> L["Leader cluster: leader index"]
    L -. "changes pulled over the network" .-> F["Follower cluster: follower index"]
    F --> READ["Applications searching nearby"]
Figure 1: The basic CCR picture. Writes go to the leader, changes flow one way to the follower, and readers can use the follower.

The dotted arrow is the heart of CCR. It is drawn from the leader toward the follower, but remember who starts the conversation: the follower reaches out and pulls. That design makes the leader simpler, because it does not have to keep track of every place its data should be pushed to.

Why the Numbering Matters

Every change on a shard gets a sequence number, counted upward like the page numbers of a diary: 1, 2, 3, and so on. Because of this, a follower can say precisely, “I have everything up to number 4,200. Please send me 4,201 onward.” There is no guessing and no need to compare whole indices. This tiny idea is what makes continuous copying both accurate and efficient.

Everyday Analogy

Imagine a friend who writes a numbered diary and lets you copy it. When you visit again, you do not reread the whole diary. You tell them the last entry number you copied, and they hand you only the newer pages. Sequence numbers do the same for shards.

4How Replication Styles Differ

Systems copy data in several different ways. Knowing the options explains why CCR was designed the way it was.

Synchronous or Asynchronous

In synchronous replication, the system does not tell the application “saved” until every copy, including the faraway one, confirms it. This gives strong safety, because no confirmed write can be lost. But every write must wait for the slowest, farthest copy. Across continents, that delay is painful, and if the far link breaks, writes may stall completely.

In asynchronous replication, the system confirms the write after saving locally, and copies to the far site afterward. Writes stay fast and keep working even when the far link is down. The price is a small window during which the far copy is slightly behind. CCR chooses this style.

Push or Pull

In a push design, the source sends changes to each destination. In a pull design, each destination asks the source. CCR is pull-based. The follower decides how fast to read, how much to read at once and when to retry after a failure, which naturally protects the leader from being overwhelmed.

StyleHow It WorksStrengthWeakness
SynchronousWrite waits until all copies confirmNo confirmed write is lostSlow over long distances and fragile when links fail
AsynchronousWrite confirms locally, copies follow laterFast writes and tolerant of link problemsThe copy can lag, so a sudden disaster may lose the last few moments
PushSource sends changes to destinationsVery low delay for simple setupsSource must track and serve every destination
PullDestination asks the source for changesDestination controls pace and retriesDelay depends on how often and how much it pulls
Everyday Analogy

Synchronous replication is like a bank clerk who refuses to hand you a receipt until the head office in another country has phoned back to confirm. Asynchronous replication is like the clerk handing you the receipt right away and sending the paperwork by courier for the head office to file. The second way is much faster, but if the courier’s truck crashes, the head office may briefly lack the newest paperwork.

*
Good to Know

Because CCR is asynchronous, the most useful number to plan around is the recovery point objective, often shortened to RPO. It answers the question “how much recent data can we afford to lose if the leader vanishes suddenly?” With CCR, that answer is usually small, often seconds, but it is never exactly zero.

5Architecture and Components

Let us open the box and look at the parts that make CCR work, from the user’s browser all the way to the shards.

The Parts of the System

Front Door

API Gateway and Load Balancer

The gateway is the single entrance for requests, handling identity checks and rate limits. The load balancer spreads requests over several application servers. During a failover, this layer is often where traffic is redirected to the other cluster.

Application

Search and Indexing Services

Your own code. Indexing services write to the leader cluster. Search services read from the leader or, for nearby reads, from the follower.

Leader Side

Leader Cluster and Shards

The cluster where data is created. Each leader shard keeps a numbered history of recent changes that followers can read.

Follower Side

Follower Cluster and Shards

The cluster that holds the copy. Each follower shard runs a small background task that reads changes from the matching leader shard and applies them.

Link

Remote Cluster Connection

A secured network link, with a saved nickname, that lets the follower cluster talk to the leader cluster. Encryption and credentials protect it.

Operations

Monitoring and Management

Stats interfaces and dashboards that show lag, errors and follower health, plus Kibana screens for managing followers and auto-follow patterns.

flowchart TD
    U["Users on web and mobile"] --> G["API Gateway"]
    G --> LB["Load Balancer"]
    LB --> IS["Indexing Service"]
    LB --> SS["Search Service"]
    IS --> LC["Leader Cluster: coordinating node"]
    LC --> LP["Leader primary shards"]
    LP --> LR["Leader replica shards"]
    RC["Remote cluster connection with TLS and credentials"]
    FT["Follower shard tasks"] --> RC
    RC --> LP
    FT --> FP["Follower primary shards"]
    FP --> FR["Follower replica shards"]
    SS --> LC
    SS -. "nearby reads" .-> FC["Follower Cluster: coordinating node"]
    FC --> FP
    MON["Monitoring: lag, errors, stats"] --> FT
Figure 2: Components of a CCR setup. Follower shard tasks pull changes from leader primary shards over the remote cluster connection.

Reading the Diagram

Follow the top of the picture first. Users reach the gateway and load balancer, which hand requests to the indexing service and the search service. The indexing service sends writes to the leader cluster. There, a coordinating node routes each write to the right primary shard, which copies it to its replicas inside the same cluster.

Now look at the follower side. For each follower shard there is a background shard follow task. This task reads changes from the leader’s primary shard through the secured remote cluster connection. It then writes those changes into its own follower primary shard, which copies them to follower replicas. The search service can send reads to the follower cluster when the follower is closer to the user. Monitoring watches the follow tasks, since they carry the health of the entire copy.

i
Design Note

The follower index must have the same number of primary shards as the leader index, because each follower shard follows exactly one leader shard. The number of replica shards on the follower, however, is your own choice and can differ from the leader. Plan the shard count carefully at the start, since it cannot be changed later without rebuilding the follower.

6Internal Working: Life of One Write

Follow one document from the moment an application saves it to the moment it is searchable in the follower cluster.

sequenceDiagram
    participant App as Application
    participant LP as Leader primary shard
    participant LR as Leader replica shard
    participant FT as Follower shard task
    participant FP as Follower primary shard
    App->>LP: Index a document
    LP->>LP: Assign sequence number and store change
    LP->>LR: Copy change to replica
    LR-->>LP: Confirm
    LP-->>App: Write acknowledged
    FT->>LP: Ask for changes after my last checkpoint
    LP-->>FT: Send batch of changes
    FT->>FP: Apply changes with same sequence numbers
    FP-->>FT: Confirm and advance checkpoint
Figure 3: The write path. The application gets its answer before the follower is involved, which is why CCR is asynchronous.
1

The write arrives on the leader

An application sends a document to the leader cluster. The coordinating node routes it to the correct primary shard, based on the document’s identifier.

2

The primary numbers and stores the change

The primary shard gives the operation the next sequence number, writes it into its transaction log for safety, and applies it to the search data.

3

Replicas inside the leader cluster copy it

The primary forwards the operation to its replica shards. Once the in-sync copies have it, the global checkpoint can move forward and the application receives its confirmation.

4

The follower task asks for news

The follower’s shard task knows the last sequence number it has applied. It sends a read request to the leader shard asking for everything newer. If there is nothing new yet, the leader holds the request open for a while and answers as soon as something arrives, instead of making the follower ask again and again.

5

The leader returns a batch of operations

The leader shard reads the requested operations from its retained history and sends them in a batch. Only operations up to its global checkpoint are handed out, so the follower never receives changes that could still be rolled back.

6

The follower applies them in place

The follower’s primary shard applies the operations, keeping the same sequence numbers the leader gave them. It then copies them to its own replicas inside the follower cluster.

7

The change becomes searchable

After the next refresh on the follower, the document can be found by searches. Then the follower’s checkpoint moves forward, and the loop starts again from step 4.

Why the Leader Must Remember History

The leader can only hand out changes that it still remembers. Elasticsearch keeps this history using a feature called soft deletes. Normally, when a document is deleted or updated, the old version simply disappears at some point. With soft deletes, the shard keeps a lightweight record of recent operations, including deletions, so that followers can read them later.

To avoid keeping history forever, Elasticsearch uses retention leases. Each follower holds a lease on the leader shard that says “please keep everything after my checkpoint.” The follower renews the lease as it reads. If a follower disappears for too long, its lease expires, by default after about twelve hours, and the leader is free to discard old history.

Everyday Analogy

A shop keeps its old receipts for a limited time. A regional auditor can ask to see the receipts since their last visit, as long as they come back within the retention period. If the auditor is away too long, the shop may have thrown the oldest receipts away, and the auditor needs a full inspection instead of a quick update. A retention lease is the auditor’s signed note saying “please keep my receipts.”

!
Watch Out

If a follower is paused or cut off for longer than the retention period, it may no longer be able to catch up from history. In that case it has to be rebuilt from scratch. Long outages of the link or long pauses are therefore a planning matter, not just a nuisance.

7Bootstrapping: Making the First Copy

A new follower cannot begin by reading a live stream of changes, because it is missing everything that happened before. It first needs a full starting copy.

Two Phases: Copy, Then Follow

When you create a follower index for an existing leader index, two phases happen. In the first phase, called remote recovery, the follower copies the leader’s data files over the network, much as a replica shard would recover from its primary. This gives the follower a snapshot of the leader as it looked at one moment.

In the second phase, the follower switches to following. It asks for every operation that happened after the moment of the snapshot, applies them, and from then on keeps reading new operations continuously. Sequence numbers make the hand-off exact, so nothing is missed and nothing is applied twice.

Everyday Analogy

Suppose you want to keep a mirror copy of a shared notebook. First you photocopy all the existing pages, which takes a while. During the copying, the owner adds new pages. After finishing, you ask for every page numbered higher than the last one you copied. From then on you check for new pages regularly. Bootstrapping is the photocopying, and following is the regular checking.

The Cost of the First Copy

For a large index, the first phase can move hundreds of gigabytes or even terabytes across the network. This takes time and uses bandwidth on both sides. Elasticsearch limits the speed of this recovery by default, so it does not swamp the leader’s network or slow its normal work. You can tune that limit, but remember that a faster copy uses more of the leader’s resources at the same moment it is serving real users.

Phase 1
bulk copy of existing data files from the leader
Phase 2
continuous reading of new operations by sequence number
Hours
typical time budget to plan for the first copy of very large indices, depending on size and bandwidth

Planning the Start

Plan bootstrapping like a move to a new house. Schedule it for a quiet time, check that the network link has enough room, and make sure the follower cluster has enough disk space for the full copy plus its own replicas. Start with the most important indices first, and add others in waves rather than all at once. Watch the recovery progress and the leader’s load while it runs.

*
Practical Tip

For a first rollout, choose one small, non-critical index and follow it end to end. Practice creating it, watching its lag, pausing it and promoting it. The skills you build on a small index carry directly to the large ones.

8Setting Up CCR Step by Step

Setting up CCR follows a clear order. Each step depends on the one before it.

1

Check the license and versions

CCR is a feature of Elastic’s paid subscription levels, and both clusters must meet its license and version requirements. Confirm this first in Elastic’s current documentation, because it decides whether the project can start at all.

2

Open the network path

The follower cluster must be able to reach the leader cluster over the network on the port used for cluster-to-cluster communication. Work with your network team on firewall rules, private links or peering between the two locations.

3

Prepare security

Encrypt the connection with TLS and create credentials for the follower to use. Give those credentials only the permissions needed to read from the leader index, following the least privilege rule.

4

Register the remote cluster

On the follower cluster, save a remote cluster connection with a short nickname and the address of the leader cluster. Elasticsearch checks the connection and reports whether it is healthy.

5

Confirm the leader index is ready

The leader index must keep its operation history, which means soft deletes must be on. In current versions this is the default for new indices. Older indices that were created without it need to be rebuilt first.

6

Create the follower index

On the follower cluster, create a follower index by naming the remote cluster and the leader index. Elasticsearch performs the bootstrapping copy and then starts following.

7

Verify and monitor

Check that the document counts match, that the lag is small and that the follow tasks show no errors. Add these checks to your monitoring before you rely on the follower for anything important.

Two Ways to Create Followers

You can create followers by hand, one index at a time, which suits a few long-lived indices such as a product catalog. Or you can define an auto-follow pattern, which suits indices that appear over time, such as one new log index each day. We look at auto-follow in the next chapter.

Using Kibana or the API

Elasticsearch exposes all of these steps through its programming interface, so they can be automated with scripts and infrastructure tools. Kibana also has a management screen for cross-cluster replication, where you can register remote clusters, create followers and auto-follow patterns, and see their status. Small teams often start with the screen to learn the concepts, and move to automation once the design is stable.

!
Watch Out

Follower index names must be different from any existing index in the follower cluster, and the shard count must match the leader. Teams often trip on naming clashes when they replicate the same index names from several leader clusters into one follower cluster. Add a prefix or suffix that identifies the source.

9Auto-Follow Patterns and Data Streams

Many real systems create new indices all the time. Creating each follower by hand would be tiring and easy to forget, so Elasticsearch can do it for you.

The Problem With Rolling Indices

Log and metric systems usually write to a new index every day or whenever the current one gets big. A team might have indices named like “logs-web-2026.09.21”, then “logs-web-2026.09.22”, and so on. If you had to create a follower for each new index by hand, someone would eventually forget, and the follower cluster would silently miss a day of data. That gap would only be discovered during a disaster, the worst possible moment.

What an Auto-Follow Pattern Does

An auto-follow pattern is a standing instruction stored on the follower cluster. It names a remote cluster and one or more name patterns with wildcards, such as “logs-web-*”. Elasticsearch checks the leader cluster regularly. Whenever it finds a new index whose name matches, it creates a follower index for it, following all the usual steps automatically. You can also decide how follower indices should be named, for example by adding a prefix that shows where they came from.

Everyday Analogy

Think of a magazine subscription. You could walk to the shop every month and buy the newest issue by hand, and one busy month you would forget. Or you can subscribe once, and every new issue is delivered automatically. An auto-follow pattern is a subscription for new indices.

Match

Name Patterns

Wildcards choose which leader indices are followed. Precise patterns avoid copying test or scratch indices that nobody needs.

Naming

Follower Names

A prefix or suffix can be added to each follower name. This keeps indices from different leader clusters apart when they land in one follower cluster.

Control

Pause and Resume

A pattern can be paused during maintenance and resumed later. While paused, new leader indices are not followed.

Visibility

Auto-Follow Stats

Elasticsearch reports how many indices were followed successfully, how many failed and the most recent errors, which helps you notice trouble early.

Data Streams

A data stream is a newer Elasticsearch structure for time-based data such as logs and metrics. Behind one name, it manages a series of hidden backing indices and creates new ones as the old ones fill up. CCR can follow data streams through auto-follow patterns, which is the recommended way to replicate time-series data. New backing indices are picked up automatically, and the follower cluster ends up with a matching data stream that readers can query by its name.

*
Practical Tip

Write your auto-follow patterns with care. A pattern that is too broad may copy huge, unimportant indices and fill the follower’s disks. A pattern that is too narrow may skip something important. Review the list of followed indices after any new naming rule is introduced on the leader side.

!
Watch Out

Auto-follow only acts when it can reach the leader. If the link is down for a while, new leader indices may appear during the outage. Check after any outage that all expected followers exist, and use the auto-follow stats to see whether any attempts failed.

10Consistency and Ordering: What You Can Promise

Because CCR is asynchronous, the follower is always slightly behind. This chapter explains exactly what is guaranteed and what is not.

Eventually Consistent

A system is eventually consistent when copies may differ for a short time, but all copies will become the same once the changes have arrived. That is the promise of CCR. If writes to the leader stopped right now, the follower would soon hold exactly the same data.

In practice this means that a document indexed on the leader a second ago might not yet be visible in the follower. During that moment, a search on the follower gives an answer that was true a short while earlier. Most search use cases accept this happily, since a product added two seconds ago missing from a far-away search is rarely a problem.

Order Is Preserved Per Shard

Each follower shard applies operations from its leader shard in sequence-number order, so the story of each individual document is told in the right order. If a document was created, updated twice and then deleted, the follower sees those events in that order and ends in the same state as the leader.

Across different shards, however, the follower does not keep a global order. Shard 1 may be a little further ahead than shard 2. A search that touches both could briefly see a mix in which shard 1 has a change that shard 2 does not yet have. For related documents that live on different shards, this can produce a momentary inconsistency.

Everyday Analogy

Imagine five clerks each copying a numbered ledger by hand from head office. Each clerk copies the pages of their own ledger in order, so each ledger is always correct up to some page. But one clerk may be on page 90 while another is only on page 85. At any given second, the five ledgers together do not show the same moment in time, although each one is internally tidy.

What You Cannot Assume

  • No read-your-own-writes on the follower. An application that writes to the leader and immediately reads the same data from the follower may not find it yet. Read from the leader when freshness matters.
  • No zero data loss on sudden failure. Operations that reached the leader but had not yet been pulled by the follower are lost if the leader is destroyed at that instant.
  • No cross-shard snapshot of a single moment. Queries that combine data from several shards see each shard at its own point.
i
Design Note

Decide early which reads can tolerate slightly old data and which cannot. Browsing a product list can be served by a follower. Checking whether a limited item is still in stock during checkout should go to the leader.

Deletes and Updates

CCR copies everything that happens to a document, including updates and deletions. This is good for fidelity and bad for protection against mistakes. If an engineer accidentally deletes documents on the leader, the follower will mirror the deletion within moments. Keep this in mind whenever someone suggests that CCR “is our backup”.

11Use Case One: Disaster Recovery and Failover

The most common reason to use CCR is to survive the loss of an entire site. This chapter shows how a follower becomes the new primary system.

Two Numbers That Matter

Disaster planning uses two standard measures. The recovery point objective (RPO) is how much recent data you can afford to lose, measured in time. The recovery time objective (RTO) is how long you can afford to be without service. A business might say, “We can lose at most a minute of data and be down for at most thirty minutes.” CCR mainly helps with the RPO, by keeping the copy fresh, and helps with the RTO, by having the copy already loaded and ready.

Everyday Analogy

Every large hospital has a backup generator. The RPO is how much of the day’s records could be lost when the power dies, and the RTO is how many seconds pass before the lights come back. A generator that is always warm and ready gives a short RTO. A backup that must first be delivered and installed gives a long one. A follower cluster is a warm generator for your search.

The Failover Story

flowchart TD
    A["Normal: writes go to Leader cluster in Region A"] --> B["Region A fails"]
    B --> C["Decide to fail over"]
    C --> D["Pause following on Region B follower indices"]
    D --> E["Close each follower index"]
    E --> F["Unfollow to turn it into a normal index"]
    F --> G["Reopen the index in Region B"]
    G --> H["Redirect applications to Region B"]
    H --> I["Region B now accepts writes"]
Figure 4: Failing over. The follower is detached from its leader and promoted into an ordinary, writable index.
1

Detect and decide

Monitoring shows the leader region is unreachable, and the people in charge decide the outage is serious enough to fail over. This decision is usually made by a person or a carefully tested automation, because switching sites by mistake causes its own trouble.

2

Stop following

On the follower cluster, each follower index is paused. This stops the background tasks that keep trying to reach the failed leader.

3

Promote the follower

The follower index is closed, then converted into a regular index by an “unfollow” step, then reopened. It now has all the data it had received, and it accepts writes like any other index.

4

Redirect the traffic

Applications are pointed to the second cluster, often by changing a DNS name, a load balancer target or a setting in the gateway. From here on, Region B is the primary.

5

Rebuild for the return

When Region A is healthy again, its copy of the data is out of date and possibly different from Region B, since some final writes may never have been copied. The clean way back is to delete the old indices in Region A and create new followers there that follow Region B. After they catch up, you can fail back or simply keep Region B as primary.

!
Watch Out

Do not let applications write to both clusters after a partial failover. If some writers still reach the old leader while others reach the promoted follower, the two copies drift apart, and merging them later is painful. Make redirection complete and quick, and stop the old writers if you can.

Practice Makes the RTO

A disaster plan that has never been rehearsed is only a hope. Schedule a game day at least once or twice a year, in which you promote the followers, redirect a test application and measure how long each step takes. Write the steps in a short runbook, a plain checklist that someone half-awake at three in the morning can follow. Time the rehearsal, and use the real time as your RTO instead of a guess.

12Use Case Two: Data Locality and Faster Reads

A second, nearby copy can make searches faster for users far from the main site, and can help meet rules about where data may live.

Distance Costs Time

Network signals travel fast, but not instantly, and long routes across oceans pass through many devices. A search that takes 20 milliseconds inside a data center can take several times longer when the user is on another continent, before we even count the time for the response to come back. Multiply that by the several requests a single web page might make, and the delay becomes something users feel.

By placing a follower cluster in each major region, searches from that region are answered close to home. Writes still go to the leader, and reads are served locally from the follower. Because most search applications read far more than they write, this sharply improves the everyday experience.

Everyday Analogy

A famous library has one original reference book. Instead of making people from every city travel to read it, the library sends photocopies to branch libraries in each city. The photocopies may be a few days behind the newest edition, but people can read them down the street. Followers are the branch copies.

Example: A Travel Booking Site

A booking site keeps its hotel catalog in a leader cluster in one region. Follower clusters in Europe, Asia and North America hold copies of the catalog index. Each region’s search service reads from its nearest follower, so a traveler in Singapore gets results quickly. Price changes and room availability that must be exact are checked against the leader at checkout, while browsing uses the local copy.

Rules About Location

Some regions require certain data to be stored or served inside their borders. A follower cluster in that region can hold a compliant local copy for serving users there. Whether a copy is allowed at all depends on the rules for your data and your industry. Speak with your legal and compliance teams before replicating personal or regulated data across borders, because the same feature that solves a speed problem can create a legal one.

!
Watch Out

Replicating data to another country is a decision about law and privacy, not only about performance. Never enable it for personal data until the right people have approved where that data may be stored and who may access it.

Use the Right Node Sizes

Follower clusters serving reads need enough memory and processing power for search traffic, but they do not need to handle heavy indexing from applications, since the only writes they perform are the replicated operations. This can allow slightly different hardware choices from the leader, tuned to reading. Many teams also choose fewer replicas on the follower in regions with lower traffic, to save cost.

13Use Case Three: Centralized Reporting

Sometimes the problem is the opposite of scattering data to many places. You have many small clusters and want everything in one.

Hub and Spoke

Imagine a retail chain with hundreds of stores, each running a small Elasticsearch cluster for local point-of-sale search and analysis. Head office wants to see sales across all stores without visiting each cluster one by one. With CCR, every store cluster acts as a leader for its own indices, and one large central cluster follows all of them. The central cluster becomes a complete, up-to-date reporting hub.

flowchart LR
    S1["Store cluster 1: leader"] --> H["Central reporting cluster: many follower indices"]
    S2["Store cluster 2: leader"] --> H
    S3["Store cluster 3: leader"] --> H
    S4["Store cluster N: leader"] --> H
    H --> D["Dashboards and analysts"]
Figure 5: Hub and spoke. Many leaders feed one follower cluster, which serves reporting for the whole organization.

A single follower cluster can follow indices from many leader clusters at once. Each follower index stands for one leader index, so you will have many follower indices in the hub, one per source index. Naming becomes important here. If every store names its sales index the same way, the hub needs a naming rule such as adding the store code as a prefix, so that names do not collide.

Everyday Analogy

Think of a national weather agency. Every local station records its own readings. Instead of asking each station by phone whenever someone wants the national picture, all stations send their data to a central office, where one team can chart the whole country. The stations stay small and simple, and the central office does the heavy analysis.

Why This Beats Visiting Each Cluster

Elasticsearch can also search many clusters at once through a different feature called cross-cluster search. That works well for occasional questions. For heavy, frequent reporting, having the data already in one place is faster, because the queries run locally in the hub without waiting for hundreds of remote clusters to answer. It also keeps reports working even if a store’s cluster is temporarily offline, since the hub already holds the latest copy.

Capacity for the Hub

The hub cluster must be able to store every followed index. Add up all the leader indices’ sizes, then the hub’s own replica shards, then room to grow. Many teams underestimate this and discover disk pressure after a few months. Combine storage planning with a retention plan, so that old data leaves the hub on a schedule.

*
Practical Tip

When following many sources, stagger the first copies. Bootstrapping fifty leaders at once creates a huge wave of network and disk activity on the hub. Start with a few, confirm that everything is healthy, and then add more in batches.

14Active-Active and Bi-Directional Patterns

So far, data flows one way for each index. Some organizations want both sites to accept writes. CCR can support this, but only with a careful design.

One Index Cannot Have Two Leaders

A follower index is read-only, and an index can only follow one leader. That means you cannot have two clusters both writing to the same index and copying to each other. If both sides could change the same document at the same time, there would be no rule to decide whose version wins, and the two copies would disagree.

The Prefix Pattern

The safe design gives each cluster its own writable index, and each cluster follows the other’s index. For example, the European cluster writes to an index whose name has a “eu” prefix. The American cluster writes to an index with a “us” prefix. Each cluster also holds a follower of the other’s index. A search that needs everything reads across both the local writable index and the follower index, usually through a shared alias or a wildcard.

Everyday Analogy

Two regional offices share a filing cabinet system. Each office writes only in its own colored folders, and copies the other office’s folders into a read-only shelf. Anyone in either office can read all the folders, and nobody ever edits the same page as someone else. Conflicts never happen, because each page has exactly one author.

ClusterWrites ToFollowsReads From
EuropeIts own index for European dataThe American indexIts own index and the follower of the American index
AmericasIts own index for American dataThe European indexIts own index and the follower of the European index

Where This Works Well

This pattern suits data with a natural owner, such as logs generated by each region, orders placed in each region, or sensor data from local sites. Each record is created in exactly one place, so nothing needs merging. It is less suited to shared records that anyone might edit, such as a single global customer profile, because those need a different design.

!
Watch Out

An active-active design multiplies the things that can go wrong: two writers, two followers, two sets of aliases and two failure modes. Use it only when you truly need writes in both regions. Many teams find that a simple active-passive setup meets their needs with far less complexity.

15CCR and Index Lifecycle Management

Data does not live forever. Elasticsearch has a feature that moves indices through stages as they age, and it interacts with CCR in useful ways.

A Quick Look at Index Lifecycle Management

Index Lifecycle Management, or ILM, is a feature that automatically moves indices through stages: hot for fresh, busy data, then warm, cold and finally deletion as data ages. It can also start a new index when the current one gets too large, an action called rollover. ILM saves money and effort by keeping recent data on fast storage and pushing old data to cheaper storage.

Leader and Follower Have Their Own Lives

The leader and the follower each live in their own cluster, and each can have its own lifecycle policy. The leader’s policy might roll over daily and delete after thirty days. The follower’s policy could keep data longer, for example ninety days, if the follower cluster is used for reporting and has cheap storage available. Retention on the follower is your own choice.

Rollover is the special case. Only the leader can roll over, because a follower is read-only. The follower gets its new indices by following the leader’s new indices, either through an auto-follow pattern or a data stream. ILM has a specific step designed for follower indices, called unfollow, which detaches a follower from its leader once the leader has finished with it. This turns an old, finished index into an ordinary index that can be moved to colder storage or deleted in the normal way.

Everyday Analogy

Consider a newspaper archive. The head office decides when to close an issue and start a new one. Branch offices receive each closed issue and then decide for themselves how long to keep it on the shelf before moving it to the basement or throwing it out. The head office does not tell them what to do with their copies.

*
Practical Tip

Make sure the leader does not delete an index before the follower has finished copying it. If your leader retention is very short and the link is slow or down, data could vanish from the leader before the follower ever received it. Keep leader retention comfortably longer than your worst expected outage.

16CCR Versus Other Ways to Move or Protect Data

Elasticsearch offers several ways to reach across clusters or to protect data. Each solves a different problem, and confusing them is a common mistake.

ApproachWhat It DoesBest ForMain Limit
Cross-Cluster ReplicationContinuously copies indices to another clusterDisaster recovery, nearby reads, central reportingFollower lags slightly, is read-only and copies mistakes too
Cross-Cluster SearchRuns one search across several clusters and merges the results, without copying dataOccasional searches over data that should stay where it isEach search depends on remote clusters being reachable and fast
SnapshotsSaves a point-in-time backup to storage such as an object storeProtection against deletion, corruption and long-term archivingRestoring takes time, and data since the last snapshot is missing
Stretched clusterOne cluster with nodes in several locationsOnly very close, low-latency sites such as nearby zonesUnsafe across long distances because of delay and split-brain risk
Dual writingThe application or a pipeline sends every write to two clustersCustom needs, such as different transformations per clusterYou must handle failures and mismatches yourself

Cross-Cluster Search in One Paragraph

Cross-cluster search leaves data where it is and travels to it. A search request goes to a local cluster, which forwards it to remote clusters, gathers their answers, and merges them into one result list. It is like phoning several offices to ask the same question. CCR, by contrast, copies the data so that the answer is already at hand. The two are often used together: CCR brings key indices close, and cross-cluster search reaches the rest.

Snapshots and CCR Are Partners

Snapshots and CCR protect against different dangers. CCR protects against the loss of a place, and it does so with small data loss and fast recovery. Snapshots protect against the loss of correctness, such as accidental deletion or corrupted data, because a snapshot can be taken before the mistake and restored afterward. A serious system uses both.

Everyday Analogy

CCR is like a live video feed to a second control room. If the first room floods, the second is already watching everything. A snapshot is like a photo album taken every night. If someone scribbles over the live records, the video feed shows the scribbles too, but the photo album still holds a clean earlier page.

*
How to Decide

Ask three questions. Do I need a live copy elsewhere? Use CCR. Do I only need to search data that lives elsewhere, occasionally? Use cross-cluster search. Do I need to undo mistakes or keep long-term history? Use snapshots. Most production systems answer yes to more than one.

17Performance and Scalability

CCR adds work to both clusters and uses the network between them. Knowing where that work comes from lets you keep it under control.

Where the Work Happens

On the leader side, the extra work is answering read requests from followers and keeping the history of recent operations. On the follower side, the work is applying those operations as if they were normal writes, then copying them to follower replicas. Between them, the network carries every change. If your leader handles a heavy stream of writes, the follower must be able to absorb the same stream.

Lever 1

Network Bandwidth

The link must carry the leader’s write volume, plus the initial copy. Undersized links are the number one cause of growing lag.

Lever 2

Read Batch Size

Followers can pull more operations per request. Bigger batches reduce round trips, but use more memory and can slow individual requests.

Lever 3

Parallel Reads

A follower shard can keep several read requests in flight at once. More parallelism helps on high-latency links, up to a point.

Lever 4

Follower Write Capacity

The follower must index operations as fast as they arrive. Fast disks and enough processing power keep it from falling behind.

Lever 5

Shard Layout

Each leader shard is followed by one follower shard, so a hot leader shard can become a bottleneck. Balanced shards spread the load.

Lever 6

Recovery Throttle

The speed limit on the initial bulk copy protects the leader. Raise it only when the leader has spare capacity.

Why Latency Matters as Much as Bandwidth

Bandwidth is how much data the link can carry at once. Latency is how long a round trip takes. A follower that reads one small batch, waits for the answer and then asks again will be limited by latency, even on a very fast link. This is why followers can keep several requests in flight and read bigger batches. Think of moving water with buckets between two villages: a wide road helps, but so does sending many carriers at the same time so nobody stands waiting.

Scaling Out

CCR scales with the number of shards. More shards mean more follow tasks working in parallel, each pulling from its own leader shard. A very busy index with many shards can therefore replicate faster than a single-shard index that funnels all changes through one task. At the same time, too many tiny shards waste memory and add overhead, so keep shard sizes in a healthy range.

!
Watch Out

Traffic spikes on the leader show up as lag on the follower a moment later. Test CCR under your busiest expected write load, not under average load. A follower that keeps up at noon on a quiet Tuesday may fall behind on the biggest sale day of the year.

Example: Fixing Growing Lag

A team notices that lag on a busy log index grows during evening peaks and only shrinks overnight. Investigation shows that the index has a single hot shard, which limits parallel copying, and that the link between regions is close to full. They split the index into more shards for new daily indices and upgrade the link. Within a week, lag stays under a few seconds even at peak.

18Data Flow and the Life of a Follower Index

A follower index moves through clear stages from birth to retirement or promotion. Knowing these stages makes operations far less mysterious.

flowchart LR
    C["Created"] --> R["Remote recovery: bulk copy"]
    R --> F["Following: reading new operations"]
    F --> P["Paused"]
    P --> F
    F --> U["Unfollowed: becomes a normal index"]
    P --> U
    F --> X["Lagging or failing: needs attention"]
    X --> F
    X --> RB["Rebuilt from scratch"]
    RB --> C
Figure 6: The life of a follower index. Most of its life is spent in the following state.
State 1

Created

The follower index is registered and linked to a leader index. No data has been copied yet.

State 2

Remote Recovery

The initial bulk copy of the leader’s data files. Searches on the follower may show incomplete data until this finishes.

State 3

Following

The normal state. Follow tasks read operations continuously, and the follower stays close to the leader.

State 4

Paused

Following is stopped on purpose, for maintenance or a failover step. The lease on the leader keeps history for a limited time only.

State 5

Unfollowed

The link to the leader is cut. The index becomes ordinary, writable and independent. This is what promotion means.

State 6

Failing or Lagging

An error such as a broken link or missing history has stopped progress. Someone needs to look at the reason and repair it.

The Data That Travels

The follower receives more than documents. When the leader’s mappings change, meaning the rules that describe the fields in the index, the follower picks up those changes as well, so both sides keep matching structures. Many index settings are also copied, so that the follower behaves like the leader. Settings that describe the follower’s own physical situation, such as the number of replica shards or where shards are placed, stay under the follower cluster’s control.

Everyday Analogy

Consider an apprentice who copies a master baker’s recipe book. The apprentice copies the recipes and any new notes, so the books match. But the apprentice’s own kitchen has its own oven, shelves and staff, so the apprentice decides where ingredients are stored. The recipes follow the master. The kitchen setup is local.

i
Design Note

Some index settings can only be changed while an index is closed. If a setting on the leader requires a closed index, the follower may need to be paused and closed briefly to accept it. Read the release notes for your version before planning changes to settings on followed indices.

19A Worked Example, Start to Finish

Concepts are easier to remember with a story. Here is a small made-up example that shows planning, lag and failover in numbers.

The Setting

An online marketplace runs its main Elasticsearch cluster in one region, which we will call Region A. It holds a product index of 400 gigabytes, and it receives about 800 changes per second on average, with peaks near 3,000 per second during sales. The business asks for a disaster recovery copy in Region B, with at most one minute of data loss and at most thirty minutes without service. All figures here are invented to illustrate the reasoning and are not from a real company.

Planning the Numbers

QuestionPlanReason
How big is the follower cluster?Disk for 400 GB, plus one replica copy, plus growth roomThe follower holds a full copy and its own replicas, so storage needs are roughly double the leader’s primary data
How long is the first copy?Plan for several hours on a link that can spare about 200 megabits per second400 GB is about 3,200 gigabits, and even at full speed the copy needs time, so it is throttled to spare the leader
What lag do we expect?A few seconds normally, and up to about 20 seconds at peakThe follower must absorb 3,000 operations per second during sales, so it is sized for the peak, not the average
Does it meet the one-minute RPO?Yes, if peak lag stays under 60 seconds, with an alert at 30 secondsIf the leader vanished at the worst moment, we would lose up to the lag, which is inside the goal
Does it meet the 30-minute RTO?Yes, if the runbook is rehearsed and takes under 15 minutesPromotion involves pausing, closing, unfollowing, reopening and redirecting traffic, which is quick for a few indices

A Day in the Life

At first, the team creates one follower index for the product catalog and watches it for a week. Lag stays around two seconds. During a big sale, lag rises to 18 seconds for a few minutes, then falls back. The team sets an alert at 30 seconds and moves on.

Three months later, a power failure takes Region A offline at 14:05. Monitoring shows the leader unreachable at 14:06. At 14:10 the incident lead decides to fail over. The team follows the runbook: pause the follower indices, close them, unfollow, reopen, and change the gateway so that applications point to Region B. By 14:22 the site accepts orders again. The last recorded lag before the failure was 4 seconds, so at most about four seconds of changes were lost, well inside the one-minute goal.

When Region A returns hours later, the team does not try to reconnect its old indices. They delete the stale ones and create new followers in Region A that follow Region B. After they catch up, the business decides to keep Region B as the primary for now and use Region A as the new standby.

i
Lesson From the Example

The goals were set first, and the design followed from them. The team sized for the peak, alerted early, rehearsed the runbook and never tried to merge the old data back in. These habits matter more than any single setting.

20Security

Two clusters joined by CCR share data over a network that may cross the public internet or a provider’s backbone. Security is not optional here.

Protect the Link

All traffic between clusters must be encrypted using TLS, the same technology that puts the padlock in a web browser. Encryption stops anyone on the path from reading or changing data in transit. Where possible, also keep the link off the public internet by using private connections, peering or virtual private networks, and limit network rules so that only the follower’s addresses can reach the leader’s cluster port.

Prove Who You Are

The follower cluster must show credentials to the leader. Depending on your version and setup, these can be certificates, or newer cross-cluster API keys that are limited to specific access on the leader. In both cases, treat the credentials like keys to a safe. Store them in a secrets manager, rotate them on a schedule, and remove them when a cluster is retired.

Give Only the Permissions Needed

Elasticsearch roles have special permissions for CCR. On the leader side, the credentials need only enough rights to read the leader index and its changes. On the follower side, the person or automation creating followers needs rights to manage follower indices and to write into them. The follower does not need any ability to change the leader, and it should not have one. This is the principle of least privilege: every person or program receives no more access than its job requires.

Everyday Analogy

A courier who collects documents from your office needs a key to the pickup room, not a key to the vault. If the courier’s key is lost, the damage is limited to what was in the pickup room. Cross-cluster credentials should be the pickup-room key.

Data Follows the Rules

A follower holds a complete copy of the leader’s data, including sensitive fields. Any access control based on documents or fields in the leader does not automatically apply to the follower unless you define roles on the follower cluster too. Set up users, roles and audit logging on the follower with the same care as on the leader. Encrypt disks in both clusters, and remember that a second copy of sensitive data doubles the places where it must be protected.

!
Watch Out

It is easy to secure the primary cluster carefully and then leave the disaster recovery cluster with weaker settings because “nobody uses it”. After a failover, that cluster becomes the production system overnight. Both sides deserve the same security standard from the start.

21High Availability, Failure Scenarios and Recovery

CCR exists to help when things fail, and CCR itself can also fail. Plan for both.

What CCR Does When the Link Breaks

If the network between clusters drops, follow tasks cannot read from the leader. They do not give up at once. They retry with growing pauses between attempts, and they resume from their last checkpoint when the link returns. Since the leader keeps history under retention leases, a short outage is repaired automatically, with lag growing during the outage and shrinking after it.

ScenarioWhat You SeeLikely CauseWhat To Do
Short network outageLag grows, then falls after the link returnsLink failure or maintenanceUsually nothing; verify that lag returns to normal
Long outage beyond retentionFollower cannot resume and reports missing historyRetention lease expired on the leaderRebuild the follower from scratch and consider longer retention
Follower falls behind at peakLag climbs during busy hoursNot enough follower write capacity or bandwidthAdd capacity, tune batch sizes or add shards for new indices
Leader cluster lostFollow tasks fail with connection errorsSite disasterFail over by promoting followers and redirecting traffic
Follower cluster lostNo follower to serve nearby reads or hold the copySite disaster on the follower sideLeader keeps working; rebuild the follower cluster and recreate followers
Credentials expiredFollow tasks fail with authorization errorsRotated or expired secretsUpdate credentials on the remote cluster settings and resume
Leader index deleted by mistakeFollower index follows the deletion or stops with errorsHuman errorRestore from a snapshot; CCR cannot undo this

Split-Brain Thinking for Two Sites

Suppose the link between regions breaks, but both regions are healthy. Each side may believe the other has failed. If your automation promotes the follower while the leader is still alive and receiving writes, you now have two writable copies that will drift apart. This is a split-brain situation. Guard against it by making failover a deliberate decision, confirming from an independent viewpoint that the leader is really down, and making sure old writers cannot keep writing after the switch.

Everyday Analogy

Two managers run branches of a shop. If the phone line between them dies, each might assume the other has closed and start ordering stock for the whole company. Both mean well, and the company ends up with double the orders. A clear rule such as “only the regional director may declare a branch closed” prevents that confusion.

*
Practical Tip

Write your failover decision rules down before you need them: who decides, which signals count, how long to wait, and what to check. A calm rule written in advance beats a panicked debate during an outage.

22Monitoring, Logging and Metrics

A follower that silently stops following looks healthy until the day you need it. Good monitoring makes silent failures loud.

The Numbers That Matter

Lag

Operations Behind

The difference between the leader’s global checkpoint and the follower’s. This is the single most important number, because it tells you how much you could lose.

Freshness

Time Since Last Read

How long ago each follow task last read from the leader. A steadily growing value means the follower has stopped receiving news.

Errors

Failed Reads and Writes

Counts of failed read requests from the leader and failed writes on the follower, with the most recent error message.

Throughput

Operations Per Second

How many operations the follower is applying. Compare it with the leader’s write rate to see whether the follower can keep up.

Coverage

Followers Versus Leaders

The count of leader indices that should be followed against those that actually are. A gap means an auto-follow pattern is missing something.

Connection

Remote Cluster Health

Whether the follower cluster can currently connect to the leader cluster, and how many connections are alive.

Where to Find Them

Elasticsearch offers statistics for individual follower indices and for the whole cluster, showing checkpoints, read and write counts, and recent exceptions. Kibana displays cross-cluster replication status in its management screens, and stack monitoring can collect metrics over time so you can chart them. Export the key numbers to the same dashboards and alerting tools that your team already uses, so CCR alerts arrive in the same place as every other alert.

Alerts That Help

  • Lag above your RPO fraction. If the RPO is 60 seconds, alert at 30, so you can act before the goal is broken.
  • Time since last read is climbing. This catches a stuck follower even when lag looks small because the leader is quiet.
  • Any fatal error on a follow task. A fatal error means the follower has stopped and will not recover on its own.
  • Missing followers. Compare the list of leader indices that match your patterns to the list of followers.
  • Remote connection down. This gives the earliest warning of link or credential problems.
!
Watch Out

Lag measured in operations can look tiny during a quiet night, even when the follower is stuck, because there is nothing to copy. Always watch the freshness of the follower’s last successful read together with the lag, so a stalled link cannot hide behind a quiet leader.

Logs and Audit Trails

Keep cluster logs for both sides, and record who created, paused, promoted or removed followers. During an incident review, these records answer questions like “who promoted the follower, and when?” Good records also help detect mistakes, such as a follower being paused for maintenance and forgotten for weeks.

23Advantages, Disadvantages and Trade-offs

CCR solves real problems, and it also brings real costs. A fair look at both sides helps you decide when it is worth using.

Advantages

  • Keeps a live copy in another cluster, protecting against loss of a whole site
  • Recovery is fast, because the copy is already loaded and searchable
  • Brings data close to users in other regions, so reads are quicker
  • Lets many small clusters feed one central reporting cluster
  • Built into Elasticsearch, with no extra pipeline software to run
  • Pull-based design protects the leader and copes with link problems
  • Copies mappings and many settings automatically, so structures stay in sync

Disadvantages

  • Asynchronous, so a sudden disaster can lose the last moments of data
  • Follower is read-only, so it cannot accept writes until it is promoted
  • Copies mistakes and deletions too, so it is not a backup
  • Needs a paid Elastic subscription level
  • Doubles storage and adds network and cross-region data transfer costs
  • Follower shard count must match the leader’s, which limits later changes
  • Failback after a disaster needs a careful rebuild

Key Trade-offs to Decide

Freshness versus cost. A faster link, more follower capacity and tighter monitoring keep lag lower, but each adds cost. Decide how fresh the copy must be and buy only that much.

Simplicity versus flexibility. One leader with one follower is easy to run and to explain. Hub-and-spoke and active-active patterns unlock more uses but need stronger discipline in naming, aliases and runbooks.

Speed of recovery versus risk of a wrong switch. Automated failover reduces the RTO, but it raises the risk of a mistaken failover or a split-brain. Manual failover is slower, but gives people a chance to confirm. Many teams choose semi-automated steps with a human decision at the start.

Local reads versus fresh reads. Serving users from a nearby follower gives speed, but with slightly old data. Serving everyone from the leader gives fresh data, but with more distance. Use each where it fits.

24Design Patterns and Anti-Patterns

Teams that have run CCR in production repeat the same good habits, and the same avoidable mistakes.

Patterns That Work

Pattern 1

Warm Standby

A follower cluster in another region receives everything and stays ready. Applications use it only after a failover, or for read-only reporting in the meantime.

Pattern 2

Regional Read Replicas

Followers in each major region serve local searches, while all writes go to one leader. Sensitive reads that need full freshness go to the leader.

Pattern 3

Hub and Spoke Reporting

Many source clusters feed one central cluster through prefixed follower names, giving analysts one place to look.

Pattern 4

CCR Plus Snapshots

A follower protects against site loss, and regular snapshots protect against mistakes and corruption. Together they cover both dangers.

Anti-Patterns to Avoid

ANTI-PATTERN 01Avoid
What people do

They treat CCR as their only backup.

Why it hurts

An accidental deletion, a bad script or corrupted data on the leader is copied to the follower within seconds. When the mistake is discovered, both copies are ruined.

Better approach

Use CCR for availability and locality, and keep regular snapshots stored separately for recovery from mistakes.

ANTI-PATTERN 02Avoid
What people do

They stretch one cluster across distant regions instead of using two clusters with CCR.

Why it hurts

High latency and network wobbles hurt cluster stability, and a broken link can leave the cluster unable to elect a leader node or protect its data safely.

Better approach

Keep each cluster inside a low-latency location and connect clusters with CCR.

ANTI-PATTERN 03Avoid
What people do

They set up CCR once and never test a failover.

Why it hurts

On the day of the disaster, the team finds missing permissions, unknown steps, forgotten indices or applications that cannot switch. The RTO turns out to be hours, not minutes.

Better approach

Rehearse failover and failback regularly, and keep the runbook current with what you learn.

ANTI-PATTERN 04Avoid
What people do

They allow writes to both the old leader and the promoted follower after a failover.

Why it hurts

The two copies diverge and cannot be merged automatically. Some data will be lost or need manual reconstruction.

Better approach

Make redirection complete, stop old writers and always rebuild the old leader as a fresh follower of the new primary.

ANTI-PATTERN 05Avoid
What people do

They never monitor lag and only check that “the follower exists”.

Why it hurts

A follower can exist and still be days behind because of a stuck task or expired credentials. The RPO is broken without anyone knowing.

Better approach

Alert on lag, on freshness of the last read and on follow task errors, not just on the existence of follower indices.

25Best Practices and Common Mistakes

A short list of habits that separate smooth CCR deployments from painful ones.

1

Write down the goals first

Set the RPO, the RTO and the purpose of each follower before choosing settings. Every later decision should trace back to those goals.

2

Decide the shard count carefully

Follower shards must match leader shards. Choose a shard layout with growth in mind, because changing it later means rebuilding.

3

Size for peak, not average

Link bandwidth, follower disks and processing power should handle the busiest hour with room to spare.

4

Automate followers with patterns

Use auto-follow patterns for rolling indices and data streams, and review the list of followers regularly.

5

Secure both sides equally

Encrypt the link, use least-privilege credentials, rotate secrets and apply the same access rules on the follower cluster as on the leader.

6

Rehearse and document

Practice failover and failback, time the steps, and keep a runbook that a tired person can follow.

Common Mistakes

  • Forgetting the license. Discovering late that the required subscription level is missing can stall an entire project.
  • Underestimating storage. The follower holds a full copy plus its own replicas, and grows as the leader grows.
  • Long pauses. Leaving a follower paused past the retention period forces a full rebuild.
  • Name collisions. Copying similarly named indices from several leaders into one follower cluster without prefixes.
  • Testing only the happy path. Trying CCR when everything is healthy, but never during a broken link or expired credential.
  • Ignoring application changes. Failing to plan how applications will find the promoted cluster, for example through DNS or gateway settings.

26Day-Two Operations: Living With CCR

Setting CCR up is one day of work. Running it for years is the rest. These are the routine tasks that keep a setup healthy.

Pausing and Resuming

Sometimes you need to stop following on purpose, for example during maintenance on the follower cluster or the network. Pausing a follower stops its read tasks but keeps its data. Resuming makes it continue from its last checkpoint. The important rule is to keep pauses short compared with the leader’s retention period. A pause of a few hours is usually harmless. A pause of several days may force a full rebuild.

Changing Mappings and Settings

When a new field is added on the leader, the follower learns about the new field automatically as part of following. You should not change the structure of a follower by hand while it is following, because the leader is the single source of truth. If you need a different structure, change it on the leader, or create a separate index on the follower side from the copied data.

Big structural changes, such as splitting an index into more shards, cannot be applied to a follower in place. The usual approach is to create a new leader index with the new layout, copy data into it and create a new follower for it. Plan these moves during quiet times, and keep the old pair running until the new pair is caught up and verified.

Rotating Credentials

Secrets should not live forever. When you rotate the credentials that the follower uses to reach the leader, update the remote cluster settings on the follower before the old credentials stop working. Check the connection afterward and confirm that lag stays low. Doing this on a schedule turns a risky emergency into a boring routine.

Everyday Analogy

Changing the lock on a shared storage room works best when you give the new key to everyone before you swap the lock. If you change the lock first, the couriers arrive to a door that will not open. Rotating credentials follows the same order: new key first, then retire the old one.

Upgrading

Cluster upgrades need special care because leader and follower must remain compatible with each other during the process. Read the upgrade notes for your versions, test in a copy of your setup, and upgrade in the order that Elastic’s documentation recommends. After each step, verify that following continues and that lag is normal. Keep the runbook for failover handy in case an upgrade goes wrong on one side.

Cleaning Up

Followers that nobody needs any more waste disk and network. Review the list of followers regularly. Remove those for retired indices, and unfollow those that are finished. Also review auto-follow patterns: an old pattern that still matches a busy naming scheme can fill the follower cluster with data that no longer has a purpose.

*
Practical Tip

Keep a short calendar of recurring CCR chores: credential rotation, failover rehearsal, follower review and upgrade planning. Tasks that are written down and scheduled actually get done, while tasks that live only in someone’s memory tend to be forgotten until they cause an incident.

27A Simple Troubleshooting Guide

When something looks wrong, a calm, ordered check finds the cause faster than guessing. Work from the outside in.

1

Is the link alive?

Check whether the follower cluster reports that its remote cluster connection to the leader is connected. If not, look at firewalls, routes, certificates and expired credentials before anything else.

2

Are the follow tasks running?

Look at the follower statistics. Each shard should have an active task. A task with a fatal error has stopped for good and needs a person to look at the message.

3

Is the lag growing or stable?

Compare the leader’s and follower’s checkpoints over a few minutes. A growing gap means the follower cannot keep up. A stable gap that is not shrinking may mean the follower is stuck.

4

Is the follower healthy on its own?

Check disk space, memory pressure, cluster health and shard allocation on the follower. A follower that cannot write because a disk is full will fall behind no matter how good the link is.

5

Is the leader still keeping history?

Confirm that soft deletes and retention leases are working for the leader index. If history was lost, the follower must be rebuilt.

6

Did anything change recently?

Recent upgrades, credential rotations, network changes, new auto-follow patterns or mapping changes are the most common triggers. Ask what changed before the trouble began.

Common Symptoms and First Guesses

SymptomFirst Thing To Check
Follower document count is lower than the leader’sWhether following is still running and whether lag is growing
Follower creation fails at the startWhether soft deletes are on for the leader index and whether the follower name already exists
Auto-follow did not create a followerWhether the name matches the pattern and whether the pattern is paused or has recent errors
Authorization errors on the follower tasksExpired or rotated credentials and missing permissions
Initial copy is very slowThe recovery speed limit, link throughput and load on the leader
Searches on the follower miss recent dataCurrent lag and the refresh interval on the follower
i
A Good Habit

Write the troubleshooting order into your runbook, and add every new lesson to it after each incident. Over time, this short document becomes the most valuable page in the whole CCR setup.

28Deployment and Cloud Considerations

Where the clusters run, and how they are connected, shapes cost, speed and risk.

Choosing Where Clusters Live

Clusters can run in your own data centers, on virtual machines in a public cloud, in containers managed by Kubernetes, or on a managed service such as Elastic Cloud. A common design places the leader in one cloud region or data center and the follower in another, far enough apart that one event cannot damage both. Choose locations with different power, network and natural risks, not just different rooms in the same building.

Connecting the Clusters

The network design has three choices. The clusters can talk over the public internet with strong encryption, over private links offered by the cloud or a telecom provider, or over a virtual private network. Private links give steadier speed and lower risk, but they cost more and take time to arrange. Whatever you choose, test the actual throughput and delay between the two sites before designing around them.

2+
separate locations with independent power and network
TLS
encryption required on every cross-cluster link
2x
rough storage footprint of a follower with its own replicas

Counting the Cost

CCR costs money in four places. First, the follower cluster needs its own servers and disks. Second, the network link may have a fixed fee. Third, many cloud providers charge for data moved between regions, and a busy index can move a lot of data over a month. Fourth, people need time to build, test and maintain the setup. Estimate all four before deciding, and compare them with the cost of an outage for your business.

Environments and Releases

Practice in a test environment that mirrors production: two small clusters, the same security settings and realistic sample data. Use it to test upgrades, credential rotation and failover. Elasticsearch versions between leader and follower must be compatible, so read the compatibility rules in Elastic’s documentation before planning upgrades. Usually, you upgrade in an order that keeps the two clusters compatible at every step, rather than upgrading both at once.

Example: A Quiet Upgrade Weekend

A team plans to upgrade both clusters to a newer version. They first read the compatibility rules, upgrade the follower cluster in the test environment, and confirm that following continues to work. On the production weekend, they upgrade in the tested order, check lag after each step, and finish with a failover rehearsal on the follower. Because the steps were practiced, the upgrade happens without any customer-visible impact.

i
Licensing Note

Cross-cluster replication is available with Elastic’s paid subscription levels, and the exact tier and version requirements have changed over time. Confirm both in Elastic’s current documentation before you commit to a design.

29Real-World Scenarios

The stories below are illustrative composites that show how organizations of different kinds use CCR. They are not accounts of specific companies.

A Global Online Retailer

A retailer with customers on three continents keeps its product catalog in one leader cluster. Follower clusters in each region serve local search, so a shopper in Sydney sees results as quickly as one in Amsterdam. Cart and stock checks go to the leader, which keeps them exact. During a regional outage, traffic can be moved to another region’s follower with only a short interruption.

A Bank Meeting Resilience Rules

A bank uses Elasticsearch for searching transaction histories and fraud alerts. Regulators ask it to prove that it can keep services running after the loss of a data center. It runs a follower cluster in a second data center, rehearses failover twice a year, and keeps signed records of the results. Snapshots stored separately protect it against mistakes and tampering.

An Energy Company With Field Sites

A company operating wind farms places a small cluster at each site to collect sensor readings, even when the connection to head office is poor. Each site is a leader. A central hub cluster follows all of them, giving engineers one dashboard for the whole fleet. When a site loses connectivity, its local cluster keeps working and the hub catches up once the link returns.

A Software Provider With Data Residency Rules

A software company serves customers in several regions, some of whom require that their data stays in their own region. It keeps separate leader indices per region and uses followers only for data that its customers have allowed to be replicated. This gives faster access for approved data while respecting local rules for everything else.

A Media Publisher During Big News

A news site sees a huge spike in searches during major events. Followers in several regions absorb most of the read traffic, keeping the leader free to accept new articles and edits. If the main region has trouble, editors continue from a promoted follower.

30Common Myths and Misunderstandings

Half-true ideas about CCR cause wasted effort and bad surprises. Here are the ones that appear most often.

Myth 1

“CCR is a backup.”

It is a live mirror. Deletions and mistakes are copied too. Use snapshots for backups.

Myth 2

“CCR means zero data loss.”

Replication is asynchronous, so the follower can be a few seconds behind. Plan for a small, measured loss window.

Myth 3

“I can write to the follower.”

Not while it is following. It is read-only until it is unfollowed and promoted.

Myth 4

“Both clusters can write to the same index.”

An index has one leader. Two-way designs use separate indices for each side, followed by the other.

Myth 5

“CCR and cross-cluster search are the same.”

CCR copies data. Cross-cluster search queries remote data without copying it. They solve different problems.

Myth 6

“Once set up, it needs no attention.”

Links break, credentials expire, indices are added. Monitoring and rehearsals are part of the deal.

Myth 7

“Failover is just flipping a switch.”

It involves several steps and a decision, and applications must be redirected. Failback needs a rebuild.

Myth 8

“Followers can have any shard layout.”

The number of primary shards must match the leader. Only replica counts and placement are free.

i
A Healthy Attitude

Treat every disaster-recovery claim as unproven until you have tested it. A design is only as good as the last rehearsal that showed it working.

31Glossary of Terms

A quick reference for the important words in this guide, in plain language.

TermPlain Meaning
Asynchronous replicationCopying data after confirming the write locally, so the copy may lag slightly
Auto-follow patternA rule that automatically creates followers for new leader indices whose names match
BootstrappingThe first bulk copy of a leader index that starts a new follower
CCRCross-Cluster Replication, the feature that copies indices between clusters continuously
ClusterA group of Elasticsearch nodes working together as one system
Cross-cluster searchA feature that searches several clusters at once without copying their data
Data streamA structure for time-based data that manages hidden backing indices behind one name
Eventual consistencyThe promise that copies may differ briefly but will match once changes arrive
FailbackReturning to the original site after a failover, usually by rebuilding it as a follower
FailoverSwitching from a failed primary system to a standby copy
Follower indexA read-only copy of a leader index in another cluster
Global checkpointThe highest sequence number safely stored on all in-sync copies of a shard
ILMIndex Lifecycle Management, which moves indices through stages as they age
Leader indexThe original, writable index that followers copy
Least privilegeGiving people and programs only the access their job needs
Remote clusterA saved connection to another cluster, with a nickname
Replication lagHow far the follower is behind the leader
Retention leaseA follower’s claim asking the leader to keep history from a certain point onward
RPORecovery point objective, the amount of recent data you can afford to lose
RTORecovery time objective, the time you can afford to be without service
Sequence numberA counter attached to each change on a shard that records the order of events
ShardOne piece of an index, stored and searched independently
Soft deletesA setting that keeps recent history, including deletions, so followers can read it later
Split-brainA situation in which two systems both believe they are the primary and accept conflicting writes
UnfollowCutting the link between a follower and its leader so the follower becomes an ordinary index

32Frequently Asked Questions

Quick answers to the questions people ask most often when they first meet CCR.

Q1Is Cross-Cluster Replication free?
No. CCR is part of Elastic’s paid subscription levels. The exact tier has changed over time, so check Elastic’s current subscription documentation before planning.
Q2Can I write to a follower index?
Not while it is following. A follower is read-only. To make it writable, you pause it, close it, unfollow it and reopen it, which turns it into a regular index.
Q3How far behind is a follower?
Usually only seconds when the network and follower are healthy. Lag grows during outages or heavy write peaks. Measure it with the follower statistics and alert on it.
Q4Does CCR replicate deletions?
Yes. It copies every operation, including updates and deletes. That is why CCR does not replace snapshots for recovering from mistakes.
Q5Can a follower have a different number of shards?
No. The follower must have the same number of primary shards as the leader. The number of replicas on the follower can be different.
Q6What is the difference between CCR and cross-cluster search?
CCR copies data into another cluster. Cross-cluster search leaves the data where it is and runs the query against several clusters. Use CCR for resilience and speed, and cross-cluster search for occasional queries across sources.
Q7What happens if the link goes down for a long time?
Follow tasks retry and resume from their checkpoint when the link returns. If the outage lasts longer than the leader’s retention period, the needed history may be gone, and the follower must be rebuilt.
Q8Can one cluster follow several other clusters?
Yes. A cluster can follow indices from many leaders, which is how hub-and-spoke reporting works. Use naming prefixes so follower index names do not collide.
Q9How do I fail over?
Pause following, close each follower index, unfollow it, reopen it and redirect applications to the new cluster. Rehearse the steps in advance and write them into a runbook.
Q10How do I fail back after the original site returns?
Do not try to reconnect the old indices, which may have diverged. Delete them and create new followers in the original site that follow the promoted cluster. Once they catch up, you can switch back if you wish.
Q11Does CCR work with data streams and rolling indices?
Yes. Auto-follow patterns can create followers for new indices and for data streams as they appear, so time-series data is replicated without manual steps.
Q12Will CCR slow down my leader cluster?
It adds some read work and history retention on the leader, but usually a small amount compared with normal traffic. The initial bulk copy can be heavier, so throttle it and schedule it during quiet hours.
Q13Can I have different retention on the leader and the follower?
Yes. Each cluster can have its own lifecycle policy. Just make sure the leader does not delete data before the follower has copied it.
Q14Do I still need snapshots if I use CCR?
Yes. CCR protects against the loss of a site. Snapshots protect against mistakes, corruption and the need for long-term history. Most serious systems use both.
Q15Can both clusters accept writes for the same data?
Not for the same index. An active-active design gives each cluster its own writable index and lets each follow the other’s index, then reads across both through aliases or name patterns.
Q16What is the biggest reason CCR projects go wrong?
Most problems come from planning rather than the feature itself: undersized links or disks, no monitoring of lag, unrehearsed failover, and treating CCR as a backup. Good goals, sizing and practice prevent nearly all of them.

33Summary and Key Takeaways

Key Takeaways

  • CCR keeps a live copy of an index in another cluster. A read-only follower pulls every change from a leader, so the copy stays close to current.
  • It is asynchronous and pull-based. Writes stay fast, links can fail without breaking the leader, and the follower runs a few seconds behind, so a sudden disaster can lose the last moments.
  • Sequence numbers, soft deletes and retention leases make it work. Followers ask for exactly the changes they are missing, and the leader keeps that history for a limited time.
  • Three uses dominate. Disaster recovery through failover, nearby reads for faster searches, and hub-and-spoke reporting from many small clusters.
  • CCR is not a backup. It copies mistakes and deletions, so pair it with snapshots, and use cross-cluster search when you only need to query remote data without copying it.
  • Plan for peak, secure both sides and monitor lag. Size links and follower capacity for the busiest hour, encrypt the connection, use least-privilege credentials, and alert on lag and stalled reads.
  • Rehearse failover and failback. Promotion is a few clear steps, failback needs a rebuild, and only regular practice turns a plan into a real recovery time.