A repository is simply a registered storage location where Elasticsearch writes and reads its backups.

Elasticsearch is a search and analytics engine. It stores your data as documents inside indices, and it spreads each index across many small pieces called shards. All of this lives on the disks of the machines (called nodes) that make up your cluster. If those machines are lost, so is your data, unless a copy exists somewhere else.

A snapshot is a backup of the data in your cluster taken at a certain moment. A snapshot repository is the place where those snapshots are stored. Before you can take any snapshot, you must first tell Elasticsearch where the repository is and how to reach it. This step is called registering a repository. After that, every snapshot you take is written into it, and every restore reads from it.

Why does Elasticsearch need a separate location?

A backup that lives next to the original is not a real backup. If a fire, a bad software update, or a wrong command hits the cluster, a copy on the same disks is hit too. That is why the repository is always something outside the normal data directories of your nodes: a shared network drive, or a cloud object store such as Amazon S3, Google Cloud Storage, or Azure Blob Storage.

Many beginners believe replica shards already protect them. They do not, and the difference matters.

Elasticsearch can keep extra copies of each shard on other nodes. These are called replica shards. If one node dies, another node holds a copy and the cluster keeps working. This protects you from hardware failure and keeps searches fast, because searches can be served by any copy.

However, replicas live inside the same cluster and follow every change instantly. If someone deletes an index by mistake, the deletion is copied to every replica within moments. If a buggy application writes garbage into your documents, the garbage is copied too. Replicas are built for availability, not for going back in time.

Six words appear again and again in any snapshot discussion. Learn them once and everything else becomes easier.

Notice how the words fit together. You register a repository. You take a snapshot into it, manually or through SLM. Later you restore from it. Underneath, the data moves as segments, and cluster internals travel as feature states.

Elasticsearch supports several kinds of repositories. Choosing the right one is your first design decision.

Who does what when a snapshot is taken? Three players are involved: the master node, the data nodes, and the repository.

The master node is the manager of the cluster. It knows which shards exist and where they live. When a snapshot is requested, the master decides which shards need to be copied and tells the right nodes to do it. It also writes the small pieces of bookkeeping that describe the snapshot.

The data nodes are the workers. Each data node holds shards, and each one uploads the files of the shards it owns directly to the repository. The data does not flow through the master. This is important because it means the work is spread across many machines in parallel.

The repository is the passive storage at the end. It does not run Elasticsearch code. It just holds files and answers read and write requests.

flowchart LR
  C["Client or SLM Policy"] --> M["Master Node"]
  M -->|"assigns shard copy tasks"| D1["Data Node 1"]
  M -->|"assigns shard copy tasks"| D2["Data Node 2"]
  M -->|"assigns shard copy tasks"| D3["Data Node 3"]
  D1 -->|"uploads new segment files"| R[("Snapshot Repository")]
  D2 -->|"uploads new segment files"| R
  D3 -->|"uploads new segment files"| R
  M -->|"writes snapshot metadata"| R
        

The diagram shows why a repository must be reachable from every node that holds data. If even one data node cannot reach it, the shards on that node cannot be saved and the snapshot will be incomplete.

A repository is not a single big file. It is an organized collection of many small files, and Elasticsearch owns their layout.

If you were to peek inside a repository, you would see a tidy structure. At the top level there is a small file that acts as the table of contents. It lists every snapshot in the repository and every index those snapshots refer to. Each time the contents change, a new version of this table of contents is written with a higher generation number, and a tiny pointer file records which version is the latest.

For each snapshot, there are files that describe it: which indices it contains, what state the snapshot finished in, and the saved cluster-level information. Then there is a folder for the actual data, organized by index, and inside each index folder, by shard. Inside the shard folders sit the segment files and a small list that says which files belong to which snapshot.

This layout is what makes snapshots cheap. Since a single data file can be referenced by many snapshots, adding a new snapshot often means uploading only a small amount of new data plus a few small metadata files.

Understanding segments is the key to understanding why snapshots are fast and small after the first one.

Inside each shard, Elasticsearch stores data in immutable segments. Immutable means a segment file is never edited after it is written. When you add or update documents, new segments are created. Over time, small segments are merged into bigger ones by a background process, and the old small ones are removed.

Because segments never change once written, a snapshot only needs to ask one question about each file: “Have I already stored this exact file in the repository?” If yes, the snapshot simply points to the stored copy. If no, the file is uploaded. That is why the first snapshot of a cluster is the biggest and later ones are much smaller.

An important consequence follows. Each snapshot is complete on its own, even though it is built incrementally. You can delete the very first snapshot and every later snapshot still restores fully. There is no “chain” you must keep intact, unlike some old-style backup tools.

Let us follow one snapshot from the moment you request it to the moment it is finished.

sequenceDiagram
  participant U as User or SLM
  participant M as Master Node
  participant D as Data Nodes
  participant R as Repository
  U->>M: Request snapshot into repository
  M->>R: Read repository index and check it is healthy
  M->>D: Start copying selected shards
  D->>R: Upload only new segment files
  D->>M: Report each shard as done
  M->>R: Write snapshot metadata and new repository index
  M->>U: Snapshot finished with a final state
        

Step by step

First, the request arrives at the master. The master checks the repository and decides which indices and shards are involved. Snapshots are taken from primary shards, so each primary must be available and assigned to a node.

Next, every data node that owns a selected primary starts copying its shard files. Before copying, the node freezes a consistent view of that shard’s files so that ongoing merges cannot delete them mid-upload. This is why the cluster keeps serving searches and accepting new documents while a snapshot is running.

When a shard is finished, its node tells the master. When all shards are done, the master writes the final metadata and the new version of the repository index. Only then is the snapshot considered complete.

Is it a perfect point in time?

Not exactly. Each shard is captured at the moment its own copy begins. Because different shards may begin a few seconds or minutes apart on a busy cluster, a snapshot may show slightly different moments for different shards. Documents indexed after a shard’s copy has begun are not in that snapshot; they will appear in the next one.

A snapshot can carry more than just your documents. You choose what to include.

By default, a snapshot of the whole cluster includes all regular indices, all data streams, and the global cluster state along with feature states. You can narrow it down to just certain indices or data streams when you only care about a few. You can also include or leave out the cluster state depending on your recovery plan.

Setting up a repository is a short sequence of careful steps. Skipping one is the most common cause of first-day failures.

Elasticsearch also offers a repository analysis operation that runs a heavier set of read and write tests against the storage. It is especially valuable when you use an S3-compatible product from a third party, because small differences in how those products behave can cause silent corruption later.

Backups only matter when you can restore them. Restore is the reason the repository exists.

To restore, you choose a snapshot, choose which indices or data streams to bring back, and decide how to treat cluster state and feature states. Elasticsearch then rebuilds each selected shard on the nodes of the target cluster by downloading its files from the repository. This looks just like normal shard recovery, except the source of the files is the repository instead of another node.

Rules that surprise beginners

You cannot restore into an open index that already has the same name. You either delete or close the existing index first, or you restore under a new name using a rename pattern. Renaming is a very popular trick, because it lets you compare the old data with the current data side by side before you decide what to do.

You can restore into a different cluster than the one that took the snapshot. This is the basis for migrations, for building test environments from production data, and for disaster recovery in another region.

flowchart TD
  A["Something goes wrong"] --> B{"What was lost?"}
  B -->|"One index deleted"| C["Restore that index under its own or a new name"]
  B -->|"Bad data written"| D["Restore old copy under new name and compare"]
  B -->|"Whole cluster lost"| E["Build new cluster and register the repository"]
  E --> F["Restore indices, cluster state and feature states"]
  C --> G["Verify counts and searches"]
  D --> G
  F --> G
        

Manual backups are forgotten backups. SLM makes backups a habit the cluster keeps for you.

Snapshot Lifecycle Management lets you describe a policy once. A policy states which repository to use, how often to take snapshots using a schedule, how to name them, and which indices to include. It also includes a retention rule that says how long snapshots may live and how many must be kept at minimum and at maximum.

Retention runs as a separate scheduled job from the snapshot job itself. By default, the retention job runs once a day, so old snapshots are not deleted the very minute they expire. This is normal and by design.

A repository is not only a safety box. It is also a building block for cost control and data movement.

Notice that searchable snapshots change how you should think about a repository. It is no longer just a cold archive that you touch only in emergencies. It becomes part of the live search path, so its speed, reliability, and permissions matter every day.

A repository holds a full copy of your data, so it must be protected as seriously as the cluster itself.

Anyone who can read the repository can read every document you ever stored, even if your cluster has strict user permissions. Anyone who can write to it can add fake data or damage backups. So security must be applied in three places: the storage, the credentials, and the cluster permissions.

An unmonitored backup system quietly fails. Watching it is part of the job.

Elasticsearch offers operations to list snapshots, to view the detailed status of running ones, and to see the history and statistics of SLM policies. From these you can learn how long each snapshot took, how much new data it uploaded, and whether any shard failed.

Alert on the age of the last success first. It is the simplest and most valuable signal, because it catches every kind of failure: a broken policy, a full disk, lost credentials, or a changed firewall rule. Then watch the trend of duration. A snapshot that used to take ten minutes and now takes two hours is warning you that data volume, network speed, or storage performance has changed.

Also keep the logs of your master node. Snapshot failures usually record a clear reason there, such as an access denied error, a timeout, or a missing file, and those reasons point straight to the fix.

Snapshots share resources with your live workload, so speed and cost need to be planned.

Speed and throttling

By default, Elasticsearch limits how fast each node uploads data to the repository, so a snapshot does not drown your search and indexing traffic. The limits for snapshot and restore speed are settings you can raise on a strong network or lower during business hours. Restoring is often more urgent than backing up, so many teams allow a faster restore rate than backup rate.

Scheduling

Take snapshots when the cluster is quiet if you can. More importantly, avoid overlapping snapshot jobs that fight for the same bandwidth. A short cycle of frequent snapshots is usually cheaper than one giant snapshot because each one only carries recent changes.

Repository size and count

Every snapshot adds a little bookkeeping to the repository. Keeping many thousands of snapshots in one repository makes list and delete operations slower. Good retention rules keep the count healthy. For very long histories, consider splitting into a few repositories by purpose.

Storage cost

Teams that run Elasticsearch for years tend to converge on the same good habits, and the same painful mistakes.

Patterns that work

Patterns to avoid

No design is perfect. Here is an honest view of what snapshot repositories give you and what they ask in return.

A useful way to think about this is in terms of two numbers that every business should know. The first is how much recent data you can afford to lose, which is decided by how often you take snapshots. The second is how long you can afford to be down, which is decided by how fast you can restore. Snapshots trade a small loss window and a moderate recovery time for low cost and high safety. If your business needs zero loss and near-zero downtime, combine snapshots with replicas and, where relevant, cross-cluster replication.

These examples show how different kinds of teams put repositories to work.

No. The repository is the storage location. Snapshots are the backups stored inside it. One repository can hold many snapshots.

Yes. Snapshots run in the background and do not block normal traffic. They do use disk, network, and some CPU, so a busy cluster may feel a small slowdown.

They are incremental in what they upload, but each one is complete on its own. You can delete the oldest snapshot and every later one still restores fully, because Elasticsearch only removes files that no other snapshot needs.

They can read from it, but only one should write. Register it as read-only on every other cluster to avoid corruption.

It depends on how much recent data you can afford to lose. Many teams take them hourly or nightly, while critical systems use much shorter intervals. Because snapshots are incremental, frequent ones are usually cheap.

Only if you include the right feature states or cluster state. Check what your snapshots contain, and test a restore to make sure logins and dashboards return as expected.

No. You can restore to the same version or to a newer supported one. Plan version paths carefully, and avoid writing to a repository with a newer cluster if older clusters still need to read it.

You must delete or close the existing index, or restore under a new name using the rename option. Renaming is the safest choice when you are unsure.

One or more shards could not be saved, often because a primary shard was unassigned or a node was unavailable during the run. Fix the shard problem and take a new snapshot.

Yes. Replicas keep the cluster available during node failures and add search speed. Snapshots protect against bigger disasters and human error. They solve different problems.

Often yes, but not all of them behave exactly like Amazon S3. Run the repository analysis operation before trusting a third-party product with real backups.

Only a restore test proves it. Restore to a scratch cluster on a regular schedule and check document counts, mappings, and a few real searches.

A snapshot repository is the registered, external storage location where Elasticsearch keeps its backups. It is organized by Elasticsearch itself, shares immutable segment files between snapshots to keep them incremental, and can be backed by a shared drive or by cloud object storage. Around it sit the tools that make it dependable: verification when you register it, lifecycle policies that automate snapshots and retention, monitoring that tells you when something drifts, and restore operations that turn stored data back into a working cluster. Treat it like the warehouse in our library story: keep it separate, keep it safe, check its contents, and practice retrieving from it before the flood arrives.