SLM is the part of Elasticsearch that runs your backup routine for you: take snapshots on time, remove old ones by rule, and keep a record of what happened.

Start with the plain-English definition

Elasticsearch is a search and analytics engine. It stores information as JSON documents inside containers called indices. Each index is cut into smaller pieces called shards, and shards are spread across the machines (called nodes) that make up a cluster.

A snapshot is a saved copy of some of that data, plus optionally some cluster settings, written to a storage location outside the cluster. That location is called a repository — think of it as a vault that lives somewhere else, such as Amazon S3, Google Cloud Storage, Azure Blob Storage, or a shared network drive.

Snapshot Lifecycle Management adds two ideas on top of plain snapshots: a policy (a saved set of instructions describing what to back up, where, and how often) and retention (rules describing how long to keep the backups). Once a policy exists, the cluster wakes up on schedule, takes the snapshot, and later deletes snapshots that are too old.

Why SLM exists

Before SLM shipped, teams wrote their own automation: a cron job on some server that called the snapshot API every night, plus a second script that deleted old snapshots. These scripts worked until the server was replaced, the script broke quietly, or the person who wrote it left the company. Nobody noticed until a restore was needed, and by then months of backups might be missing.

SLM moved this job inside the cluster itself. The schedule, the retention rules, and the success and failure history now live in cluster state, so they survive node replacements and are visible through the same tools already used to run Elasticsearch day to day. SLM first shipped in the Elasticsearch 7.x series and is a standard feature in current versions.

The three jobs of SLM

A few numbers worth remembering

These are cluster-level defaults administrators can change. They resurface in later chapters, so it helps to meet them early.

The big picture

Here is the whole story in one diagram. A policy lives in cluster state. The elected master node — the single node currently acting as the cluster’s coordinator — runs the SLM scheduler. When the schedule fires, the master asks the relevant data nodes to copy their shard files into the repository. A separate retention job later removes old snapshots.

flowchart LR
  P["SLM Policy
schedule, scope, retention"] --> M["Elected Master Node
SLM scheduler"]
  M -->|"cron fires"| S["Snapshot Service"]
  S --> D1["Data Node 1"]
  S --> D2["Data Node 2"]
  S --> D3["Data Node 3"]
  D1 --> R[("Snapshot Repository
S3, GCS, Azure, shared FS")]
  D2 --> R
  D3 --> R
  M -->|"daily retention job"| R

Read the diagram left to right. The policy is only a set of instructions. The master node is the manager who reads those instructions. The data nodes are the workers who carry the boxes. The repository is the warehouse where the boxes are stored. The bottom arrow shows a second manager task: walking through the warehouse and removing boxes that are past their date.

What SLM does not do

Data loss rarely comes from one dramatic event. It comes from ordinary mistakes, ordinary bugs, and ordinary hardware, meeting a cluster with no recent backup.

Six ways data gets lost

Replicas are not backups

Elasticsearch already keeps copies of every shard. A primary shard holds the main copy, and a replica shard holds another live copy on a different node. If a node dies, a replica takes over within seconds — this is called high availability, and it is genuinely valuable.

But a replica updates the instant the primary changes. If the primary is deleted, the replica is deleted too. If a bug writes garbage data, the replica receives the same garbage. A replica is a mirror, not a memory.

Two numbers every backup plan needs

Recovery Point Objective (RPO) answers: how much recent data can we afford to lose? If snapshots run every hour, the worst case is losing a little more than an hour of changes. Recovery Time Objective (RTO) answers: how long can the system stay down while restoring? Restoring a large cluster from object storage takes time proportional to its size, so RTO must be measured by testing, not guessed.

Why automation beats memory

Humans forget. Scripts break without telling anyone. A backup process depending on one person or one server is a backup process waiting to fail. SLM keeps its schedule inside the cluster, tracks every run, and can be watched by the same monitoring already used for everything else. It turns “someone should back this up” into “the cluster backs this up, and we get alerted if it stops.”

SLM only automates snapshots, so understanding what a snapshot contains, where it lives, and why the second one costs so much less than the first, makes everything else click.

What is inside a snapshot

A snapshot can contain three kinds of things. First, index data: the shard files for the indices selected. Second, the global state: cluster-wide items such as persistent settings, index templates, ingest pipelines, and lifecycle policies. Third, feature states: the internal system indices that features and tools such as Kibana and security keep for themselves.

The policy chooses which of these to include. A common pattern backs up business indices every hour without global state, and takes a separate, less frequent snapshot that includes global state so the cluster’s configuration can be rebuilt from scratch if needed.

The repository: your vault

Before any snapshot can run, a repository must be registered. This tells Elasticsearch which storage system to use and how to reach it. Registration also runs a verification step where every node checks that it can actually write to that location.

Segments: the secret behind cheap snapshots

Inside each shard, Elasticsearch relies on a library called Lucene, which writes data into files called segments. The key rule: a segment is never changed after it is written. New documents go into new segments. Deleted documents are only marked as deleted. Periodically, small segments merge into larger new segments and the old ones are discarded.

This rule makes incremental backups almost free. When a snapshot runs, Elasticsearch checks which segment files already exist in the repository and copies only the ones missing. Everything else is simply referenced by pointer.

flowchart TD
  SA["Snapshot Monday"] --> F1["Segment file 1"]
  SA --> F2["Segment file 2"]
  SB["Snapshot Tuesday"] --> F1
  SB --> F2
  SB --> F3["Segment file 3, added Tuesday"]
  SC["Snapshot Wednesday"] --> F2
  SC --> F3
  SC --> F4["Segment file 4, added Wednesday"]

This picture explains two important behaviors. First, the repository grows by roughly the amount of changed data each time, not by the full index size — a 100 GB index that changes by 5 GB overnight adds about 5 GB to the next snapshot, not another 100 GB. Second, deleting a snapshot never removes a file another snapshot still needs. In the diagram, deleting Monday’s snapshot leaves file 1 in place because Tuesday still points at it; only once Tuesday is also deleted does file 1 become unused and get removed.

Point-in-time, but not perfectly simultaneous

A snapshot does not freeze the cluster. Indexing and searching continue while it runs. Each shard is captured as it looked at the moment that shard’s copy began. Shards starting a few seconds later may include a few seconds more data. Each shard is internally consistent, but the snapshot is not one perfectly synchronized instant across every shard.

The most important repository rule

A policy is a small saved recipe with five ingredients. Once each ingredient is understood, every policy anyone writes becomes easy to read.

The five parts

Policy ID versus snapshot name

Newcomers often mix these two up. The policy ID is the permanent name of the policy itself, for example nightly-critical. The snapshot name is generated fresh at each run from the name pattern, for example nightly-2026.09.21-x7f3k9q. One policy produces many snapshots over its lifetime.

SLM also stamps each snapshot with metadata containing the policy ID. This stamp is how the retention job later knows which snapshots belong to which policy. Snapshots created manually, without the stamp, are never touched by SLM retention.

What the config section controls

A policy explained in words

Suppose a team writes a policy called nightly-critical. Its schedule fires at 02:00 UTC every day. Its repository is a bucket reserved for production backups. Its config includes the orders-* and customers-* indices and leaves out global state. Its retention says expire after 30 days, always keep at least 7 snapshots, and never keep more than 60.

Every night the cluster creates a fresh snapshot of those indices. The next morning’s retention job — which runs on its own schedule, not at 02:00 — scans all snapshots stamped nightly-critical and removes those that break the rules. The team touches nothing after setup.

Following one policy run from the scheduled second to the finished snapshot shows exactly which cluster components talk to each other and in what order.

Step by step

sequenceDiagram
  participant C as Cron Scheduler
  participant M as Master Node
  participant D as Data Nodes
  participant Repo as Repository
  C->>M: Schedule matches now
  M->>M: Generate snapshot name
  M->>D: Create snapshot request
  D->>Repo: Upload new segment files
  D->>M: Shard snapshot complete
  M->>Repo: Write snapshot manifest
  M->>M: Record success in policy status

Why only the master schedules, but any node can copy

Elasticsearch avoids two nodes trying to run the same scheduled job at once by giving scheduling duties to the single elected master node at any moment. If that master fails mid-run, a newly elected master takes over SLM’s bookkeeping using the same cluster state, so a policy is never silently lost, though an in-flight snapshot at the moment of failover may need to be retried.

Copying the actual shard data, however, is spread across every relevant data node, since each node holds a different slice of the index. This keeps a snapshot from becoming a single-node bottleneck.

What “success” really means

Zooming out from one run to the whole lifecycle shows how creation, listing, restoring, and deletion fit together as one continuous flow.

The four lifecycle stages

flowchart LR
  A["Create snapshot
scheduled or manual"] --> B["Store in repository"]
  B --> C{"Restore needed?"}
  C -->|"Yes"| D["Restore into a cluster"]
  C -->|"No"| E["Wait"]
  E --> F["Daily retention check"]
  F --> G{"Meets retention rule?"}
  G -->|"No"| H["Delete snapshot"]
  G -->|"Yes"| E

Restoring is its own operation, not part of SLM

It surprises many newcomers that SLM has nothing to do with restoring data. SLM only creates and expires snapshots. Restoring uses the ordinary restore-snapshot API, which can restore an entire snapshot or only specific indices, and can rename indices during the restore so the original data is not overwritten.

A typical safe pattern is to restore into freshly named indices — for example, restoring orders as orders-restored-2026-09-21 — verify the restored data looks correct, and only then swap it into place, rather than restoring directly over a live index and hoping for the best.

Data streams flow through the same pipeline

A data stream is Elasticsearch’s structure for time-ordered data, such as logs or metrics, made of multiple underlying indices behind one name. SLM can back up data streams directly; each underlying index is snapshotted like any other index, and a restore recreates the stream structure so time-based queries keep working afterward.

Retention is where good backup plans quietly go wrong. The three retention settings interact, and understanding that interaction prevents two very different failure modes: keeping too little, or keeping — and paying for — far too much.

The three retention settings

How the three rules combine

The retention job evaluates every snapshot in a policy together, oldest first, and a snapshot is deleted only when doing so does not violate min_count, and either it is older than expire_after or the total count still exceeds max_count. min_count always wins: even a five-year-old snapshot survives if deleting it would drop the cluster below the configured floor.

Worked example

A policy takes hourly snapshots, with expire_after: 7d, min_count: 24, max_count: 200. After one week, roughly 168 snapshots exist. All of them are within seven days old, so none expire by age yet, and 168 is comfortably under the 200 maximum, so nothing is deleted. By day nine, the oldest snapshots pass seven days old and begin expiring one by one, but never below 24 remaining, guaranteeing at least a full day of hourly granularity always stays available even under a very aggressive expiry setting.

Real clusters rarely run one policy. Different data deserves different frequency, different retention, and sometimes a different repository entirely.

Why one policy is usually not enough

A single cluster commonly holds customer-facing transactional data, internal analytics, and raw application logs side by side. Backing all three up on the same hourly, thirty-day schedule wastes money on the logs and under-protects the transactional data if a shorter interval is really warranted. Splitting into several policies lets each dataset get exactly the protection it needs, no more and no less.

Separate repositories for separate risk

Some organizations register a second repository in a different cloud region, or even a different cloud provider, and run a lower-frequency policy against it purely as a disaster-recovery copy. If the primary repository’s account or region becomes unreachable, this second copy is what stands between the organization and a total loss.

flowchart TB
  Cluster["Production Cluster"] --> R1[("Primary Repository
Region A")]
  Cluster -->|"lower frequency"| R2[("DR Repository
Region B, different account")]
  R1 -.->|"read-only mount"| DRCluster["Standby Cluster, Region B"]
  R2 -.->|"read-only mount"| DRCluster

Snapshots are cheap thanks to incremental copying, but they are not free. Understanding what actually consumes resources keeps SLM from becoming an unwelcome surprise on a busy cluster.

Where the cost really goes

Snapshots scale with change, not with total size

Because of the segment-sharing behavior covered earlier, a snapshot’s cost during a normal run is proportional to how much data changed since the previous snapshot, not to the full size of the index. A 10 TB index that changes by 50 GB a day produces roughly 50 GB snapshots most days, which is why SLM remains practical even on very large clusters.

The first snapshot of a brand-new large index is the expensive exception, since there is nothing yet to reuse — every segment must be uploaded. Teams bringing a large existing cluster under SLM for the first time should expect that first run to take noticeably longer and to schedule it at a quiet time.

Concurrency limits protect the cluster

Elasticsearch limits how many snapshot operations can run at once, through settings controlling maximum concurrent snapshots per repository and per cluster. These limits exist because unlimited concurrent snapshot activity could saturate disks and network links that indexing and search also depend on.

What an interviewer may ask

SLM is built to keep working through node failures and master elections, but a few realities are worth knowing before relying on it in a crisis.

Master failover mid-snapshot

If the elected master node fails while a snapshot is in progress, a new master is elected using the cluster’s normal election process, and it takes over SLM duties immediately because policy definitions and schedules live in replicated cluster state, not on any single node’s local disk. A snapshot that was actively uploading at the exact moment of failover, however, may be left in an incomplete state and typically needs to be retried on the next scheduled run or manually.

Repository outages

If the repository itself becomes unreachable — a cloud storage outage, expired credentials, a network partition — snapshot attempts fail, and SLM records the failure in the policy’s status rather than crashing anything else in the cluster. Indexing and search continue completely unaffected, since snapshotting is a background activity layered on top of normal cluster operation, not a blocking one.

flowchart TD
  A["Snapshot in progress"] --> B{"Master fails?"}
  B -->|"Yes"| C["New master elected"]
  C --> D["SLM resumes from cluster state"]
  D --> E["In-flight snapshot marked incomplete"]
  E --> F["Retried on next scheduled run"]
  B -->|"No"| G["Snapshot completes normally"]

Reliability comes from testing, not just scheduling

Mature teams schedule periodic restore drills: restore a recent snapshot into an isolated test cluster, verify document counts and a sample of records, and time how long the process takes. This is the only reliable way to know that RTO assumptions from earlier chapters actually hold true.

A backup system that anyone can read or delete is not really a safety net — it is a second attack surface. SLM’s security model layers on top of Elasticsearch’s regular access control.

Permissions that matter

Splitting these permissions matters in practice. A monitoring dashboard service account typically needs only read_slm. A platform team automating policy rollout needs manage_slm. Very few people, ideally, should hold the ability to delete or reconfigure the repository itself, since that is the one action capable of destroying every existing backup at once.

Protecting the repository from the cluster it protects against

Encryption

Data in transit to cloud repositories is encrypted using the storage provider’s standard TLS connections. Data at rest depends on the repository: cloud object stores typically offer server-side encryption by default, and Elasticsearch can also work with repositories layered under disk-level or provider-managed encryption for stricter compliance needs.

What an interviewer may ask

A policy nobody watches is a policy that can silently fail for weeks. SLM exposes enough status information to build real alerting around it.

What to watch

A simple, effective alert

The most valuable single alert most teams can add is straightforward: compare “time since last successful snapshot” against the policy’s expected interval, and page someone once that gap exceeds roughly two to three times the scheduled frequency. An hourly policy silent for six hours deserves attention long before it becomes a genuine emergency.

flowchart LR
  A["SLM Policy Status API"] --> B["Monitoring system polls status"]
  B --> C{"Last success older than threshold?"}
  C -->|"Yes"| D["Alert on-call engineer"]
  C -->|"No"| E["No action"]
  A --> F["SLM History Index"]
  F --> G["Dashboard: success rate, duration trend"]

SLM behaves slightly differently depending on whether Elasticsearch runs self-managed, on Elastic Cloud, or inside a Kubernetes deployment.

Cross-cloud and hybrid repositories

A cluster running in one cloud provider can still snapshot into another provider’s object storage, since repository plugins connect over the network rather than requiring the storage to be local. This is a common way to build the cross-region or cross-provider disaster-recovery pattern introduced in Chapter 8, keeping backup infrastructure independent from a single provider’s outage blast radius.

Years of production use have surfaced a handful of patterns that work well, and an equally memorable set of mistakes that keep recurring.

Patterns worth copying

Anti-patterns to avoid

A concise, practical checklist distilled from the chapters above, organized as pros to lean into and cons to actively guard against.

Every SLM decision trades one good thing for another. Naming the trade-off explicitly leads to better decisions than treating any single setting as simply “correct.”

There is no universally correct answer to any row in this table — only the answer that fits a given system’s actual RPO and RTO targets, its budget, and how quickly problems in that particular dataset tend to be noticed.

SLM’s design choices make far more sense once seen against how organizations actually run Elasticsearch at meaningful scale.

Recent Elasticsearch versions extend what a snapshot can do beyond pure backup, blurring the line between “backed up” and “still queryable.”

Searchable snapshots

A searchable snapshot lets a snapshot in a repository be mounted directly as a queryable index, without fully restoring every byte back onto local disk first. Data is read from the repository on demand, with local caching for frequently accessed portions. This effectively turns the repository itself into a very cheap, very cold storage tier that can still answer occasional queries, most commonly used for older log or metrics data that is unlikely to be searched often but must remain available.

How this connects back to SLM

SLM itself does not change when searchable snapshots are used — it still creates ordinary snapshots on schedule. What changes is what happens to a snapshot afterward: instead of only sitting in the repository as a pure backup, older data’s snapshot can also be mounted as a cold-tier searchable index, letting one snapshot serve two purposes at once, backup and cheap long-term query access.

flowchart LR
  SLM["SLM creates
scheduled snapshot"] --> Repo[("Repository")]
  Repo -->|"full restore"| Hot["Restored index
fully on local disk"]
  Repo -->|"mount as searchable snapshot"| Cold["Cold-tier queryable index
data stays in repository"]

Working through a few concrete failure stories ties every earlier chapter together into practiced judgment rather than isolated facts.

Scenario one: the wildcard delete

An engineer meant to delete a single test index but a shell wildcard expanded to match twelve production indices as well. Replicas offered no protection, since the delete propagated everywhere instantly. The response: identify the most recent snapshot from before the delete using the policy history from Chapter 12, restore the affected indices under temporary names as described in Chapter 6, verify document counts against known application metrics, then rename them back into production use.

Scenario two: the silent repository failure

A cloud credential rotation broke the repository’s write access, but nobody had wired up the alerting from Chapter 12. Three weeks passed with every SLM run failing quietly. The gap was only discovered during an unrelated audit. The fix going forward: the time-since-last-success alert became a required part of every new cluster’s setup checklist, not an optional extra.

Scenario three: retention too aggressive for the problem

A subtle ingestion bug corrupted a small percentage of documents over several weeks before being noticed. Because retention had been set to keep only ten days for cost reasons, every remaining snapshot already contained corrupted data. The only path forward was rebuilding from the original source system, since Elasticsearch itself had no clean copy left. This is precisely the anti-pattern flagged in Chapter 14, and afterward the team extended retention specifically for indices fed by that ingestion path.

flowchart TD
  A["Incident detected"] --> B["Check SLM policy status and history"]
  B --> C["Identify last known-good snapshot"]
  C --> D["Restore into temporary index name"]
  D --> E["Verify data with counts and spot checks"]
  E --> F{"Data looks correct?"}
  F -->|"Yes"| G["Swap into production use"]
  F -->|"No"| H["Try an earlier snapshot, or rebuild from source"]

SLM did not appear in a vacuum. Comparing it against the approaches teams used before, and against approaches from outside Elasticsearch entirely, makes clear exactly what problem it solves and where its boundaries sit.

SLM versus Curator

Before SLM existed, a popular open-source tool called Curator handled index maintenance tasks, including triggering snapshots and deleting old ones based on age or count. Curator ran as an external process, usually on a cron schedule of its own, completely outside the Elasticsearch cluster it managed. This meant Curator’s schedule, its credentials, and its own uptime were all a separate thing that had to be deployed, patched, and monitored, on top of the cluster itself.

SLM absorbed Curator’s snapshot-management responsibilities directly into Elasticsearch. There is no external process to deploy, no separate credential set to secure, and no separate monitoring system needed just to know whether the backup tool itself is still running. The trade-off is flexibility: Curator could be scripted to do highly custom, cross-tool orchestration, while SLM intentionally stays focused and declarative, covering the common cases well rather than trying to cover everything.

SLM versus filesystem or volume-level backups

Some teams instead back up the raw data directories on disk, or take storage-volume snapshots at the infrastructure layer, such as cloud block-storage snapshots. This approach can look appealing because it needs no Elasticsearch-specific tooling at all. It carries a serious risk, though: Elasticsearch writes across many files as documents are indexed, and a volume snapshot taken mid-write, without the cluster’s cooperation, can capture an inconsistent, unusable state unless the node is fully stopped or specifically flushed and locked first, which is disruptive in a live production system.

SLM avoids this problem entirely because it works through Elasticsearch’s own snapshot API, which coordinates with the cluster to capture a consistent view of each shard while it continues serving live traffic, as explained back in Chapter 3.

SLM versus third-party backup products

Commercial backup products sometimes wrap Elasticsearch’s own snapshot API with a friendlier interface, extra scheduling features, or cross-product orchestration across several different databases at once. Under the hood, most of them still call the same underlying repository and snapshot mechanisms this guide has described. For a team running only Elasticsearch, SLM alone is usually sufficient and avoids an extra product to license and maintain; for a team standardizing backup tooling across many different database technologies, a third-party product’s unified interface can be worth the added cost and complexity.

Bringing everything together, here is the order a team typically follows when adopting SLM on a cluster for the first time, from a blank slate to a monitored, tested backup routine.