What Is Reindexing in Elasticsearch?

What Is Reindexing in Elasticsearch?

The safe, standard way to fix a mistake, upgrade a schema, or move data — without ever touching your live index in place.

Sooner or later, every Elasticsearch user hits the same wall: a field was mapped the wrong way, or a schema needs to change, or documents need to move to a different index entirely — and it turns out you simply cannot edit an existing field’s type in place. The answer to all of these situations is the same operation: reindexing. It sounds intimidating the first time you hear it, but it’s really just “copy every document from one index into another, transforming it along the way if needed.” This article explains exactly what reindexing is, how it works internally, and how production teams use it without downtime.

1Core Concepts

What reindexing actually means, before any architecture talk.

Reindexing is the process of copying documents from a source index into a destination index, re-running them through the indexing pipeline along the way. It’s built into Elasticsearch as a dedicated API — you don’t have to manually export and re-import data yourself. The destination index can have a completely different mapping, different settings, or even live on a different cluster, and the documents can optionally be transformed (filtered, renamed, or reshaped) as they move.

Everyday Analogy

Imagine you’ve outgrown your filing cabinet and its folders are badly organized — some labeled wrong, some missing dividers you now need. Reindexing is like buying a brand-new, better-organized cabinet, and carefully transferring every single folder into it, fixing labels and adding dividers along the way, rather than trying to reorganize the old cabinet while it’s still in daily use.

What it is: a built-in operation that reads documents from one index and writes them into another, applying the destination’s mapping and any requested transformations.

Why it exists: because a field’s type, an index’s shard count, and several other settings cannot be changed on a live index — reindexing is the only supported path to apply those kinds of structural changes.

Where it’s used: mapping corrections, shard count changes, Elasticsearch version upgrades, splitting or merging indices, and migrating data between clusters or environments.

i
Key Idea

Reindexing never modifies the source index. The original data stays completely untouched and searchable throughout the entire process — the new, corrected copy is built alongside it in a brand-new index.

Why Can’t Elasticsearch Just “Fix” the Field in Place?

Recall that Elasticsearch’s storage engine, Lucene, writes data into immutable segments — files that, once written, are never edited, only eventually merged or deleted. A field’s type governs exactly how its values were physically written into those segments (for example, as searchable text tokens versus as a raw number). There’s no way to reinterpret already-written bytes as a different type after the fact, so the only reliable fix is to read the original values back out and write them fresh, under the new, corrected rules — which is exactly what reindexing does.

2Architecture & Components

The moving pieces involved in a reindex operation.
Source

Source Index

The existing index being copied from — can even be an index on a remote cluster, allowing cross-cluster migrations.

Destination

Destination Index

A new (or existing) index with its own mapping and settings, created ahead of time so the corrected schema is already in place before data arrives.

Transform

Optional Script / Query

An optional filter query narrows which documents are copied; an optional script can reshape, rename, or compute new field values on the way through.

Coordination

Reindex Task

The operation runs as a trackable, cancellable background task, reporting progress, throughput, and any per-document failures as it proceeds.

graph LR
    A[Source Index
old_products_v1] -->|Read documents
in batches| B[Reindex Task] B -->|Optional script:
transform fields| C{Transform?} C -- Yes --> D[Apply script
per document] C -- No --> E[Pass through unchanged] D --> F[Destination Index
products_v2] E --> F F --> G[Alias 'products'
repointed to v2]
Fig. 1 — A typical reindex flow: documents stream from the old index, through an optional transform, into the new index, with an alias switched over at the end.

Notice the alias at the end of the diagram. Production teams almost never point application code directly at a raw index name like “products_v2” — instead, they point it at an alias like “products”, which can be instantly repointed from the old index to the new one the moment reindexing finishes. This is the architectural trick that makes reindexing feel “zero-downtime” from the application’s point of view.

3Internal Working

What Elasticsearch is actually doing, step by step, while a reindex runs.
1

Scroll / Batch Read

Elasticsearch reads documents from the source index in efficient batches (internally similar to a scrolling search), rather than loading the entire index into memory at once.

2

Optional Script Execution

If a script was supplied, it runs against each document in the batch — renaming fields, dropping ones no longer needed, or computing new derived values.

3

Bulk Write to Destination

The (possibly transformed) batch of documents is written into the destination index using the Bulk API internally, going through the destination’s own mapping validation and analysis exactly as if freshly submitted.

4

Progress Tracking & Retry

The task tracks how many documents have been processed, created, updated, or failed, and can automatically retry certain transient failures (like temporary resource pressure) without aborting the whole job.

5

Completion

Once every batch has been processed, the task reports a final summary — total documents copied, any failures, and total time taken — and the destination index is now a fully independent, searchable copy.

!
Point-in-Time Snapshot Behavior

A reindex operation captures the source index as it looked at the moment reindexing started. If new documents are written to the source index while reindexing is still running, those very latest writes may not automatically appear in the destination unless the reindex is run again or combined with a strategy to catch up on the gap.

4Data Flow & Lifecycle

How a reindex fits into the broader lifecycle of an index migration.

Step 1 — Create the Destination Index

Define the corrected mapping and settings up front, before any data arrives — this is the whole point of the exercise, so it must be right before reindexing begins.

Step 2 — Run the Reindex Operation

Kick off the copy, optionally scoped with a query (to migrate only recent documents first, for example) and optionally with a script to reshape data along the way.

Step 3 — Catch Up on New Writes

If the source index kept receiving writes during the reindex, run a follow-up reindex scoped to only documents changed after the first run started, to close the gap.

Step 4 — Switch the Alias

Atomically repoint the application-facing alias from the old index to the new one — this single action is what makes the cutover instantaneous from the application’s perspective.

Step 5 — Retire the Old Index

Once confidence is high that the new index is correct and complete, the old index can be safely deleted, freeing its disk space.

Everyday Analogy

This mirrors moving to a new house while still living in the old one: you first prepare the new place (Step 1), gradually move your things over (Step 2), grab anything you brought home in the meantime (Step 3), officially change your mailing address (Step 4), and only then give up the lease on the old house (Step 5).

5Advantages, Disadvantages & Trade-offs

Advantages

  • The source index is never at risk — it stays fully intact and searchable throughout.
  • Enables changes that are otherwise completely impossible on a live index, like fixing a field’s type or shard count.
  • Supports on-the-fly transformation, so cleanup and migration can happen in the same step.
  • Combined with aliases, achieves an effectively zero-downtime cutover for applications.

Disadvantages

  • Temporarily doubles disk usage, since both the old and new index exist simultaneously.
  • Can be time-consuming and resource-intensive for very large indices, consuming CPU and I/O on the cluster.
  • Requires careful handling of documents written to the source after the reindex started, or data can be silently missed.
  • Any downstream systems referencing the raw index name directly (instead of an alias) require extra coordination during cutover.

The core trade-off is safety versus cost: reindexing trades additional disk space and processing time for the guarantee that your live, serving index is never put at risk while structural changes are made.

6Security

Access control considerations specific to reindexing.
Permissions

Read + Write Privileges

Reindexing requires read privileges on the source index and write/create privileges on the destination — a role missing either will fail partway through or be blocked outright.

Remote Reindex

Cross-Cluster Trust

Reindexing from a remote cluster requires that remote cluster to be explicitly allow-listed and reachable, preventing arbitrary external clusters from being pulled from.

Data Exposure

Field-Level Security During Migration

If a transform script inadvertently copies a field that was meant to be excluded via field-level security, that field can end up freshly exposed in the destination index — transform scripts should be reviewed with the same care as the mapping itself.

Everyday Analogy

Reindexing across clusters is like moving sensitive paper files between two different offices: you need permission to take the files out of the first office, permission to bring them into the second, and a secure, trusted route between the two — skipping any of that is how records go missing or end up somewhere they shouldn’t.

7Monitoring, Logging & Metrics

Because reindexing can run for minutes, hours, or occasionally longer on huge datasets, keeping an eye on it as it runs is essential:

Docs/sec
REINDEX THROUGHPUT
Failures
PER-DOCUMENT WRITE ERRORS
Task Duration
ELAPSED VS. ESTIMATED TIME
  • Task status API: Elasticsearch exposes a running task’s progress — total documents, documents processed so far, and any failures — so operators can watch it live rather than waiting blindly for completion.
  • Cluster resource usage: reindexing generates significant CPU, memory, and I/O load; monitoring node-level metrics during a large reindex helps avoid impacting other live traffic on the same cluster.
  • Document count parity: comparing the document count in the source and destination indices after completion is a simple, essential sanity check before cutting over.
  • Failure logs: individual document failures (for example, from a script error or a type mismatch) are reported in the task’s response and should be reviewed, not ignored, before trusting the new index.

8Design Patterns & Anti-patterns

Pattern: Alias-Based Zero-Downtime Cutover

Always point applications at an alias rather than a raw index name, so reindexing to a new underlying index can be swapped in atomically, with zero code changes and no visible downtime.

Pattern: Reindex Then Reconcile

Run an initial bulk reindex for historical data, then run a smaller, scoped follow-up reindex (filtered to “changed since” the first run’s start time) to catch any writes that happened during the migration window.

Pattern: Throttled Reindexing

Deliberately cap the reindex operation’s throughput (requests per second) during business hours to avoid competing for cluster resources with live production traffic, running unthrottled only during low-traffic windows.

ANTI-PATTERN-01 Avoid
Reindexing Without Catching Up on New Writes

Running a single reindex and immediately cutting over the alias, without accounting for documents written to the source during the operation, silently drops the most recent data from the new index.

Skipping Document Count Verification

Cutting over to the new index without comparing document counts (and spot-checking a sample of documents) risks discovering a transform script bug only after it’s already serving production traffic.

9Best Practices & Common Mistakes

Best Practices

  • Always use an alias in front of your indices, even from day one, so future reindexing is painless.
  • Validate the destination mapping thoroughly before starting a large reindex — mistakes here just repeat the same problem.
  • Throttle large reindex operations to protect live cluster performance during peak hours.
  • Run a “catch-up” reindex for any documents written after the main operation started, before cutting over.
  • Verify document counts and spot-check a sample of transformed documents before deleting the old index.

Common Mistakes

  • Deleting the old index immediately after cutover, before fully confirming the new index is correct and complete.
  • Running a massive, unthrottled reindex during peak production hours and starving other operations of resources.
  • Forgetting that transform scripts run per document and can silently drop or mis-map fields if not carefully tested.
  • Pointing application code directly at a raw index name, making every future reindex require a code deployment.

10Real-World & Industry Examples

E-commerce

Fixing a Mis-Mapped Field

A retailer that accidentally mapped “sku” as “text” instead of “keyword” reindexes the entire product catalog into a corrected index, fixing broken exact-match filtering without ever taking the storefront’s search offline.

Observability

Version Upgrades

Teams upgrading to a newer Elasticsearch major version commonly reindex older indices created under incompatible settings into fresh, version-compatible indices as part of the upgrade process.

Media & Streaming

Shard Count Correction

A media platform whose content index was originally created with too few shards for its later scale reindexes into a new index with a properly sized shard count, improving search performance under heavier load.

Finance

Cross-Cluster Migration

Financial platforms migrating between data centers or cloud regions use remote reindexing to pull data from an old cluster into a newly provisioned one, as part of a broader infrastructure migration.

“Reindexing isn’t a workaround for Elasticsearch’s limitations — it’s the designed, supported way structural change happens safely.”

11Frequently Asked Questions

Q1Does reindexing affect or lock the source index?
No. The source index remains fully readable and writable throughout — reindexing only reads from it, it never modifies or locks it.
Q2Do I need to create the destination index myself first?
It’s strongly recommended. If the destination doesn’t exist, Elasticsearch will create it automatically with dynamic mapping, which usually defeats the purpose if you were reindexing specifically to fix a mapping problem.
Q3Can reindexing change field names or values, not just types?
Yes — an optional script supplied to the reindex operation can rename fields, drop them, or compute entirely new values as each document passes through.
Q4How do I avoid downtime for my application during a reindex?
Point your application at an alias rather than the raw index name; once reindexing finishes, atomically repoint the alias to the new index so the application never notices the switch.
Q5What happens to documents written to the source index while a reindex is in progress?
They aren’t automatically captured by that reindex run. A common pattern is to run a smaller follow-up reindex, scoped to documents changed after the first run’s start time, to close the gap before cutting over.
Q6Can reindexing move data between two different Elasticsearch clusters?
Yes, using remote reindexing, as long as the source cluster is explicitly configured as a trusted remote host on the destination cluster.

12Summary and Key Takeaways

Key Takeaways

  • Reindexing copies documents from a source index into a destination index, optionally transforming them along the way.
  • It exists because certain structural changes — like fixing a field’s type or changing shard count — cannot be made on a live index.
  • The source index is never modified or put at risk; the destination is built alongside it as an independent copy.
  • Combining reindexing with an alias lets applications cut over instantly, achieving effectively zero downtime.
  • Documents written to the source during the operation need a catch-up reindex to avoid being silently missed.
  • Reindexing is resource-intensive and should be throttled and monitored, especially on large indices under live production load.
  • Always verify document counts and sample data correctness in the destination before deleting the original index.