What Is an Update Operation in Elasticsearch?
A beginner-friendly, deep walkthrough of how Elasticsearch changes part of a document without touching the rest — what really happens behind the scenes, and why it isn't quite what it looks like.
Imagine you have a printed contract sitting in a filing cabinet, and you need to correct just the phone number on page one. You can’t simply erase and rewrite that one line on the original paper — instead, you photocopy the whole contract, cross out the old number, write in the new one, staple the corrected copy together, and file that as the new official version, shredding the old one. In Elasticsearch, an update operation works in almost exactly this way, and understanding this “copy, change, replace” reality is the key to using updates correctly.
1What Exactly Is an Update Operation?
Before anything else, let’s build a clear, simple definition.
An update operation in Elasticsearch lets you change part of an existing document — such as correcting one field’s value — without needing to resend the document’s entire contents. From the outside, it feels exactly like editing one cell in a spreadsheet. From the inside, however, Elasticsearch doesn’t actually modify anything in place at all.
Instead, an update works by fetching the document’s current _source, merging your requested change into it in memory, and then writing the result back as an entirely new version of the document. The old version is marked as deleted and eventually cleaned up, while the new, merged version becomes the one that’s searchable going forward.
Think of updating a document in Elasticsearch like editing a shared online document that automatically saves a brand-new version every time you make a change, rather than overwriting the same file in place. Each edit produces a fresh version, and the system quietly retires the older ones, even though to you it feels like you simply “changed” the document.
Partial Document
The small set of fields you send in an update request, describing only what should change.
Version Number
An internal counter that increases every time a document is changed, used to detect conflicts.
Upsert
A special update option that creates a new document if one with the given ID doesn’t already exist.
Scripted Update
An update that uses a small script to compute a change, rather than supplying a static field value.
This distinction matters because it explains behavior beginners often find surprising: an update operation is never truly a lightweight, in-place edit. It’s a complete read, merge, and rewrite cycle disguised behind a simple, convenient API.
2Why Does This Concept Exist?
Every technology exists to solve a real, painful problem. Here is the one updates solve.
Without an update operation, changing even one field of a document would require the calling application to first fetch the entire current document, modify the relevant field itself, and then resend the whole thing as a brand-new index request. For large documents, or for applications making frequent small changes, this becomes clumsy, wasteful, and error-prone.
The Core Problem
Applications frequently need to change just a small part of a document — incrementing a view counter, updating a status field, correcting one typo — without forcing every caller to manually fetch, merge, and resend the entire document every single time.
The update API solves this by handling the fetch-merge-rewrite cycle internally, on the server side, so the application only needs to describe what should change, not manage the entire document lifecycle itself. This keeps application code simpler and reduces the risk of accidentally overwriting fields the caller didn’t even know about.
Without server-side updates, two different parts of an application changing different fields on the same document at nearly the same time would risk one overwriting the other’s change entirely, since each would be working from its own separately fetched, now-outdated copy.
Updates also solve a second, related problem: safe concurrent modification. Because Elasticsearch tracks a version number for every document, it can detect when an update is based on stale data and either retry automatically or reject the change — protecting against silent, accidental data loss when multiple updates race against each other.
3Architecture and Components
Understanding the pieces involved in every update, and how they relate.
_source
The current stored document, read at the start of every update to serve as the base for merging.
Merge Logic
The process that combines your partial change with the existing document to produce the new version.
Version Control
Tracks document versions and detects conflicting, concurrent update attempts.
retry_on_conflict
A setting that tells Elasticsearch to automatically retry an update if a version conflict occurs.
graph TD
A[Update request: change status field] --> B[Fetch current _source]
B --> C[Merge new field into existing document]
C --> D[Write merged document as new version]
D --> E[Old version marked deleted]
D --> F[New version becomes searchable]
Picture a bank ledger where every transaction gets its own new entry rather than erasing old numbers — even a small correction becomes a fresh line item, while the outdated one is simply crossed out and ignored going forward. The full history technically still exists internally for a short time, even though only the newest, correct entry is what everyone actually sees and uses.
This architecture is precisely why updates rely so heavily on _source being enabled, as discussed in the earlier _source tutorial. Without a stored original document to merge changes into, the update operation would have nothing to base its new version on.
4How It Works Internally
A precise, step-by-step look at what happens on the shard.
When an update request arrives, Elasticsearch performs what is often called a “read-modify-write” cycle entirely within the primary shard responsible for that document, ensuring the whole operation is handled consistently in one place.
Fetch the Current Document
The primary shard retrieves the document’s current _source and its current version number.
Apply the Change
The partial fields you provided (or a script’s result) are merged into the fetched document.
Reindex the Merged Document
The resulting, fully merged document is indexed again as if it were a brand-new write.
Increment the Version
A new, higher version number is assigned, and the previous version is marked obsolete.
Because the entire document is reindexed, not just the changed field, an update’s cost scales with the full size of the document, not just the size of the change. Updating one small field on a very large document still requires rewriting the entire thing.
This is also where version conflicts become visible. If another update or index request modified the same document between when this update’s fetch happened and when its write attempt occurs, Elasticsearch detects the mismatch and either retries the whole cycle automatically (if configured to do so) or returns a conflict error for the application to handle.
5Data Flow and Lifecycle
Tracing one update request from the client all the way to a confirmed new version.
sequenceDiagram
participant Client
participant Primary as Primary Shard
participant Replica as Replica Shard
Client->>Primary: Update request (partial doc or script)
Primary->>Primary: Fetch current _source and version
Primary->>Primary: Merge change into fetched document
Primary->>Primary: Reindex as new version
Primary->>Replica: Replicate the fully merged new document
Replica-->>Primary: Acknowledge
Primary-->>Client: Return success and new version number
Notice that what gets replicated to other shards isn’t the small partial change you sent — it’s the entire, freshly merged document. This ensures every replica ends up with a fully consistent, complete copy, rather than needing to independently perform the same merge logic themselves.
Successful Update
The merge succeeds, a new version is created, and the response confirms the updated version number.
Version Conflict
Another change happened concurrently; the update either retries automatically or fails, depending on settings.
Document Not Found
Without an upsert configured, updating a nonexistent document simply returns a not-found response.
Upsert Created
With upsert configured, a brand-new document is created using the provided fallback content instead.
6Common Uses and Real-World Examples
How update operations actually show up in everyday application design.
Incrementing a Counter
A content platform tracking view counts on articles commonly uses a scripted update to increment a numeric field by one each time an article is viewed, without needing to know or resend the article’s full text.
Changing a Status Field
An order-processing system updating an order’s status from “pending” to “shipped” typically uses a simple partial update, changing only that one field while leaving the rest of the order document untouched.
Upserting a User Preference
An application saving a user’s settings might use upsert so that the first time a user changes a preference, a new document is created automatically, while every later change simply updates the existing one.
Conditional Updates with Retry
A high-traffic like-counter feature, updated by many users simultaneously, commonly configures retry_on_conflict so that brief version clashes are resolved automatically rather than surfacing as errors to end users.
| Update Type | Purpose | Typical Use |
|---|---|---|
| Partial Document Update | Replace specific field values directly | Status changes, corrections |
| Scripted Update | Compute a new value based on the existing one | Counters, calculated fields |
| Upsert | Create the document if it doesn’t already exist | First-time preference saves |
7Advantages, Disadvantages and Trade-offs
Updates are convenient, but the underlying reindex cost is real.
Advantages
- Lets applications change part of a document without managing the full fetch-merge-resend cycle themselves.
- Built-in version checking helps prevent silent, accidental overwrites from concurrent changes.
- Upsert conveniently handles both “create” and “update” cases with a single request.
- Scripted updates allow computed changes, like incrementing counters, without extra round trips.
- retry_on_conflict smooths over brief, high-contention update races automatically.
Disadvantages / Trade-offs
- Every update reindexes the entire document, not just the changed field, regardless of change size.
- Frequent updates to very large documents can become surprisingly expensive at scale.
- High-contention documents updated very frequently can experience repeated version conflicts.
- Update logic adds a small amount of latency compared to a plain, direct index write.
The trade-off in short: updates buy convenience and safety at the cost of hidden full-document rewrite overhead. For small documents and moderate update frequency, this cost is negligible; for very large documents updated very often, it can become a genuine performance bottleneck worth redesigning around.
8Security Considerations
Modifying existing data carries its own distinct security responsibilities.
Because an update can silently create a completely new version of a document, controlling who is allowed to perform updates — and on which fields — is just as important as controlling who can read or search the data in the first place.
Restrict Update Permissions
Grant update rights only to roles or applications that genuinely need to modify specific indices.
Validate Scripted Updates
Carefully review any scripts used in updates, since they execute directly against live document data.
Audit Update Activity
Log which users or services perform updates on sensitive indices, and when, for later review.
Guard Against Unintended Upserts
Be deliberate about enabling upsert, since it can silently create new records if misused or misconfigured.
Granting broad update permissions to an application component that only ever needs to change one specific, low-risk field, opening the door for that component to unintentionally or maliciously alter other, more sensitive fields on the same documents.
9Monitoring, Logging and Metrics
Signals worth watching to keep update-heavy workloads healthy.
Version Conflict Rate
A rising rate signals growing contention on specific, heavily-updated documents.
Update Latency
Since updates involve a fetch-merge-write cycle, their latency is a useful signal of document size pressure.
Retry Count
Tracks how often retry_on_conflict actually triggers, hinting at hot, contended documents.
Merge and Segment Growth
Frequent updates generate many new document versions, increasing background merge activity.
An index that seems to consume more disk and merge activity than its raw document count would suggest is often a sign of a heavily update-driven workload, since every update leaves behind an old, soon-to-be-cleaned-up version in the background.
10Design Patterns and Anti-patterns
Sensible approaches to using updates, and common mistakes worth avoiding.
Batching Frequent Small Changes
For extremely high-frequency changes, like real-time counters, a proven pattern is to batch increments in an external, faster store and periodically flush the aggregated total into Elasticsearch via a single update, rather than issuing one update per individual event.
Using retry_on_conflict Deliberately
Setting a reasonable retry count on genuinely high-contention documents lets Elasticsearch absorb brief update races automatically, rather than pushing every conflict back to application-level retry logic.
Problem
Performing very frequent updates on a single, large document that many concurrent processes are also trying to change at the same time.
Why It’s Harmful
Since every update reindexes the whole document, both the resource cost per update and the frequency of version conflicts scale up together, often turning one document into a serious performance bottleneck.
Correct Approach
Consider splitting frequently-changing, high-volume data into smaller, separate documents, or aggregating rapid changes elsewhere before applying them to Elasticsearch in less frequent batches.
Problem
Using update requests as the default way to write all data, even for brand-new documents that could simply be indexed directly.
Why It’s Harmful
Update requests carry extra fetch-and-merge overhead that a plain index request doesn’t need to pay when there’s genuinely nothing to merge against.
Correct Approach
Use a direct index request for genuinely new documents, and reserve the update API specifically for cases where an existing document needs a partial, targeted change.
11Best Practices and Common Mistakes
Practical, field-tested guidance for working with updates day to day.
Keep documents reasonably sized
Smaller documents make every update cheaper, since the whole document is rewritten on each change.
Use retry_on_conflict for hot documents
Let Elasticsearch handle brief contention automatically instead of surfacing every conflict as an error.
Reserve upsert for genuine create-or-update cases
Enable it deliberately, not as a default habit, to avoid accidentally creating unintended documents.
Monitor conflict rates
Treat a rising version conflict rate as an early signal to reconsider a document’s update pattern.
Before designing a feature around frequent updates, ask whether the same outcome could be achieved by indexing new, smaller documents instead — such as one document per event rather than one giant, constantly-updated summary document.
A frequent beginner mistake is assuming updates are inherently “cheap” simply because they only send a small amount of data over the network. The network payload may be small, but the actual work performed on the shard — fetch, merge, full reindex — scales with the size of the whole document, not the size of the change.
Another common mistake is ignoring version conflicts entirely rather than handling them gracefully, leading to silently dropped updates in high-concurrency scenarios. Either configuring retry_on_conflict or explicitly handling conflict responses in application logic avoids this quiet, hard-to-diagnose failure mode.
12Frequently Asked Questions
Quick, direct answers to the questions beginners ask most often about updates.
No. Internally, Elasticsearch fetches the existing document, merges in the change, and writes an entirely new version, marking the old one obsolete.
It can save the application from having to fetch and reconstruct the whole document itself, but on the shard, the underlying cost is similar, since the entire document is rewritten either way.
Without an upsert configured, the update simply returns a not-found response; with upsert, a new document is created using the provided fallback content instead.
It occurs when the document changed between the moment an update fetched it and the moment it tried to write the result back, indicating a concurrent modification happened in between.
It increases the chance of success under brief contention by retrying automatically, but it doesn’t guarantee success indefinitely — a fixed retry limit still applies.
Yes, scripted updates run against the current fetched document, allowing computed changes like incrementing a counter based on its existing value.
13Summary and Key Takeaways
An update operation in Elasticsearch offers the convenience of changing part of a document without the calling application having to manage the entire fetch-merge-resend cycle itself — but underneath that convenience sits a genuine full-document read, merge, and reindex process every single time. Understanding this reality helps explain both why updates are so useful for everyday partial changes, and why they aren’t a free, lightweight operation for very large or extremely frequently changed documents.
Key Takeaways
- Updates are read-merge-write cycles — never true in-place edits, despite how simple they appear from the outside.
- The whole document is rewritten each time — cost scales with document size, not with the size of the actual change.
- Version numbers protect against silent overwrites — conflicts are detected and can be retried automatically.
- Upsert conveniently handles create-or-update logic — but should be enabled deliberately, not by default.
- Scripted updates allow computed changes — such as incrementing counters based on existing values.
- Frequent updates to hot, large documents are costly — consider smaller documents or batching for high-frequency changes.
- Security still matters — update permissions and scripts deserve the same scrutiny as any other write access.