The Refresh Interval in Elasticsearch
How Elasticsearch decides when a newly written document actually becomes visible to a search query — and why that single setting quietly controls the trade-off between "instant" search results and a fast, stable cluster.
Imagine a busy newsroom. A reporter types a breaking story and hits “publish” — but the printing press only runs once every minute. Between the moment the story is written and the moment the press actually runs, the story exists, it is saved, it is safe — but nobody outside the newsroom can read it yet. Elasticsearch works in almost exactly this way. When you send a document to be indexed, it is accepted and safely recorded immediately, but it does not become searchable the instant it arrives. It becomes searchable only at the next “press run” — and the refresh interval is the dial that controls how often that press runs. This single, small setting shapes how “real-time” your search application feels, and it is one of the first things every Elasticsearch engineer eventually has to understand deeply.
1What Exactly Is the Refresh Interval?
Before diving into internals, let’s build a rock-solid definition in plain language.
The refresh interval is a per-index setting in Elasticsearch that controls how frequently the search engine takes everything that has been indexed recently — but is still sitting in a temporary, in-memory holding area — and makes it visible to search queries. By default, this happens automatically every 1 second. That one-second delay is why Elasticsearch is officially described as “near real-time” (NRT) rather than “real-time.” A document is durably stored the moment it is accepted, but it is only searchable after the next refresh.
Think of a library with a “New Arrivals” cart. When a new book arrives at the front desk, it is immediately checked in and physically exists in the building — but a visitor browsing the shelves can’t find it until a librarian wheels the cart out and places the books on the shelves. The librarian doesn’t do this continuously; she does a “shelving round” on a fixed schedule. The refresh interval is exactly that schedule — how often the librarian does her round.
Why This Matters for Everyday Developers
If you have ever indexed a document in Elasticsearch and then immediately searched for it, only to find nothing — and then searched again a moment later and found it — you have experienced the refresh interval directly. It is not a bug. It is Elasticsearch deliberately trading a tiny amount of freshness for a large amount of indexing speed and cluster stability.
1 Second
Out of the box, every index refreshes automatically once per second, making new documents visible almost instantly without any tuning.
index.refresh_interval
A dynamic index-level setting — it can be changed on a live index at any time, without closing or re-creating it.
-1
Disables scheduled automatic refreshing entirely. Documents still get indexed, but stop becoming searchable on a timer.
The Refresh API
A refresh can always be requested on demand for an index, regardless of the scheduled interval, when a specific document must be searchable right now.
2Why Elasticsearch Needs a Refresh at All
To understand refresh, you first have to understand what Elasticsearch is actually built on top of.
Underneath every Elasticsearch index sits Apache Lucene, a lower-level search library that does the actual heavy lifting of storing and searching text. Lucene organizes data into immutable, read-only files called segments. “Immutable” is the key word here — once a segment is written, it is never edited again. This design choice is what makes Lucene searches extremely fast, because a search never has to worry about data changing underneath it mid-query.
But this creates an obvious problem: if segments can never be changed, how does Elasticsearch handle new documents arriving every second? The answer is that it doesn’t edit existing segments — it creates brand-new, small segments to hold the new data, and later merges many small segments into fewer, larger ones in the background. The refresh interval controls exactly one thing in this picture: how often a new segment gets created and opened up for searching.
Indexing a document and making it searchable are two separate steps in Elasticsearch. Indexing writes the document into an in-memory buffer almost instantly. Refreshing is the separate step that converts that buffer into a searchable Lucene segment. Refresh interval controls only the second step.
The Cost of Making Everything Instant
You might reasonably ask: why not just refresh after every single document, so everything is searchable the instant it’s written? Elasticsearch actually supports this — you can set the refresh interval to a very small number, or even request a refresh after every single write. The problem is cost. Creating a new Lucene segment is not free. Each new segment consumes file handles, memory for its internal data structures, and CPU cycles. If you create thousands of tiny segments per second, the cluster spends more time managing segments than it does actually serving useful search traffic. The one-second default is a carefully chosen middle ground between “fresh enough for almost every use case” and “not so frequent that it wrecks performance.”
3Architecture & Components Involved
Four moving parts work together to make refresh possible. Understanding each one individually makes the whole picture click.
1. The In-Memory Indexing Buffer
Every shard has a small area of RAM set aside for newly indexed documents. When your application sends a document to Elasticsearch, it lands here first, in a format Lucene understands internally but has not yet turned into a proper, queryable segment.
2. The Transaction Log (Translog)
Because RAM is volatile — a crash or restart would wipe it — every write is also appended to a translog on disk. This is purely for durability and crash recovery; the translog is not what search queries read from, and it is not affected by the refresh interval at all.
3. The Filesystem Cache
When a refresh happens, the new segment is written out, but critically it is written to the operating system’s filesystem cache, not necessarily flushed all the way to physical disk with a full fsync. This is the trick that makes refresh cheap: writing to OS memory is fast, while forcing data to durable disk is comparatively slow.
4. Lucene Segments
The end product of a refresh. A new, small, immutable, searchable unit that Elasticsearch’s query layer can now include in search results. Over time, a background merge process combines many small segments into fewer, larger ones for efficiency.
Refresh and flush are not the same operation. Refresh makes data searchable by opening a new segment in the filesystem cache. Flush makes data durable on disk by fsyncing segments and clearing the translog. A refresh can happen many times between flushes. Confusing the two is one of the most common misunderstandings among engineers new to Elasticsearch.
4Internal Working — What Actually Happens During a Refresh
Let’s walk through the exact sequence of internal steps, from the moment the timer fires to the moment a document becomes searchable.
Timer Fires (or Manual Call Arrives)
Once the configured interval elapses — or someone explicitly calls the refresh API — Elasticsearch begins the refresh process for that shard.
The In-Memory Buffer Is “Frozen”
The current contents of the indexing buffer are taken as a snapshot. Any documents that arrive after this exact moment will simply wait for the next refresh cycle.
A New Lucene Segment Is Written
Lucene converts the frozen buffer into a proper segment structure, including its inverted index (the data structure that makes text search fast) and other supporting files.
The Segment Is Written to the Filesystem Cache
The new segment is handed to the operating system, which keeps it in page cache (RAM-backed) rather than forcing an immediate, expensive write to physical disk.
The Segment Is Opened for Reading
Elasticsearch’s internal reader is updated to include this new segment alongside all previously existing ones. From this instant, any search query touching this shard will consider the new documents.
The Indexing Buffer Is Cleared
Now that the data has a permanent home as a segment, the temporary in-memory buffer space is freed up, ready to accept the next batch of documents.
Notice something important: the translog is not cleared during this process. The translog only gets trimmed during a separate operation called a flush. This is exactly why refresh is considered a “cheap and frequent” operation while flush is a “heavier and less frequent” one — refresh only touches memory-backed structures, while flush deals with guaranteed durability on physical disk.
5Data Flow & Lifecycle of a Single Document
Following one document from the moment your application sends it to the moment it’s fully durable and searchable ties everything together.
flowchart TD
A["App sends document
(index request)"] --> B["Written to In-Memory Buffer"]
B --> C["Appended to Translog on disk
(durability guaranteed)"]
C --> D{"Refresh Interval
elapsed?"}
D -- "No, wait" --> D
D -- "Yes" --> E["New Lucene Segment created
from buffer contents"]
E --> F["Segment written to
Filesystem Cache"]
F --> G["Segment opened for search
Document is now SEARCHABLE"]
G --> H{"Flush interval /
translog size limit reached?"}
H -- "No, wait" --> H
H -- "Yes" --> I["Segments fsynced to disk,
Translog cleared"]
Two milestones matter most for engineers to remember. The first milestone is durability — reached the instant the document is written to the translog, seconds before it is even searchable. Even if the node crashes right after this point, the document will not be lost; it will be replayed from the translog on restart. The second milestone is searchability — reached only after the next refresh completes and opens a new segment. These two milestones are often reached at almost the same time in practice, but they are conceptually and mechanically distinct events.
6Configuring the Refresh Interval
This setting is deliberately easy to change, because different indices in the same cluster often need very different freshness guarantees.
The refresh interval is configured per index using the setting index.refresh_interval. It is a dynamic setting, which means it can be updated on an index that is already live, receiving traffic, and serving queries — no downtime, no reindexing, no closing the index required. It accepts a time value such as 1s, 30s, 5m, or the special disabling value -1.
| Value | Meaning | Typical Use Case |
|---|---|---|
1s (default) | Refresh automatically every second | General-purpose search applications, product catalogs, content search |
30s or 60s | Refresh less often, in exchange for higher indexing throughput | Log analytics, time-series data, metrics ingestion at high volume |
-1 | Disable scheduled automatic refresh entirely | Large one-time bulk imports, initial data migrations, reindexing jobs |
| Manual refresh call | Force an immediate refresh regardless of the interval | Automated tests, “read your own write” requirements right after indexing |
A very common, well-documented pattern is to set index.refresh_interval to -1 before a large bulk import, run the import at full speed, and then set it back to its normal value (or explicitly call a refresh) once the import finishes. This avoids the cluster wasting effort creating and merging thousands of tiny segments during the bulk load.
Index-Level, Not Cluster-Level
It’s worth emphasizing that this setting lives at the index level, not the cluster level. A single Elasticsearch cluster can happily run one index with a 1-second refresh interval for a live customer-facing search feature, right alongside another index with a 30-second interval for background log ingestion — each tuned to what that specific workload actually needs.
7Advantages, Disadvantages & Trade-offs
Every value you choose for the refresh interval is really a trade-off along the same dial — freshness on one end, throughput and stability on the other.
Advantages of a Short Interval
- Documents become searchable almost immediately after being indexed
- Users see near-instant results, important for chat apps, live dashboards, and order tracking
- Simplifies application logic, since developers rarely need to worry about visibility delay
- Works well out of the box for low-to-medium indexing volumes
Disadvantages of a Short Interval
- Many small segments are created, increasing merge activity in the background
- More CPU and I/O spent on segment creation instead of serving queries
- Indexing throughput drops under heavy write load
- More open file handles and memory overhead per shard
Advantages of a Long Interval (or Disabled)
- Significantly higher indexing throughput for bulk and high-volume workloads
- Fewer, larger segments created, reducing merge overhead
- Lower CPU and memory pressure during heavy write bursts
- Ideal for pipelines where data doesn’t need to be searchable instantly
Disadvantages of a Long Interval
- Noticeable delay between indexing a document and being able to find it
- Can confuse users or break “read your own write” expectations
- Requires explicit manual refresh calls in scenarios needing instant visibility
- Adds complexity to testing, since assertions right after writes may fail
8Monitoring, Logging & Metrics
Refresh activity is not invisible — Elasticsearch exposes detailed statistics so engineers can see exactly how much work refreshing is costing the cluster.
Both the index-stats and node-stats APIs expose a dedicated refresh section. This section reports how many refreshes have happened, how much cumulative time has been spent refreshing, and whether any refreshes are currently in progress. Watching these numbers over time is the standard way to tell whether your refresh interval is well-tuned or is quietly eating into cluster capacity.
Rapidly Rising Refresh Count
If the refresh count climbs far faster than expected for your configured interval, something (often application code) may be triggering excessive manual refreshes.
High Total Refresh Time
A large cumulative refresh time relative to uptime suggests refreshing is consuming a meaningful share of node resources — a strong hint to widen the interval.
Segment Count Growth
Pair refresh metrics with segment-count metrics; a fast-growing, never-shrinking segment count often means merges can’t keep up with refresh frequency.
Cluster Monitoring Dashboards
Most production teams surface these stats through a monitoring stack, watching refresh time alongside indexing rate and merge time on the same timeline.
9Design Patterns & Anti-patterns
Over years of production use, the Elasticsearch community has converged on a few clear “do this” and “don’t do this” patterns around refresh.
Pattern: Disable-Then-Restore for Bulk Loads
Set refresh_interval to -1 immediately before a large, one-time bulk import; restore it to its normal value (or trigger a single manual refresh) right after the import finishes. This is the single most recommended pattern for speeding up large data migrations.
Pattern: Longer Intervals for Write-Heavy, Read-Light Indices
Indices that mostly absorb high-volume writes (logs, metrics, event streams) and are queried in aggregate rather than instantly are excellent candidates for a 30s–60s refresh interval, trading a little freshness for a lot of throughput.
Pattern: Targeted Manual Refresh Instead of a Global Short Interval
Rather than lowering the interval cluster-wide to “always be fresh,” many teams keep the default interval and call a manual refresh only on the specific index, only at the specific moment freshness is actually required (for example, right before running an automated test assertion).
Problem
Setting refresh_interval to an extremely small value (such as tens of milliseconds) on a high-write-volume index, in an attempt to make search feel “instant.”
Why It’s Harmful
It forces Elasticsearch to create an enormous number of tiny segments per second. The background merge process can’t keep up, segment count balloons, memory and file-handle usage climb, and indexing throughput collapses — often making the application slower overall, not faster.
Correct Approach
Keep the default 1-second interval for most workloads, or use targeted manual refresh calls for the rare cases that genuinely need instant visibility, rather than lowering the scheduled interval globally.
Problem
Calling a manual refresh after every single bulk indexing request “just to be safe.”
Why It’s Harmful
This defeats the entire purpose of batching writes together. Each manual refresh forces immediate segment creation, undoing the throughput benefits that bulk indexing was meant to provide.
Correct Approach
Let the scheduled refresh interval handle normal visibility, and reserve manual refresh calls only for the small number of code paths that truly require read-your-own-write behavior right after a write.
10Best Practices & Common Mistakes
A short, practical checklist that summarizes what experienced Elasticsearch operators actually do differently from beginners.
Leave the Default Alone Unless You Measure a Problem
The 1-second default is well-tuned for most applications. Change it only after monitoring shows refresh activity is genuinely a bottleneck.
Tune Per-Index, Not Cluster-Wide
Different indices have different freshness needs. Apply the setting deliberately to the specific index whose workload actually calls for it.
Restore Settings After Bulk Loads
Always remember to re-enable the normal refresh interval after a bulk import that temporarily disabled it — forgetting this step silently degrades search freshness indefinitely.
Prefer Manual Refresh in Tests
In automated test suites, call a manual refresh right after indexing test data instead of adding arbitrary sleep delays or lowering the global interval.
Confusing Refresh With Flush
Assuming a refresh guarantees durability on disk. It doesn’t — that’s the flush operation’s job, backed by the translog.
Searching Immediately After Indexing Without Refreshing
Writing code that indexes a document and immediately queries for it, then treating the empty result as a bug, instead of accounting for the refresh delay.
11Real-World & Industry Examples
Different industries lean on different points along the refresh-interval trade-off, depending on what their users actually need.
E-Commerce Product Search
Online retailers typically keep the default 1-second interval on their product catalog index, since customers expect newly published or updated products (price changes, stock updates) to be reflected in search results almost immediately.
Centralized Log Analytics Platforms
Teams running large-scale log ingestion pipelines commonly widen the refresh interval to 30 seconds or more on their logging indices, since engineers analyzing logs rarely need sub-second freshness, and the ingestion volume is far higher than typical search workloads.
One-Time Data Migrations
Organizations migrating historical data from another database into Elasticsearch routinely disable refresh entirely during the migration window, then re-enable it once the migration completes, to maximize bulk write throughput.
Live Chat and Messaging Search
Applications where users search their own recent conversations often rely on a manual refresh immediately after a message is sent, guaranteeing the message is searchable the instant it appears in the UI, without waiting on the scheduled interval.
12Frequently Asked Questions
No. Durability comes from the translog, which is written on every index request regardless of the refresh interval. Refresh only affects when data becomes searchable, not whether it’s safely stored.
They are still safely indexed and durable via the translog, but they will not become searchable until a manual refresh is triggered or the setting is changed back to a scheduled interval.
Yes. It is a per-index setting, so each index in a cluster can have its own value, chosen based on that index’s specific freshness and throughput needs.
Generally no, for high-volume workloads, because it forces constant small-segment creation and removes the batching benefits of bulk indexing. It’s better reserved for specific cases genuinely requiring instant visibility.
No. It’s a dynamic setting and can be updated on a live, running index without any downtime.
Because refreshing has a real resource cost (creating new Lucene segments). One second was chosen as a balance point that feels instant to most users while keeping segment-creation overhead manageable for typical workloads.
13Summary and Key Takeaways
The refresh interval is one of the smallest settings in Elasticsearch, and yet it governs one of the biggest architectural trade-offs in the entire system: freshness versus throughput. It controls how often the in-memory buffer of newly indexed documents is turned into a new, searchable Lucene segment — completely separate from durability, which is handled by the translog and the flush process. Understanding this distinction clearly is often the difference between an engineer who fights confusing “missing document” bugs and one who tunes Elasticsearch confidently for exactly the workload in front of them.
Key Takeaways
- Refresh ≠ Durability — refresh controls searchability; the translog and flush handle durability.
- 1 second is the default — a deliberate balance between near-instant results and manageable resource cost.
- -1 disables scheduled refresh — commonly used to speed up large bulk imports, then restored afterward.
- It’s a dynamic, per-index setting — changeable live, and tunable differently across indices in the same cluster.
- Shorter intervals mean more, smaller segments — trading indexing throughput for freshness.
- Longer intervals favor high-volume writes — ideal for logs, metrics, and bulk-loaded data.
- Manual refresh is a scalpel, not a hammer — best used sparingly, for specific moments that truly need instant visibility.