The Request Cache in Elasticsearch
A complete, friendly guide to how Elasticsearch remembers the answers to repeated searches, when that memory saves you real money and time, when it quietly does nothing, and how to measure and tune it in production.
Picture a school hallway at lunchtime. Hundreds of students walk up to the same notice board and ask the same question: what is on the menu today? If the office staff had to walk to the kitchen and read the menu aloud every single time, the line would never end. So the staff write the menu on a board once, and everyone simply reads it. The board stays up until the kitchen changes the menu, and then it gets wiped and rewritten.
Elasticsearch has a board just like that. It is called the request cache. It keeps the finished answers to searches that people ask again and again, and it throws those answers away the moment the data underneath them changes. Used well, it turns slow, expensive dashboard queries into instant replies. Used badly, it sits in memory doing nothing at all.
In this guide you will learn what the request cache is, what exactly it stores, how Elasticsearch decides whether an old answer is still safe to reuse, and how to measure and tune it. You do not need any coding background. Every new word is explained the first time it appears, and every big idea comes with an everyday comparison.
1What Is the Request Cache?
The shortest possible answer: it is a memory area that remembers finished search answers for each shard, so repeated searches can skip the hard work.
Let us build the definition slowly. Elasticsearch is a search and analytics engine. Companies use it to search products, explore logs, count events, and draw charts. You send it a question, called a search request, and it sends back an answer, called a response.
Elasticsearch stores data in an index, which you can think of as a big labeled cabinet of documents. To handle large amounts of data, it slices each index into pieces called shards. Each shard is a small, complete search engine on its own. When you ask a question, every relevant shard works on its own slice and produces a partial answer. Those partial answers are then combined into the final response.
The request cache, whose official name is the shard request cache, remembers the partial answer that each shard produced. The next time the very same request arrives and the shard’s data has not changed, the shard does not repeat the work. It simply pulls the stored partial answer out of memory and hands it back.
Imagine a library with many shelves. A helpful librarian tends each shelf. People keep asking, “How many mystery books are on your shelf, and how many were written in each decade?” Counting takes ten minutes. So each librarian writes the answer on a sticky note and attaches it to the shelf. The next person who asks the same question gets the note in one second. When a new book arrives on that shelf, the librarian throws the note away, because the numbers may have changed. The shelf is the shard, the sticky note is the cache entry, and the arrival of new books is a data change.
What does the cache actually store?
By default, the request cache stores the results of searches that ask for no individual documents. These are searches where the size of the result list is set to zero. A search like that still returns useful things: the total number of matching documents, the results of aggregations (which are Elasticsearch’s way of grouping and counting data, like “sales per country” or “errors per hour”), and suggestions (spelling or autocomplete hints). Those three kinds of answers are exactly what dashboards and analytics screens ask for all day long.
The cache does not store the documents themselves in the default setup. If your search asks for the top ten matching documents, the request cache normally steps aside. You will see later how that rule can be changed and why you should be careful about doing so.
People say “request cache,” “shard request cache,” and sometimes the old name “shard query cache.” They all refer to the same thing. It is not the same as the node query cache, which stores filter results and is covered in Chapter 4.
Why the word “request” matters
The cache is keyed by the request. That means the identity of a cached answer is built from the exact text of the search you sent. If two requests are word-for-word identical, they share an answer. If they differ in even a small way, they are treated as strangers. This single fact explains most of the surprises people meet when they first try to get good hit rates, and Chapter 6 is entirely about it.
Where you will meet it
You will meet the request cache in Kibana dashboards that refresh every thirty seconds, in web shops that show product counts next to every filter, in monitoring screens that draw the same twelve charts for a whole operations team, and in reporting jobs that run the same summary every hour. Anywhere the same analytical question is asked repeatedly against data that changes slowly, this cache can shine.
2Why It Exists: The Problem of Repeated Work
Search engines are fast, but analytics questions can be heavy. The request cache exists because the same heavy question is often asked many times in a row.
Finding a single document by a keyword is usually cheap. Elasticsearch keeps a special structure called an inverted index, which works like the index at the back of a textbook: it lists each word and the pages where that word appears. Looking up a word is quick.
Aggregations are a different story. To answer “how many orders came from each of the two hundred countries in the last year,” the engine must visit the matching documents, read one field from each, and tally the results. If a shard holds fifty million documents, that tally takes real CPU time and real memory. When the same tally is requested again a second later, redoing it is pure waste.
The pattern of repetition
Repetition in search traffic is very common. A few examples show why:
- A team dashboard has twelve panels. Twenty people keep it open on a wall screen and on their laptops. Each browser tab refreshes every thirty seconds. That is 240 identical panel requests every thirty seconds.
- A web shop shows how many products match each brand and each price band on the left side of a category page. Thousands of shoppers open the same category every hour.
- A finance report counts transactions per day for the past ninety days. Ninety of those days are closed and will never change again, yet the report recounts them every time.
In each case the answer is the same many times over. The request cache lets Elasticsearch do the heavy work once and reuse the result for as long as the underlying data stays the same.
A teacher hands out the same quiz to five classes and has the answer key in a drawer. Instead of solving every question again for each class, she grades using the key. Solving the quiz is the query. The answer key is the cache. If the teacher changes even one question, the key must be rewritten, which is exactly what happens when a shard’s data changes.
An illustrative story
BookNook, an imaginary online bookshop
BookNook stores ten million book records across ten shards. Every category page asks Elasticsearch for the number of books per language, per price band, and per rating. Each of those questions takes about 120 milliseconds on a shard when the cache is cold. During a sale, 800 shoppers per minute open the “Science Fiction” page. Without the cache, every visit costs the cluster ten shard computations. With the cache, only the first visit after each data change pays that cost, and the other 799 visits read the notes from memory. This story is invented to make the numbers easy to follow, but the shape of the saving is real.
A short history
The feature began life in the 1.x era of Elasticsearch under the name shard query cache. It was later renamed the shard request cache, and in modern versions it is switched on by default. The design goal never changed: reuse finished shard answers, but never risk showing an outdated one. That second half of the goal is what makes the cache safe to leave on.
The cache only helps when questions repeat and data stays still for a while. If every request is unique, or if documents are being written constantly, the cache will hold answers that are almost never reused. Later chapters show you how to check whether your workload is a good fit.
3The Building Blocks You Need to Know First
To understand the cache you need six simple ideas: node, index, shard, replica, segment, and refresh. Each one is explained here with a plain comparison.
A running copy of Elasticsearch
A node is one server process. A group of nodes working together is a cluster. Think of nodes as the branches of a library chain.
A collection of similar documents
An index might hold all products, or all log lines for one day. It is like one large cabinet with a label on the front.
One slice of an index
Elasticsearch splits an index into shards so the work can be spread over many nodes. Each shard is a mini search engine.
A spare copy of a shard
Replicas protect against failure and share the reading load. Each copy is a separate, independent shard in its own right.
An immutable file bundle inside a shard
A shard stores its data in segments. Once written, a segment is never edited. New documents go into new segments.
The moment new data becomes searchable
Newly indexed documents are invisible to searches until a refresh happens. By default this occurs about once per second on active indices.
Primary shards and replica shards
Every shard has one primary copy, which receives new writes first. It can also have zero or more replica copies. Replicas hold the same data as the primary, live on different nodes, and can answer searches. This matters for caching because each copy keeps its own cache entries. A note stuck to the shelf in one branch does not appear in another branch, even if the books are identical.
Why segments and refresh matter so much
Segments never change after they are written. When you delete or update a document, Elasticsearch marks the old version as deleted and writes a new version somewhere else. Background jobs called merges later combine small segments into bigger ones and clean up deleted data. A refresh is the step that opens a fresh, consistent view of the current segments so that searches can see recent changes.
That fresh view is called a reader. A reader is a frozen snapshot of the shard at one moment in time. Searches run against a reader. When a refresh finds new data, a new reader is opened. The request cache attaches its notes to a specific reader, which is the trick that keeps it safe. You will see this in detail in Chapter 8.
Who does what during a search
When a search request arrives, it first lands on a coordinating node. That node forwards the request to one copy of each relevant shard, waits for the partial answers, merges them, and replies to the client. In most production setups the client talks to the cluster through a load balancer or an API gateway, which spreads incoming requests across nodes.
flowchart LR
A["Client application"] --> B["Load balancer or API gateway"]
B --> C["Coordinating node"]
C --> D1["Shard 1 on Data Node A"]
C --> D2["Shard 2 on Data Node B"]
C --> D3["Shard 3 on Data Node C"]
D1 --> E["Coordinating node merges partial results"]
D2 --> E
D3 --> E
E --> F["Final response to client"]
Look closely at Figure 1. The request cache is not sitting at the load balancer or at the coordinating node. It sits with the shards on the data nodes. Each shard’s partial answer is remembered separately. This has an important consequence: if only one shard receives new documents, only that shard’s notes are thrown away, and the other shards can still answer from memory.
Elasticsearch also has a separate “final answer” stage in which the coordinating node fetches the actual document contents. The request cache works on the earlier stage, where each shard computes its own partial result. Chapter 5 walks through both stages.
4How the Request Cache Differs From Other Caches
Elasticsearch has several caches, and beginners often mix them up. Each one remembers something different and forgets it for a different reason.
Four caches matter for search speed: the shard request cache, the node query cache, the fielddata cache, and the operating system’s file system cache, often called the page cache. Let us meet them one by one.
The node query cache
The node query cache, sometimes called the filter cache, remembers which documents matched a filter. A filter is a yes-or-no test such as “status is published” or “price is under 20.” The cache stores the matching documents as a compact bit map for each segment. By default, it uses 10 percent of the heap on each node, and only large segments are eligible: a segment needs more than ten thousand documents and must hold at least three percent of the shard’s documents. Its entries are dropped when the segment goes away, for example after a merge.
The fielddata cache
Fielddata is an in-memory table that lets Elasticsearch sort or aggregate on text fields when someone explicitly allows it. It is expensive and is loaded on demand. Most modern data uses doc values instead, which live on disk and are read through the page cache. Fielddata is a special-purpose tool and is usually left off.
The page cache
The operating system keeps recently read pieces of files in spare memory. Elasticsearch relies on this heavily, which is why guidance says to leave a large share of a server’s memory outside the Java heap. The page cache does not know anything about searches. It only speeds up reading bytes from disk.
Side-by-side comparison
| Cache | Remembers | Lives in | Forgotten when |
|---|---|---|---|
| Shard request cache | The whole partial answer of a shard: counts, aggregations, suggestions | Java heap, one shared pool per node | The shard opens a new reader after a change, the cache fills up, or you clear it |
| Node query cache | Which documents matched a reusable filter, per segment | Java heap, one shared pool per node | The segment disappears, or the cache fills up |
| Fielddata cache | Field values loaded for sorting or grouping on text | Java heap | The segment disappears, or a limit forces eviction |
| Page cache | Raw bytes of index files | Operating system memory outside the heap | The operating system needs the memory for something else |
Think of a restaurant kitchen. The page cache is the counter where ingredients were recently put out, so the cook does not walk to the pantry each time. The node query cache is a tray of pre-chopped onions, ready for any recipe that needs them. The request cache is a finished dish under a heat lamp, ready to serve to the next customer who orders exactly that dish. The finished dish saves the most work, but it only helps when someone orders the identical meal and nothing in the kitchen has changed.
flowchart TD
A["Search request arrives at a shard"] --> B["Request cache: whole shard answer"]
B -->|Miss| C["Node query cache: reusable filter results"]
C --> D["Segment files read through the OS page cache"]
D --> E["Partial answer built and returned"]
B -->|Hit| E
How they work together
These caches are teammates, not rivals. The request cache is the outermost layer: if it has a matching entry, none of the others are needed. If it misses, the shard runs the query, and the query cache may still speed up the filters inside it. Whatever data must be read from disk is served from the page cache when possible. A well-tuned cluster uses all four.
Use the request cache for repeated whole-question answers, the query cache for repeated filter conditions across different questions, and leave plenty of memory free for the page cache. When someone says “Elasticsearch is slow the first time and fast the second time,” it is worth asking which of these caches is doing the work.
5The Journey of a Search and Where the Cache Is Checked
A search has two big stages. The request cache plugs into the first one, on every shard, before any real work starts.
Elasticsearch normally answers searches in two stages, known as query then fetch. In the query phase, each shard figures out which documents match, counts them, ranks them, and computes any aggregations. It returns a small package of results, mostly ids and numbers. In the fetch phase, the coordinating node asks the shards for the full contents of only the winning documents.
When a request asks for zero documents, there is nothing to fetch. The whole search is really just the query phase, and the request cache stores the outcome of that phase for each shard.
The request arrives
A client sends a search. A load balancer forwards it to any node, which becomes the coordinating node for this request.
The coordinator picks shard copies
For each shard in the target indices, it selects one copy, primary or replica. The selection is normally rotated so load is shared.
Each shard decides if the request is cacheable
The shard checks the index setting, the request parameter, the result size, and whether the request contains anything that changes from moment to moment.
The shard looks up the cache
It builds a key from its current reader and the request, then asks the cache if a stored answer exists.
Hit or miss
On a hit, the stored partial answer is returned right away. On a miss, the shard runs the full query phase and, if the request was cacheable, stores the new answer.
The coordinator merges everything
Partial answers from all shards, some from cache and some freshly computed, are merged into one response for the client.
flowchart TD
A["Shard receives a search request"] --> B{"Is the request cacheable?"}
B -->|No| G["Run the full query phase"]
B -->|Yes| C{"Entry found for this reader and this request?"}
C -->|Hit| D["Return the stored result"]
C -->|Miss| G
G --> H["Store the result in the cache when allowed"]
H --> I["Return the fresh result"]
A mixed result is normal
Because each shard decides for itself, a single search can be a hit on some shards and a miss on others. Suppose an index has ten shards and only shard four received new documents since the last identical search. Nine shards answer from memory, and shard four recomputes. The total time is close to the time of that one shard, not ten. This is one of the reasons why splitting data into many shards, and keeping old shards untouched, works so well with this cache.
What a hit really saves
A hit saves the shard from walking through its matching documents, from reading field values, and from building aggregation structures in memory. It also reduces pressure on other resources such as CPU caches and the page cache, because less data is touched. What a hit does not save is the trip across the network and the merge step on the coordinating node. Those still happen for every request.
The lookup happens on the data node that hosts the chosen shard copy, not on the coordinating node. If a request is sent to the same shard copy each time, it is more likely to meet a warm cache. Chapter 11 explains how to encourage that.
6Inside the Cache: The Key and the Value
Every cache is a lookup table. The key says which question you are asking, and the value is the stored answer. Understanding both explains nearly every hit and every miss.
The key: who asked what, and about which data
A cache key is like the label on a jar in a pantry. To find the right jar quickly, you need a label that is exact. The request cache key has two important ingredients.
- The shard’s current reader. This identifies the exact snapshot of the data the answer was computed from. If the shard opens a new reader, the old label no longer matches anything.
- The request itself. The search body, meaning the query, the aggregations, and the other options, is turned into bytes and used as part of the key.
Two searches share an entry only if both ingredients match exactly. The same request run against a different reader is a different key. A slightly different request run against the same reader is also a different key.
Why word-for-word matters
Because the request is used as raw bytes, the cache does not understand the meaning of your search. It does not know that two differently ordered lists of conditions are logically the same. It does not ignore extra spaces or reordered fields. Two requests that mean the same thing to a human but are written a little differently will not share an entry.
A vending machine only accepts an exact button code. Pressing “B7” gives the snack you want, but pressing “7B” gives nothing. The machine does not care that you meant the same snack. The request cache has the same stubbornness: the text of the request is the button code.
Applications that build search bodies from maps or dictionaries sometimes write the fields in a random order. That single detail can turn a great hit rate into almost none. Make your application produce the same text every time for the same question. Chapter 13 shows this as a design pattern.
The value: the shard’s answer, stored compactly
The value stored in the cache is the shard’s query-phase result. It includes the total hit count, any aggregation results in their partial form, and any suggestions. Elasticsearch keeps this result in a serialized form, which means it is packed into a compact block of bytes, the same way it would be packed to travel over a network. When a hit occurs, the block is unpacked and handed to the coordinating node.
Storing bytes rather than live objects has two benefits. Memory use is predictable and easy to measure, and unpacking is very fast compared with recomputing. The trade-off is that large aggregation results, such as a grouping with thousands of buckets, produce large entries, and a few big entries can crowd out many small ones.
Where the entries live
Although each shard’s answers are separate, all of them on one node live in a single shared pool. The pool has a memory limit, and when it is full, Elasticsearch removes the least recently used entries to make room. That rule is called LRU. It means entries that nobody has asked for lately are the first to go. A background cleaner also removes entries that belong to readers that no longer exist, so stale notes do not waste space.
7What Gets Cached and What Does Not
The cache is careful. It refuses to store answers that could be wrong or that would almost never be reused. Knowing the rules saves hours of confusion.
The default rule: only size zero
By default, a request is stored only when it asks for no documents back. The total count, aggregations, and suggestions are cached. The reason is practical. Result lists are large, they depend on ranking scores, and they are rarely repeated exactly. Aggregation and count results are compact and often repeated.
You can override this rule for a single request by setting the request-level parameter named request_cache to true. That forces caching even when documents are returned. This can help for a small, hot, repeated query, and it can hurt for a large result list, because each entry then eats a lot of memory.
Requests that are never cached
Some requests cannot be cached safely, because their answer would differ every time or because the cache cannot represent them correctly.
| Situation | Cached? | Why |
|---|---|---|
| Size zero, ordinary query and aggregations | Yes, by default | Compact and often repeated |
| Size above zero | No, unless forced with the request parameter | Result lists are large and rarely repeated exactly |
Query uses the current time keyword now | No | The meaning of “now” changes every millisecond, so the answer would be stale immediately |
| Script uses randomness or the current clock | No | The script gives a different result each run |
| Scroll requests | No | Scrolling walks through a frozen result set in pages, which is a different pattern |
| Profile requested | No | Profiling measures real execution, so a stored answer would defeat its purpose |
| Search type that pre-collects global term statistics | No | The answer depends on statistics gathered from other shards |
| Index setting turns the cache off | No | You disabled it for that index |
The “now” problem, explained simply
Many dashboards ask for “the last fifteen minutes.” In the search body this is often written using the keyword now. Since now is a different moment on every request, no two requests are ever identical, and Elasticsearch refuses to cache them at all. A related trap is a client that converts “last fifteen minutes” into exact timestamps before sending. Those timestamps change every second, so the key changes every second, and hit rates fall to almost zero even though the request cache is technically allowed.
The fix is rounding. If your dashboard asks for “the last day, rounded down to the start of the hour,” then every request within the same hour uses the same boundaries and shares an answer. Elasticsearch date math supports rounding, and rounded expressions are the standard way to make time-based dashboards cache-friendly. You trade a small amount of precision for a large amount of speed.
Ask yourself whether a business user would notice if the chart lagged by up to one minute or one hour. If not, round your time boundaries to that unit. Your cache hit rate, and your cluster bill, will thank you.
Turning caching off for one request
Setting request_cache to false skips the cache for that one request. This is useful when you are measuring the true cost of a query, or when you know a particular request will never repeat and you do not want it to consume space.
Details such as which search types are excluded can change between Elasticsearch releases. Treat the table above as a reliable guide to the main rules and confirm edge cases in the documentation for the version you run.
8Invalidation: How Elasticsearch Keeps Answers Fresh
A cache is only useful if it never lies. Elasticsearch guarantees that a cached answer is the same answer you would get without the cache.
There is an old saying in computer science that cache invalidation is one of the hardest problems in the field. Invalidation means deciding when a stored answer is no longer trustworthy and must be thrown away. Many systems solve this with timers: keep the answer for sixty seconds and hope for the best. Elasticsearch takes a smarter path for the request cache, and it does not need a timer at all.
The reader is the trick
Remember that every cached entry is tied to a specific reader, the frozen snapshot of the shard. New documents, updates, and deletions do not change that snapshot. They only show up after a refresh opens a new reader. Since the reader is part of the cache key, the arrival of a new reader means every older key stops matching. The old answers are no longer reachable, and a background cleaner soon removes them from memory.
This is neat because there is no risk of forgetting to clear something. The data can only change visibly by opening a new reader, and opening a new reader automatically retires the old notes. The cached answer and the fresh answer are the same, or the entry is gone.
Picture a museum guide who prints a fact sheet for each exhibit hall and stamps it with the hall’s “edition number.” Whenever curators change anything in the hall, they raise the edition number. A visitor holding a sheet with the old edition number knows at once that it is out of date, because the number no longer matches. Nobody has to walk around collecting old sheets. The reader is the edition number.
sequenceDiagram
participant Q as Search client
participant S as Shard
participant R as Request cache
participant W as Indexing client
Q->>S: Search request one
S->>R: Look up answer for current reader
R-->>S: Miss
S->>S: Run the query on the current reader
S->>R: Store answer for current reader
Q->>S: Same search again
S->>R: Look up answer
R-->>S: Hit
W->>S: Index a new document
S->>S: Refresh opens a new reader
Q->>S: Same search again
S->>R: Look up answer for the new reader
R-->>S: Miss because the reader changed
Only real changes matter
A refresh does not always create a new reader. If nothing changed since the last refresh, Elasticsearch keeps the existing reader, and the cached answers stay valid. This is why a read-mostly index keeps its notes for hours or days, even though refresh is technically scheduled every second. It also means that an index receiving one new document per minute loses its notes about once per minute, but not once per second.
Other events that clear entries
- Mapping updates. A mapping describes the fields of an index and how they are treated. Changing it can alter how a query is understood, so cached results are invalidated.
- Merges. When background merges rewrite segments, the next refresh opens a reader over the new set of segments, and old entries fall away. Merges that touch old data can therefore reset notes for a shard even when no documents changed.
- Closing or deleting an index, or moving a shard. The entries belong to that shard on that node and disappear with it.
- Manual clearing. An administrator can clear the request cache for an index. Chapter 9 explains how.
- Node restart. The cache lives in heap memory only. A restarted node begins with an empty cache and warms up again as requests arrive.
The refresh interval and its side effect
The setting that controls how often refresh happens is called the refresh interval, and it defaults to one second for indices that are being actively searched. A short interval gives near-instant visibility of new data but also means new readers, and therefore lost notes, very frequently. On a busy write index, the request cache seldom gets a chance to be useful. Some clusters stretch the interval to five, thirty, or sixty seconds for analytics indices, accepting slightly delayed visibility in return for more cache reuse and cheaper indexing.
Elasticsearch also has a feature for search idle shards. If a shard has not been searched for a while, its scheduled refreshes are paused until the next search arrives. This saves work on quiet shards and prevents pointless reader churn. It does not replace good interval settings on busy indices, but it helps large time-series clusters where most indices are rarely read.
Optional expiry
Some releases also let you attach an expiry time to cache entries. This is rarely needed, because automatic invalidation is already precise. Adding a timer only makes sense in unusual situations where you want to bound memory use for entries that would otherwise sit unused for very long periods. When in doubt, leave it alone.
9Configuration and Control
You can control the request cache at three levels: the node, the index, and the individual request. There is also a tool to clear it by hand.
Cache size
The setting indices.requests.cache.size controls how much heap the cache may use on each node. The default is 1 percent of the heap. It is a static setting, so it belongs in the node’s configuration file and needs a restart to change.
Enable or disable
The setting index.requests.cache.enable turns the cache on or off for an index. It is on by default and can be changed at any time on a live index.
The request_cache parameter
Set to true, it forces caching for that request, even when documents are returned. Set to false, it skips the cache for that request.
Clear cache
The clear cache operation, with its request option turned on, removes stored answers for chosen indices. It is useful for benchmarking and for recovering from odd behavior.
Choosing a size
One percent of the heap is a sensible default. A node with a 30 GB heap gets roughly 300 MB of cache. That is enough for tens of thousands of small entries. Increasing it is tempting, but the heap is shared with many other needs such as aggregations in progress, the node query cache, and internal data structures. Memory given to the request cache is memory taken from somewhere else.
Before increasing the size, look at the statistics described in Chapter 10. If evictions are low and memory use is far below the limit, a bigger cache brings nothing. If evictions are high but your hit rate is also high, a modest increase may help. If evictions are high and your hit rate is low, the problem is your requests or your refresh pattern, not the size.
Disabling per index
Turning off the cache for an index is reasonable when that index is written constantly and every query is different, for example a raw event stream searched by unique identifiers. In that case the cache does nothing but consume memory and CPU to serialize answers that will never be read. Turn it off for such indices, and keep it on for the ones that serve dashboards.
Forcing caching for one request
The request parameter is your precision tool. A very small, very popular search that returns a handful of documents, such as the ten most recent announcements shown on every home page load, can be worth forcing into the cache, provided the index does not change every second. A large search returning a hundred full documents is almost never worth it.
Clearing the cache by hand
Clearing is a blunt instrument. It empties entries for one or more indices at once, and the next requests must recompute. It is most useful when measuring: before timing a query, clear the caches so you know you are measuring the real cost. In normal operation you should never need it, because invalidation is automatic.
Index settings can usually be changed while the cluster runs. Node settings that are marked static, such as the cache size, are read only at startup. Plan a rolling restart, one node at a time, when you change them.
10Monitoring: Is the Cache Actually Helping?
A cache you never measure is a cache you cannot trust. Elasticsearch exposes four numbers that tell you almost everything.
Statistics about the request cache are available per index and per node through the stats operations, and a summary is also shown in the compact tabular views used by operators. The same figures can be collected by monitoring tools and drawn on dashboards next to CPU, heap, and search latency.
| Statistic | Meaning | What to look for |
|---|---|---|
| memory_size | How many bytes the cache currently holds | Compare it with the configured limit. Very small values mean little is being stored. |
| hit_count | How many lookups found a stored answer | Should grow steadily on dashboard-heavy clusters |
| miss_count | How many lookups found nothing | Every miss is a full query. A huge count means most requests are unique or data changes often. |
| evictions | How many entries were removed to make room | Rising quickly means the cache is too small for the working set, or full of large useless entries |
Computing the hit ratio
The most useful single number is the hit ratio. To find it, divide the hit count by the sum of hits and misses. A ratio of 0.9 means nine out of ten cacheable lookups were answered from memory. Because the statistics are counters that only ever grow, calculate the ratio over a time window by comparing two readings, otherwise old history can hide today’s behavior.
These bands are rules of thumb from common practice, not official thresholds. Your workload defines what is good. A cluster with a ratio of 40 percent that is saving thousands of expensive aggregations per minute may be doing very well.
Reading the signs
Low memory, few hits
The cache is barely used. Requests may contain now, return documents, or change text on every call.
Full memory, many evictions, low hits
Thrashing. Many large, unique entries are pushing each other out. Narrow what gets cached.
Full memory, many evictions, high hits
A healthy but crowded cache. A modest size increase may help.
High hits, still slow searches
The slow work is elsewhere: fetch phase, network, coordinating node merges, or uncached indices.
Watching per index
Node-level totals can hide problems. One noisy index can fill the shared pool and squeeze out useful entries from others. Look at per-index numbers to see who is consuming memory and whether that index deserves it. If a raw event index uses half the cache and shows a hit ratio near zero, disable the cache on it and free the space for dashboards.
Add the four numbers to your regular monitoring dashboard and set an alert on eviction rate. Sudden changes usually point to a release that changed how your application builds search requests, or to a new dashboard that was added without cache-friendly time rounding.
11Performance and Scalability
The cache scales with your cluster, not against it. But its benefits depend on your shard layout, your replicas, and how your data ages.
The cost model
Think of two costs. A miss costs the normal query time plus a small extra to pack the answer into bytes and store it. A hit costs a hash lookup plus unpacking the answer, which usually takes microseconds to a few milliseconds. If the average query costs 100 milliseconds and the average hit costs 1 millisecond, then a hit ratio of 90 percent brings the average shard cost down to about 11 milliseconds. Sharper savings appear when queries are heavy aggregations and blunt savings when queries are already cheap.
Time-series data is the sweet spot
Many clusters store logs, metrics, or events in one index per day or per week. Only the newest index receives writes. Older indices are fixed. The request cache loves this layout: a dashboard covering the last thirty days touches thirty indices, but only one of them keeps changing. The other twenty-nine answer from memory after the first run, and the cost of the dashboard is roughly the cost of the newest index.
flowchart LR
A["Dashboard query for the last 30 days"] --> B["Index for today: actively written"]
A --> C["Index for yesterday: no more writes"]
A --> D["Index for last week: no more writes"]
B --> E["Mostly cache misses"]
C --> F["Mostly cache hits"]
D --> F
E --> G["Merged into one chart"]
F --> G
You can push this further. Once an index is no longer written, mark it read-only and consider merging its segments down to a small number. A read-only index stops opening new readers, and a merged index has less to scan when a miss does happen. Index lifecycle management features automate these steps as data ages.
More shards mean more entries
Each shard stores its own answer, so a search across 200 shards produces up to 200 entries for a single request. That is not a problem for a few requests, but it is a real factor for memory when hundreds of distinct dashboards run against many shards. It is also a reminder that having too many small shards wastes resources in general. A sensible shard size, often tens of gigabytes, keeps both the shard count and the cache footprint under control.
Replicas and cache warmth
Every replica keeps its own cache, and its own segments. Even though a replica holds the same documents as its primary, its readers and segment layout can differ, so an entry created on one copy is of no use to another copy. If the coordinating node rotates requests across copies, the same question might be a miss on the primary, then a miss on the replica, before it finally hits on the primary again.
The search request has an option called preference that steers which shard copies handle it. Sending the same preference value with related requests, for example a session identifier for one user or a fixed word for one dashboard, routes those requests to the same copies each time and warms the cache faster. The trade-off is slightly less even load distribution, so use meaningful preference values with a large variety, not a single global constant.
CPU, heap, and garbage collection
The cache lives in the Java heap. Storing answers as bytes keeps the pressure on garbage collection modest, but cache size still counts as memory you cannot use for something else. If you see long garbage collection pauses, check whether the cache is oversized or filled with big entries from unusual aggregations.
Always compare search latency and CPU usage before and after a caching change. A high hit ratio is a good sign, but the real goal is lower latency and lower cost. If those did not move, the cache was not the bottleneck.
12Advantages, Disadvantages, and Trade-offs
No cache is free. Here is an honest scorecard so you can decide when to lean on it and when to look elsewhere.
Advantages
- Skips the entire query phase on a hit, which is the biggest possible saving for aggregations.
- Always consistent: the answer matches an uncached search on the same data.
- Works automatically and is on by default, so beginners benefit without setup.
- No timers to tune, because invalidation follows data changes.
- Works per shard, so a change in one shard leaves the others warm.
- Excellent for time-series layouts where old data never changes.
- Reduces CPU load, letting the cluster serve more users with the same hardware.
Disadvantages
- Useless for frequently updated indices, because each refresh retires the notes.
- Requires byte-for-byte identical requests, which is easy to break by accident.
- Skips any request using
nowor non-deterministic scripts. - Takes heap memory, and large aggregation results can crowd out other entries.
- Each replica keeps its own cache, so entries are not shared across copies.
- Restarts and shard moves start with an empty cache.
- Does not help the fetch phase, network time, or coordinating node merges.
The central trade-off: freshness against reuse
The more often data becomes visible, the fewer chances the cache has to be useful. If you need every new document to be searchable within a second, you accept that the request cache will rarely help on that index. If your users can wait thirty seconds for fresh numbers, you can lengthen the refresh interval and enjoy far more hits. This is a business decision as much as a technical one, so talk to the people who read the dashboards.
The second trade-off: precision against reuse
Rounding time boundaries is the same trade in a different costume. Rounding to the minute or the hour makes requests repeat and makes charts slightly less exact. For monitoring, that is almost always acceptable. For a billing report that must include the last second, it is not, and that report should skip the cache.
The third trade-off: memory against everything else
The heap is a shared budget. Every megabyte of cache is a megabyte not available to aggregations in flight, the node query cache, or internal bookkeeping. Because the default is only one percent, the risk of overspending is small, but the opportunity to gain from raising it is also usually small.
When the request cache is the wrong tool
If your problem is that a single unique query is slow, the request cache will not help, because it only accelerates repetition. In that case look at the query itself, the mapping, the number of shards, the hardware, and options such as pre-computing summaries with rollup or transform jobs. If your problem is that many users request different filtered variants of a dashboard, work on reducing variety: fewer free-form filters, shared time ranges, or pre-defined segments.
13Design Patterns for Cache-Friendly Searching
Good hit rates rarely happen by accident. These six habits make your requests and your data easy for the cache to reuse.
Round your time
Round time boundaries to the coarsest unit your users can tolerate: the minute, the five minutes, the hour, or the day. All requests inside that window then share one answer.
Freeze the past
Once an index stops receiving writes, make it read-only and merge its segments. It will stop opening new readers, and its notes will last until eviction or restart.
Canonical requests
Make your application write each search body in a fixed field order with fixed spacing. Two calls for the same question should be byte-for-byte identical.
Sticky preference
Send a stable preference value for related requests so they reach the same shard copies and find a warm cache instead of a cold sibling.
Separate fixed from moving
Split one large question into a stable part, like all closed days, and a moving part, like today. The stable part is answered from cache and only the moving part is computed.
Aggregate, do not list
Ask for counts and groups with size set to zero. Keep document lists in separate requests so that the analytical part remains cacheable.
Pattern 5 in action
Suppose a finance dashboard shows revenue per day for the last ninety days. A naive request covers all ninety days in one go with a moving end time, so it can never be cached. A cache-friendly design sends two requests. The first covers days one through eighty-nine using fixed, rounded boundaries and gets answered from cache almost every time. The second covers today only and is computed fresh. The application then stitches the two results together on screen. The heavy part is now nearly free.
Pattern 2 in action
A logging platform keeps daily indices. At midnight a new index starts receiving writes. A short time later the previous day’s index is switched to read-only and merged into a small number of segments. From that moment its notes remain valid. Every dashboard that includes yesterday gets that day’s counts instantly for the rest of its life.
Index lifecycle management can roll over the active index, set old indices to read-only, and merge them on a schedule. That turns pattern 2 from a manual chore into an automatic feature of your platform.
A pattern for shared dashboards
If many users share the same dashboard, pass shared filters from a pre-defined list instead of free text where possible. A drop-down with twenty choices creates twenty possible request texts. A free-text box creates infinitely many. Fewer variations mean more overlap, and more overlap means more hits.
14Anti-Patterns and Common Mistakes
These are the traps that quietly ruin hit rates. Each one is common in real projects and easy to avoid once you know it.
What people do
They write time ranges like “from now minus fifteen minutes to now” and assume the cache will help.
Why it hurts
The current time keyword makes the request non-cacheable, and application-generated exact timestamps make every request unique. The result is zero reuse and a dashboard that recomputes everything on every refresh.
Better approach
Round the boundaries to a unit that makes sense, such as the minute or the hour, and keep the text identical within that window.
What people do
They put a unique tracking value, such as a request id or a timestamp, inside the search body for logging purposes.
Why it hurts
Any extra content inside the body becomes part of the key, so no two requests match. The cache fills with entries that will never be read again.
Better approach
Carry tracking values in request headers or in client-side logs, not inside the search body.
What people do
They set the request parameter to true on every search, including large result lists, hoping to cache everything.
Why it hurts
Big entries fill the pool quickly and push out the small aggregation answers that actually give the best return. Evictions rise and hit rates for useful queries fall.
Better approach
Force caching only for small, hot, repeated searches, and let the default rule handle the rest.
What people do
They raise the cache size again and again whenever evictions appear.
Why it hurts
Evictions can be caused by unique requests or rapid reader changes, which a bigger cache cannot fix. The extra memory is taken from other parts of the heap and can cause slower garbage collection.
Better approach
Look at the hit ratio and the request shapes first. Grow the cache only when hits are high and space is truly tight.
What people do
They run a one-second refresh interval on every index by habit, including analytics indices where nobody needs second-level freshness.
Why it hurts
Frequent refreshes create new readers constantly, retiring cached answers before they can be reused, and they also make indexing more expensive.
Better approach
Ask what freshness the business really needs and set the interval accordingly, per index.
Other frequent mistakes
- Testing performance without clearing caches, then mistaking a warm result for the true cost of a query.
- Assuming the cache works across replicas, when in fact each copy has its own.
- Forgetting that a restart wipes the cache and that a first request after a restart is always slow.
- Believing that a hit removes all latency, when the network and merge steps remain.
- Comparing hit ratios across clusters without considering that the workloads are different.
15Security and Multi-Tenant Considerations
A cache must never show one person data that belongs to someone else. Here is how the request cache fits into that promise.
Many organizations put several teams, customers, or roles on one cluster. Elasticsearch’s security features can limit which documents and which fields a given user is allowed to see. These are known as document level security and field level security. A user might be allowed to see only documents from their own department, for instance.
This raises a fair question. If one user’s search answer is stored in the cache, could another user with different permissions accidentally receive it? The design answer is no. The permissions that apply to a request are folded into what makes the cache entry unique, so two users with different restrictions do not share an entry. A cached answer is only handed to a request that would have produced that same answer under the same restrictions.
Picture a hotel with several floors of guest safes. A guest with a key for floor three should never be able to open a safe on floor five, even if the safes look the same. The request cache is like a set of private lockers: your answer sits in a locker labeled with your access level, and a guest with a different level is sent to a different locker.
Consequences for hit rates
The safety comes at a natural cost. If every user has a unique permission filter, then every user has a unique key, and hardly any two requests share an answer. In multi-tenant systems with one filter per customer, hit ratios are limited by how often the same customer repeats the same question. That is still valuable, because a single customer’s dashboard is often refreshed many times, but it is not the same as a global shared cache.
Memory and privacy
- Heap only. The cache is held in memory and is not written to disk by Elasticsearch. It disappears on restart.
- Shard-local. Entries stay on the node holding the shard. They are not sent to other nodes except as part of a normal search response.
- Content is aggregated. By default, the cached content consists of counts, grouped numbers, and suggestions, not the full documents, which reduces exposure.
- Clearing on demand. Administrators can clear entries for chosen indices if a data correction or compliance action requires it.
If you delete personal data to meet a legal request, remember that new readers retire old answers on their own after the next refresh. For immediate certainty in unusual cases, you can also clear the cache for the affected index. Always confirm your exact obligations with your security and legal teams.
Operational security
Restrict who may call the clear cache and settings operations, because they change performance characteristics for everyone. Treat changes to node-level settings like any other production change: review, roll out gradually, and monitor.
16Real-World Scenarios
Here are four common situations, with rough numbers, showing how the request cache behaves in practice. The companies are imaginary, but the patterns are the ones teams meet every day.
Scenario 1: A wall dashboard for an operations team
A team of 25 engineers opens a Kibana-style dashboard with 12 panels, and each browser refreshes every 30 seconds. That is about 600 panel requests per minute. Each panel searches 30 daily indices with 2 shards each, so every request touches 60 shards. Only today’s index changes. With rounded time boundaries, 58 of the 60 shards answer from cache on most refreshes, and the cluster is doing roughly one thirtieth of the raw work.
Scenario 2: A web shop with live facet counts
The imaginary shop ShopWave shows counts next to every brand and price filter. Its product index has 8 shards and receives batch updates every night, with only rare price edits during the day. Refreshes find no changes for hours, so entries live long. A category page that appears in tens of thousands of visits per hour costs the cluster almost nothing after the first visit. The team keeps its search bodies canonical by generating them with a fixed template.
Scenario 3: A security analytics team with unique queries
A team hunts through a stream of network events with hand-written questions. Nearly every query is new, and the newest index receives tens of thousands of documents per second. Here the request cache shows a hit ratio under 5 percent. The team disables it on the live index, freeing memory, and keeps it enabled for closed indices where saved reports are re-run weekly.
Scenario 4: An internal reporting API
A company exposes a reporting API used by many internal apps. Different apps write the same request with fields in different orders and with different spacing, so hit rates are low. After the platform team publishes a shared client library that generates a canonical request body, hit rates climb from 15 percent to more than 70 percent without any change to the cluster.
A worked example with numbers
Set the scene
A dashboard panel searches an index with 10 shards. A miss costs 150 ms per shard, shards run in parallel, and each hit costs 2 ms.
Cold start
The first person opens the panel. All 10 shards miss. The panel takes about 150 ms of shard time, plus network and merging.
Warm reads
The next 100 people open it. All 10 shards hit. Each panel load takes about 2 ms of shard time. Total shard work fell from 100 times 10 times 150 ms to a tiny fraction.
One shard changes
New documents arrive at one shard, and a refresh opens a new reader there. The next load has 9 hits and 1 miss, so the panel takes about 150 ms once, then returns to 2 ms.
Steady state
If data changes in one shard every minute, one refresh cycle per minute costs 150 ms on one shard, while hundreds of loads per minute cost almost nothing.
17Troubleshooting: My Cache Never Hits
When hit counts stay flat, work through this checklist from the most common cause to the least common.
Check the result size
Confirm that your searches ask for zero documents. If they return hits, the default rule skips them. Split analytics into separate size-zero requests, or force caching for small hot searches.
Look for the now keyword
Search for any use of the current time keyword or exact, constantly changing timestamps in your request bodies. Replace them with rounded boundaries.
Check index and request settings
Verify that the index setting is enabled and that no client is sending the request parameter set to false.
Compare request texts
Capture two supposedly identical requests and compare them character by character. Look for different field order, spacing, or hidden extra fields.
Check the write rate and refresh interval
If the index is written every second, readers change every second. Lengthen the interval if the business can accept it, or split moving data into its own index.
Check for scripts, profiling, or scroll
Non-deterministic scripts, the profile option, and scroll requests are never cached. Remove them from the hot path.
Look at replicas and routing
Requests bouncing across shard copies meet cold caches. Use a stable preference value to steer related requests.
Check evictions and node restarts
High evictions mean useful entries are being pushed out. Frequent restarts or shard moves mean the cache is always warming up.
To prove the cache is working, clear it for one index, send one identical size-zero request twice, and compare the statistics before and after. You should see one miss and then one hit for each shard involved.
18Sizing and Capacity Planning
How big should the cache be? You can answer that with simple arithmetic instead of guesswork.
The request cache holds one entry for every combination of a shard reader and a distinct request. To estimate how much memory you need, you need to know three things: how many distinct requests your users send, how many shards each request touches, and how big an average answer is. The product of those three is your working set, the amount of data the cache would need to hold to answer almost everything from memory.
How big is one entry?
Entry size depends on the shape of the answer. A request that only asks for a total count produces a tiny entry, often a few hundred bytes. A request that groups documents into a hundred buckets, each with a couple of numbers, produces an entry of a few kilobytes. A request that groups into ten thousand buckets can produce hundreds of kilobytes for every shard. Because of this wide range, a handful of heavy aggregations can use more memory than thousands of simple counts.
The best way to learn your real sizes is to measure them. Clear the cache for one index, send one typical request, and read the memory figure from the statistics. The number you see is roughly the entry size for that request across the shards involved.
List the distinct requests
Count how many different search bodies your dashboards and applications produce during a normal day, after rounding time boundaries and making the text canonical.
Count the shards each one touches
A request across 30 daily indices with 2 shards each touches 60 shards. Each shard stores its own entry.
Measure the average entry size
Clear, run, and read memory usage for a few typical requests, then average them, giving extra weight to the heavy ones.
Spread across nodes
The entries are spread over the nodes that host the shards. Divide the total by the number of data nodes that serve these searches, and remember that replicas each keep their own copy of any entry they answer.
Add headroom and compare with the limit
Add roughly thirty percent for growth and uneven distribution, then compare the result with one percent of the heap on each node.
A worked example
Sizing the cache for a dashboard platform
Imagine 40 distinct dashboard panels, each request touching 50 shards, with an average entry size of 5 KB. The working set is 40 times 50 times 5 KB, which is 10,000 KB, or about 10 MB. Even if the number of distinct requests rises to 1,000 because users apply many filters, the working set is about 250 MB across the whole cluster. Spread over ten data nodes, that is about 25 MB per node, which is far below the default limit of roughly 160 MB on a node with a 16 GB heap. In this case the default is more than enough, and the cache should not be enlarged.
Now imagine the same platform adds a panel that groups by a field with 50,000 different values. Each entry could be several megabytes. Multiply by 50 shards and by a few filter variations, and this one panel could fill the cache alone. The right response is not to raise the limit. It is to redesign the panel, for example by showing only the top values, so the entries stay small.
General Elasticsearch guidance is to give the heap no more than about half of a machine’s memory and to stay below the size at which the Java virtual machine loses its compact pointer optimization, which is around thirty gigabytes. Do not grow the heap just to grow the request cache. Leave the rest of memory to the page cache, which serves every search.
Watch the trend, not just the snapshot
Sizing is not a one-time job. New dashboards, new indices, and new filters change the working set. Review the cache statistics after each major release of your application and after adding large indices. If memory use stays close to the limit and evictions climb while hits are high, it is time to look at either the request shapes or a modest size increase.
19Failure Scenarios and Recovery
A cache lives in memory, so anything that disturbs a node or a shard also disturbs the notes. Knowing what to expect keeps surprises small.
Node restart
When a node restarts, its cache is empty. All shards on it begin with misses. During a rolling restart, where nodes are restarted one after another for an upgrade, each node goes through this cold period in turn. If many dashboards refresh at the same time, a burst of full queries hits the freshly restarted node and can cause a short spike in CPU and latency. This burst is sometimes called a thundering herd, because many clients arrive together at a place where nobody has stored the answer yet.
Shard relocation and node failure
Cache entries belong to a shard copy on a specific node. If Elasticsearch moves a shard to another node, or promotes a replica after a node fails, the new home starts with its own cache, which may hold older entries or nothing at all. Rebalancing after adding nodes therefore causes temporary cold periods for the moved shards. The effect is limited to the shards that moved, not the whole cluster.
Bulk loads and heavy merges
A large bulk import or a big merge changes segments and forces new readers. Every shard touched by such a job loses its notes. If a nightly job loads data into indices that dashboards read in the morning, schedule the job so that the last refresh finishes before people arrive, and consider running a quick warm-up pass afterward.
Memory pressure
If heap use runs high, entries can be evicted more often, and useful notes vanish just when the cluster is under load. This can make a bad situation worse, because more misses mean more work. Watching heap, evictions, and garbage collection together helps you spot this loop early.
A school moves the notice board to a new hallway over the weekend. On Monday morning, the board is blank, and the office has to answer every student in person until the notes are pinned up again. If everyone arrives at the same minute, the office is swamped. If they arrive gradually, the notes go up one by one and the pressure spreads out. Restarts and relocations behave the same way.
Ways to soften the cold start
- Stagger restarts. Restart one node at a time and wait for the cluster to settle before moving on.
- Spread dashboard refreshes. Add a small random delay to auto-refresh timers so browsers do not all fire in the same second.
- Warm up on purpose. After a restart or a large load, run a short script that sends your most important dashboard requests. Older versions of Elasticsearch had a built-in warmers feature for this, but it was removed long ago, so teams use their own scheduled jobs.
- Keep enough replicas. Replicas take over when a node is lost. Their caches may be cold at that moment, but the service stays available.
- Protect the cluster. Use sensible timeouts, limits on expensive aggregations, and a queue or rate limit in front of the cluster so a burst does not overwhelm it.
A healthy system should still work when the cache is empty. Treat the cache as a bonus that lowers cost and latency, not as a requirement for staying alive. If your cluster only survives with a warm cache, it is under-provisioned.
20Testing and Benchmarking the Cache
Trust numbers, not hopes. A short, careful experiment tells you exactly what the cache is worth on your data.
Every search response reports a field named took, which is the time in milliseconds that Elasticsearch spent on the search. It is the simplest measuring stick for cache experiments. Compare it across runs, and also compare the end-to-end time your application sees, since network and merge time are outside the cache’s reach.
Four test conditions
| Condition | How to create it | What it tells you |
|---|---|---|
| Baseline, cache off | Send requests with the request cache parameter set to false | The true cost of each query with no help from the request cache |
| Cold, cache on | Clear the request cache first, then send the request once | The cost of a miss, including storing the answer |
| Warm, cache on | Send the identical request again | The cost of a hit, plus network and merge time |
| Under change | Run warm requests while a steady stream of documents is being indexed | How much of the benefit survives real refresh activity |
Measure percentiles, not just averages
Averages hide the slowest requests, and the slowest requests are what users complain about. Record the median, the 95th percentile, and the 99th percentile. A cache typically lowers all three, but the biggest gain is often in the upper percentiles, because those are the heavy aggregations that occasionally pile up.
Test with a realistic mix
A benchmark that sends one request ten thousand times proves nothing except that a hit is fast. Build a mix from real traffic: the same distribution of dashboards, the same time ranges, and the same variety of filters. Replay a recorded hour of production traffic if possible. The hit ratio in the test should resemble what you expect in production.
Test the awkward moments
- After a node restart, to see how long the cold period lasts and how large the spike is.
- During a bulk load, to see how much refresh activity erodes hit rates.
- With different refresh intervals, to find the best balance of freshness and reuse.
- With rounded and unrounded time boundaries, to prove the value of rounding to your stakeholders.
Avoid common testing errors
Do not run tests on a tiny dataset and assume the results scale. Do not compare a warm run with a cold run of a different query. Do not forget that the page cache also warms up, so a second run may be faster for reasons unrelated to the request cache. To separate the two, use the request cache parameter to switch only that cache on and off while everything else stays the same.
A good finding sounds like this: “With rounded time boundaries and a thirty-second refresh interval, the dashboard’s 95th percentile fell from 1.8 seconds to 0.3 seconds, and CPU use dropped by 40 percent at the same traffic.” That sentence is easy for anyone to check and act on.
21Where the Request Cache Fits in a Whole System
The request cache is one layer among many. A good architecture uses each layer for what it does best.
Real products rarely let users talk to Elasticsearch directly. A typical path goes from a browser through a content delivery network, an API gateway, an application service, and maybe an application cache such as Redis, before a search reaches the cluster. Each layer can remember answers, and each remembers them at a different scale and with different rules.
flowchart TD
U["User browser"] --> CDN["Browser cache or CDN"]
CDN --> GW["API gateway"]
GW --> APP["Application service"]
APP --> RC["Application cache such as Redis"]
RC -->|Miss| CN["Elasticsearch coordinating node"]
CN --> SH["Data nodes: shard request cache and node query cache"]
SH --> DK["Segment files on disk through the page cache"]
What each layer is good at
Browser cache and CDN
Best for content that is identical for everyone and can be a few minutes old. It saves the network trip entirely.
Application cache
Stores the final merged answer, often with a time limit you choose. It saves the trip to Elasticsearch, but you must manage freshness yourself.
Shard request cache
Saves shard-level work and stays correct automatically. It cannot save network hops or the coordinator’s merge step.
Query cache and page cache
Speed up reusable filters and raw file reads for the requests that still have to run.
Choosing between an application cache and the request cache
An application cache can be faster, because it skips the cluster altogether. But it returns whatever it stored until its timer expires, so users may see numbers that are a few minutes old even when the data has changed. It also needs its own code, monitoring, and invalidation rules. The request cache is slower than an application cache on a hit, because the request still travels to the cluster, but it is always consistent and needs no code.
Many teams use both. The application cache protects the cluster from the biggest bursts of identical traffic for a short time, and the request cache makes every request that does reach the cluster as cheap as possible. When the two are combined, keep in mind that a short application cache time can hide the request cache’s statistics, because fewer repeated requests arrive at all.
Think of a chain of stores supplying a neighborhood. The corner shop keeps a few popular items on the shelf, which is like the application cache. The regional warehouse keeps ready-packed boxes for common orders, which is like the request cache. The main factory can make anything but takes the longest, which is the shard doing a full query. You want most customers served from the corner shop or the warehouse, and only unusual orders sent to the factory.
Gateway and load balancer roles
The API gateway and load balancer do not read search results, but they shape traffic. A gateway can apply rate limits, authentication, and request normalization. The last of these is especially useful: if the gateway or a small service in front of the cluster rewrites every search into a canonical form, request texts become identical across applications, and the request cache benefits without any change to the clients.
Put the normalization step, meaning rounded times, fixed field order, and pre-defined filters, in one shared place. That single decision often does more for cache hit rates than any tuning inside the cluster.
22Frequently Asked Questions
23Summary and Key Takeaways
The request cache is a simple idea with big payoffs when your workload fits it. Here is everything to remember.
Key Takeaways
- What it is: a per-shard memory of finished search answers, so repeated requests skip the query phase.
- What it stores: by default, only searches that return no documents, meaning counts, aggregations, and suggestions.
- How it stays correct: entries are tied to a shard reader, and a new reader after a refresh retires the old entries automatically.
- What breaks it: the
nowkeyword, non-deterministic scripts, scroll, profiling, changing request text, and constantly changing data. - How to help it: round time boundaries, keep request bodies canonical, freeze old indices, and use a stable preference value.
- How to measure it: watch memory size, hits, misses, and evictions, and compute the hit ratio over a time window.
- How to size it: start with the default of one percent of the heap and grow it only when hits are high and space is tight.
Keep in mind that details can change between Elasticsearch versions, so confirm exact behavior against the documentation of the release you run. With that habit, and the ideas in this guide, you can turn repeated heavy searches into fast, cheap answers.