What Is the Query Cache in Elasticsearch?
A complete, plain-English guide to the memory area that lets Elasticsearch answer repeated yes-or-no questions almost instantly: how it works inside, when it helps, when it hurts, and how to run it in production.
Picture a librarian who is asked the same question fifty times every minute: “Which books on the third floor are still on the shelf?” Each time she climbs the stairs, walks past every shelf, checks every book, and comes back down with the answer. After a while she has a better idea. She writes the answer on a small card and keeps it at her desk. The next time someone asks, she reads the card. No stairs, no walking, no checking.
That small card is the heart of the Elasticsearch Query Cache. It is a quiet feature. Most people never touch it, and many engineers do not even know it exists. Yet on busy clusters it can decide whether a dashboard loads in a blink or drags for seconds. In this guide we start from zero, build up one idea at a time, look inside the machine, and finish with the settings, the numbers, and the mistakes that matter in production.
1What the Query Cache Actually Is
Before any settings or diagrams, let us get one clear picture of what this thing is and what it is not.
Elasticsearch is a search engine. You give it many documents, such as products, news articles, or log lines. Later you ask it questions like “show me red shoes under fifty dollars.” To answer fast, Elasticsearch splits the documents into smaller pieces called shards, and those shards live on machines called nodes.
Every question has two kinds of parts. Some parts are about ranking: “how well does this document match the words I typed?” Other parts are plain yes-or-no checks: “is the price under fifty?”, “is the item in stock?”, “was it published this year?” These yes-or-no checks are called filters. A filter does not care how good a match is. A document either passes or it does not.
The Query Cache is a memory area inside each Elasticsearch data node that remembers the answers to filters. It is also called the node query cache or, in older writing, the filter cache. When the same filter shows up again against the same data, Elasticsearch skips the checking work and reuses the remembered answer.
Think of a teacher with a class of 300 students spread over ten notebooks. Every morning the principal asks, “Who passed maths?” The first time, the teacher reads every line in every notebook. Then she makes a small list for each notebook: a row of tick marks, one per student. The next morning she just reads the tick-mark lists. The lists are the cache. They are tiny compared with the notebooks, and reading them is much faster than reading every line again.
What it remembers, and what it does not
This detail surprises many people. The query cache does not remember your documents. It does not remember your search results in the way a browser remembers a web page. It only remembers which documents matched a filter, stored as a compact list of yes-or-no marks. That is why it can hold a huge number of answers in a modest amount of memory.
Elasticsearch has several caches and the names are confusing. The query cache stores filter results per segment. The shard request cache stores whole shard-level responses. The fielddata cache and the operating system’s file cache are different again. Chapter 8 puts them side by side.
The one-sentence version
If you remember only one line from this chapter, make it this: the query cache is an automatic, node-wide, least-recently-used memory of “which documents passed this filter” for each large data segment, and it speeds up filters that repeat.
Who benefits and who does not
It is worth being clear early about where this cache shines and where it does nothing. It shines for applications that filter a lot, meaning they use many yes-or-no conditions such as category, status, region, tenant, or a date range, and that repeat those conditions across many requests. It does little for applications that mostly do free-text search ranked by relevance, because ranking is scoring work and scoring work is not cached. It also does little for small indices, because they are already fast.
Most real systems are a mix. A shop has free-text search for the search box and filters for the side menu. A log system has free-text for exploring and filters for dashboards. The query cache accelerates the filter half of the mix and leaves the text half alone. Knowing which half your workload leans on tells you how much to expect from it.
2Why It Exists: The Problem of Repeated Work
Every cache is born from one annoying fact: someone keeps asking the same thing.
Look at how people really use search. An online shop shows the same left-hand menu on every page: “In stock only”, “Free delivery”, “Category: Laptops”. A monitoring dashboard refreshes every ten seconds and asks for “errors from the last hour” over and over. A software-as-a-service product adds “tenant equals company 4471” to every query so customers only see their own data. In all these cases, a small set of filters is used again and again, by many users, all day.
Without a cache, Elasticsearch repeats the full checking work each time. For a filter like “price between 20 and 50,” it must walk through the data of every segment in every shard involved and mark which documents fit. On a shard with tens of millions of documents that is real work. Multiply it by hundreds of requests per second across many shards, and the cluster spends much of its time re-answering questions it already answered a moment ago.
A busy restaurant does not chop onions from scratch for every single order. The cooks prepare a big tray of chopped onions before the rush and scoop from it all evening. The query cache is that tray: work done once, reused many times.
A short history
Caching filters is an old idea in search. Early versions of Elasticsearch had a filter cache of their own, and users often had to decide by hand which filters should be cached. Get it wrong and you either wasted memory or lost speed. Over time the project moved toward caching decisions made by the system itself, using the way filters are actually used.
Early days: manual filter caching
Users flagged filters as cacheable or not. Powerful, but easy to misuse.
Around Elasticsearch 2.0: caching goes automatic
Elasticsearch adopted the caching machinery built into Lucene, the search library underneath. The cache watches how often a filter is used and decides for itself. The old name “filter cache” gradually gave way to “query cache,” while the older shard-level cache was renamed the shard request cache.
Later releases: smarter rules
Rules were added to skip tiny segments and very cheap filters, because caching them costs more than it saves.
Today: mostly hands-off
Most clusters run it with defaults. Your job is mainly to write queries that let it work and to watch its numbers.
The exact version details of this history are worth double-checking against the release notes of the version you run, but the direction is clear: the system moved from “you decide” to “the engine decides based on evidence.”
Why not cache everything?
A fair question. If caching is good, why not remember every filter forever? Because memory is limited and because building a cache entry is not free. To store the answer for a filter, Elasticsearch must evaluate it over the whole segment, not just the few documents it would otherwise have needed to check. If the filter is never used again, that effort is wasted. So the cache waits until a filter has proven that it repeats. That patience is the reason the query cache works well without any tuning.
A quick numbers walkthrough
To feel how large the savings can be, imagine some invented numbers. Suppose a costly range filter takes 4 milliseconds to evaluate on one large segment. Suppose each search touches 20 large segments across its shards, and the cluster receives 500 searches per second that use this filter. That is 500 times 20 times 4 milliseconds, which equals 40 seconds of processing work for every single second on the clock. In other words, about 40 processor cores would be busy just re-answering that one question.
Now suppose the cache turns each of those evaluations into a look-up that takes 0.05 milliseconds. The same traffic now needs about half a second of processing per second. The exact figures in real systems will differ, but the shape of the result is what matters: for repeated, costly filters, the difference between recomputing and remembering can be more than a factor of fifty.
3Core Concepts You Need First
A handful of building blocks explain almost everything about the query cache. Learn these and the rest reads like common sense.
Node
One running Elasticsearch process on one machine. Data nodes hold shards and do the searching. The query cache lives inside each data node.
Shard
A slice of an index. Each shard is a complete mini search engine. Replicas are extra copies of a shard on other nodes.
Segment
A small, sealed, read-only chunk of data inside a shard. Each shard is made of many segments. The cache stores its answers per segment.
Filter context
Yes-or-no checking with no scoring. Only work done in filter context is eligible for the query cache.
Query context
Checking plus scoring: “how relevant is this document?” Scores are not stored by the query cache.
Bitset
A row of switches, one per document in a segment. ON means the document passed the filter. This is what the cache stores.
Hit and miss
A hit means the answer was already remembered. A miss means it was not, so the work had to be done.
Eviction
Throwing out the least recently used entries to make room when the cache is full.
Segments: the notebooks that never get erased
This is the most important idea for understanding the query cache. When Elasticsearch stores documents, it writes them into segments, and a segment is never changed after it is sealed. If you update a document, Elasticsearch marks the old copy as deleted and writes a new copy into a newer segment. If you delete a document, it is marked as deleted. Now and then, background workers merge several small segments into one bigger segment and throw the old ones away.
Imagine a school register written in ink. You never scribble over a page. To fix a mistake you cross out the line and add the correct line on a fresh page. Once in a while the office staff copy several thin registers into one thick clean register and recycle the thin ones. Each register, once written, always says the same thing, which makes it perfect for caching.
Because a sealed segment never changes, an answer computed for that segment stays correct for the segment’s whole life. Nothing has to be “refreshed” or “invalidated” when a document is edited elsewhere. The cache entry simply disappears when the segment disappears. This tidy property is what makes per-segment caching safe and cheap.
Filter context versus query context
In the search language of Elasticsearch, a compound query can have clauses that must match and contribute to the score, and clauses that only filter. The “filter” clauses and the “must not” clauses run in filter context. Filter aggregations and constant-score wrappers also run filters without scoring. These are the candidates for the query cache. Clauses in the ordinary “must” and “should” positions run in query context, because they feed the relevance score, which depends on the specific document and on statistics that the cache does not store.
Putting a yes-or-no check like “status is active” in the scoring part of a query gives you nothing extra in relevance and makes it ineligible for the cache. If you do not need a score from it, use filter context.
Bitsets: one switch per document
A bitset is the cache’s storage format. Suppose a segment holds 5 million documents. A filter result is stored as a row of 5 million switches, each ON or OFF. In a plain dense form that costs one bit per document, so roughly 625 kilobytes. When only a few documents match, Lucene uses a cleverer sparse layout that takes even less memory. Either way, checking “did document number 4,000,123 pass?” is one quick look-up.
Quick glossary
| Word | Plain meaning |
|---|---|
| Cache | A place where answers are kept so they can be reused instead of recomputed. |
| Heap | The working memory of the Java program that Elasticsearch runs in. |
| Merge | Combining several small segments into one bigger segment in the background. |
| Refresh | The moment when newly indexed documents become visible to searches, usually by creating a new segment. |
| Hit ratio | The share of look-ups that found a ready answer. |
| LRU | Least recently used: the rule that removes the entries untouched for the longest time first. |
| Static setting | A setting that can only change when a node restarts or an index is closed. |
| Cold and warm | Cold means the cache is empty or nearly so. Warm means it holds the entries the workload needs. |
4Architecture and Components
Where does the query cache sit inside a real cluster, and which parts work together to make it happen?
A production Elasticsearch cluster usually has a front door (a load balancer), a few coordinating or client-facing nodes, and many data nodes that hold the shards. A search request enters at the front, is fanned out to the shards that hold the data, and each shard does its part of the work on its segments. The query cache is consulted at that last step, deep inside the data node.
flowchart TB
Client["Client or Kibana dashboard"] --> LB["Load Balancer"]
LB --> Coord["Coordinating Node"]
Coord --> ShardA["Shard 1 copy on Data Node A"]
Coord --> ShardB["Shard 2 copy on Data Node B"]
subgraph NodeA["Data Node A"]
ShardA --> SegA1["Large Segment"]
ShardA --> SegA2["Small Segment"]
QCA["Query Cache: one per node, shared by all shards, lives in JVM heap"]
SegA1 -.-> QCA
end
subgraph NodeB["Data Node B"]
ShardB --> SegB1["Large Segment"]
QCB["Query Cache of Node B"]
SegB1 -.-> QCB
end
Read the picture from top to bottom. The client’s request travels through the load balancer to a coordinating node. That node knows which shards hold the data and forwards the request to a copy of each needed shard. Inside each data node, the shard searches its segments. When a filter must be evaluated against a large segment, the node looks in its own query cache first. The dotted lines show that relationship. Notice that the small segment has no dotted line: segments below the size thresholds are skipped by the cache.
The main components
The cache itself
A least-recently-used store built on Lucene’s query cache, wrapped by Elasticsearch so that it can share one memory budget across all shards on the node.
Caching policy
Tracks how often each filter has been used recently and decides whether it is worth caching yet. Costly filters qualify sooner than cheap ones.
Segment size rule
Refuses to cache for tiny segments. A segment needs at least 10,000 documents and at least 3 percent of its shard’s documents.
Cached bitsets
One entry per pair of (filter, segment). The key is the filter itself plus the identity of the segment.
Statistics counters
Hits, misses, cached counts, evictions, and memory used. These are tracked so you can see how healthy the cache is.
Settings and clear API
A node-level size setting, an index-level on and off switch, and a clear-cache endpoint that can target the query cache alone.
One cache per node, not per index
A common misunderstanding is that every index or every shard gets its own private cache. It does not. There is one cache per node. All shards on that node, from all indices, share one memory budget. Elasticsearch still keeps statistics per shard so you can see who is using what, but the memory pool is common.
It is like a shared whiteboard in an office. Every team on the floor can write answers on it, but the board has a fixed size. When it fills up, the oldest, least-used notes are wiped to make room. A team that writes a lot can push out another team’s notes.
The query cache lives in the Java heap of the data node, the same memory that the rest of Elasticsearch needs to run. That is why its default is a percentage of heap, and why giving it too much can starve other parts of the node.
Each shard copy has its own segments
A replica is not a byte-for-byte copy of its primary shard. Each copy indexes documents on its own and merges segments on its own schedule, so the segments on two copies of the same shard can differ. Since cache entries belong to segments, cached answers on one copy do not help another copy. Chapter 10 shows how to use this fact to your advantage when routing searches.
Walking one request through the machine
Let us follow a shopper who clicks “Laptops, in stock, under 1,000 dollars” on a store’s website.
The website builds a search request
The words go into the scoring part. The category, the stock status, and the price limit go into the filter part.
The request reaches a coordinating node
It decides which shards hold product data and sends the request to one copy of each.
Each data node searches its segments
For each large segment, the three filters are checked against the node’s query cache.
Cached filters answer immediately
The stock filter and the price filter were used by thousands of earlier shoppers, so their bitsets are already stored. The cheap category term is evaluated directly.
The scoring work happens on the survivors
Only documents that passed every filter are scored for how well they match the typed words. The node returns its best few to the coordinating node.
The coordinating node merges the answers
It picks the overall best results and returns them to the website, which shows the page.
5Internal Working: What Happens on Each Lookup
Now we open the box. Every time a filter runs against a segment, a small chain of decisions plays out in a fraction of a millisecond.
When a shard searches one of its segments, Lucene builds a helper object for each filter clause. The query cache wraps that helper. The wrapper does not run the filter immediately. It first asks a few questions, in order. The flowchart below shows the path.
flowchart TD
A["Filter clause arrives at one segment"] --> B{"Is the query cache enabled for this index?"}
B -- No --> Z["Run the filter normally"]
B -- Yes --> C{"Is the segment big enough to be eligible?"}
C -- No --> Z
C -- Yes --> D{"Is the answer already in the cache?"}
D -- Yes --> H["Cache hit: reuse the stored bitset"]
D -- No --> E{"Has this filter been used often enough recently?"}
E -- No --> Z
E -- Yes --> F["Run the filter over the whole segment and store the bitset"]
F --> G["Use the bitset to answer"]
Z --> M["Count a miss"]
Check the switch
If caching is turned off for the index, the filter just runs. Nothing is stored or looked up.
Check the segment size
Only segments with at least 10,000 documents, holding at least 3 percent of the shard’s documents, are eligible. This keeps tiny, short-lived segments from cluttering the cache.
Look up the entry
The cache is asked: do you have a stored answer for this exact filter on this exact segment? If yes, that is a hit, and the stored bitset is used directly.
Ask the usage tracker
If there is no stored answer, the policy checks how many times this filter has been seen recently. A short rolling memory of recent queries is kept for this purpose.
Decide whether to build an entry
Costly filters, such as ranges over numbers and dates or lookups of large lists of terms, are cached after only a couple of recent uses. Cheap filters need more repeats, and the very cheapest, like a lone term look-up, are generally not worth caching at all. These thresholds are a Lucene detail and can shift between versions.
Build and store
The filter is evaluated over every document in the segment, the resulting bitset is stored, and the cache’s memory counter goes up. If the cache is over its limit, the least recently used entries are evicted.
A shop owner notices that customers keep asking, “Which items are on sale?” The first few times, she just answers from memory of the shelves. Once she sees the question is a regular, she spends an evening going through the whole store and writes a full sale list. Writing the list costs a lot of effort, so she only does it for questions that keep coming back.
Why building an entry is more expensive than one normal run
In a normal run, a filter often only has to check documents that other clauses have already narrowed down. Suppose a query asks for “laptops” and also “in stock.” The engine may find laptops first and then check “in stock” for only those. To build a cache entry, however, the engine has to evaluate “in stock” for every document in the segment, because the stored answer must be complete and reusable for any later query. That extra effort is the price of the entry, and it is why the policy insists on proof of repeat use.
What the stored answer looks like in memory
Lucene picks the storage shape based on how many documents matched. If few documents matched, it stores a compact, sparse structure. If many matched, it stores a plain row of bits. The size of a dense entry is about the number of documents in the segment divided by eight bytes. A segment of 100 million documents therefore costs around 12.5 megabytes per dense entry. That is why a few very large segments with many different filters can fill a cache quickly.
Least recently used eviction
When the total memory of all entries passes the limit, the cache removes entries that were used the longest time ago. The idea is simple: what you have not needed lately, you probably will not need soon. Popular filters stay, one-off filters fade away.
A little deeper: the role of Lucene
Elasticsearch did not invent this cache from scratch. Lucene, the open-source search library at the core of every shard, ships with its own query cache, and Elasticsearch wraps it with extra rules: the segment-size gate, node-wide memory accounting, and per-shard statistics. This layered design means that improvements made in Lucene arrive in Elasticsearch through upgrades, which is one reason the exact thresholds can shift from one version to the next.
Weights and scorers, in plain words
When a search begins, each clause is turned into a weight, which is a kind of prepared plan that knows how to search this particular index. Then, for each segment, the weight produces a scorer, a small object that walks through the matching documents one after another. The cache steps in at exactly this point. Instead of handing back the scorer that walks the segment the slow way, it may hand back a scorer that walks the stored bitset. The rest of the search does not know the difference. That is what makes the design elegant: caching is invisible to the rest of the engine.
A navigation app can either work out a route from scratch or show a route it saved earlier. Either way, the driver follows the same kind of turn-by-turn instructions and never needs to know which one the app used. The saved route just appears sooner.
Working together with other clauses
A typical search combines several clauses, and the engine intersects them, meaning it looks for documents that appear in all of them. A cached bitset is very friendly to this job. It can jump straight to the next matching document without any checking, so it works well whether it leads the intersection or follows another clause. When several filters are cached, the intersection becomes a fast pass over a few stored rows.
Many threads at once
A data node answers many searches at the same time, using many threads. The cache is designed for that world. Reads are frequent and writes, meaning the building of new entries, are rare in comparison. As entries are added and removed, the cache keeps a running total of memory used, and this total is what the size limit is compared against. Elasticsearch also keeps hit and miss counts per shard so that you can see which indices are getting the most value.
6Data Flow and Lifecycle of a Cache Entry
Cache entries are born, used, and eventually die. Understanding that life story helps you predict when the cache will be warm and when it will be cold.
flowchart LR
R1["First requests: cache miss"] --> R2["Filter seen often: bitset stored"]
R2 --> R3["Later requests: cache hits"]
R3 --> R4{"What happens next?"}
R4 -- "Memory limit reached" --> R5["Least recently used entries evicted"]
R4 -- "Segment merged away" --> R6["Entries of the old segment dropped"]
R4 -- "New refresh adds a small segment" --> R7["New segment may be too small to cache"]
R5 --> R1
R6 --> R1
A day in the life of one filter
Follow one filter, “status is published,” on a large segment in a busy news search cluster.
Requests one and two: cold
The filter has no history. It runs normally and is counted as a miss. The usage tracker writes it down.
Request three or so: promoted
The tracker has now seen the filter enough times. This request pays the extra price and builds the bitset for the whole segment.
Requests four onward: fast
Every request finds the stored bitset. These are hits. The filter now costs almost nothing, no matter how many users repeat it.
New articles arrive: new segments
Publishing creates new small segments at each refresh. They are below the size threshold, so the cache ignores them and they get searched the normal way. That is fine, because they are small and cheap to scan.
A merge happens
Background merging combines segments into a larger one. The old segments vanish and their cache entries are dropped. The new bigger segment starts with no entry, so the cycle begins again for it.
Pressure or silence: eviction
If other filters need room, or nobody uses this filter for a long while, its entry is evicted and it must earn its place again.
What refreshes, updates, and deletes do
A refresh makes newly indexed documents searchable by writing them into a new segment. It does not touch existing segments, so it does not invalidate existing cache entries. This is a huge advantage over caches that must be wiped whenever data changes.
An update is really a delete of the old document plus an insert of the new document into a fresh segment. A delete marks the document as gone. Deleted documents are removed from results separately when the search combines its pieces, so a cached answer for an older segment remains valid even after some of its documents are deleted. In short, ordinary writes do not spoil the cache. What hurts is merging away large segments that were popular.
Indices that no longer receive writes, such as last month’s logs, stop producing new segments and merges. Their entries live as long as memory allows. That is the best case for the query cache: stable segments, repeated filters, and a lot of reuse.
Cold starts
After a node restarts, the cache is empty. The query cache lives in memory and is not saved to disk. Filters warm up again as they are used. After a big merge or a shard moving to a new node, the same thing happens for the affected segments. Cold periods are normal. Plan for them, and do not panic when latency is a little higher for the first minutes after a restart.
Restarts, rebalancing, and shard moves
Cold starts do not only happen after a restart. Whenever a shard moves from one node to another, the old node drops that shard’s entries and the new node starts with nothing for it. Shards move when you add or remove nodes, when the cluster rebalances itself, when a node fails and its shards are rebuilt elsewhere, and during rolling upgrades.
If you add three new nodes at the height of the busy hour, the rebalancing that follows can drag down search speed for a while, because many shards start cold at once. Whenever possible, schedule planned changes for quieter periods, and limit how many shards may move at the same time so that the cluster warms up gradually rather than all at once.
7What Gets Cached and What Does Not
The rules are simple once you see them as a checklist. A filter must be a filter, the segment must be big enough, and the filter must repeat.
| Situation | Eligible? | Why |
|---|---|---|
| Clause in the “filter” part of a compound query | Yes | Filter context, no scoring needed. |
| Clause in the “must not” part | Yes | Also runs in filter context. |
| Query wrapped as a constant score | Yes | Scoring is replaced by a fixed value. |
| Filter-style aggregations | Often | They use the same filter machinery. |
| Clause in “must” or “should” that produces a score | No | Scores depend on statistics the cache does not store. |
| Segment below 10,000 documents or below 3 percent of the shard | No | Too small to be worth the memory and effort. |
| Very cheap filters such as a lone term look-up | Usually not | Faster to recompute than to store and fetch. |
| Costly filters such as ranges and large term lists | Yes, quickly | Expensive to compute, so caching pays off soon. |
| Index with the cache turned off | No | Switch set to false. |
| Filter that is never repeated | Not for long | The usage tracker never sees enough repeats. |
The “now” problem
Dashboards love the phrase “the last 15 minutes.” Under the hood, the word “now” is turned into an exact moment in time, down to the millisecond, each time the query runs. Two requests one second apart therefore ask about two slightly different ranges. Because the cache key contains the exact filter, the two do not match, and every request looks brand new. The filter is technically cacheable but almost never repeats, so it never gets reused.
The fix is rounding. If you write the time range so that it rounds to the minute, the hour, or the day, all requests within that period ask the identical question and share one cache entry. A dashboard that rounds to the nearest minute might trade a few seconds of freshness for a much better hit rate.
Imagine asking a friend, “What happened in the last 3 minutes and 12 seconds?” and then, one second later, “What happened in the last 3 minutes and 13 seconds?” She has to think again every time. If you both agree to ask, “What happened this minute?” she can reuse her notes for the whole minute.
Filters that embed a different random value each time, such as a unique session token or a request id used as a filter, are guaranteed misses. They cannot repeat, so they only waste effort. Keep such values out of filter clauses whenever you can.
Why scoring queries are left out
A score answers “how well does this document match?” and depends on things like word frequencies across the whole index. A bitset can only say “yes” or “no.” Trying to cache scores would need much more memory per entry, and the scores would go stale as data changes. By restricting itself to yes-or-no answers, the cache stays small and always correct.
Cheap versus costly filters, in plain words
The cache treats filters differently depending on how hard they are to compute, so it helps to know which are which. This classification is done by Lucene, and the details can vary a little between versions, but the general picture is stable.
A cheap filter is one that the index can answer almost immediately. Looking up a single exact value in a keyword field is the classic example. The index keeps a sorted dictionary of all values and, for each value, a ready-made list of documents. Finding one entry in that dictionary is very fast, so caching it would barely help.
A costly filter is one that must visit many places. Number and date ranges have to gather documents from many blocks of the index. Prefix, wildcard, and pattern matches have to walk through many dictionary entries. Lists containing many values have to combine many document lists. Filters on geographic areas are also heavy. Those are the filters that repay caching quickly.
Finding one person in a phone book is quick: you flip to the name. But “list everyone whose surname starts with Sm” or “everyone whose surname falls between Ma and Mo” means reading many pages. The second kind of question is the one where a saved list is worth having.
8Query Cache Versus the Other Caches
Elasticsearch has several layers of memory tricks. Knowing which layer does what stops you from tuning the wrong one.
flowchart TB
Q["Search request"] --> RC["Shard Request Cache: whole shard-level responses"]
RC -->|miss| QC["Query Cache: filter bitsets per segment"]
QC --> FD["Fielddata cache and doc values: field data structures"]
FD --> OS["Operating system file cache: raw index files"]
OS --> DISK["Disk"]
The picture is a simplification, since the layers do not always run strictly in this order, but it gives the right feeling: the higher a cache sits, the more work it can save when it hits.
| Cache | What it stores | Level | Typical default | Emptied when |
|---|---|---|---|---|
| Query cache | Which documents passed a filter | Per segment, node-wide pool | 10 percent of heap | Segment disappears, memory pressure, or manual clear |
| Shard request cache | Complete shard-level results, such as aggregation results and hit counts, for requests that return no hits | Per shard | 1 percent of heap | The shard’s data changes at a refresh, or manual clear |
| Fielddata cache | In-memory value tables for sorting and grouping on certain text fields | Per node, in heap | Unbounded unless limited | Memory limit, or manual clear |
| Operating system file cache | Raw bytes of index files, including doc values | Machine, outside the heap | Whatever free memory the machine has | The operating system decides |
A kitchen has a fridge, a counter, a spice rack, and a pantry. The shard request cache is a finished dish waiting on the counter, ready to serve. The query cache is a tray of prepped ingredients. The fielddata and doc values are the spice rack laid out for common recipes. The operating system cache is the pantry near the door, so you do not have to go to the warehouse (the disk) each time.
How to think about the difference
The shard request cache is the biggest shortcut. When a whole request is repeated and the shard has not changed, the answer is returned right away. It works best for aggregation-heavy dashboards on data that rarely changes. But it is thrown away whenever the shard changes at a refresh, so on a busy live index it may seldom get to help.
The query cache is smaller in scope but far more resilient. It works on pieces of a query, on individual segments, and it survives writes because sealed segments do not change. Two totally different requests can share a query cache entry as long as they contain the same filter.
The operating system file cache is the unsung hero. It keeps recently read index files in memory outside the Java heap. It is the reason Elasticsearch guidance says to leave a large portion of machine memory, often at least half, to the operating system rather than giving it all to the heap.
Elasticsearch has a clear-cache endpoint, and you can tell it to clear only the query cache, only the request cache, or only fielddata. Use the narrow option. Clearing everything on a busy production cluster forces every node to start cold at the same time.
Which cache should you look at first?
When search feels slow, it is tempting to reach for the cache you know best. A better habit is to ask three questions in order. First: is the same whole request repeated on data that rarely changes, such as a dashboard with many aggregations on old indices? If so, the shard request cache is the first place to look. Second: is the request full of repeated filters over large segments? Then the query cache is your friend. Third: are reads going to disk again and again? Then the operating system file cache, and the memory left for it, is the real problem.
Tuning the wrong cache is a classic time-waster. An engineer who enlarges the query cache to fix a problem caused by too little file-system memory will only make the problem worse, because the enlarged heap takes memory away from the very cache that mattered.
9Configuration and Sizing
The query cache has very few knobs. That is by design. Knowing exactly what each one does and when to leave it alone is most of the skill.
| Setting or tool | Scope | What it controls | Default |
|---|---|---|---|
indices.queries.cache.size | Node, static | Maximum memory of the query cache. Accepts a percentage of heap or an exact size such as 512mb. | 10 percent of heap |
index.queries.cache.enabled | Index, static | Turns query caching on or off for that index. | true |
| Clear cache endpoint | Index or cluster, on demand | Empties caches, and can be limited to the query cache only. | Not applicable |
“Static” means the setting cannot be changed on a running node or an open index. The node-level size lives in the node’s configuration file, and it takes effect after that node is restarted. The index-level switch can be set when the index is created, or changed while the index is closed. There is no dynamic dial you can turn during a production incident, so decisions here should be made calmly, ahead of time.
Doing the heap arithmetic
Suppose a data node has 64 gigabytes of RAM and you give the Java heap about 30 gigabytes, which stays under the range where Java can use compact object pointers. The default cache limit is 10 percent of that heap, so about 3 gigabytes. If the average dense entry on a big segment costs 12 megabytes, the cache holds around 250 such entries before it starts to evict. That gives a feel for why a few huge segments with many different filters can fill it quickly.
When to raise the size
- Your monitoring shows a steady stream of evictions while the hit ratio is decent.
- The workload has many different but repeating filters, such as a multi-tenant product with hundreds of active tenants.
- The heap has clear room to spare and garbage collection is calm.
When to lower the size or turn caching off
- The hit ratio is very low and the cache is full of entries that are never reused.
- The heap is under pressure and you need memory for other things.
- An index is write-heavy with constantly changing large segments, or it holds only small segments, so the cache barely helps.
Raising the cache to a large share of heap feels harmless, but every byte given to the cache is a byte the rest of Elasticsearch cannot use. Aggregations, indexing buffers, and the garbage collector all need heap. Raise in small steps and watch memory and pause times after each change.
Think of a backpack of fixed size. The cache is one pocket. Making the pocket larger squeezes the space for your lunch, water, and books. The right size is not the biggest one; it is the one that leaves every other item room to fit.
A sensible starting rule
Leave the defaults on. Watch the metrics for a few weeks of real traffic. Change one thing at a time. Write down what you changed and why, so that a teammate next year can understand the reasoning.
A worked capacity example
Numbers make ideas real, so here is a made-up but realistic scenario. An online shop runs six data nodes, each with a 30 gigabyte heap, so each node has roughly a 3 gigabyte query cache and the cluster has about 18 gigabytes in total. The product index has 12 shards with one replica each, which means 24 shard copies spread over six nodes, or four copies per node.
Say each shard copy holds about 16 million documents, mostly in four large segments of around 4 million documents. A dense cache entry for one filter on one such segment needs about 4 million divided by 8, so roughly 500 kilobytes. One filter on one shard copy therefore costs about 2 megabytes when dense, and one filter across the four copies on a node costs about 8 megabytes.
| Step | Calculation | Result |
|---|---|---|
| Entry per large segment | 4 million documents divided by 8 | About 500 KB |
| Entry per shard copy | Four large segments | About 2 MB |
| Entry per node for one filter | Four shard copies | About 8 MB |
| Filters that fit in 3 GB | 3,000 MB divided by 8 MB | Around 375 filters |
If the shop has about 150 frequently used filters, the cache fits them comfortably, with room to spare. But if the shop suddenly lets shoppers pick any exact price range, thousands of slightly different range filters appear and the cache begins to churn. Sparse entries take less room than dense ones, so the real capacity is usually a bit higher, but the lesson holds: the number of distinct filters matters more than the number of requests.
Rather than growing the cache to fit every custom price range, the shop can offer fixed price bands such as under 25, 25 to 50, and 50 to 100. The number of distinct filters drops from thousands to a handful, and the hit ratio jumps without adding any memory.
Mixed workloads and node roles
Many teams run indexing, heavy aggregations, and interactive searches on the same nodes. Each of those activities wants heap: indexing needs buffers, aggregations need working memory, and the query cache wants its share. When the heap is tight, they compete, and the query cache is a convenient place to blame even when the real culprit is a heavy aggregation.
Separating roles can help. Dedicated nodes for interactive search keep the cache warm for the users who care most. Dedicated nodes for heavy batch analytics keep expensive work away from those users. On a smaller cluster this may not be affordable, and that is fine. The important thing is to know that the cache shares a heap, and to look at heap usage as a whole before judging the cache in isolation.
Checking your version before you tune
The names and defaults in this chapter match the documentation for recent Elasticsearch releases, but software changes. Before you act on any number, do a short check. Open the reference documentation for the exact version you run and read the pages on the query cache and on node settings. Look at the release notes of the versions between yours and the newest one, in case a threshold or a setting was changed. And look at what your own cluster reports, because the statistics output is the most reliable description of what your nodes are actually doing.
This habit pays off during upgrades too. A major version upgrade can rebuild segments, restart every node, and reset every cache, so you should expect a temporary slowdown right after it and not mistake that for a permanent regression.
10Performance and Scalability
Where does the speed-up come from, how big can it be, and what limits it as your cluster grows?
The gain comes from replacing a scan with a look-up. A costly filter over a large segment may take milliseconds of processing. Reading a stored bitset takes a tiny fraction of that. When a request contains several filters and each one is cached, combining them is just a fast bitwise operation over the stored rows. On filter-heavy searches, such as product catalogs with many facets, this can be the difference between a request that feels instant and one that feels sluggish. The exact size of the improvement depends on your data and filters, so measure it on your own workload rather than trusting a number from a blog post.
Combining cached filters is like laying several transparent sheets, each with holes punched at matching positions, on top of each other. Wherever light shines through every sheet, the document passes all filters. Lining up sheets is far quicker than re-checking every document by hand.
What makes the cache perform well
Repeating filters
A small set of filters used by many requests gives a high hit ratio. Random filters give almost none.
Large, stable segments
Old, read-only indices with a few big segments hold entries for a long time. Constant merging keeps resetting them.
Sticky shard copies
Sending the same user or session to the same shard copies helps the same cache entries get reused.
Rounded time ranges
Rounding “now” turns a stream of unique filters into a few repeating ones.
Scaling out
Because there is one cache per node, adding nodes adds cache capacity. Twenty nodes with 3 gigabytes each give the cluster far more total cache than three nodes. However, spreading the same data over more nodes also spreads the filters, so each node sees fewer repeats of each filter, which can lower the hit ratio a bit. In practice the extra capacity usually wins.
Replicas and the preference trick
Recall that each shard copy has its own segments and therefore its own cache entries. If a user’s requests bounce randomly between the primary and its replicas, the same filter has to be learned and stored separately on each copy. By passing a preference value, such as a session or user identifier, with the search request, you make the same user go to the same copies again and again. The cache entries built by the first request are then reused by the next. Elasticsearch’s own tuning advice lists this as a way to improve cache use.
Force merging read-only data
For indices that will never be written again, merging their segments into fewer, larger ones can improve search speed and give the query cache fewer, bigger segments to hold entries for. Do this only on data that is truly finished, because a force merge on an index that still receives writes can create huge segments that are expensive to maintain.
The cost side
- Fill cost: the first build of an entry scans the whole segment.
- Memory cost: entries occupy heap that other features might want.
- Churn cost: after merges or restarts, entries must be rebuilt.
A healthy cache pays these costs back many times over. An unhealthy one, filled with never-reused entries, only pays them and returns nothing.
The best test of the cache’s value is a before-and-after comparison on real traffic, looking at search latency and at the cache’s own hit and eviction numbers. Chapter 13 shows which numbers to read.
Refresh, merges, and how they shape the cache
Every refresh turns freshly indexed documents into a new small segment. On an index that takes many writes, dozens of tiny segments can pile up. Elasticsearch’s default merge policy, which is tiered, keeps combining segments of similar size into bigger ones. Small segments are merged often and quickly. Large segments are merged rarely, because merging them is expensive.
This pattern happens to suit the query cache. The tiny, fast-changing segments are exactly the ones the cache ignores. The large, slow-changing segments are exactly the ones it stores entries for. Trouble only starts when a big merge replaces a popular large segment, because the new segment has no entries and needs to build them again.
A longer refresh interval on write-heavy indices means fewer tiny segments and fewer merges, which lowers churn overall. It also means new documents become searchable a little later, so this is a business decision as well as a technical one. Ask what freshness users actually need before changing it.
Think of a warehouse with a small “incoming” table and big shelves. New parcels land on the table and are sorted onto shelves later. Nobody bothers to write a catalogue for the incoming table, because its contents change every minute. The catalogue is written for the shelves, which stay the same for weeks. Only when the warehouse reorganizes an entire aisle does a catalogue need rewriting.
11Advantages, Disadvantages, and Trade-offs
No cache is free. Here is the honest balance sheet.
Advantages
- Automatic: no manual flags for most workloads.
- Safe: sealed segments never change, so entries are never stale.
- Compact: stores only bitsets, not documents or scores.
- Shared across requests: different searches with the same filter reuse one entry.
- Survives ordinary writes: refreshes, updates, and deletes do not wipe it.
- Scales with the cluster: more nodes give more total cache.
Disadvantages
- Uses Java heap, which is precious.
- Useless for scoring clauses and for filters that never repeat.
- Cold after restarts, shard moves, and big merges.
- Does not help small segments or very cheap filters.
- Static settings: no on-the-fly tuning in an emergency.
- One noisy tenant can push another tenant’s entries out of the shared pool.
| Trade-off | Leaning one way | Leaning the other way |
|---|---|---|
| Cache size | Bigger cache: more hits, less heap for everything else | Smaller cache: safer heap, more recomputation |
| Freshness of time ranges | Rounded to the minute: high reuse, results a bit older | Exact “now”: always fresh, almost no reuse |
| Segment size | Fewer big segments: better caching, costlier merges | Many small segments: cheap merges, nothing cached |
| Replica routing | Sticky preference: better reuse, possible uneven load | Random spread: even load, fragmented cache |
| Cache on or off per index | On: helps repeating filters | Off: saves memory on indices where it cannot help |
A bigger desk lets you keep more notes within reach, but a big desk crowds the room. Sticky notes are quick, but a wall covered in notes nobody reads slows everyone down. Good caching is about keeping only the notes you really use.
How much should you lean on the query cache?
| Workload | Expected benefit | Reason |
|---|---|---|
| Faceted search with a small set of repeated filters | High | Many repeats over large, stable segments. |
| Dashboards over older, read-only indices | High | Segments never change, filters repeat every refresh. |
| Dashboards over the newest, fast-changing index | Medium | Large segments still help, small new ones do not. |
| Pure full-text search ranked by relevance | Low | Most work is scoring, which is not cached. |
| Tiny indices | None | Segments fall below the size gate. |
| Filters with unique values on every request | Negative | Entries are built and never reused. |
12Security Considerations
A cache is a shortcut, and shortcuts deserve a second look from a security point of view.
What is inside the cache
The query cache holds bitsets: marks that say which document positions matched a filter. It does not hold the text of your documents. That already limits what could leak. It lives in the memory of the Java process, so anyone who can read that process’s memory or take a heap dump has access to much more than the cache anyway. Protecting the machine and the process is the first layer of defense.
Different users, different views
In systems that use document-level security, different users see different subsets of the same index. The cache key is built from the exact filter and the segment, so entries that came from different filters do not get mixed up. Elasticsearch’s security features are designed so that one user’s restricted view is not handed to another. If your design depends on this, test it with two roles that have different permissions and confirm the results, and read the documentation for your specific version.
Never assume. In a multi-tenant system, write a test that runs the same search as two different tenants and checks that neither ever sees the other’s documents, both with a cold cache and a warm one.
Resource abuse: the shared whiteboard problem
The more realistic risk is not data leakage but resource pressure. Because all shards on a node share one pool, someone who sends thousands of different filters against large segments can push other people’s useful entries out and burn CPU building bitsets. Ways to reduce this risk:
- Put rate limits and query timeouts in front of the cluster.
- Give heavy or untrusted workloads their own nodes or their own cluster.
- Restrict which users can send arbitrary, very expensive queries.
- Watch eviction counts for sudden spikes.
Timing differences
A cached request answers faster than an uncached one. In theory, a very determined observer could notice timing differences and learn something about what others recently searched. For most business systems this is a low risk, and it is one more reason to keep access to the search API behind proper authentication and to avoid exposing it directly to the public internet.
If you notice that a shop assistant answers one question instantly, you can guess that someone else asked it a moment ago. The information you gain is small, but in a place where even small hints matter, you would not let strangers stand at the counter.
Security checklist
- Turn on authentication and encrypted connections between clients and nodes.
- Use role-based access rather than sharing a superuser account.
- Separate high-risk tenants or workloads physically when the stakes are high.
- Include cache metrics in your alerts, because a sudden change can be the first sign of abuse.
Most of the security value around the query cache comes from ordinary good practice: authentication, encryption, role-based access, and limits on expensive queries. The cache itself needs no special treatment beyond being included in your monitoring.
13Monitoring, Logging, and Metrics
A cache you cannot see is a cache you cannot trust. These are the numbers to watch and how to read them.
Elasticsearch reports query cache statistics through its node and index statistics endpoints, and the compact table view for nodes offers short columns for the same values. Most monitoring tools, including Elastic’s own, can chart them over time.
| Metric | Meaning | What to look for |
|---|---|---|
| Memory size | Bytes currently used by entries | Near the limit for long periods suggests pressure. |
| Hit count | Lookups answered from the cache | Should grow steadily on a healthy workload. |
| Miss count | Lookups that were not found | Grows quickly if filters never repeat. |
| Total count | Hits plus misses | Used to compute the ratio. |
| Cache count | Entries ever added | Compare with cache size to see churn. |
| Cache size | Entries currently stored | A flat line at a ceiling means the cache is full. |
| Evictions | Entries thrown out for lack of room or age | Sustained high growth means thrashing. |
The hit ratio
The single most useful number is the hit ratio: hits divided by hits plus misses. A high ratio means that the cache is doing its job. A low ratio is not always a problem, because cheap filters and small segments are not supposed to be cached and simply count as misses. Compare the ratio with your own past behavior rather than with an ideal figure.
Three patterns and what they mean
High hits, few evictions
Memory sits comfortably below the limit. Nothing to change.
Full cache, many evictions
Entries are built and discarded before they can be reused. Consider a larger size, fewer unique filters, or rounding time ranges.
Tiny cache, few hits
Segments too small, filters too cheap, or queries using scoring context. Check the query design.
Steady rise then sudden drop
Large merges or restarts are wiping entries. Look at merge activity and node restarts.
A troubleshooting playbook
Confirm the symptom
Is search slower, or is only one dashboard slow? Compare latency before and after the change you suspect.
Read the cache numbers per node
Look for one node with very different values. A hot node points to uneven routing or shard placement.
Inspect the slow queries
The search slow log records requests that pass a time threshold you choose. Look for filters using “now” without rounding, scoring clauses that should be filters, and one-of-a-kind values.
Fix the query first, the size second
Better filter design gives bigger and safer wins than a bigger cache.
Change one setting and re-measure
If you must change the size or the index switch, change one thing, wait through a full traffic cycle, and compare.
Alert on sustained eviction growth, on heap usage approaching its ceiling, and on a sharp drop in hit ratio. These three together catch most cache-related trouble early.
How to benchmark the query cache honestly
Claims about caches are easy to make and easy to get wrong, so test them. A careful benchmark has a few simple steps.
Collect real requests
Take a sample of genuine search requests from production logs. Invented requests tend to repeat too neatly or not at all, which hides the real behavior.
Test cold first
On a test cluster, clear the query cache only, or restart the node, and replay the requests. Record latency at the median and at the slow end, such as the 95th and 99th percentile, not just the average.
Test warm second
Replay the same requests for long enough that entries have had time to be built. Record the same numbers and the cache’s hit, miss, and eviction counts.
Compare with the cache switched off
On a copy of the index with query caching disabled, run the same test. The gap between the warm run and this run is the real value of the cache for your workload.
Change one thing and repeat
Try rounding time ranges, or moving a condition to filter context, and measure again. Small query changes often beat settings changes.
Do not benchmark on an idle machine with a single user and then expect the same numbers in production. Do not forget that the operating system file cache also warms up during your test, so a warm run is faster for more than one reason. Repeat runs to separate the effects.
14Design Patterns and Anti-patterns
Good habits make the cache work for you. Bad habits quietly make it useless.
Patterns that work
Filter first, score second
Put every pure yes-or-no condition in the filter part of the query. Keep only the clauses that need relevance in the scoring part.
A small vocabulary of filters
Design your application so that filters come from a limited, stable set: category, status, region, tenant. Repetition is what the cache feeds on.
Rounded time windows
Round relative time ranges to a sensible unit so that many requests share one filter.
Time-based indices with a hot and cold split
Write to the newest index and let older indices go read-only. Old indices keep stable segments and reward the cache with reuse.
Sticky routing for sessions
Use a preference value per user or session so repeated searches hit the same shard copies.
Tenant filter as a standard building block
In multi-tenant systems, the tenant filter is added to every request. Being identical for all requests of a tenant, it is an ideal cache candidate.
A good library uses the same few labels on every book: fiction, science, history, children’s. Because the labels are few and consistent, the librarian’s index cards get reused thousands of times. If every book had its own unique label, the cards would be useless.
Anti-patterns to avoid
What people do
They add a unique value to every filter: a request id, a random token, or an exact timestamp taken at the moment of the request.
Why it hurts
No two requests share the same filter, so the cache can never reuse anything. Worse, if the filter is costly enough to be cached, entries get built for nothing and push out useful ones.
Better approach
Keep unique values out of filters. Round time. If you need to trace a request, use headers or logs instead of query content.
What people do
They place every condition, including simple yes-or-no checks, in the scoring part of the query because it is the first thing the tutorial showed them.
Why it hurts
Those conditions cannot use the query cache, and they waste effort on scores that make no difference to ranking.
Better approach
Move every pure yes-or-no condition into filter context and keep scoring for the text matching that truly needs it.
What people do
They see a low hit ratio and immediately grow the cache to a very large share of the heap.
Why it hurts
If the filters do not repeat, a bigger cache just holds more useless entries and steals memory from aggregations, indexing, and garbage collection.
Better approach
Find out why the ratio is low first: unrounded time, unique values, tiny segments, or scoring clauses. Fix the cause before touching the size.
What people do
They clear all caches every time performance looks odd, or on a schedule “just in case.”
Why it hurts
Every clear sends the node back to cold, and the next requests pay full price to rebuild everything. On a busy cluster this can cause a latency spike.
Better approach
Clear only when you have a specific reason, only the cache you mean, and preferably on one index at a time.
Modeling data so that filters are easy to cache
The way you shape your data decides how nicely filters behave. Three ideas are worth knowing.
Keep the filter vocabulary small. If a field can only take twenty values, then only twenty possible filters exist on it, and the cache will remember all of them. If a field can take twenty million values, as with a user identifier, filtering by it produces almost no repeats. Use such fields for lookups, not for shared filters.
Pre-compute buckets for ranges. If users mostly choose from a few price bands or age groups, store the band as its own keyword field when you index. A filter on the band becomes a cheap exact-value look-up, which is fast even without caching. You trade a little storage and indexing work for very cheap searches.
Use the right field type. Numbers that are only ever matched exactly, such as an order number or a postal code, often work better as keyword fields than as numeric fields, because numeric fields are built to answer range questions and keyword fields are built to answer exact-value questions. Picking the right type makes the filter cheap to compute, whether or not it gets cached.
If a school sorts its students into a few clear houses at the start of the year, asking “Who is in the blue house?” is trivial from then on. If instead the school records only each student’s exact height and asks “Who is between 140.2 and 143.9 centimetres?”, every question is a fresh piece of work.
15Best Practices and Common Mistakes
A short list you can hand to a teammate on day one.
Best practices
Use filter context by default
If you do not need a score from a condition, do not ask for one.
Round dates
Choose the coarsest time unit your users will accept.
Keep defaults until data says otherwise
Collect a few weeks of cache metrics before touching settings.
Protect the heap
Leave generous memory to the operating system file cache and keep the heap moderate.
Let old data settle
Roll over to new indices and let old ones become read-only, then consider merging them.
Use preference for sessions
Consistent routing makes repeated searches land where their entries already live.
Watch the trio
Hit ratio, evictions, and memory size tell you almost everything.
Test cold and warm
Load-test both states, because production sees both after every restart.
Common mistakes
- Confusing the query cache with the request cache. They have different sizes, different rules, and different reasons for being emptied.
- Expecting instant wins on small indices. If every segment is under the size threshold, the query cache never engages.
- Judging by a single request. The cache only pays off over many requests, so a one-off test tells you nothing.
- Forgetting the cold start. Rolling restarts empty the cache node by node, so schedule them outside peak times when possible.
- Assuming a disabled cache has no cost. It saves memory but may raise CPU use for repeated filters. Decide with measurements.
- Setting the size as a percentage without checking the resulting bytes. Ten percent of a small heap is small, and ten percent of a large heap is large.
When search is slow, do not start with the cache settings. Start with the queries. Bad query design defeats every cache, while good query design makes even a small cache shine.
Myths versus facts
| Myth | Fact |
|---|---|
| The query cache stores my search results. | It stores which documents passed a filter. Whole responses are the job of a different cache. |
| Every filter is cached the first time it runs. | Filters must prove they repeat, and cheap ones may never be cached. |
| Indexing new data wipes the cache. | Sealed segments do not change, so ordinary writes leave entries alone. |
| A bigger cache is always better. | A bigger cache takes heap from other features and helps only if filters repeat. |
| Clearing caches is a good routine fix. | It makes the next requests slower and rarely fixes anything by itself. |
| Each index has its own private cache. | There is one shared pool per node. |
| Small indices benefit the most. | Segments below the size gate are skipped, so small indices see little effect. |
16Real-World and Industry Examples
Where does the query cache quietly earn its keep?
Many well-known organizations have publicly said that Elasticsearch is part of their stack. Wikipedia’s search, Netflix, Uber, and Stack Overflow are among the names that appear in public talks and engineering write-ups. They rarely publish their cache settings, so the scenarios below are typical patterns for this kind of workload, not descriptions of anyone’s private configuration.
Large online store: faceted product search
Think of a marketplace the size of Amazon or eBay. Almost every search carries the same handful of filters: in stock, free shipping, brand, category, price band. Shoppers apply them thousands of times per minute. Those filters repeat massively, and the catalog changes slowly compared with how often it is read. This is the textbook case for the query cache, provided the product index has large segments.
Log and observability dashboards
Engineering teams at streaming and ride-hailing companies watch logs through dashboards that refresh every few seconds. Panels repeat filters such as “service equals checkout” and “level equals error.” Old daily indices are read-only, so their segments never change. The trap here is the time range: unrounded “last 15 minutes” filters look unique on every refresh, so teams round to the minute to enable reuse.
Multi-tenant business software
A customer-support platform adds the tenant identifier to every query so that each company only sees its own tickets. The tenant filter is identical across all of one company’s users all day, so it is cached and reused heavily. The risk to manage is a single very heavy tenant crowding the shared pool.
Encyclopedia and content search
A site like a large public encyclopedia has a few filters used constantly: language, namespace, page type. These are cheap for tiny indices but valuable on the big ones. Since content changes continuously in small pieces, the many small new segments are ignored by the cache while the big old segments carry the load.
Security analytics
Analysts search huge volumes of events using repeated filters: an event category, a network zone, a severity band. Investigations often hammer the same filters again and again over the same read-only historical indices, which suits the cache well.
Job boards and real-estate listings
Filters like “remote,” “full-time,” “city,” or “bedrooms” are repeated by countless visitors. The number of distinct filters is small, so entries stay hot and the hit ratio is high.
Streaming catalog browsing
A video service lets viewers browse by genre, language, and maturity rating. These filters are shared by millions of viewers and change only when the catalog is updated. A small number of distinct filters and a huge number of repeats is close to the ideal shape for a cache.
Travel booking search
Travelers commonly filter by “direct flights only” or “free cancellation.” Those switches are identical across huge numbers of searches, while the free-text or date-based parts are not. The switches are cached and reused, and the changing parts are computed fresh.
Every good example has the same three ingredients: few distinct filters, many repeats, and data that does not churn too fast. If your workload has all three, the query cache is likely a quiet hero. If it has none, do not expect much from it.
17Frequently Asked Questions
Short answers to the questions that come up most often.
18Summary and Key Takeaways
If you skim only one part of this guide, make it this one.
The Elasticsearch query cache is a small, clever memory that remembers which documents passed a yes-or-no filter. It works per segment, is shared by all shards on a node, and quietly removes repeated work from busy clusters. It asks very little of you: write your filters well, watch a few numbers, and leave the rest to the engine.
Key Takeaways
- What it stores: bitsets showing which documents passed a filter, not documents, not scores, not final results.
- What qualifies: filters in filter context, on segments with at least 10,000 documents and at least 3 percent of the shard, that repeat often enough to earn a cache entry.
- Why it is safe: sealed segments never change, so entries are never stale and simply vanish when their segment is merged away.
- Where it lives: one shared, least-recently-used pool per node in the Java heap, sized at 10 percent of heap by default with static settings.
- How to help it: use filter context, round time ranges, avoid unique values in filters, use sticky routing, and let old data settle into large read-only segments.
- How to watch it: track hit ratio, evictions, and memory size, and fix query design before growing the cache.
- How to keep it in perspective: it is one layer beside the shard request cache, fielddata, and the operating system file cache, and healthy clusters need all of them working together.