What Is a Sub-aggregation in Elasticsearch?
A complete, zero-to-solid guide to nesting one summary inside another — and how this single idea turns Elasticsearch from a search engine into a genuine analytics engine capable of answering multi-layered business questions in a single request.
Imagine you ask a store manager one simple question: “How many sales did we make this month?” That’s a fine answer on its own, but the moment you follow up with “…broken down by store location, and within each location, broken down by product category, and within each category, what was the average sale price?” — you’re no longer asking for one number. You’re asking for a nested, layered breakdown: a summary inside a summary inside a summary. Elasticsearch handles exactly this kind of layered question through a feature called a sub-aggregation — an aggregation nested inside another aggregation, letting you drill down through your data one level at a time, all within a single request. This article walks through what sub-aggregations are, how they work internally, why they exist, and how real companies use them to build the dashboards, reports, and analytics experiences that power modern data-driven products.
1Core Concepts
Before sub-aggregations make sense on their own, we need to be crystal clear on what a plain aggregation is, and what “bucket” and “metric” mean in Elasticsearch’s aggregation vocabulary.
What is an aggregation, in plain terms?
An aggregation is Elasticsearch’s way of summarizing data across many documents, rather than returning the documents themselves. Instead of getting back a list of a million individual sales records, you might ask Elasticsearch to hand you a single number: the total revenue. That’s an aggregation — a computed summary, not a list of raw rows.
Two families of aggregation: bucket and metric
Elasticsearch’s aggregations split into two broad families. A metric aggregation computes a single number (or a small handful of numbers) from a set of documents — the average price, the total count, the minimum and maximum values. A bucket aggregation instead sorts documents into groups, called buckets, based on some criterion — one bucket per store location, one bucket per price range, one bucket per day. Crucially, bucket aggregations don’t just count documents; they create a genuine grouping that other computations can then be run against.
A sub-aggregation is any aggregation defined inside a bucket aggregation, so that instead of computing one overall summary, Elasticsearch computes that same summary separately, once for each individual bucket the outer aggregation created.
Why “sub” only makes sense underneath a bucket
This is the single most important structural rule in this entire topic: sub-aggregations can only be nested inside bucket aggregations, never inside metric aggregations. A metric aggregation produces one final number — there’s no “group” left underneath it to further subdivide. A bucket aggregation, by contrast, produces multiple distinct groups, and each of those groups is itself a smaller set of documents that can be summarized all over again, using exactly the same aggregation toolkit available at the top level.
Think of a school report card. The top-level bucket aggregation is “group students by grade level” — 9th grade, 10th grade, 11th grade, 12th grade. Inside each grade-level bucket, a sub-aggregation might further group students “by subject” — Math, Science, English — and inside each of those buckets, a metric aggregation computes the average test score. Three layers deep, and every layer builds directly on the grouping the layer above it already established.
Why this matters more than it first appears
Without sub-aggregations, answering a layered business question would require running many separate queries — one per store location, say — and stitching the results together in application code afterward. That’s slow, wasteful of round-trips to the cluster, and awkward to keep consistent if the underlying data changes between queries. Sub-aggregations let Elasticsearch compute the entire nested breakdown in a single pass over the data, in a single request, which is both dramatically faster and far simpler to reason about.
A first, concrete example
Suppose an online retailer wants to know: “for each product category, what’s the average order value?” Without sub-aggregations, this would mean running one query per category, filtering to just that category, and computing an average each time — ten categories means ten separate round-trips. With a sub-aggregation, a single request defines a terms bucket aggregation grouping orders by category, and nests an avg metric aggregation directly inside it. Elasticsearch returns, in one response, one bucket per category, each already carrying its own average order value — computed correctly and consistently from the exact same underlying snapshot of data.
How sub-aggregations relate to the response shape
It helps to picture the aggregation response as a tree that exactly mirrors the request’s own nesting. The outer bucket aggregation’s response is a list of buckets; each bucket object contains its own doc_count, plus, nested directly inside that same bucket object, the results of any sub-aggregation defined for it — which might itself be another list of even smaller buckets, each again containing their own further nested results. Reading a nested aggregation response is, in this sense, simply reading the same tree shape you wrote in the request, just filled in with real numbers.
2Architecture & Components
A sub-aggregation isn’t a special, separate feature with its own syntax — it’s simply an aggregation object nested inside another aggregation’s own body. Let’s look at the pieces that make this nesting possible.
terms
Creates one bucket per distinct value of a field — for example, one bucket per store location or per product category.
date_histogram
Creates one bucket per fixed time interval — daily, weekly, monthly — ideal for time-series breakdowns.
range / histogram
Creates buckets from numeric ranges you define yourself, or from a fixed numeric interval Elasticsearch computes automatically.
filters
Creates one bucket per named query filter you supply, useful when your groupings aren’t simply “one bucket per field value.”
avg / sum / min / max
Single-value metric aggregations, commonly placed as the innermost layer to summarize each bucket’s contents with one number.
cardinality
Estimates the number of distinct values of a field within a bucket — for example, unique visitors per day.
The nesting structure, spelled out
Structurally, every aggregation request has an aggs (or aggregations) object at the top level. Inside it, each named aggregation can itself contain its own nested aggs object, and that nested object can contain further aggregations, each of which can have its own nested aggs object, and so on. There is no special keyword for “sub-aggregation” — it is simply the same aggs block, appearing one level deeper, attached to whichever bucket aggregation you want to drill further into.
How many levels deep can this go?
Technically, there’s no hard-coded limit on nesting depth baked into the aggregation syntax itself — you could nest ten or twenty levels deep if you genuinely needed to. In practice, Elasticsearch does enforce a maximum overall aggregation “bucket depth” as a cluster-level safety setting, specifically to prevent a runaway nested aggregation from generating an explosive number of buckets and overwhelming cluster memory. Real-world queries rarely go past three or four levels deep, both because the resulting bucket count grows multiplicatively with each additional layer, and because a report with more than a few layers of nesting usually becomes hard for a human to actually read and use.
Metric sub-aggregations versus bucket sub-aggregations, side by side
It’s worth being explicit about the two very different roles a sub-aggregation can play, because they answer fundamentally different questions. A metric sub-aggregation nested inside a bucket answers “what single number summarizes this bucket’s contents” — the average, the sum, the maximum. A bucket sub-aggregation nested inside another bucket answers “how should this bucket’s contents themselves be further subdivided” — creating an entirely new layer of smaller buckets underneath the parent. Both are written using the identical aggs syntax; the only difference is which aggregation type you choose to place there, and that choice alone determines whether the nesting stops at that level or continues one level deeper.
Multiple sub-aggregations under the same parent bucket
A single bucket aggregation isn’t limited to hosting just one sub-aggregation — its aggs block can contain several sibling sub-aggregations side by side, each computed independently against that same bucket’s documents. This is extremely common in practice: a single terms bucket on product category might simultaneously host an avg sub-aggregation for average price, a sum sub-aggregation for total revenue, and a cardinality sub-aggregation for unique customer count, all three computed in parallel from the same underlying bucket, in the same single request.
| Aggregation Type | Family | Can Host Sub-aggregations? |
|---|---|---|
| terms | Bucket | Yes |
| date_histogram | Bucket | Yes |
| range / histogram | Bucket | Yes |
| filters | Bucket | Yes |
| avg / sum / min / max | Metric | No |
| cardinality | Metric | No |
3Internal Working
Here’s the part most tutorials skip: what actually happens, step by step, inside the aggregation engine when it evaluates a nested chain of bucket and metric aggregations.
flowchart TD
A["Query runs, producing a
set of matching documents"] --> B["Outer bucket aggregation
groups documents into buckets
(e.g. one per store location)"]
B --> C["For EACH bucket independently"]
C --> D["The matching documents
belonging to that bucket
are collected"]
D --> E{"Does this bucket contain
a sub-aggregation?"}
E -- "No" --> F["Bucket's document count
and doc_count are finalized"]
E -- "Yes" --> G["Sub-aggregation runs
ONLY against this bucket's documents"]
G --> H["If the sub-aggregation is itself
a bucket aggregation, repeat
the same process one level deeper"]
H --> C
F --> I["All buckets' results are
assembled into the final,
nested response tree"]
Fig. 1 — How Elasticsearch evaluates a nested chain of sub-aggregations
Step-by-step breakdown
The query runs first
Before any aggregation logic begins, Elasticsearch first identifies the full set of documents matching the request’s query portion — aggregations operate only over this matching document set, never over the entire index unconditionally.
The outer bucket aggregation partitions the matches
The top-level bucket aggregation examines every matching document and assigns each one to exactly one bucket (in the common case) based on the bucket’s defining criterion — a field value, a date range, a numeric range.
Each bucket becomes its own miniature document set
Conceptually, every bucket now holds its own private subset of the original matching documents — exactly the ones that were assigned to it.
Sub-aggregations run independently, per bucket
Any sub-aggregation nested under that bucket aggregation is executed separately for every single bucket, using only that bucket’s own subset of documents — a sub-aggregation never sees documents belonging to a sibling bucket.
Deeper nesting repeats the exact same process
If a sub-aggregation is itself a bucket aggregation, it creates its own further sub-buckets from its parent bucket’s documents, and any aggregation nested beneath that runs the same way, one level deeper still.
Results assemble into a nested tree
The final aggregation response mirrors the exact nesting structure of the request — each bucket’s JSON object contains its own metrics and its own nested buckets, precisely matching the request’s shape.
A sub-aggregation does not re-run the original search query. It only operates on whichever documents already ended up inside its specific parent bucket. A sub-aggregation’s world is strictly limited to that one bucket’s contents — nothing more, nothing less.
How document counts scope down at each level
Every bucket, regardless of how deeply nested, carries its own doc_count — the number of documents that landed in it. This count strictly narrows as you go deeper: the doc_count of a sub-bucket can never exceed the doc_count of its parent bucket, since a sub-bucket’s documents are, by definition, a subset of its parent’s documents. This nesting property is what makes reading a multi-level aggregation response intuitive once you understand it — the numbers only ever shrink or stay the same as you drill down, never grow.
What happens when a document doesn’t fit any bucket
Not every document necessarily lands in a bucket. A terms aggregation on a field that’s missing from some documents simply excludes those documents from every bucket by default, unless a missing parameter is explicitly configured to assign them to a designated fallback bucket instead. This matters directly for sub-aggregations: a document excluded from its would-be parent bucket never reaches any sub-aggregation nested beneath it either, since it was never assigned to that bucket’s document subset in the first place. Being aware of this is important when a nested aggregation’s totals don’t quite add up to the overall document count from the top-level query — the gap is often exactly these unbucketed documents.
How a two-level worked example actually resolves
Consider a concrete two-level request: an outer terms aggregation bucketing documents by store_location, with an inner avg sub-aggregation on sale_price. For the “Downtown” bucket, Elasticsearch first identifies every matching document whose store_location equals “Downtown.” It then runs the avg sub-aggregation purely against that isolated set — summing every one of those documents’ sale_price values and dividing by their count — completely independently of how the “Uptown” or “Suburb” buckets’ own average gets computed. Each bucket’s sub-aggregation result is, in this very literal sense, its own self-contained calculation.
4Data Flow & Lifecycle
A sub-aggregation doesn’t operate in isolation — it’s one stage in a broader request lifecycle that spans the cluster’s distributed architecture. Let’s trace that full journey.
sequenceDiagram
participant App as Application
participant Coord as Coordinating Node
participant S1 as Shard 1
participant S2 as Shard 2
App->>Coord: Send search request with
nested aggregations
Coord->>S1: Forward query + aggregation tree
Coord->>S2: Forward query + aggregation tree
S1->>S1: Compute buckets and
sub-aggregations locally
S2->>S2: Compute buckets and
sub-aggregations locally
S1-->>Coord: Return this shard's partial
nested bucket results
S2-->>Coord: Return this shard's partial
nested bucket results
Coord->>Coord: Merge buckets with matching
keys across all shards
Coord->>Coord: Merge nested sub-aggregation
results within each merged bucket
Coord-->>App: Return final, merged
nested aggregation tree
Fig. 2 — How sub-aggregations are computed and merged across a distributed cluster
Why distributed merging matters for sub-aggregations
Because an Elasticsearch index is typically split across multiple shards, and each shard only holds a portion of the total documents, every shard independently computes its own local version of the entire nested aggregation tree — buckets, sub-buckets, and all — before the coordinating node combines them. For a terms aggregation, this means the coordinating node has to merge buckets that share the same key (say, the same store location) from every shard’s separate response, summing their document counts and, critically, also merging their respective sub-aggregation results together, recursively, at every nested level.
Why terms aggregations can sometimes be approximate
This distributed merging process is also the root cause of a subtlety experienced Elasticsearch users eventually run into: a terms aggregation’s top buckets can, in some circumstances, be slightly inaccurate, because each shard only returns its own top few buckets by default, and a value that ranks highly overall but only appears on a few documents per shard can be undercounted or missed entirely in the final merge. Elasticsearch exposes a doc_count_error_upper_bound figure specifically so users can see how much uncertainty might exist in a given result, and offers tuning knobs (like increasing the shard_size parameter) to reduce this risk when precision genuinely matters.
How the shard_size parameter trades accuracy for cost
The shard_size parameter controls how many candidate buckets each individual shard considers before sending its results up to the coordinating node for merging — a value larger than the requested final size. Raising shard_size gives the coordinating node a wider pool of candidates to merge from, meaningfully reducing the risk that a genuinely popular value gets missed simply because it was spread thinly across many shards, but this comes at the direct cost of more data being computed and transferred from every shard on every request. Choosing an appropriate shard_size is itself a precision-versus-cost trade-off, not a setting with one universally correct value.
The lifecycle of a dashboard built on sub-aggregations
In a typical analytics dashboard, a single sub-aggregation request often powers an entire visualization — a stacked bar chart showing sales by region, further broken down by product line, is usually the direct, literal output of one two-level nested aggregation, reshaped slightly for the charting library. As the underlying data updates — new sales come in — the exact same request, re-run, produces an updated nested tree reflecting the latest state, with no separate caching or precomputation layer required unless the dataset has grown large enough that query latency becomes a genuine concern.
How sub-aggregations interact with the search request’s own filters
Everything discussed so far assumes a query has already narrowed the document set before any bucketing begins. In practice, most dashboards let users apply their own filters — a date range picker, a category dropdown — which change the query portion of the request, and the entire nested aggregation tree recomputes freshly against that new, narrower set of matching documents. This is one of the more elegant properties of the design: filtering and nested aggregation are cleanly separated concerns, and changing one never requires restructuring the other.
How refresh intervals affect what a sub-aggregation “sees”
Because aggregations, like any other Elasticsearch query, only see documents that have already become searchable — which happens at the index’s configured refresh interval, not the instant a document is written — a sub-aggregation computed immediately after a burst of new writes may not yet reflect the very latest documents. For dashboards where near-real-time accuracy matters, this refresh interval becomes a meaningful part of the overall data lifecycle, sitting upstream of every aggregation request and directly bounding how fresh any nested breakdown can possibly be.
How sub-aggregations behave across a rolling time-series index pattern
Many of the real-world examples in this article — log analytics, sales trends, security dashboards — run against data spread across a rolling series of time-based indices, exactly the pattern index templates were built to automate. A nested aggregation targeting an index pattern rather than a single named index transparently spans every matching underlying index, with the coordinating node merging results across shards from all of them exactly as it would for a single index — meaning a dashboard’s nested aggregation continues working correctly, without modification, as new daily or monthly indices are created behind the scenes.
5Design Patterns & Anti-Patterns
Once the mechanics are clear, the real skill is structuring nested aggregations that answer real business questions efficiently, without accidentally overwhelming the cluster.
Pattern: Bucket-then-metric layering
The single most common pattern in real-world usage is exactly two layers deep: an outer bucket aggregation (group by category, by region, by day) with a small number of metric sub-aggregations nested directly underneath (average, sum, count). This shape answers the overwhelming majority of “break this number down by X” business questions cleanly, without introducing unnecessary complexity.
Pattern: Multi-level drill-down for hierarchical data
For genuinely hierarchical business questions — region, then store, then product category — nesting three terms aggregations deep, with a metric aggregation as the innermost layer, mirrors the natural structure of the question directly. This is the pattern behind most “drill-down” style dashboard experiences, where a user clicks into a bar in a chart and sees it expand into its own further breakdown.
Real Example
An e-commerce analytics dashboard might nest a date_histogram (bucket by month) as the outer layer, a terms aggregation on product category as the middle layer, and an avg metric on order value as the innermost layer — answering “what was the average order value per category, per month” in a single request.
Anti-pattern: Uncontrolled cardinality explosion
Nesting a terms aggregation on a high-cardinality field (like a user ID, with millions of distinct values) underneath another terms aggregation multiplies the total bucket count dramatically — ten outer buckets times a hundred thousand inner buckets is a million total buckets, computed and held in memory simultaneously. This is one of the fastest ways to turn a seemingly innocent aggregation request into a serious cluster performance problem.
Context
An analytics team nests a terms aggregation on user_id directly underneath a terms aggregation on country, intending to compute a per-country, per-user summary.
Problem
With millions of distinct users, the total number of nested buckets explodes into the millions, consuming enormous amounts of heap memory on every data node and causing noticeably slower, sometimes failing, aggregation queries cluster-wide.
Consequence
Rather than nesting a raw high-cardinality field, prefer a cardinality metric aggregation (which estimates unique counts without materializing every individual value as a bucket), or apply a size limit and accept an approximate top-N breakdown instead of an exhaustive one.
Pattern: Using size and min_doc_count to keep buckets manageable
Almost every bucket aggregation accepts a size parameter limiting how many buckets it returns, and a min_doc_count parameter suppressing buckets that contain too few documents to be meaningful. Applying both deliberately at every nesting level — not just the outermost one — is one of the most effective, simple guardrails against runaway bucket counts in a multi-level aggregation.
Pattern: Choosing the right outer bucket for the question being asked
The single biggest design decision in any nested aggregation is which field becomes the outer bucket versus which becomes the inner one, and getting this backwards produces a technically correct but practically useless breakdown. If the real business question is “for each region, what’s the trend over time,” the region should be the outer bucket and the date histogram the inner one — reversing the order instead answers “for each time period, how do regions compare,” which is a genuinely different question even though it touches the exact same two fields. Being deliberate about which question is actually being asked, before writing the nested request, avoids a surprisingly common source of confusing dashboard output.
Pattern: Combining terms and filters aggregations for hybrid groupings
Sometimes a grouping isn’t cleanly expressible as “one bucket per distinct field value.” A filters aggregation, nested as a sub-aggregation or used at the top level, lets you define custom, named groups from arbitrary query logic — “high value orders,” “first-time customers,” “returning customers” — that a plain terms aggregation on a single field could never express directly, since these groupings depend on combining several conditions together rather than a single field’s raw value.
Anti-pattern: Nesting metric aggregations under other metric aggregations
Because only bucket aggregations create the grouping structure that a sub-aggregation can meaningfully operate within, attempting to nest an aggregation underneath a plain metric aggregation like avg is simply not supported — there’s no group left to further divide. Beginners sometimes reach for this instinctively when they actually want a different, more advanced construct (like a pipeline aggregation, covered briefly in Chapter 9) rather than a literal nested sub-aggregation.
Pattern: Reusing a consistent nesting convention across a whole product
Larger analytics products built on Elasticsearch often standardize a house convention for how nesting order should generally be chosen — for instance, “time period is always the outermost bucket, unless the dashboard is explicitly framed around comparing categories” — so that engineers building new dashboards aren’t re-deriving nesting-order decisions from scratch every time, and so that the resulting dashboards feel consistent to end users navigating between them. This kind of convention doesn’t remove the need to think carefully about a specific dashboard’s actual question, but it does provide a sensible, well-reasoned default to start from.
6Advantages, Disadvantages & Trade-offs
Sub-aggregations are powerful, but that power comes with real resource costs that scale multiplicatively as nesting deepens — understanding this trade-off is essential to using them well.
Advantages
- Answers deeply layered analytical questions in a single request, avoiding slow, awkward multi-query round-trips.
- Mirrors how humans naturally think about breakdowns — by category, then by sub-category, then by metric.
- Keeps data consistent across every layer of the breakdown, since it’s all computed from one single, atomic snapshot of matching documents.
- Composes cleanly with filters, sorting, and further pipeline aggregations for even richer analytical output.
- Scales naturally across a distributed cluster, since each shard computes its own local nested tree before a lightweight merge step.
Disadvantages & Trade-offs
- Total bucket count multiplies across nesting levels, and memory usage grows accordingly, sometimes explosively.
- Deep nesting on high-cardinality fields can degrade cluster performance significantly if not carefully bounded.
terms-based buckets can be approximate, not exact, under certain distributed-merge conditions.- Very deeply nested responses become harder for both humans and downstream code to parse and reason about.
- Aggregation performance is sensitive to field mapping choices (keyword vs. text, doc_values settings) in ways that aren’t always obvious upfront.
The multiplicative cost, made concrete
If an outer bucket aggregation produces 50 buckets, and each of those contains a sub-aggregation producing 100 buckets of its own, the total bucket count for that two-level structure is 5,000 — not 150. Add one more level with 20 buckets each, and the total becomes 100,000. This multiplicative growth is the single most important number to keep in mind when designing any nested aggregation, since it directly predicts memory and computation cost far more reliably than simply eyeballing how “deep” the nesting looks on paper.
Trade-off: readability versus completeness
A deeply nested, exhaustive breakdown is technically more complete than a shallow one, but completeness and human readability pull in opposite directions past a certain point. A three-level nested table with hundreds of rows is already pushing the limits of what a person can meaningfully scan, and a fourth or fifth level rarely adds proportional insight — it mostly adds scrolling. Many well-designed analytics products deliberately cap visible nesting at two or three levels in the user interface, even when the underlying aggregation capability could technically go deeper, precisely because usability, not raw computational capability, becomes the real constraint at that point.
Trade-off: real-time computation versus precomputed rollups
Computing a nested aggregation fresh, on every request, guarantees the result always reflects the very latest data, but it also means every dashboard view pays the full computational cost of the aggregation each time it loads. For extremely large datasets or extremely high-traffic dashboards, some teams instead precompute and store rollup summaries on a schedule, trading a small amount of data freshness for dramatically lower per-request cost. Sub-aggregations remain relevant even in this precomputed world, since the rollup-generation job itself is typically still built from the exact same nested aggregation request run periodically in the background, rather than at the moment a user opens the dashboard.
7Security
Sub-aggregations are primarily an analytics tool, but they intersect with security and data-governance concerns in a few important ways worth understanding.
Aggregations bypass document-level filtering differently than expected
It’s worth stating this clearly: aggregations, including sub-aggregations, only ever operate over documents that survived the request’s query portion — so any document-level security filter correctly applied there is correctly respected by every level of nested aggregation underneath it. The risk arises when access control is applied inconsistently — for instance, enforced on the main search results but forgotten on a separate aggregation-only request — which can let an aggregation reveal patterns (bucket keys, counts) drawn from documents a user shouldn’t otherwise be able to see, even if the raw documents themselves stay hidden.
Information leakage through bucket keys and counts
Even when individual documents are properly filtered out, a bucket’s key and count can themselves leak sensitive information. A terms aggregation on department that surfaces a bucket named “Confidential Merger Project” with a document count of 3, for a user who shouldn’t know that project exists at all, is a real and easy-to-overlook risk in enterprise search and analytics platforms. This is precisely why field-level security and document-level security need to be considered together, not treated as independent, redundant layers.
Resource-exhaustion risk from unbounded nested aggregations
Because deeply nested aggregations on high-cardinality fields can consume very large amounts of cluster memory, as covered in Chapter 5, an unauthenticated or loosely-validated endpoint that allows arbitrary user-constructed aggregation requests is a genuine denial-of-service risk. Production systems that expose any aggregation capability to end users typically validate and cap nesting depth, bucket size, and which fields are even eligible for aggregation, rather than passing a user-supplied aggregation request through to Elasticsearch unmodified.
Least-privilege design for self-service analytics tools
Self-service analytics tools that let business users build their own nested breakdowns without writing raw Elasticsearch queries typically constrain the underlying aggregation capability to a safe, pre-approved subset — a fixed list of fields eligible for bucketing, a hard cap on nesting depth, sensible default size limits — rather than exposing the full expressive power of the aggregation framework directly. This mirrors the least-privilege principle used elsewhere in security design: giving users the specific analytical capability they need, without handing them the ability to accidentally (or deliberately) construct a request expensive enough to degrade the cluster for everyone else.
Auditing who has access to aggregation-only endpoints
Because an aggregation-only response can still reveal meaningful patterns about a dataset — bucket keys and counts — without ever returning individual documents, it’s worth treating access to aggregation endpoints with the same seriousness as access to the underlying search endpoint itself, rather than assuming that “just an aggregation, not the raw documents” is inherently lower risk. Some organizations apply identical role-based access controls to both endpoint types for exactly this reason.
8Monitoring, Logging & Metrics
Nested aggregations that quietly grow too expensive rarely announce themselves loudly — they show up first as slow dashboards and gradually rising cluster memory pressure. Here’s what to watch.
Aggregation request latency
Tracked per endpoint, a rising trend often signals a nested aggregation whose bucket count is quietly growing as the underlying data grows.
JVM heap usage during aggregation-heavy periods
Nested bucket structures live in memory during computation, so correlating heap pressure spikes with dashboard refresh times often points directly at the culprit query.
doc_count_error_upper_bound
Watching this value on terms-based sub-aggregations flags when approximate bucket counts might be materially inaccurate for a given query.
Total bucket count returned
Logging the actual number of buckets a nested aggregation produced (not just how many were requested) helps catch cardinality growth before it becomes a performance incident.
Slow-query logging for aggregation-heavy requests
Elasticsearch’s slow-query logging can be configured to capture requests exceeding a latency threshold, and for analytics-heavy deployments, it’s worth reviewing these logs specifically for aggregation shape — how many levels deep, which fields, what size values — rather than only looking at raw latency numbers. A query that was fast last month but has slowly crept past the slow-query threshold is often a direct symptom of naturally growing cardinality in a nested terms aggregation, exactly the kind of drift that’s easy to miss without deliberately watching for it.
Capacity planning for growing dashboards
Because bucket counts scale multiplicatively with data growth in ways that aren’t always intuitive, teams building long-lived analytics dashboards benefit from periodically re-estimating the worst-case bucket count a given nested aggregation could produce as the underlying dataset continues to grow, rather than only validating performance once at launch. A dashboard that performed perfectly well against a young dataset with modest cardinality can degrade meaningfully within a year or two of organic growth if this isn’t revisited.
Alerting on aggregation-specific failure modes
Beyond generic latency and memory dashboards, mature clusters wire up alerts for failure modes specific to nested aggregations: circuit-breaker exceptions triggered when an aggregation request would have exceeded a configured memory limit, a rising rate of aggregation requests being rejected or truncated, and a growing gap between the number of buckets requested via size and the number actually returned, which can indicate the underlying cardinality has grown past what the current configuration comfortably handles. Each of these signals turns a slow-burning capacity problem into something an on-call engineer notices proactively, rather than discovering it only after a dashboard has already started failing for end users.
Logging aggregation shape for later analysis
Just as query-log analysis helps tune minimum_should_match thresholds, capturing a sample of the actual aggregation request bodies hitting a cluster — which fields, how many nesting levels, what size values — over time gives teams a concrete, evidence-based picture of how aggregation usage is evolving, rather than relying on assumptions formed when a dashboard was first built. This is particularly valuable in self-service analytics tools, where end users construct their own breakdowns and the resulting aggregation shapes can drift meaningfully from what the original tool designers anticipated.
9Best Practices & Common Mistakes
These are the lessons teams building serious analytics on Elasticsearch usually learn the hard way — collected here so you don’t have to relearn them from a production incident.
Best practices
- Apply
sizeandmin_doc_countdeliberately at every nesting level, not just the outermost bucket aggregation. - Prefer a
cardinalitymetric aggregation over nesting a raw high-cardinality field as a fulltermsbucket, whenever you only need a unique count rather than every individual value. - Keep nesting depth to what a human reading the dashboard actually needs — most real business questions resolve cleanly in two or three levels.
- Map fields intended for aggregation as
keywordrather thantext, since aggregating on analyzed text fields is either unsupported or behaves in ways that rarely match business intent. - Watch
doc_count_error_upper_boundon terms-based sub-aggregations whenever precise counts genuinely matter, and increaseshard_sizeif the reported error is unacceptably large. - Validate and cap aggregation shape (depth, bucket size, allowed fields) at the application layer for any endpoint exposing aggregation capability to end users.
- Choose the outer-versus-inner bucket order deliberately based on the exact business question being asked, since reversing the order answers a genuinely different question even with identical fields.
- Adopt a consistent house convention for common nesting patterns across a product’s dashboards, so engineers aren’t re-deriving the same design decisions repeatedly.
Common mistakes
Nesting metric aggregations under metric aggregations
Attempting to nest an aggregation under avg or sum, not realizing that only bucket aggregations can host sub-aggregations at all.
Assuming terms buckets are always exactly accurate
Trusting a distributed terms aggregation’s top buckets as perfectly precise without checking doc_count_error_upper_bound, especially in reports where exact ranking genuinely matters.
Letting bucket counts grow silently with the data
Shipping a nested aggregation that performs fine at launch, then never revisiting it as underlying cardinality naturally grows over months or years.
Confusing what a sub-aggregation reaches versus what filters reach
Expecting a sub-aggregation to somehow “see” documents outside its parent bucket, or expecting a sibling bucket’s data to influence a bucket that shouldn’t be able to see it at all.
A short checklist before shipping a nested aggregation to production
Before deploying a new or changed multi-level aggregation, it’s worth running through a short mental checklist: what is the realistic worst-case total bucket count across every level, multiplied together, not just at the top level? Are size and min_doc_count set deliberately at every level rather than left at defaults everywhere? Are any high-cardinality fields being nested as full terms buckets when a cardinality metric would answer the actual business question instead? And has the query been tested against a dataset that’s representative of where the underlying cardinality is headed, not just where it stands today?
Reviewing dashboard queries as part of a broader relevance and performance practice
Teams that treat aggregation-heavy dashboards as genuine production infrastructure, rather than a one-off reporting script, typically fold nested aggregation design into the same review discipline used elsewhere — a second engineer reviewing a new dashboard’s underlying query for bucket-count risk before it ships, the same way a database team might review a new query’s execution plan before approving a schema change. This habit catches the overwhelming majority of cardinality-explosion mistakes well before they reach a production cluster serving real user traffic.
Documenting the intent behind a nested aggregation’s structure
Because a multi-level aggregation’s JSON structure doesn’t inherently explain why a particular field was chosen as the outer bucket versus the inner one, leaving a short comment or a linked design note alongside a dashboard’s underlying query pays off considerably the next time someone needs to modify it. Without that context, a future engineer changing the nesting order to “simplify” the query can inadvertently answer a subtly different business question than the one the dashboard was originally built to answer — exactly the kind of silent, hard-to-notice regression that documentation is cheap insurance against.
Onboarding new engineers to nested aggregation design
Because the multiplicative bucket-count behavior described in Chapter 6 isn’t always intuitive on first exposure, teams that maintain nontrivial analytics products often benefit from a short, hands-on onboarding exercise for new engineers — walking through a real nested aggregation, computing the actual worst-case bucket count by hand, and comparing that estimate against what the cluster actually reports. This kind of concrete, worked exercise tends to build lasting intuition far more effectively than reading about the concept alone, and it directly reduces the odds of a new engineer unknowingly introducing a cardinality-explosion anti-pattern in their first few weeks of building dashboards.
10Real-World & Industry Examples
Sub-aggregations aren’t an academic corner case — they’re the quiet computational engine behind an enormous number of dashboards, reports, and analytics products in daily use.
E-commerce sales analytics
Online retailers commonly nest a date_histogram for monthly trend lines, a terms aggregation on product category for the breakdown within each month, and metric aggregations for revenue and units sold — producing a full, multi-dimensional sales report from a single request rather than a slow patchwork of separate queries.
Application performance monitoring
Observability platforms tracking application latency typically nest a terms aggregation on service name, with a further terms aggregation on HTTP status code beneath it, and a percentile metric aggregation as the innermost layer — instantly surfacing which specific service and status code combination is driving latency problems across an entire fleet of services.
Security event and SIEM dashboards
Security teams building detection dashboards frequently nest a date_histogram for time trends, a terms aggregation on source IP or user account beneath it, and a cardinality metric to estimate how many distinct systems a single account touched — a structure directly useful for spotting anomalous, wide-ranging account activity.
Ride-sharing and logistics operations dashboards
Companies coordinating large vehicle fleets commonly nest a terms aggregation on city or region, with a further range aggregation on trip duration beneath it, letting operations teams instantly see how trip-length distribution differs across markets without building a separate report per city.
Content and media consumption analytics
Streaming and publishing platforms nest a terms aggregation on content category with a date_histogram beneath it, tracking how viewership or readership within each category trends over time, feeding directly into content-strategy decisions about what to commission or promote next.
What these examples have in common
Across every one of these industries, the underlying need is identical: a single, layered question — “break this down by X, and within each X, break it down further by Y” — that would otherwise require multiple round-trips and manual stitching in application code. Sub-aggregations collapse that entire layered question into one request, computed consistently from one snapshot of the underlying data.
Financial services risk and transaction analytics
Banks and payment processors analyzing transaction patterns commonly nest a terms aggregation on merchant category with a range aggregation on transaction amount beneath it, and a cardinality metric to estimate how many distinct cards were used within each combination — a structure directly useful for surfacing unusual concentration patterns worth a closer manual review.
Human resources and workforce analytics
Organizations tracking workforce composition often nest a terms aggregation on department with a terms aggregation on job level beneath it, and a metric aggregation on tenure or compensation range as the innermost layer, giving leadership a single-request view into how the organization’s structure breaks down across multiple dimensions simultaneously.
Why the “right” nesting order varies so much by industry
Comparing these examples side by side reinforces the design point made in Chapter 5: the outer-versus-inner bucket choice always follows from which dimension the primary business question is organized around, not from any inherent property of the fields themselves. The same two fields — merchant category and transaction amount, say — can be nested in either order depending on whether the question is “how does spending vary by merchant category” or “for a given spending range, which merchant categories dominate.” Both are legitimate, useful questions; they simply require the nesting order to be reversed to answer correctly.
11Frequently Asked Questions
12Summary & Key Takeaways
Key Takeaways
- A sub-aggregation is any aggregation nested inside a bucket aggregation, computed separately once for each bucket the outer aggregation produces.
- Sub-aggregations can only be nested under bucket aggregations, never under metric aggregations, since a metric produces no further grouping to subdivide.
- Each sub-aggregation only operates on the specific subset of documents belonging to its own parent bucket — never on sibling buckets’ documents.
- Bucket counts grow multiplicatively, not additively, with every added nesting level — the single most important number to estimate before shipping a deep nested aggregation.
- Distributed
terms-based buckets can be approximate; watchdoc_count_error_upper_boundwhen precision genuinely matters. - Nesting a raw high-cardinality field as a full
termsbucket is a common performance anti-pattern — acardinalitymetric is often the better fit. - Sub-aggregations power the vast majority of layered dashboards and reports across e-commerce, observability, security, logistics, and media analytics.
- Healthy production systems apply size and min_doc_count limits at every nesting level and monitor bucket-count growth as data volume increases over time.