What Is Pagination (from/size) in Elasticsearch?
A ground-up, zero-jargon walkthrough of how search results get split into pages — and the hidden cost that makes "just skip ahead 500,000 results" far more expensive than it sounds.
Imagine a search engine finds ten thousand matching results for your query, but your screen only has room to show ten at a time. Something has to decide which ten to show first, and how to fetch the next ten when you click “next.” In Elasticsearch, that job belongs to pagination, most commonly controlled through two simple parameters called from and size. This guide walks through exactly how they work, and why a seemingly small design detail becomes a serious performance concern once you scroll far enough.
ACore Concepts: What Is Pagination?
Let’s define the idea plainly before looking at how it’s implemented.
Pagination is the practice of breaking a large set of search results into smaller, manageable “pages,” so a client only ever receives and displays a small chunk at a time, rather than the entire result set at once. In Elasticsearch, the most common way to achieve this is with two request parameters: size, which controls how many results to return per page, and from, which controls how many results to skip before starting that page — effectively saying “start counting from result number X, and give me the next Y of them.”
Think of flipping through a phone book to find every “Smith” listed. You don’t photocopy the entire book and hand it over — you turn to roughly the right page, read a handful of names, and flip forward as needed. “from” is like saying “skip past the first 40 names,” and “size” is like saying “then show me the next 10.” The phone book itself doesn’t change; you’re just choosing which small window of it to look at right now.
size
How many results to return in this response — the “page length.”
from
How many results to skip over before returning results — the “starting offset.”
Deep Pagination
Requesting results very far into a large result set (a high “from” value), which becomes progressively more expensive the deeper it goes.
search_after
A different, more efficient pagination technique for scrolling deeply through large result sets, used as a preferred alternative to very high “from” values.
Pagination sounds like a trivial, purely cosmetic feature — until a result set grows large and a user (or, more often, an automated script) tries to page deep into it. At that point, understanding how from/size actually works internally becomes the difference between a healthy cluster and one straining under unnecessary load.
BArchitecture & Components
Let’s see how from/size interacts with the shard-based architecture covered in earlier guides.
Remember that a search request fans out to every shard holding a piece of the target index, and each shard scores and ranks its own local matches before the coordinating node merges everything into one final, globally ranked list. Pagination has to happen after that merge, because only the coordinating node has the complete, correctly ordered picture across every shard — no single shard alone knows the true global rank of its own results relative to every other shard’s results.
Per-Shard Result Set
Each shard independently gathers and ranks its own top matching documents before sending them onward — critically, it must gather enough to cover from + size, not just size.
Coordinating Node Merge
Combines every shard’s candidates into one globally sorted list, then trims it down to exactly the requested page (skipping “from,” keeping “size”).
from + size Window
The true amount of data every shard must gather and sort for a single page request — not just the page itself, but everything from the very beginning up through the end of that page.
FIG 1 — Requesting page 101 (from=1000, size=10) forces every shard to gather and sort 1,010 candidates each, even though only 10 documents ever make it into the final response.
CInternal Working: Why “Skip 1,000” Isn’t Free
Let’s dig into exactly why deep pagination gets expensive, rather than just accepting it as a rule of thumb.
Here is the crucial internal detail: no single shard can simply “skip ahead” to result 1,000 and start counting from there, because a shard has no way of knowing, in advance, how many of the top-ranked documents overall are going to come from other shards versus its own. To guarantee a correct answer, every shard must gather and sort its own top from + size results, and only then does the coordinating node merge all those candidates together and trim the final list down to just the requested page.
This means requesting page 1 (from=0, size=10) is genuinely cheap — each shard only needs its top 10. But requesting page 101 (from=1000, size=10) forces every single shard to gather and sort its top 1,010 candidates each, even though 1,000 of those, from every shard, will ultimately be thrown away once the final trim happens. The deeper the requested page, the larger this wasted, discarded portion becomes — and the cost grows roughly in proportion to how deep you page, not how many results you actually display.
Imagine four separate relay teams, each independently ranking their own runners from fastest to slowest, and you want to know who placed 1,001st through 1,010th overall, combining all four teams. There’s no shortcut — each team has to fully rank at least its own top 1,010 runners first, because any of them could plausibly be the 1,001st fastest person overall once all four teams’ results are combined. Only after every team has done that ranking work can the results be merged and the correct 1,001st-to-1,010th slice identified.
It’s tempting to assume “from” works like flipping to a specific page in a book — an instant, direct jump. Internally, it behaves more like “re-sort everything up to and including this page, every single time,” which is why Elasticsearch places a default safety limit on how deep from/size pagination is allowed to go by default.
DData Flow & Lifecycle
Let’s trace a single paginated request from submission to final response.
Client Requests a Page
A search request is sent with a chosen from and size value, alongside the usual query.
Coordinating Node Forwards from + size
The coordinating node tells every shard how many total top results it needs to gather (from + size), not just the final page size.
Each Shard Gathers and Sorts
Every shard independently finds, scores, and sorts its own top candidates up to that combined number.
Candidates Sent to Coordinating Node
Each shard sends its full set of gathered candidates back, not just the final trimmed page.
Global Merge and Trim
The coordinating node merges all shards’ candidates into one correctly ordered list, skips the first “from” entries, and keeps exactly “size” of them.
Final Page Returned
Only the requested page — the small, final slice — is sent back to the client, with everything else discarded after having been fully sorted for nothing.
This lifecycle makes clear that from/size pagination is well-suited to shallow, user-facing browsing (page 1, 2, maybe 3), but poorly suited to programmatically walking through an entire large dataset page by page, since each successive page gets progressively more expensive rather than staying constant.
EAdvantages, Disadvantages & Trade-offs
from/size is deliberately simple — and that simplicity carries a specific, well-defined limitation.
Advantages
- Extremely simple to use — just two straightforward numbers
- Perfect fit for typical user-facing pagination (browsing the first several pages of results)
- Allows jumping directly to an arbitrary page number, unlike some alternative techniques
- No extra setup or state needs to be maintained between requests
Disadvantages
- Becomes progressively more expensive the deeper you page (“deep pagination”)
- Has a default safety limit on how far it can go, to protect cluster resources
- Poorly suited to exporting or processing an entire large dataset page by page
- Wastes real, computed work on every request, since every shard must fully sort results it will ultimately discard
The trade-off in one sentence: from/size optimizes for simplicity and arbitrary jump-to-page access at shallow depths, and deliberately becomes uneconomical at deep depths — which is exactly why Elasticsearch offers a different, purpose-built alternative for deep, sequential scrolling through very large result sets.
FSecurity
A seemingly harmless pagination parameter can, if left unchecked, become a real resource-abuse vector.
Built-In Depth Limit
Elasticsearch enforces a default maximum on how large from + size is allowed to be, specifically to prevent a single request from forcing every shard to sort an unreasonably huge number of candidates.
Unbounded Client-Controlled Pagination
If an application directly exposes raw from/size values to end users or external clients without any upper bound of its own, a malicious or careless caller could repeatedly request deep pages to intentionally strain cluster resources.
Sensible Caps
Applications should generally impose their own, tighter limits on how deep a user is allowed to paginate through a public-facing search feature, well below the cluster’s absolute default safety limit.
A public search box rarely needs to let users page past the first handful of pages anyway — most real users never scroll that far — so capping from/size well below the cluster’s built-in limit is both a security best practice and a reasonable reflection of actual usage patterns.
GMonitoring, Logging & Metrics
Because deep pagination’s cost is easy to overlook until it’s already causing problems, it deserves deliberate, ongoing observation.
Teams commonly track the distribution of “from” values actually being requested across real traffic, watching in particular for unusually high values that suggest either a legitimate but poorly-designed export process, or something closer to abusive scraping behavior. Query latency broken down specifically by pagination depth is also revealing, since it often shows a clear, sharply rising cost curve as “from” increases — a pattern that a flat, undifferentiated latency average would otherwise hide.
A store that never tracks how far customers actually browse into its catalog might assume deep pages matter more than they do, or fail to notice when a single automated bot is repeatedly requesting page 900 over and over. Watching actual pagination depth in practice reveals both how real users behave and where genuine misuse might be hiding.
HDesign Patterns & Anti-Patterns
One pattern is the right tool for deep, sequential access; one anti-pattern is a well-known source of avoidable cluster strain.
Pattern: search_after for Deep or Full Dataset Walks
For genuinely deep pagination, or for walking through an entire large dataset from start to finish (such as a full export), a cursor-based technique remembers a marker from the last document of the previous page, and asks only for results that come “after” that marker — avoiding the need for every shard to re-sort a huge, ever-growing number of candidates on each successive page. This keeps the cost of each page roughly constant, no matter how deep the walk goes.
The Anti-Pattern
Using from/size to sequentially walk through an entire large result set page by page — for example, an automated export job that loops from=0, then from=10, then from=20, and so on, all the way through hundreds of thousands of documents.
Why It Fails
Each successive page becomes progressively more expensive, since every shard must fully gather and sort an ever-larger number of candidates purely to discard most of them. This pattern predictably degrades in performance as it progresses, and will eventually hit the built-in depth safety limit and fail outright.
Better Approach
Switch to the cursor-based search_after technique (or a dedicated bulk-export mechanism, for genuinely massive one-time exports), both of which are purpose-built to walk through large result sets efficiently without the escalating cost that plain from/size incurs.
IBest Practices & Common Mistakes
These habits consistently separate pagination that stays fast from pagination that quietly becomes a cluster liability.
Best Practices
- Use from/size for shallow, user-facing browsing of the first several pages
- Switch to a cursor-based technique for anything deeper or for full dataset walks
- Impose an application-level cap on how deep users can paginate, tighter than the cluster’s built-in default
- Monitor real pagination depth in production traffic, not just in theory
Common Mistakes
- Building an export or sync job that loops through from/size all the way to the end of a large dataset
- Exposing raw, unbounded from/size values directly to end users or external API callers
- Assuming pagination cost is constant per page, regardless of depth
- Only discovering the built-in depth safety limit in production, rather than designing around it from the start
JReal-World & Industry Examples
The from/size trade-off shows up constantly in everyday search-driven products.
E-Commerce Search Result Pages
Online storefronts use from/size for the classic “page 1, 2, 3…” browsing experience shoppers expect, relying on the fact that the overwhelming majority of real shoppers never browse past the first few pages, keeping typical usage well within the cheap, shallow range.
Support Ticket Dashboards
Internal support tools paginating through open tickets for agents typically stay shallow by design, since agents work through a manageable queue rather than scrolling thousands of pages deep, making plain from/size a perfectly reasonable fit.
Data Export and Migration Pipelines
Teams building bulk export or reindexing pipelines that need to walk through an entire large index from start to finish deliberately avoid from/size for this exact reason, favoring cursor-based or dedicated bulk-scan mechanisms instead to keep performance steady across the entire dataset.
KFrequently Asked Questions
“size” is how many results to return, and “from” is how many results to skip before returning them — together they define exactly which slice of the ranked results comes back.
Because every shard must gather and sort its own top from + size candidates before the coordinating node can merge and trim them down to the final page — the deeper the page, the more each shard has to sort, even though most of that work is discarded.
Yes — Elasticsearch enforces a default maximum on from + size specifically to prevent excessive resource use from very deep pagination requests.
A cursor-based technique like search_after, which remembers a marker from the previous page and fetches only what comes after it, keeping the cost of each page roughly constant no matter how deep you go.
Not at all — for shallow, everyday browsing of the first several pages, which is how most real users actually behave, from/size is simple, effective, and perfectly appropriate.
LSummary and Key Takeaways
Key Takeaways
- Pagination splits a large result set into manageable pages, and from/size is Elasticsearch’s simplest built-in tool for doing so.
- size controls how many results come back; from controls how many are skipped before that page starts.
- Every shard must gather and sort its own top from + size candidates before the coordinating node merges and trims the final list — deeper pages cost more, not less, per request.
- This is why Elasticsearch enforces a default depth limit, protecting the cluster from excessively deep pagination requests.
- from/size is the right tool for shallow, user-facing browsing, and the wrong tool for walking through an entire large dataset sequentially.
- search_after is the purpose-built alternative for deep or full-dataset pagination, keeping per-page cost roughly constant regardless of depth.
- Public-facing applications should impose their own tighter pagination caps, both for performance and to prevent resource-abuse scenarios.
- Real systems — e-commerce browsing, support dashboards, and bulk export pipelines — each choose the pagination technique that actually fits how deep their users or processes really need to go.