Before we can talk about how runtime fields work, we need to agree on what a “field” is in Elasticsearch at all, and what makes the runtime kind different.

What Elasticsearch Normally Does With a Field

Elasticsearch stores information as JSON documents inside an index, which you can think of as a giant, searchable filing cabinet. Every document has fields — a name, a price, a timestamp, and so on. Normally, when you add a document, Elasticsearch immediately looks at each field, decides how to store it efficiently, and writes that decision down permanently in something called the mapping. From that point forward, that field is “baked in.” If you later realize you needed the price stored differently, or you want a brand-new field that combines two existing ones, you traditionally had only one real option: reprocess and reindex all your data from scratch, which for large datasets can take hours or days.

The Official Definition

A runtime field is a field that is evaluated at query time rather than at indexing time. Instead of being computed once when a document is written and stored on disk, its value is calculated on the fly, using a script, whenever a search, aggregation, or filter actually needs it. Elasticsearch does not store the resulting value anywhere; it recalculates it fresh for every query that touches it.

Why “Runtime” Is the Right Word

The word “runtime” refers to the period while a program is actually running, as opposed to when it was written or compiled. In this context, it means the field only comes into existence in the runtime of a query — the brief window while Elasticsearch is actively processing your search request. Before that moment, and after it, the field has no stored representation anywhere in the index.

A Concrete Mental Picture

Suppose every document in your index has a field called full_name, storing something like “Amit Verma.” You never anticipated needing to search only by first name. With a runtime field, you can define a field called first_name whose rule is simply “take everything in full_name before the first space.” The moment you define that rule, you can search, sort, or aggregate by first_name as if it had always existed — even though, physically, nothing was ever written to disk for it. Delete millions of documents, add a hundred more tomorrow, and the rule still applies automatically to every one of them, old and new, without ever touching the index again.

Runtime fields don’t float around on their own — they slot into specific places inside Elasticsearch’s existing architecture. Let’s look at where.

Where Runtime Fields Live

Every Elasticsearch index has a mapping, which is essentially a schema describing every field’s name and data type. Runtime fields live inside this same mapping, in a dedicated section usually named runtime. This is important: they are declared alongside your ordinary indexed fields, and from a search request’s point of view, they look and behave almost identically to a “real” field. The difference is invisible to the person writing the search — it only matters to Elasticsearch internally, when it decides how to fetch the value.

Two Ways to Declare a Runtime Field

There are two places you can define a runtime field, and this choice matters for how flexible and how fast it is.

How Runtime Fields Fit Beside Normal Fields

An index can freely mix indexed fields and runtime fields. In fact, a very common pattern is a “shadow” runtime field that has the same name conceptually as an indexed field but recalculates it differently — for instance, an indexed field storing a raw string price like “$45.00”, paired with a runtime field that strips the currency symbol and returns it as a proper decimal number for math and sorting.

This is the heart of the topic: how does Elasticsearch actually turn a script into a usable field, on demand, for potentially millions of documents?

The Role of Painless

Painless is a scripting language that Elastic built specifically for Elasticsearch. It looks similar to Java in its syntax, but it is deliberately restricted and sandboxed — it cannot read files, open network connections, or do anything that could destabilize the cluster. When you define a runtime field, you write a small Painless script that describes how to compute the value, using a special object called emit to hand the result back to Elasticsearch.

Step by Step: What Happens When a Query Touches a Runtime Field

flowchart TD
    A[Search request references runtime field] --> B{Mapping lookup}
    B --> C[Runtime field script found]
    C --> D[Narrow candidate documents using indexed fields]
    D --> E[For each candidate document]
    E --> F[Run Painless script against _source / doc values]
    F --> G[emit computed value]
    G --> H[Use value in filter, sort, or aggregation]
    H --> I[Return response]
    I --> J[Computed value discarded]
        

Reading From Doc Values Instead of _source

By default, runtime field scripts commonly read from doc values — a special columnar, on-disk data structure Elasticsearch already builds for sorting and aggregations — rather than parsing the full JSON _source every time. Reading from doc values is considerably faster than re-parsing JSON text, because doc values are already stored in a compact, typed, ready-to-use format. This detail matters a lot for performance, which we’ll return to later.

Zooming out from a single query, let’s trace how a runtime field’s role changes across the entire lifetime of a piece of data, from the moment it’s created to the moment it’s searched.

The Journey of an Indexed Field

A traditional indexed field’s journey is short and final. A document arrives, Elasticsearch’s analysis and mapping logic processes each field once, and the resulting representation is written permanently into segments on disk. From that instant onward, every future query simply reads the already-computed result. The computation happens exactly once per document, no matter how many times it’s later searched.

The Journey of a Runtime Field

A runtime field’s journey looks completely different, because it has no “birth” tied to document indexing at all. Its only real event happens later and repeatedly: every time a query needs it, the script runs again, against whatever data currently exists for that document. If you update the document tomorrow, the runtime field automatically reflects the new value the very next time it’s queried — with zero extra indexing work.

sequenceDiagram
    participant U as User/App
    participant ES as Elasticsearch
    participant IDX as Index (disk)
    U->>ES: Index a document
    ES->>IDX: Write indexed fields (computed once, stored)
    Note over IDX: Runtime field NOT computed here
    U->>ES: Search request using a runtime field
    ES->>IDX: Read _source / doc values for candidates
    ES->>ES: Run Painless script, emit() value
    ES-->>U: Return results using freshly computed value
    Note over ES: Computed value is discarded after response
        

What Happens When You “Promote” a Runtime Field

A very common real-world pattern is to start a field’s life as a runtime field, and only later, once you’re confident the logic is correct and it’s being queried heavily, convert it into a genuine indexed field through reindexing. This gives teams a safe way to experiment with new derived fields without committing to a slow, disk-hungry reindex until the value has been proven.

Lifecycle Across Schema Changes

Because a runtime field’s definition lives in the mapping but not in the stored document data, changing its script is as simple as updating the mapping — there is no reindexing step at all. This is one of the single biggest lifecycle advantages: a mistake in the logic, or a new business requirement, can be fixed in seconds rather than by rewriting an entire index.

Nothing in engineering is free. Runtime fields trade one resource for another, and knowing exactly what you’re trading is essential to using them well.

The Core Trade-off, in Plain Terms

Indexed fields spend disk space and indexing time up front so that every future query is as fast as possible. Runtime fields spend nothing up front and instead pay a small computational cost on every query. If a field is searched rarely, or its logic changes often, runtime is usually the smarter trade. If a field is searched constantly by thousands of users a second, paying the indexing cost once is usually the smarter trade.

Because runtime fields involve running scripts, it’s natural to ask whether they open any security risks — and Elasticsearch has specific protections built in for exactly this reason.

Sandboxing Through Painless

Painless was designed from the start to be a safe scripting language for a multi-tenant search engine. It has no access to the file system, no network access, no ability to spawn processes, and no reflection tricks that could reach outside its sandbox. This means a runtime field script, even a poorly written one, cannot be used to read arbitrary files on the server or reach out to other systems.

Denial-of-Service Style Risks

The realistic security concern with runtime fields is less about data theft and more about resource exhaustion. Because a script runs per document, a poorly written or maliciously crafted runtime field query — especially one with an expensive script and a weak filter — could consume significant CPU across a cluster. This is why Elasticsearch enforces script complexity and execution limits, and why access to define arbitrary runtime fields, particularly ad-hoc ones inside search requests, should be restricted to trusted users or applications.

Sensitive Data Exposure

Because runtime field scripts typically read from _source, they can technically access any field in the original document, including ones that may be intended to stay hidden from search results. Field-level security controls should be reviewed carefully, since a cleverly written runtime field could otherwise expose data that a role is supposed to be restricted from seeing.

Since runtime fields shift cost from indexing time to query time, keeping an eye on that cost becomes an ongoing operational responsibility rather than a one-time concern.

What to Watch

Using the Profile API

Elasticsearch offers a built-in profiling capability that breaks down exactly how much time each part of a query took, including time spent inside runtime field scripts specifically. This is the most direct way to confirm whether a slow query is actually being caused by a runtime field, rather than guessing.

Logging Practices

It’s good practice to log slow queries — queries that exceed some acceptable time threshold — with enough detail to see which fields, including runtime fields, were involved. Over time, this log becomes a valuable dataset for deciding which runtime fields are hot enough to be worth promoting into real indexed fields.

Over time, the Elasticsearch community has settled on a handful of patterns that work well with runtime fields, and a handful of traps that are easy to fall into.

Good Patterns

Anti-Patterns to Avoid

A short, practical checklist for using runtime fields effectively in real systems.

Best Practices

Common Mistakes

Runtime fields are widely used in observability, security, and analytics platforms built on Elasticsearch. Here are common, realistic scenarios.

Across all of these cases, the underlying motivation is the same: the cost of being wrong about a schema decision made months ago should not require reprocessing an enormous amount of historical data. Runtime fields turn “we should have indexed it differently” from a multi-day data engineering project into a mapping edit that takes effect immediately.

Yes. Runtime fields can be used inside most aggregation types, though because the value must be computed per document on the fly, aggregations over very large document sets can be noticeably slower than the same aggregation over an equivalent indexed field.

Yes, a runtime field can shadow an existing indexed field by sharing its name, effectively overriding how that field behaves for queries, while the original indexed data remains untouched underneath.

The execution cost is essentially the same either way. The difference is scope and reusability — a mapping-level definition is available to every future query automatically, while a query-level definition only exists for that one request.

They are generally better suited to filtering, sorting, and simple transformations rather than full-text search, since they lack the specialized inverted index structures that make text search fast on genuinely indexed text fields.

Elasticsearch provides ways to handle this gracefully within the script logic itself, such as checking whether a source value exists before using it. If unhandled, an error on one document can affect the entire query’s results for that shard.

Ask how often it will be queried and how likely its logic is to change. Frequently queried, stable logic belongs as an indexed field. Rarely queried, evolving, or exploratory logic belongs as a runtime field.

Runtime fields support a defined but somewhat narrower set of types, covering the most common cases such as text, numbers, dates, booleans, and a few others, which is usually sufficient for most derived-value use cases.

A runtime field flips the usual order of operations in Elasticsearch: instead of deciding what a field means before storing data and paying that cost once, you decide what it means the moment you need an answer, and pay a small cost every time you ask. That single shift unlocks enormous flexibility — instant schema changes, retroactive application to historical data, and a safe space to experiment — at the price of extra computation on every query that uses it. Used deliberately, alongside ordinary indexed fields rather than as a wholesale replacement for them, runtime fields let teams evolve their data model at the speed of a business question, instead of at the speed of a reindexing job.