Before we look at n-grams specifically, we need to understand what a tokenizer does in general, and why the standard word-splitting approach falls short for partial matching.

What is a tokenizer?

A tokenizer is the component within Elasticsearch’s text-analysis pipeline responsible for breaking a stream of raw text into individual, indexable units called tokens. The most familiar and commonly used tokenizer splits text on whitespace and punctuation, producing one token per whole word — so “wireless mouse” becomes the two tokens “wireless” and “mouse.” This whole-word approach works beautifully for typical full-text search, but it has a clear blind spot: it can only ever match complete words, never fragments or substrings within a word.

So what is an n-gram?

An n-gram is a short, contiguous sequence of characters (or sometimes words, though character n-grams are far more common in this context) taken from a larger piece of text, where “n” simply refers to the length of that sequence. An n-gram tokenizer, instead of producing one token per whole word, slides a small window of a fixed character length across the text, producing a token for every possible position of that window, resulting in many small, overlapping fragments rather than a handful of whole-word tokens.

Two related but distinct flavors

Elasticsearch offers two closely related tools worth distinguishing clearly from the start. A plain n-gram tokenizer generates fragments starting from every position within a word, useful for matching a substring located anywhere inside a larger word. An edge n-gram tokenizer generates fragments only from the beginning of a word, growing outward one character at a time — producing, for the word “search,” fragments like “se,” “sea,” “sear,” “searc,” and “search.” Edge n-grams are specifically what powers the familiar experience of search suggestions appearing as a user types character by character.

Let’s look at where an n-gram tokenizer fits inside the broader text-analysis pipeline, and the settings that shape its behavior.

The analysis pipeline it belongs to

Recall that an analyzer is generally made up of a character filter stage, a tokenizer stage, and a token filter stage. An n-gram tokenizer is simply a specific choice made at the tokenizer stage, slotted into this same overall pipeline just as the standard whole-word tokenizer would be. This means an n-gram-based field can still be combined with the same lowercase normalization, stop-word handling, or other token filters used elsewhere, if desired, applied on top of the raw n-gram fragments it produces.

Minimum and maximum gram length

An n-gram tokenizer’s behavior is primarily controlled by two settings: a minimum fragment length and a maximum fragment length. Rather than producing fragments of only one fixed size, it typically produces fragments across a configured range — for example, a minimum of 2 and a maximum of 3 characters would produce both 2-character and 3-character fragments for every applicable position in the text. Choosing this range carefully is one of the most important decisions when configuring n-gram-based search, since it directly controls the trade-off between matching flexibility and index size.

Where n-grams are typically applied: multi-fields

N-gram-based analysis is rarely applied to a field’s only representation. It is far more common to add an n-gram-analyzed sub-field alongside a normally-analyzed field (using the multi-field concept covered elsewhere in this reference library), so that ordinary full-text search continues to use standard whole-word matching, while a dedicated sub-field is available specifically for partial-match or autocomplete-style queries when needed.

flowchart TB
    A["Raw Text Input
'search'"] --> B["N-gram Tokenizer
min: 2, max: 3"]
    B --> C["se"]
    B --> D["sea"]
    B --> E["ea"]
    B --> F["ear"]
    B --> G["ar"]
    B --> H["arc"]
    B --> I["...and more overlapping fragments"]
    C --> J["Inverted Index
of Overlapping Fragments"]
    D --> J
    E --> J
    F --> J
    G --> J
    H --> J
    I --> J
    J --> K["Substring Query Matches Fragment
Directly, No Wildcard Scan Needed"]
    

Let’s trace, step by step, exactly what happens the moment a document containing n-gram-analyzed text is indexed, and later searched.

N-gram tokenization shapes both how data grows on disk over its lifetime and how a search experience feels character by character.

Index growth over time

Because n-gram tokenization multiplies a single word into many overlapping fragment tokens, an n-gram-analyzed field naturally grows the size of an index more than an equivalent standard-analyzed field would, and this effect compounds as more documents are indexed over the index’s lifetime. This growth is a direct, predictable consequence of the fragment-length range chosen; wider ranges producing proportionally more fragments per word.

The search-as-you-type lifecycle

In a typical autocomplete experience built on edge n-grams, a user’s keystrokes drive an evolving sequence of queries: after one character, results reflect an extremely broad match; after several more characters, the effective match narrows considerably, purely because the query text itself is growing longer and more specific with each keystroke, while the underlying edge-n-gram-indexed fragments remain unchanged. This lifecycle — broad to narrow, driven entirely by growing query length — is what gives autocomplete its familiar, responsive, narrowing-as-you-type feel.

Reindexing implications

Just like other analyzer choices, changing an n-gram tokenizer’s minimum or maximum length settings on an already-populated field requires a reindex, since the previously generated fragments were built using the old configuration and cannot be retroactively regenerated in place.

N-gram tokenization solves a genuine problem well, but it is one of the more resource-intensive text-analysis choices available, and that trade-off deserves careful thought.

N-gram tokenization is not primarily a security feature, but it has a few notable implications worth understanding.

Increased partial-match discoverability

Because n-gram tokenization deliberately makes substrings of a value independently searchable, a sensitive field indexed this way becomes discoverable through partial fragments as well as whole values, which can meaningfully widen the practical surface through which sensitive information might be inferred or located, compared to a standard whole-word analyzed field. This is a reason to apply n-gram analysis specifically to fields where partial matching is a genuine, deliberate requirement, rather than broadly across all text fields by default.

Resource consumption as an availability concern

Because indexing n-gram-analyzed content is comparatively expensive, a system that allows uncontrolled or excessively large text input into an n-gram-analyzed field can experience disproportionate indexing load compared to the same input on a standard field, which is worth factoring into capacity planning and input-size validation for any user-facing system accepting free-form text into such a field.

Because n-gram tokenization’s costs are largely structural (disk size, indexing throughput), watching a few specific signals goes a long way toward catching problems early.

What to watch

• Index size per field: comparing the storage footprint of n-gram-analyzed fields against comparable standard-analyzed fields reveals the true, ongoing cost of the chosen gram-length range in production.
• Indexing throughput and latency: tracking whether documents containing n-gram-analyzed fields take noticeably longer to index than similar documents without them helps quantify the real-world indexing overhead.
• Autocomplete query latency by keystroke position: for search-as-you-type experiences, monitoring latency at each stage of a growing query (one character, two characters, and so on) helps confirm the feature remains responsive throughout.
• Field usage relative to cost: periodically confirming that an n-gram-analyzed field is genuinely being queried in ways that rely on its partial-matching capability, rather than sitting unused while still incurring its full indexing and storage cost.

Certain n-gram usage patterns are considered established best practice, and certain shortcuts consistently cause avoidable pain.

Established patterns

Anti-patterns to avoid

A short, practical checklist distilled from how experienced Elasticsearch practitioners use n-gram tokenizers well.

N-gram tokenization quietly powers many of the responsive, forgiving search experiences people interact with every day.