To appreciate kNN search, we first need to see what ordinary search does well and where it runs out of road.

How keyword search works

Traditional Elasticsearch search is built on the inverted index, which lists, for every word, the documents that contain it. When you search for “cheap laptop,” Elasticsearch finds documents containing those words and ranks them with a scoring formula that rewards rare words, frequent occurrences and short fields. This approach, often called lexical search, is fast, precise and easy to explain. If a document contains your words, it can be found.

The weakness is that lexical search only understands words, not ideas. A document about an “inexpensive notebook computer” describes exactly what the user wants, yet it shares no important words with the query. Synonyms, paraphrases, spelling variations, questions phrased in everyday language, and images all fall outside what an inverted index can handle without heavy manual effort.

The idea behind searching by meaning

The trick is to describe every document, and every question, as a point in a kind of map where similar meanings sit close together. A sentence about laptops lands near other sentences about laptops. A photo of a beach lands near other beach photos. Once everything is a point on this map, finding relevant results becomes a geometry problem: given the point for the question, find the closest points. Those closest points are the nearest neighbors.

graph LR
  subgraph Lexical["Keyword search"]
    Q1["Query words"] --> II["Inverted index: word to documents"]
    II --> R1["Documents containing the words"]
  end
  subgraph Semantic["kNN search"]
    Q2["Query turned into a vector"] --> VI["Vector index: points on a map"]
    VI --> R2["Documents whose meaning is closest"]
  end
        

Everything in kNN search depends on one idea: describing things with lists of numbers.

What is a vector?

A vector is simply an ordered list of numbers. You already know a tiny one: a location on a map can be described by two numbers, how far east and how far north. A point in a room needs three numbers. In machine learning, vectors are much longer, often hundreds or even thousands of numbers, and each number captures some faint aspect of the thing being described. Nobody labels the aspects by hand; the model discovers them during training.

What is an embedding?

An embedding is a vector produced by a machine learning model for a piece of data such as a sentence, a paragraph, a product photo or an audio clip. The model has been trained on huge amounts of examples, so that things with similar meaning receive similar lists of numbers. The word “car” and the word “automobile” get vectors that are very close, while “car” and “banana” get vectors that are far apart.

Dimensions

The number of values in a vector is called its number of dimensions. Common text embedding models produce vectors with a few hundred to a couple of thousand dimensions. More dimensions can capture finer detail but cost more memory and more computing effort. Every vector in the same field must have exactly the same number of dimensions, and the query vector must match too.

flowchart LR
  T["Text: cheap laptop for students"] --> M["Embedding model"]
  M --> V["Vector: 0.12, -0.53, 0.88, ... hundreds more"]
  P["Photo of a beach"] --> M2["Image embedding model"]
  M2 --> V2["Vector: 0.71, 0.02, -0.34, ..."]
        

To find the nearest neighbors, we need a clear rule for how near two vectors are.

Three popular ways to measure

There are several ways to measure closeness, and Elasticsearch supports the main ones. Each is a small formula that takes two vectors and returns one number. Choosing the wrong one for your model gives poor results, so it is important to know what each means and which your embedding model expects.

Which one should I use?

Use the metric recommended by the documentation of your embedding model. Many text models are trained with cosine similarity or with dot product on normalized vectors. Normalized means each vector has been scaled to a length of exactly one. When vectors are normalized, cosine similarity and dot product give the same ranking, and dot product is cheaper to compute, so it is a favorite for performance.

Now we can put the pieces together into a precise definition.

A precise definition

k-nearest neighbor search, written kNN, is the task of finding the k stored items whose vectors are closest to a given query vector, according to a chosen similarity measure. The letter k is simply how many results you want, such as the ten nearest neighbors. In Elasticsearch, the stored items are documents with a vector field, the query vector represents the user’s question, and the result is a ranked list of the k most similar documents.

The name comes from an old and simple idea in data science: to decide something about a new item, look at the items nearest to it. If you want to guess what kind of film someone will like, look at the films closest to the one they just enjoyed. That basic idea powers modern semantic search, recommendations and question answering.

graph TD
  Q["Query point"] --> N1["Neighbor 1: distance 0.10"]
  Q --> N2["Neighbor 2: distance 0.14"]
  Q --> N3["Neighbor 3: distance 0.19"]
  Q -.-> F1["Far point: distance 0.90"]
  Q -.-> F2["Far point: distance 1.10"]
        

What kNN search is used for

The simplest way to find nearest neighbors is to check everything. It is accurate, and it is also slow at scale.

The brute-force method

Exact kNN, sometimes called brute force or flat search, compares the query vector with every stored vector, computes the similarity for each, and keeps the best k. There is no cleverness involved, which is its strength: the answer is always perfectly correct, and there is no index to build or tune. For small collections, or when a filter has already reduced the candidates to a few thousand documents, it is an excellent choice.

Why it breaks down

The work grows in direct proportion to the number of documents and the number of dimensions. Comparing one query against ten thousand vectors of 768 dimensions is quick. Comparing it against fifty million vectors means tens of billions of arithmetic operations for a single query, and that is far too slow if hundreds of users search at once. Reading all those vectors from memory or disk adds delay as well.

To go fast, we accept a tiny chance of missing the perfect answer in exchange for looking at only a small part of the data.

The bargain: approximate search

Approximate nearest neighbor search, or ANN, finds neighbors that are very likely, though not guaranteed, to be the true nearest ones. By building a special index structure in advance, it can find good answers while examining only a tiny fraction of the vectors. The measure of how often it finds the true nearest neighbors is called recall. A recall of ninety-five percent means that, on average, ninety-five of the true top hundred neighbors are found.

How HNSW works

Elasticsearch uses an algorithm called HNSW, short for Hierarchical Navigable Small World. It organises vectors into a graph, a network in which each vector is connected to a handful of its nearby friends. Searching the graph means starting somewhere, looking at the friends of the current point, stepping to whichever friend is closer to the query, and repeating until no closer friend can be found.

The hierarchical part adds layers. The top layer contains only a few vectors with long-distance links, like motorways. Lower layers contain more vectors with shorter links, like main roads, and the bottom layer contains every vector with local links, like streets. A search starts at the top, takes big strides toward the right neighborhood, then drops down a layer at a time, taking smaller and smaller steps until it reaches the streets around the target.

graph TD
  L2["Layer 2 (few points, long links)"] --> L1["Layer 1 (more points, medium links)"]
  L1 --> L0["Layer 0 (all points, short links)"]
  L0 --> R["Closest neighbors found"]
  Q["Query vector"] --> L2
        

The two build-time knobs

The details of storage explain nearly every performance and memory question you will meet later.

One graph per segment

Elasticsearch stores each index in shards, and each shard is made of immutable pieces called segments. The HNSW graph is built separately for each segment. When new documents are written and a new segment is created, a new graph is built for it. When segments are merged in the background, the graphs of the merging segments are combined into a new graph for the merged segment, which is real work for the node.

This has two consequences. First, a search must visit the graph in every segment of every shard involved, so having many small segments makes searches slower. Second, indexing vectors is more expensive than indexing plain fields, because building and merging graphs takes processing time.

Vectors live outside the Java heap

The vectors and the graph structure are stored in files on disk and are read through the operating system’s file cache, not held in the Java heap. For fast searches, the parts of the graph that queries touch should be present in that cache, because reading them from disk for every hop would be slow. This is why memory planning for kNN is about leaving enough free memory outside the heap for the file cache, a point we will return to in the capacity planning chapters.

graph LR
  subgraph Shard["One shard"]
    S1["Segment 1: vectors and HNSW graph"]
    S2["Segment 2: vectors and HNSW graph"]
    S3["Segment 3: vectors and HNSW graph"]
  end
  Q["kNN query"] --> S1
  Q --> S2
  Q --> S3
  S1 --> M["Merge best candidates"]
  S2 --> M
  S3 --> M
        

Before you can search vectors, you must tell Elasticsearch which field holds them and how to treat them.

The field type

Vectors are stored in a field of type dense_vector. The word dense means that every position in the list holds a value, as opposed to sparse representations in which most positions are empty. You declare this in the index mapping, along with a few properties that control how the field behaves.

Decisions you cannot easily undo

Several choices in the mapping are effectively permanent for an index. The number of dimensions cannot be changed once documents exist. The similarity metric is part of how the graph is built, so changing it means reindexing. Because of this, it is wise to test with a small sample of data and your chosen model before creating a large production index.

Elasticsearch stores and searches vectors, but the vectors have to come from somewhere. There are several routes.

Route one: create vectors outside and send them in

You can run an embedding model in your own application, or call an external service, and then index each document together with its vector. This gives you complete freedom over the model, and it is the approach used when you have a specialised or fine-tuned model. The responsibility for keeping the model, its version and its dimensions consistent lies with you.

Route two: let Elasticsearch call a model

Elasticsearch has an inference feature that connects to models, either hosted inside the cluster on machine learning nodes or provided by external services. An ingest pipeline can call the model for each incoming document and store the resulting vector automatically. At query time, the same model can convert the user’s text into a vector on the fly, so the application sends plain text and never handles vectors at all.

Route three: semantic fields

Newer versions offer a semantic text field type that handles much of the work automatically. It splits long text into chunks, creates embeddings for each chunk using a configured inference endpoint, and stores them for searching. This reduces the amount of code needed for the common case of semantic search over text.

sequenceDiagram
  participant App as Application
  participant ES as Elasticsearch
  participant Model as Embedding model
  App->>ES: Index document text
  ES->>Model: Create embedding for the text
  Model-->>ES: Vector
  ES-->>App: Stored with vector
  App->>ES: Search with question text
  ES->>Model: Create embedding for the question
  Model-->>ES: Query vector
  ES-->>App: Nearest documents
        

A kNN search request has only a handful of parts. Understand each and you can read any example.

Reading a request in words

In plain language, a typical request says: “In the field called description_vector, find the ten documents closest to this query vector, considering fifty candidates on each shard, and only among documents whose category is shoes.” The answer comes back in the usual search response format, with the hits ordered by score, and each hit carrying its document source. You can request only some fields, add highlighting to other parts of a combined request, and use all the familiar options for paging and sorting.

A knn option and a knn query

Elasticsearch offers kNN as a top-level option of the search request and also as a query type that can be placed inside the query section. The top-level option is the classic route and is well suited to standalone kNN or to combining kNN with ordinary queries. The query form makes it possible to use kNN inside compound queries and other places where queries are accepted. Both use the same underlying graph search.

Following one request through the cluster shows where time and memory are spent.

sequenceDiagram
  participant C as Client
  participant N as Coordinating node
  participant S1 as Shard 1
  participant S2 as Shard 2
  C->>N: kNN request (k=10, num_candidates=50)
  N->>S1: Search graphs, return top candidates
  N->>S2: Search graphs, return top candidates
  S1-->>N: 50 candidates with scores
  S2-->>N: 50 candidates with scores
  N->>N: Merge 100 candidates, choose best 10
  N->>S1: Fetch documents
  N->>S2: Fetch documents
  N-->>C: Top 10 hits
        

Two numbers in every request quietly control the balance between fast answers and accurate answers.

What each number does

The number k is the size of the final answer: how many results the user will receive. The number num_candidates is how many nearest neighbor candidates each shard considers internally before it hands its best ones to the coordinating node. Candidates are the pool from which the answer is picked. A larger pool means the search explores more of the graph and is more likely to include the true nearest neighbors, but it also visits more vectors and therefore takes longer.

The rule is that num_candidates must be at least as large as k, and there is an upper limit on how large it can be. In practice, a value several times larger than k is common. Raising it is the simplest lever for improving recall when results look poor, and lowering it is the simplest lever for improving speed when latency is too high.

A tuning recipe

Real applications almost never want “the nearest thing overall.” They want the nearest thing that also meets some condition.

Why filtering is tricky in vector search

Suppose a shopper wants shoes similar to a photo, but only in their size and only in stock. There are two obvious ways to combine similarity with conditions. One is post-filtering: find the nearest neighbors first, then throw away those that fail the condition. The trouble is that you may throw away most or all of them and end up with too few results, or none.

The other is pre-filtering: apply the condition first, and find the nearest neighbors only among the documents that pass. This guarantees that the requested number of matching results are returned, as long as enough documents match. Elasticsearch supports this by letting you place a filter inside the kNN search, so the graph search itself respects the condition as it walks.

flowchart LR
  subgraph Post["Post-filtering: risky"]
    A1["Find 10 nearest"] --> A2["Remove non-matching"] --> A3["Maybe only 2 remain"]
  end
  subgraph Pre["Filter inside the kNN search: safer"]
    B1["Search graph among matching documents only"] --> B2["Return 10 matching neighbors"]
  end
        

What happens behind the scenes

While walking the graph, the search simply ignores documents that do not pass the filter when collecting results, though it may still pass through them on the way. If the filter is extremely restrictive and matches only a small number of documents, walking the graph can become less efficient than simply checking those few documents directly, and the engine may fall back to an exact comparison over them. This is often a pleasant surprise: highly selective filters can make searches both faster and perfectly accurate.

Elasticsearch ranks all results by a relevance score, so raw similarity values are converted into scores that are always positive.

From similarity to score

Elasticsearch requires scores to be non-negative, and higher scores mean better matches. For cosine similarity and dot product, the raw value can be negative, so it is shifted and scaled: one is added to the similarity and the result is halved, mapping the range from minus one to plus one into the range from zero to one. For L2 distance, the score is one divided by one plus the squared distance, so a distance of zero gives a perfect score of one and greater distances give smaller scores.

These conversions keep the ordering the same as the raw metric. The closest vector always has the highest score. What changes is the scale, which matters when you combine kNN scores with scores from ordinary queries.

The similarity threshold

Because kNN always returns the k closest items, you might receive poor matches when nothing good exists. The similarity option lets you set a minimum quality: documents less similar than the threshold are not counted as matches, even if they are among the k nearest. Note that the threshold is expressed in the raw units of the metric, not in the converted score. For cosine similarity it is a cosine value between minus one and one; for L2 distance it is a maximum distance.

Meaning and words each catch things the other misses, so the best systems use both.

Why hybrid?

Semantic search is excellent at finding paraphrases and related ideas, but it can stumble on exact terms such as product codes, names, error numbers and rare jargon, which an embedding model may blur. Lexical search nails exact terms but misses paraphrases. Hybrid search runs both and combines the results, so that a document that ranks well on either measure has a chance to appear, and documents strong on both rise to the top.

Ways to combine

The simplest method is to include both a kNN section and a normal query in one request. Elasticsearch adds the scores together, and each part can be given a boost to control how much it matters. Because the two scores are on different scales, tuning boosts takes some experimentation. A more robust method is reciprocal rank fusion, which ignores raw scores and combines results using only their positions in each ranked list: an item near the top of either list receives a good combined rank. Retrievers, a newer request style, let you describe such combinations clearly and even chain reranking steps.

graph TD
  Q["User question"] --> L["Keyword query on text fields"]
  Q --> E["Embedding of the question"]
  E --> K["kNN search on vector field"]
  L --> F["Combine: weighted scores or rank fusion"]
  K --> F
  F --> R["Final ranked results"]
        

Sometimes brute force is exactly what you want, and Elasticsearch supports it directly.

How it works

Besides the graph-based search, Elasticsearch can compute vector similarity inside a scripted scoring query. A normal query first selects a set of documents, and then a script computes the similarity between the query vector and each selected document’s vector, using built-in functions for cosine similarity, dot product and distances. Because every selected document is scored, the result is exact.

This approach does not need the graph index at all, so it works with vector fields that are not indexed for approximate search. Its cost is proportional to the number of documents scored, which means it is only sensible when the pre-selected set is modest, such as a few thousand documents narrowed down by a strong filter.

Real documents are long, and one vector rarely captures a whole long document well.

Why chunking helps

An embedding model compresses text into a fixed-size vector. For a short sentence, the vector can be rich and precise. For a long report, the single vector is a blurry average of many topics, and a question about one detail may not match it well. The remedy is chunking: split the document into passages of a sensible size, embed each passage, and search over passages.

Good chunks respect natural boundaries such as paragraphs and sections, are small enough for the model to represent faithfully, and often overlap slightly so that an idea split across a boundary is still captured whole in at least one chunk. Each chunk should keep a reference to its parent document, so that results can be grouped and displayed properly.

Ways to store chunks

Vectors are big. Quantization is the main tool for making them smaller without ruining search quality.

The size of a vector

A typical vector stores each number as a four-byte decimal value. A vector with 768 dimensions therefore occupies about three thousand bytes, roughly three kilobytes. A collection of ten million such vectors holds around thirty gigabytes of raw vector data before counting the graph. To search quickly, the hot parts of that data should sit in the file cache, so large collections demand large amounts of memory.

What quantization does

Quantization stores each number with less precision, using fewer bits. Instead of a four-byte decimal, a number might be squeezed into a single byte, a half byte or even a single bit. Comparing the compressed vectors is much faster and needs far less memory, at the cost of slightly less precise similarity values. Elasticsearch typically keeps the original full-precision vectors on disk too, so that a final rescoring step can correct the small errors for the top candidates.

A little arithmetic tells you whether a plan is realistic before you buy any machines.

The scenario

A company wants semantic search over ten million product descriptions. Their embedding model produces 768-dimensional vectors, and each vector is stored as four-byte decimals. They want fast searches, meaning the vectors should live in the file cache. Let us estimate what that requires.

Loading millions of vectors takes planning. A few habits make it much smoother.

Why indexing is slower

Each new vector must be linked into the graph of its segment by searching for its best neighbors, which is real computation. Later, when segments merge, graphs must be rebuilt or combined. Creating embeddings, if you do it during ingestion, adds further time because a model must run for every document. On big loads, these costs add up to hours rather than minutes.

Practical tips

Fast vector search comes from a few well-understood conditions. Check them one by one when latency disappoints.

flowchart TD
  A["Search is slow"] --> B{"Is the time spent creating the query vector?"}
  B -->|"Yes"| C["Cache embeddings, scale the model service"]
  B -->|"No"| D{"Are vector files in the file cache?"}
  D -->|"No"| E["Add memory, use quantization, reduce data per node"]
  D -->|"Yes"| F{"Many segments or shards?"}
  F -->|"Yes"| G["Merge segments, reduce shard count"]
  F -->|"No"| H["Lower num_candidates or tighten filters"]
        

A fast search that returns the wrong things is worthless. Quality needs to be measured, not assumed.

Two kinds of quality

The first kind is recall of the approximation: does the fast approximate search find the same neighbors that an exact search would? This measures the cost of the shortcut and can be computed automatically by comparing the two on sample queries. The second kind is relevance: are the nearest neighbors actually useful to the user? This depends on the embedding model, the chunking, the filters and the hybrid combination, and it must be judged by people or by labelled examples.

A simple evaluation routine

Elasticsearch supports two families of vector-based search, and it helps to know how they differ.

Dense vectors

Everything so far has used dense vectors, in which every position holds a number and the whole vector describes the overall meaning of the text. kNN search operates on dense vectors. They are compact and excellent at capturing broad semantic similarity, but individual numbers have no human-readable meaning, so it is hard to explain why two items matched.

Sparse vectors

A sparse vector has a huge number of possible positions, but only a few are filled. Each position usually corresponds to a word or word-like term, and the value says how important that term is to the text. Learned sparse models expand a text with related terms, so a document about “laptops” might also carry weights for “notebook” and “computer.” Sparse vectors are searched using the inverted index machinery rather than graph-based kNN, and they retain some of the explainability of keyword search, since you can see which terms contributed.

Vectors are derived from your data, so the usual rules of data protection apply, with a few extra points.

Access control still applies

Role-based access, document-level security and field-level security work with vector searches just as they do with ordinary searches. A user restricted to certain documents can only receive neighbors among those documents. Make sure that any restriction is enforced by the security features or by filters inside the kNN search, not by hiding results in the user interface.

Vectors are not anonymous

It is tempting to think that a list of numbers cannot reveal the original text. Research has shown that, in some circumstances, information about the source text can be partially recovered from embeddings. Treat vectors that come from sensitive content as sensitive too, and protect them with the same access rules, encryption and retention policies as the source documents.

Model and service credentials

When Elasticsearch calls an external model service, it needs credentials such as keys. Store them using the secure mechanisms provided, restrict who can create or change inference endpoints, and rotate keys regularly. Remember also that text sent to an external service leaves your cluster, which may matter for privacy and compliance rules.

Vector workloads have their own warning signs. Watch them and you will catch trouble early.

Where to look

Node and index statistics give search and indexing rates and times, segment counts and merge activity. The disk usage analysis tool reports how much space each field, including vector fields, consumes. Operating system metrics reveal file cache behaviour and disk reads. Slow logs highlight the slowest searches. Your application should also log the time of each stage: creating the query vector, running the search and post-processing.

Let us follow a realistic project from idea to launch to see how the pieces fit together.

The scenario

A software company has a help centre with fifty thousand articles. Customers search with questions such as “why can’t I log in after changing my phone,” but the articles use titles like “Two-factor authentication device reset.” Keyword search returns poor results because few words overlap. The team decides to add semantic search with kNN and combine it with their existing keyword search.

graph TD
  A["Help articles"] --> B["Split into overlapping chunks"]
  B --> C["Ingest pipeline creates embeddings"]
  C --> D["Index: text fields and vector field"]
  Q["Customer question"] --> E["Embed the question"]
  E --> F["kNN search with language filter"]
  Q --> G["Keyword search"]
  D --> F
  D --> G
  F --> H["Fuse rankings"]
  G --> H
  H --> I["Show best passages and article links"]
        

These habits have proved themselves across many production systems.

Recognise these mistakes early, and you will avoid the most common kNN disappointments.

Every technology is a bargain. Here is what kNN search gives and what it costs.

Keyword search and kNN search side by side

Here is how organisations in different fields put nearest neighbor search to work.

The lesson across all these examples

In every case, the value came from letting users express what they mean, rather than forcing them to guess the words a document uses. And in every case, the successful teams paired the technology with filters, keyword matching, careful evaluation and sensible capacity planning, instead of relying on the vectors alone.

A few misunderstandings appear again and again. Let us set them straight.

A short history helps you read older articles and understand why the feature looks the way it does.

Short answers to the questions beginners ask most often about kNN search.

It finds the k stored items whose vectors are closest to the vector of a query, so results are chosen by similarity of meaning rather than by matching exact words.

An embedding is a list of numbers produced by a machine learning model that represents the meaning of a piece of data, so that similar items have similar lists.

Exact kNN compares the query with every candidate vector and is perfectly accurate but slow at scale. Approximate kNN uses an HNSW graph to examine only a small part of the data, trading a tiny chance of missing a true neighbor for much greater speed.

It sets how many candidates each shard considers before the best results are chosen. Higher values improve accuracy and slow the search; lower values do the opposite. It must be at least as large as k.

Use the one recommended for your embedding model. Cosine is a safe default. Dot product is fast when vectors are normalized to unit length. L2 distance suits models trained with Euclidean distance.

Not on an existing field that already holds data. Create a new index with the new mapping and reindex, generating vectors with the corresponding model.

If the condition is applied after retrieving the k neighbors, many may be discarded. Put the filter inside the kNN search so that the neighbors are chosen among matching documents only.

No. They are stored in files and read through the operating system’s file cache. Leave enough free memory outside the heap for the cache to hold the hot vector data.

It is storing vector numbers with fewer bits, which shrinks the memory needed and speeds comparisons at a small cost in precision. Full-precision originals are usually kept for rescoring.

For text, usually yes. Hybrid search handles paraphrases through vectors and exact terms through keywords, and typically beats either alone.

Because it returns the closest items whatever their distance. Use a similarity threshold, and check quality on real queries, so that weak matches are not presented as good ones.

Split them into passages, embed each passage, and keep a reference to the parent document. Search over passages and group or display results by parent.

Measure recall against exact search to check the approximation, and measure relevance against labelled queries to check usefulness. Repeat whenever the model, data or settings change.

Yes, but it requires re-embedding all documents and creating a new index. Vectors from different models must never be mixed. Use an alias to switch over smoothly.

No. Vectors can reveal information about their source text, so protect them with the same access controls, encryption and retention rules as the documents they came from.

k-nearest neighbor search finds the items whose vectors lie closest to a query vector, letting Elasticsearch match by meaning instead of by words. Embedding models turn text, images and other data into vectors, similarity metrics define what near means, and an HNSW graph in each segment makes the search fast by examining only a small part of the data. The controls that matter most are the choice of model and metric, the values of k and num_candidates, filters applied inside the search, hybrid combination with keyword retrieval, and quantization for memory. Success depends as much on discipline as on technology: chunk documents sensibly, never mix models, leave memory for the file cache, protect vectors like the data they came from, and measure recall and relevance on honest queries before and after every change.