Semantic search is a way of searching where the engine tries to understand what you mean, not only which words you typed.

The word semantic comes from the study of meaning. In classic search, the engine compares the letters and words of your question with the letters and words in documents. If they match, the document is a candidate. In semantic search, the engine compares the meaning of your question with the meaning of documents. Two texts can share no words at all and still be judged very close, because they talk about the same idea.

Elasticsearch has always been excellent at the first kind of search, which is called lexical search. Over the last few years it has added many features for the second kind: fields that store meaning as numbers, models that turn text into those numbers, fast search over them, and ways to blend both kinds of search into one result list. This guide walks through all of it in simple steps.

People say the same thing in many ways. Search that only sees words is easily fooled.

Human language is full of synonyms, different phrasings, spelling styles, and context. A customer may search for “cheap phone with good battery” while the product page says “budget smartphone, long-lasting power”. Almost no words match, yet the product is exactly what the customer wants. This gap between how people ask and how documents are written is called the vocabulary mismatch problem.

The opposite problem also exists. One word can have several meanings. A search for “jaguar speed” could mean the animal or the car. Keyword search returns both, because the word is the same, and it cannot tell which one you care about.

Older tricks tried to reduce the gap: synonym lists, stemming, spelling correction, and hand-written rules. They help, but they need constant human work, and they never cover everything. Semantic search attacks the root of the problem instead.

To appreciate semantic search, we should remember what the traditional approach does well.

When you index a document, Elasticsearch breaks the text into small pieces called tokens, usually words. It builds an inverted index, which is like the index at the back of a textbook: for each word, a list of documents that contain it. When you search, the engine finds the documents that contain your words and then scores them using a formula called BM25.

BM25 gives a higher score when your words appear more often in a document, when the words are rare across all documents, and when the document is short. Rare words such as “photosynthesis” matter more than common words such as “the”. This approach is fast, cheap, transparent, and very precise for exact terms.

The central idea of semantic search is the embedding. Everything else in this guide is built on it.

An embedding is a list of numbers, called a vector, that represents the meaning of a piece of text. For example, a sentence might become a list of 384 or 768 numbers. A machine learning model, trained on huge amounts of text, decides what those numbers should be. The training teaches the model that texts with similar meanings should produce similar lists of numbers.

You never need to read the numbers yourself. What matters is the relationship: if two texts mean nearly the same thing, their numbers will be close to each other. If they mean very different things, their numbers will be far apart.

flowchart LR
  T1["Text: repairing a dripping faucet"] --> M["Embedding model"]
  T2["Text: how to fix a leaking tap"] --> M
  T3["Text: history of the Roman empire"] --> M
  M --> V1["Vector A: 0.21, -0.43, 0.88 and more"]
  M --> V2["Vector B: 0.20, -0.41, 0.90 and more"]
  M --> V3["Vector C: -0.75, 0.32, 0.10 and more"]
        

In the picture, vectors A and B are almost the same because the two texts describe the same task. Vector C is very different because it is about something else. Searching then becomes a simple question: which stored vectors are closest to the vector of the question?

If you could see all embeddings at once, related ideas would form neighborhoods.

Imagine a huge invisible map where every sentence has a location. Sentences about cooking gather in one region. Sentences about football gather in another. Sentences about banking are in a third. A question about “boiling pasta” lands in the cooking region, so the closest documents are recipes and kitchen tips, even if none of them uses the word “boiling”.

This map has many more than two directions. Each of the hundreds of numbers is one direction. Humans cannot picture that, but computers handle it easily, and the same ideas of “near” and “far” still work.

flowchart TD
  Q["Question: how long to boil pasta"] --> N{"Nearest neighbors in vector space"}
  N --> D1["Recipe: cooking spaghetti"]
  N --> D2["Guide: al dente timing"]
  N --> D3["Tips: salting water"]
  N -.->|"far away"| D4["Article: stock market news"]
        

Because the map is built once and reused, adding a new document just means placing a new point on it. The rest of the map does not change.

To find the nearest neighbors, the engine needs a way to measure how close two vectors are.

Elasticsearch supports a few standard measures. Each turns two vectors into a single similarity number. The right choice depends on how the embedding model was trained, so the model’s documentation should guide you.

Elasticsearch converts these measures into a positive score so that higher always means better, which lets vector results be shown and combined like any other search result.

A trained model creates them. Choosing that model is the most important quality decision you will make.

Embedding models are neural networks, usually of the transformer kind, trained on large collections of text. Given a piece of text, the model outputs its vector. Models differ in the size of the vector they produce, the languages they know, the length of text they can read, the domains they were trained on, and the computing power they need.

What to look for in a model

First, the language: a model trained mostly on English text will do poorly on Hindi or Japanese. Second, the maximum text length: most models read a limited number of tokens, so long documents must be cut into pieces. Third, the vector size: bigger vectors can hold more detail but cost more storage and time. Fourth, the domain: a model trained on general web text may not understand legal or medical language well. Finally, cost and speed: larger models are slower and need stronger hardware.

Elasticsearch supports two very different shapes of embedding. Knowing both prevents a lot of confusion.

A dense vector is a fixed-length list of numbers where almost every position holds a value, such as 768 numbers. Each position has no human meaning by itself. The meaning lives in the pattern of all numbers together. Dense vectors are searched with nearest neighbor search.

A sparse vector is a very long list where almost all positions are zero and only a few hundred carry weights. In the sparse approach used by Elastic’s own model, each position stands for an actual word or word piece. The model expands a text into related words with weights. A text about “cheap phones” might get weight on words such as “budget”, “smartphone”, “affordable”, and “mobile”, even though those words were never written. Sparse vectors are searched using the same inverted index technology as keyword search.

Here is the toolbox. Each block will get its own chapter, so treat this as a map of what is coming.

A semantic search system has a few moving parts. Seeing them together shows how they cooperate.

flowchart TD
  APP["Application"] --> CO["Coordinating node"]
  CO --> ING["Ingest node: pipelines and inference"]
  ING --> ML["Machine learning node or external model service"]
  ML --> ING
  ING --> DN["Data nodes: shards with text, vectors and HNSW graphs"]
  CO --> DN
  CO -.->|"embed the question"| ML
  DN --> CO
  CO --> APP
        

Because the model is used twice, once for documents and once for each user question, the model’s speed and availability affect both indexing speed and search latency.

The inference API gives every model the same simple front door, no matter where the model lives.

An inference endpoint is a named connection to a model. You create it once, telling Elasticsearch which task it performs, such as producing dense embeddings, producing sparse embeddings, or reranking, and which service provides it. After that, other features refer to the endpoint by name and do not care how the model is run.

The benefit is flexibility. You can start with an endpoint that runs a model inside your cluster, and later switch to a hosted provider, or the other way around, by changing the endpoint and re-embedding. Your search definitions can stay much the same.

ELSER is a model made by Elastic for semantic search that works well without any tuning.

ELSER stands for Elastic Learned Sparse EncodeR. It is a sparse model for English text. It reads a document and produces a set of weighted terms that describe it, including terms that were never in the original text. For example, a paragraph about a car engine might get weight on “motor”, “vehicle”, and “mechanic”. At query time, the question is expanded in the same way, and matching is done between the expanded terms.

Because the output is a list of real words with weights, it fits neatly into the same inverted index technology that powers keyword search. That makes it fast, reasonably cheap in memory, and easier to understand than dense vectors, since you can inspect which terms contributed to a match.

sequenceDiagram
  participant D as Document text
  participant E as ELSER model
  participant I as Index
  D->>E: cheap budget phone with long battery
  E->>I: Weighted terms: phone, cheap, budget, smartphone, battery, power, affordable, mobile
  Note over I: Stored as a sparse vector using the inverted index
        

Dense models are the most widely used family of embedding models, and they cover many languages and domains.

A dense embedding model reads text and outputs a fixed-length vector. Popular families include the E5 models, which have multilingual versions, and the many models offered by AI providers. Elasticsearch can run some of these inside the cluster, and can call the others through the inference API.

This is the field type that stores dense embeddings and makes them searchable.

When you define a dense_vector field in a mapping, you set a few things. The number of dimensions must equal the size of the vectors your model produces. The similarity measure tells the engine how to compare vectors. And the index options choose how vectors are stored and searched, including whether a graph is built and whether the numbers are compressed.

Also note that vectors take room. In many recent versions, the raw vector values are left out of the returned source by default, so that responses stay small, while the vectors are still stored and searchable. If you need them back, there are options to retrieve them explicitly.

If you want semantic search without plumbing, semantic text is the feature designed for you.

A semantic_text field hides the technical steps. You declare the field and, optionally, the inference endpoint. Then you simply index normal text into it. Elasticsearch takes care of splitting long text into chunks, calling the model for each chunk, and storing the resulting vectors in the right form, whether sparse or dense. At search time, you use a semantic query on the field with a plain text question, and Elasticsearch embeds the question with the same endpoint.

Models can only read a limited amount of text at once. Long documents must be cut into chunks.

Every embedding model has a maximum input length, measured in tokens, which are small pieces of words. Many models handle a few hundred tokens. If you feed a whole book chapter into such a model, it either refuses or quietly reads only the beginning and ignores the rest. Your search would then miss anything that appears later in the document.

The cure is chunking: splitting the document into smaller passages, embedding each passage separately, and searching over the passages. When a question matches a passage, the engine can return the parent document, and often the exact passage that matched, which is very useful for showing snippets or feeding answer generation.

flowchart TD
  DOC["Long document"] --> C1["Chunk 1"]
  DOC --> C2["Chunk 2"]
  DOC --> C3["Chunk 3"]
  C1 --> E1["Vector 1"]
  C2 --> E2["Vector 2"]
  C3 --> E3["Vector 3"]
  Q["Question vector"] -.->|"closest"| E2
  E2 --> R["Return document and best chunk"]
        

Let us follow one document from the moment it arrives to the moment it is searchable by meaning.

sequenceDiagram
  participant App as Application
  participant Co as Coordinating Node
  participant In as Ingest or Field Logic
  participant Mo as Model
  participant Sh as Shard
  App->>Co: Index a document with text
  Co->>In: Run ingest steps or semantic field logic
  In->>In: Split long text into chunks
  In->>Mo: Ask for embeddings of each chunk
  Mo->>In: Return vectors
  In->>Sh: Store text, vectors and metadata
  Sh->>Sh: Add vectors to the search graph at refresh and merge
  Sh->>App: Document is searchable
        

What this means in practice

Indexing semantic data is slower than indexing plain text. Each chunk needs a model call, and the model may run on a machine learning node that has limited capacity. Bulk indexing at full speed can fill the model’s waiting line and cause rejections. Plan your indexing rate around the model throughput, not only around disk and network.

The search structures for dense vectors are also built during segment creation and merging, which uses extra processor time. A big initial load can keep merges busy for a long while after the last document arrives.

Now the reverse: a user types a question and expects the right documents back in a fraction of a second.

sequenceDiagram
  participant U as User
  participant Co as Coordinating Node
  participant Mo as Model
  participant Sh as Each Shard
  U->>Co: Question in plain text
  Co->>Mo: Embed the question
  Mo->>Co: Question vector
  Co->>Sh: Find nearest vectors, filtered if needed
  Sh->>Co: Top candidates per shard
  Co->>Co: Merge candidates from all shards
  Co->>U: Ranked results
        

The extra step compared with keyword search is the model call for the question. It is one short text, so it is fast, but it still adds time, often tens of milliseconds for a local model and more for a remote service that is reached over the internet. It also creates a dependency: if the model service is down, semantic queries fail even though the index is healthy.

There are two ways to find the nearest vectors. One is perfect but slow, and the other is nearly perfect but fast.

Exact search, also called brute force, compares the question vector with every stored vector and keeps the best. It always finds the true nearest neighbors. But its cost grows with the number of vectors, so it is only practical for small sets, or when a filter has already reduced the candidates to a small number.

Approximate nearest neighbor search, shortened to ANN, uses a clever structure to visit only a small part of the vectors. It may miss a true neighbor once in a while, but it is much faster, even for millions of vectors. In Elasticsearch, ANN search uses a graph structure called HNSW, which comes from the Lucene engine.

HNSW is the engine of fast vector search. Its idea is simple and beautiful.

HNSW stands for Hierarchical Navigable Small World. Every vector becomes a node in a graph, and each node is connected to a handful of its close neighbors. The graph has several layers. The top layer has only a few nodes with long-distance links, like highways. The lower layers have more nodes with shorter links, like local roads. The bottom layer contains every vector.

To search, the engine starts at the top layer and moves greedily toward the node closest to the question, following links. When it cannot get closer, it drops one layer down and continues from there. At the bottom, it explores the neighborhood around the best node and collects the closest candidates. The search touches only a tiny fraction of all vectors.

flowchart TD
  T["Top layer: few nodes, long jumps"] --> M["Middle layer: more nodes, medium jumps"]
  M --> B["Bottom layer: all vectors, short links"]
  B --> R["Collect the closest candidates"]
  Q["Question vector"] --> T
        

Graph settings that matter

Two numbers control most of what happens in an approximate vector search.

Real searches are rarely pure meaning. Users also say things like “only in stock” or “from this year”.

Most applications need to combine semantic closeness with hard conditions: the product must be available, the document must belong to this customer, the date must be recent, the language must match. These conditions are called filters. In vector search, filtering has a special difficulty. If you first take the nearest vectors and then apply the filter, you may throw away most of them and be left with too few results.

Elasticsearch solves this by applying the filter while walking the graph, so that the search keeps looking until it has found enough vectors that pass the filter. Because of this, you can rely on receiving the requested number of results, as long as enough documents qualify.

flowchart LR
  A["Nearest first, filter after"] --> B["Few or no results left: bad"]
  C["Filter during graph search"] --> D["Enough matching results: good"]
        

Because graphs live inside segments, the physical layout of your index changes the speed of vector search.

A shard is made of many segments, and each segment holds its own HNSW graph. A vector search must query the graph of every segment, then merge the results. If a shard has forty small segments, forty graphs are searched. If merges have combined them into five larger segments, only five graphs are searched, and each search is faster and often more accurate.

Also, a search runs on every shard of the index and merges the per-shard results. Too many shards therefore multiply the work, just as they do for other searches.

Semantic search is powerful but not perfect. Keyword search is limited but reliable. Together they are better than either alone.

Semantic search shines with long, natural questions and with paraphrases. But it can stumble on exact things: a product code such as “X200-B”, a person’s rare name, an error number, or a specific legal clause number. The embedding may blur exact strings into a general idea, and the results become vague. Keyword search handles such exact strings perfectly.

Keyword search, on the other hand, fails when users use different words than documents. So each method covers the weakness of the other. Hybrid search runs both and merges their results into one list. Studies and real-world experience repeatedly show that hybrid ranking beats either method alone on many kinds of content.

Merging two ranked lists sounds easy, but their scores are not comparable. Rank fusion solves this problem.

Keyword search returns BM25 scores that might range from zero to thirty. A vector search returns similarity scores that are usually between zero and one. Adding them directly makes no sense, since one scale would dominate. One approach is to normalize each score and combine them with weights. Another, very popular approach, is Reciprocal Rank Fusion, or RRF.

RRF ignores the scores and looks only at ranks. Each document gets a small number of points from each list, based on its position: high positions give more points, lower positions fewer, using a simple formula where the points are one divided by a constant plus the rank. The points from all lists are added, and the documents are sorted by their total. A document that appears near the top of both lists wins clearly.

flowchart TD
  Q["User question"] --> K["Keyword search: ranked list A"]
  Q --> V["Vector search: ranked list B"]
  K --> F["Reciprocal rank fusion: add points by rank"]
  V --> F
  F --> R["One combined ranked list"]
        

Fast retrieval finds a good candidate set. A stronger, slower model can then reorder the best few with more care.

The first stage of search must be very fast, because it looks at millions of documents. It uses simple comparisons of vectors or terms. The second stage, reranking, takes only the top few dozen or hundred results and examines each together with the question, using a more powerful model that reads both texts side by side. This kind of model, often called a cross-encoder, understands the relationship between the question and each passage much better than comparing two separate vectors.

flowchart LR
  A["Millions of documents"] -->|"fast retrieval: keyword, vector, or hybrid"| B["Top 100 candidates"]
  B -->|"slower, smarter reranking model"| C["Top 10 reordered results"]
        

Retrievers let you describe a whole search pipeline as a tree of simple steps.

A retriever is a building block that produces a ranked list of documents. Some retrievers do the actual searching, such as the standard retriever for keyword queries and the kNN retriever for vector search. Others take lists from child retrievers and transform them, such as the rank fusion retriever, which merges lists, and the reranking retriever, which reorders the top results using a model.

Because retrievers nest, you can express a complete modern search in one readable structure: get candidates by keyword and by meaning, fuse them, and rerank the top. This is easier to read and maintain than hand-mixing many separate clauses and boosts.

flowchart TD
  RR["Reranking retriever"] --> FU["Rank fusion retriever"]
  FU --> ST["Standard retriever: keyword query"]
  FU --> KN["kNN retriever: vector query"]
        

Many teams adopt semantic search because they want a chat assistant that answers from their own documents.

Large language models write fluent answers, but they do not know your private documents, and they can invent facts. Retrieval-augmented generation, or RAG, fixes this. First, a retrieval step finds the passages of your own content that are most relevant to the user’s question. Then those passages are placed in the prompt of a language model, which writes an answer using only that material and can cite its sources.

Elasticsearch is a natural retrieval engine for this job. It stores your chunks, finds them by meaning and by keywords, filters them by permissions, and returns the best passages. The quality of the final answer depends heavily on the quality of the retrieval, which makes everything in this guide directly relevant.

sequenceDiagram
  participant U as User
  participant A as Application
  participant E as Elasticsearch
  participant L as Language model
  U->>A: Ask a question
  A->>E: Hybrid search with permission filters
  E->>A: Best passages
  A->>L: Question plus passages as context
  L->>A: Answer with citations
  A->>U: Show answer and sources
        

Vectors are big. Quantization compresses them so more data fits in fast memory.

A vector of 768 numbers stored as 32-bit floats takes about three kilobytes. For ten million chunks, that is around thirty gigabytes for the raw vectors alone, before counting the graph. Fast search needs the vectors and the graph to sit in the operating system page cache, so memory is often the deciding cost of a semantic search system.

Quantization reduces the precision of each number. Instead of a 32-bit float, each number is stored in eight bits, four bits, or even a single bit. Accuracy drops only a little, because the original full-precision vectors can still be used to rescore the best candidates, while the memory needed for the main search falls dramatically.

Semantic search adds new costs. Estimate them early so that the project does not surprise you.

Semantic search touches several node roles. Giving each its own resources keeps the cluster healthy.

The most important separation is between model work and shard work. A busy model can consume processors that data nodes need for searches, and heavy vector merges can slow down model responses. Putting models on their own machine learning nodes prevents them from fighting. Trained model deployments also let you control how many allocations, which are copies of the model, and how many threads per allocation, so you can scale throughput or latency according to your needs.

With so many options, a simple decision path helps you pick a starting point.

flowchart TD
  S["Start: need better search"] --> A{"Mostly English text and want quick results?"}
  A -->|"Yes"| B["semantic_text with ELSER"]
  A -->|"No: other languages or special domain"| C{"Want simplicity or full control?"}
  C -->|"Simplicity"| D["semantic_text with a multilingual dense endpoint"]
  C -->|"Full control"| E["dense_vector with your chosen model and settings"]
  B --> H["Add hybrid search with keyword"]
  D --> H
  E --> H
  H --> R{"Need the best precision at the top?"}
  R -->|"Yes"| RR["Add a reranker"]
  R -->|"No"| DONE["Evaluate and tune"]
  RR --> DONE
        

You cannot improve what you cannot measure. Relevance needs numbers as much as speed does.

To evaluate search, you need a test set: a list of real questions, each with the documents that a human judged to be good answers. Even fifty to a hundred well-chosen questions are a great start. You then run each search method on the same questions and compute measures that reflect how high the good documents appear.

Elasticsearch offers a ranking evaluation operation that runs a set of queries against judged documents and reports these metrics, so you can compare keyword, semantic, and hybrid setups on equal terms. Also measure the recall of the approximate vector search against exact search, since that tells you whether your candidate settings are too aggressive.

When quality or speed is not good enough, work through these levers in order, from cheapest to most costly.

Embeddings are derived from your data, so they deserve the same protection as the data itself.

Controls

Use role based access control and document or field level security so that queries only see allowed data. Keep the model inside your cluster when the data is sensitive. Encrypt traffic to and from model services. Limit who can create inference endpoints and deploy models, because those actions can send data outside or consume large resources. Audit access to sensitive indices.

A semantic system has two health stories: the search cluster and the model. Watch both.

Business level signals

Also watch how users behave. Track the share of searches that end with a click, searches that return nothing, and searches that users immediately rephrase. A rise in rephrasing often shows that results have become worse. Keep a sample of queries and results for periodic human review.

Most problems come from a small set of causes. This table maps symptoms to first checks.

These habits show up in most successful semantic search projects.

These mistakes appear again and again in real projects.

A checklist to keep beside you when building semantic search.

Semantic search is a strong tool, but it is not free.

The trade is usually worth it where users ask questions in their own words and documents are written in many styles. It is less compelling for tiny catalogs with clean structured data, where good filters and keyword search already work well.

These stories show semantic search at work in familiar settings.

You do not need to rebuild everything. A step-by-step path keeps risk low.

Some beliefs about semantic search cause bad decisions. Let us test them.

Keep these plain-language definitions handy. Shared vocabulary makes design discussions shorter and clearer.

When a teammate says “our recall is low”, they mean the approximate search is missing true neighbors, and the candidate number may need to grow. When they say “we need to chunk”, they mean long documents are being cut off by the model. When they say “let us go hybrid”, they mean keyword and semantic results should be merged. Knowing these phrases turns long meetings into short ones.

It is search based on meaning. Text is turned into vectors by a model, and Elasticsearch finds the documents whose vectors are closest to the vector of the question, even if they share no words.

An embedding is a list of numbers produced by a model that represents the meaning of a text. Texts with similar meanings get similar lists of numbers.

Dense vectors are fixed-length lists where nearly all values are used and are searched by nearest neighbor search. Sparse vectors are long lists with mostly zeros, where each position stands for a word, and they are searched with inverted index techniques.

ELSER is Elastic’s learned sparse model for English text. It expands text into weighted related terms and works well out of the box without tuning.

It is a field type that automates chunking, embedding, and storage. You index plain text, and search with a semantic query using plain text, without building pipelines yourself.

Yes, in most cases. Keyword search is excellent for exact codes, names, and short queries. Hybrid search that combines both usually gives the best results.

Hybrid search runs keyword and vector searches and merges the results. Reciprocal rank fusion is a popular way to merge by rank position, which avoids comparing scores that are on different scales.

The value k is how many nearest neighbors you want returned overall. The value num_candidates is how many candidates each shard examines. More candidates give better recall but slower searches.

Models read only a limited amount of text, and one vector cannot represent many topics well. Chunking makes every part of a document searchable and improves accuracy.

Yes, but you must re-embed all your data into a new index, since vectors from different models cannot be compared. Use versioned indices and aliases to make the switch safe.

Enough for the vectors and graphs of the data you search often to sit in the operating system page cache. Estimate chunks times dimensions times bytes per number, adjust for compression, and add the graph overhead.

Only if you choose an outside embedding provider. Models that run inside your cluster keep the text within it. Check your provider terms for anything sensitive.

Reranking reorders the top results using a stronger model that reads the question and each passage together. Use it when top precision matters, such as question answering, and you can afford the extra delay.

Build a judged set of real questions, compute metrics such as precision, recall, and NDCG for each approach, and confirm with an experiment on real users.

Retrieval-augmented generation uses search to find relevant passages, and then a language model writes an answer from them. Elasticsearch supplies the retrieval, so good semantic and hybrid search directly improves answer quality.

Semantic queries and semantic indexing fail or slow down. Plan a fallback to keyword search, and monitor the model endpoints and their queues.

Semantic search in Elasticsearch finds documents by meaning. A model turns text into vectors, either dense lists of numbers searched through a fast graph, or sparse weighted terms searched through the inverted index. The semantic text field makes the setup easy, the inference API connects many kinds of models, hybrid search with rank fusion combines meaning with exact words, and reranking polishes the top of the list. Around all of this sit the practical concerns of chunking, memory, model capacity, security, evaluation, and monitoring. Like the librarian in our opening story, a good search system understands what you mean, but still checks the shelf labels when you ask for a book by its exact title.