The shortest possible answer: it is a set of log files where Elasticsearch records searches and document writes that were slower than a limit you set.

Let us build the definition slowly. Elasticsearch is a search and analytics engine. Companies use it to search products, explore application logs, count events, and draw charts. You send it a question, called a search request, or you send it new data, called an indexing request. Elasticsearch stores data in an index, which is like a big labeled cabinet of documents, and it slices each index into pieces called shards so the work can be shared across many servers.

A log is a diary kept by a program. Each line says what happened and when. Elasticsearch already writes a general diary about its own health, but that diary does not say which particular search was slow. The slow log fills that gap. It is a special pair of diaries, one for searches and one for indexing, that only records events that crossed a threshold. A threshold is simply a time limit, such as “five seconds.”

Two logs, not one

Elasticsearch offers two separate slow logs. The search slow log records slow searches. The indexing slow log records slow document writes. They have their own settings, their own files, and their own thresholds, because searching and writing are very different kinds of work. You can switch on one, both, or neither.

Off until you turn it on

By default, the slow log is disabled. Every threshold starts at a special value that means “never log.” This is on purpose. A slow log with no limits would log every operation, which would fill your disks and slow down the cluster. You choose the limits, and you can change them at any time without restarting anything.

What the slow log is not

The slow log is not a live monitor. It writes a line only when an operation has finished, so it cannot tell you about a search that is stuck right now. It is not a profiler, so it does not explain why something was slow. It is not an alarm, so it does not send messages. It is a factual record: which index, which shard, how long, and what was asked. Everything else, such as explaining and alerting, is built on top of that record.

Where you will meet it

You will meet the slow log when a dashboard becomes sluggish and someone asks “which query is doing this?”, when a nightly data load takes twice as long as last week, when a cluster costs too much and you want to find the heaviest queries, and when a release changes the shape of your search requests and you want proof of the effect. In each case, the slow log turns a feeling into evidence.

A cluster can be busy and still look healthy on the surface. The slow log exists because averages hide the few requests that hurt the most.

Most monitoring shows averages: the average search time, the average indexing rate, the average CPU use. Averages are useful, but they smooth over trouble. If ninety-nine searches finish in 20 milliseconds and one takes 30 seconds, the average looks fine. Yet the user who waited 30 seconds sees a broken product, and that single search may have consumed as much CPU as the other ninety-nine combined.

The needle in a haystack

A busy cluster handles thousands of requests every second. You cannot read them all, and you should not want to. What you need is a filter that says: show me only the slow ones, and give me enough detail to recognize them. That is the job of the slow log.

Slowness has many possible owners

When Elasticsearch feels slow, the cause could be almost anything: one badly written query, a growing index, too many shards, a burst of writes, a garbage collection pause, a busy disk, or a noisy neighbor on shared hardware. Guessing is expensive. A slow log line points at facts that narrow the list quickly: which index was involved, how many shards, which phase was slow, and what the request looked like.

An illustrative story

Writes can be slow too

Searches get most of the attention, but slow writes hurt as well. If documents take too long to index, data arrives late, queues grow, and dashboards show yesterday’s numbers. The indexing slow log catches writes that were slower than expected, often exposing oversized documents, huge numbers of fields, or a disk that is struggling.

A short history

The slow log has been part of Elasticsearch for many years and has grown gradually. Early versions wrote plain text lines. Modern versions write structured JSON lines that are easier for machines to read, and newer releases can add details such as which user sent the request. The core idea has not changed: a per-index, threshold-based diary of slow work.

To read the slow log with confidence you need seven simple ideas: node, index, shard, search phases, indexing operation, threshold, and log level.

Primary shards and replica shards

Every shard has one primary copy, which receives new writes first, and it can have replica copies on other nodes. Replicas hold the same data and can answer searches. This matters for the slow log because each shard copy writes to the log of the node it lives on. The evidence for one slow search may be spread across several nodes, and you need to collect logs from all of them to see the whole picture.

Query phase and fetch phase

Elasticsearch normally answers a search in two steps. In the query phase, each shard figures out which documents match, counts them, ranks them, and computes any aggregations. Aggregations are Elasticsearch’s way of grouping and counting data, such as “orders per country.” The shard returns a small package of results. In the fetch phase, the coordinating node asks the shards for the full contents of the winning documents. The search slow log measures these two phases separately, so you can tell whether the finding was slow or the collecting was slow.

Who does what during a search

A search request first lands on a coordinating node. That node forwards the request to one copy of each relevant shard, waits for the partial answers, merges them, and replies to the client. In most production setups the client talks to the cluster through a load balancer or an API gateway.

flowchart LR
    A["Client application"] --> B["Load balancer or API gateway"]
    B --> C["Coordinating node"]
    C --> D1["Shard 1 on Data Node A"]
    C --> D2["Shard 2 on Data Node B"]
    C --> D3["Shard 3 on Data Node C"]
    D1 --> E["Coordinating node merges partial results"]
    D2 --> E
    D3 --> E
    E --> F["Final response to client"]
    D1 -.-> L1["Slow log entry on Node A"]
    D2 -.-> L2["Slow log entry on Node B"]
    D3 -.-> L3["Slow log entry on Node C"]

Look closely at Figure 1. The dotted lines show that the slow log entries are written next to the shards, not at the load balancer or the coordinating node. Two consequences follow. First, one slow search can produce several log lines, one per slow shard. Second, the time you see in a line is the shard’s own working time, not the total time the client waited.

Indexing an operation

When you add or change a document, Elasticsearch sends the write to the right primary shard, which processes it and then forwards it to the replicas. Each shard copy that processes the write does its own work and can write its own slow log line. Applications often send many documents in one bulk request to save network trips. The indexing slow log still measures each document on its own, so one bulk can produce many lines or none at all.

Elasticsearch offers several ways to look at performance. Each answers a different question, and using the right one saves hours.

Beginners often ask why they need a slow log when there are already statistics, monitoring dashboards, and a profiler. The answer is that each tool shows a different angle. Let us meet them one by one.

Node and index statistics

The statistics operations return counters such as the total number of searches, the total time spent searching, and the number of indexing operations. They are great for trends and averages. They cannot tell you which specific request was slow, because counters only add things up.

The profile option

A search request can ask Elasticsearch to profile itself. The response then includes a detailed breakdown of how much time each part of the query took inside each shard. Profiling is superb for understanding one query, but it only works on a request you already know about and choose to profile. It also adds some overhead, so it is not something you leave on for all traffic.

The task management operations

The tasks operations list requests that are running right now, and they can cancel some of them. They are the right tool for a stuck search. They cannot show you a search that ended an hour ago.

The hot threads operation

Hot threads takes a quick snapshot of what the busiest threads inside a node are doing at this moment. It is helpful during a live slowdown to see what the CPU is busy with, but it too is a snapshot, not a history.

Side-by-side comparison

flowchart TD
    A["Users report slowness"] --> B["Monitoring dashboards: is the whole cluster stressed?"]
    B --> C["Slow log: which requests were slow?"]
    C --> D["Profile option: why was that request slow?"]
    D --> E["Fix the query, mapping, or settings"]
    E --> F["Slow log and dashboards: confirm the improvement"]
    B --> G["Tasks and hot threads: what is happening right now?"]

How they work together

These tools are teammates, not rivals. A healthy workflow starts wide and narrows down. Dashboards say that latency rose at three o’clock. The slow log says that a certain index and a certain query shape caused most of the slow entries. The profile option, run on a copy of that query, shows that one clause dominates the time. After the fix, the slow log goes quiet, and the dashboards confirm it.

The number in a slow log line is not the time your user waited. It is the time one shard spent on one piece of work. Knowing the difference prevents many wrong conclusions.

The search slow log

The search slow log watches the two phases of a search, one shard at a time. For the query phase, it measures how long the shard took to find, rank, and aggregate its matches. For the fetch phase, it measures how long the shard took to load the documents that were chosen. Each phase has its own set of limits, because they usually run at very different speeds. A query phase of a few hundred milliseconds may be normal, while a fetch phase of that length would be alarming.

Because the measurement is per shard, a search against an index with ten shards is judged ten separate times. Two of the ten shards may cross the limit and write lines, while eight stay quiet. The slow lines you see are the slowest pieces of the search, not necessarily the whole search.

The indexing slow log

The indexing slow log watches individual document write operations on a shard. When a shard finishes processing a document and the time it took is over the limit, a line is written. The measured time covers the shard’s own work: reading the document, applying the field rules, and adding it to the index structures. It is measured for each document separately, even when the document arrived as part of a large bulk request. Deletions are not the focus of this log; it is about writing documents.

What is counted and what is not

sequenceDiagram
    participant C as Client
    participant N as Coordinating node
    participant S as Data node shard
    participant L as Slow log file
    C->>N: Search request
    N->>S: Run the query phase
    S->>S: Find, rank, and aggregate
    S->>L: Write a line if the time is over a query limit
    S-->>N: Partial result
    N->>S: Run the fetch phase for chosen documents
    S->>S: Load documents
    S->>L: Write a line if the time is over a fetch limit
    S-->>N: Documents
    N-->>C: Merged response

Why this matters in practice

If users complain about slowness but the slow log is empty, do not conclude that nothing is wrong. The delay may be in the queue, the network, the coordinating node, the client application, or an ingest pipeline. Thread pool queues and rejected requests, which are visible in statistics and monitoring, are the usual next place to look. The slow log is powerful but narrow, and treating it as the whole truth is a classic mistake.

Every slow log line comes from a simple comparison: the measured time against a set of limits. Understanding the rule makes the settings easy to plan.

For each kind of measurement, you can set up to four thresholds, one for each log level: warn, info, debug, and trace. A log level is a label that says how serious or how detailed a line is. Usually, warn is the most serious and trace is the most detailed. In the slow log, the four levels act as four tiers of slowness.

The rule

When an operation finishes, Elasticsearch compares its time against the thresholds, starting from the most serious level. If the time is greater than or equal to the warn threshold, the line is written at the warn level. If not, it checks the info threshold, then debug, then trace. The first one that matches decides the level of the line, and only one line is written per operation. If the time is below every threshold, nothing is written.

That means you should usually set the warn threshold as the largest number and trace as the smallest. For example, warn at ten seconds, info at five, debug at two, and trace at half a second. A search that takes seven seconds gets an info line. One that takes twelve seconds gets a warn line. One that takes one second gets nothing.

flowchart TD
    A["Operation finishes with a measured time"] --> B{"Time at or above the warn limit?"}
    B -->|Yes| W["Write the line at warn level"]
    B -->|No| C{"Time at or above the info limit?"}
    C -->|Yes| I["Write the line at info level"]
    C -->|No| D{"Time at or above the debug limit?"}
    D -->|Yes| G["Write the line at debug level"]
    D -->|No| E{"Time at or above the trace limit?"}
    E -->|Yes| T["Write the line at trace level"]
    E -->|No| N["Write nothing"]

Special values

• Minus one switches a threshold off. This is the default for all of them, which is why a fresh cluster writes no slow log lines.
• Zero milliseconds means “log everything at this level.” This is useful for a short experiment, because it captures every operation. It is dangerous to leave on, because the volume can be enormous.
• Time units such as 500ms, 2s, and 1m are all accepted, so you can write limits the way you speak.

Why four tiers?

The tiers let one setup serve many purposes. The warn tier can trigger alerts, because anything at that level is really bad. The info tier is a daily review list. The debug and trace tiers are gentler limits for hunting slightly slow operations when you have time or when the cluster is quiet. Later, when you ship logs to a central place, you can filter by level to keep the noisy tiers out of your alerting.

Thresholds apply to time, not to importance

A slow log line tells you that something was slow, not that it was important. A monthly finance report that legitimately takes forty seconds will appear in the log every month, and it is not a problem. Good teams learn the usual suspects and mark them as expected, so their attention goes to the surprises.

Sample values from the official documentation

The Elasticsearch documentation shows a set of example limits that many teams use as a starting point. For the query phase, warn at ten seconds, info at five seconds, debug at two seconds, and trace at half a second. For the fetch phase, warn at one second, info at eight hundred milliseconds, debug at five hundred milliseconds, and trace at two hundred milliseconds. For indexing, the same shape as the query limits is used. These are examples, not rules. Your own limits should come from your own expectations.

The slow log is controlled by index settings, so you can tune it per index and change it on a live cluster without a restart.

Index settings are dynamic

All of these are dynamic settings, which means you can change them while the index is in use. You change them through the update index settings operation, and the new values take effect right away. To go back to the default, you set the value to null, or to minus one for a threshold you want disabled.

Choosing which indices to cover

Settings belong to individual indices, but you can apply them to many at once by using an index name pattern with a wildcard. For a family of daily indices, you can set limits once for all current ones. For future ones, the limits must live in an index template, which is a saved recipe that Elasticsearch applies to each new matching index when it is created. A template does not change indices that already exist, so a complete setup needs both: an update for current indices and a template for new ones.

The indexing source settings

An indexing slow log line can include a copy of the document that was slow. This helps you recognize the culprit, but documents can be big and can contain private information. The source setting lets you choose. By default, only the first thousand characters are included. You can turn it on fully, cut it shorter, or switch it off entirely with false or zero. The reformat setting, which is on by default, squeezes the copy onto a single line so that one log entry is one text line. Turning it off keeps the original layout, which may spread one entry across several lines and complicate log processing.

Including who asked

Newer versions can add information about the authenticated user to slow log lines through an include-user setting on each of the two logs. It is off by default. When it is on, you can tell which user or key sent a slow request, which is very helpful in shared clusters where many teams send traffic.

Log files and the logging configuration

Where the lines end up is decided by the node’s logging configuration, not by index settings. Elasticsearch uses a logging library named log4j2, and its configuration file defines two named loggers for the slow logs: one for searches and one for indexing. Each is attached to its own rolling file, which means the file is closed and a new one is started when it becomes too large or too old. Administrators can change these details, for example to write into a different folder or keep more history, by editing the logging configuration on every node and restarting.

Each line packs many facts into a small space. Knowing what each field means lets you read a line like a sentence.

Modern Elasticsearch writes slow log lines as JSON. JSON is a simple way to write labeled information, where every fact has a name and a value, so machines can read it easily. A line also has the usual log fields, such as the time it was written, the level, the name of the node, and the name of the cluster. On top of these, the slow log adds fields that begin with elasticsearch.slowlog.

Fields in a search slow log line

Fields in an indexing slow log line

Reading a line step by step

Slow log lines are written on the node where the shard lives. In a real cluster you need a plan to gather them into one place.

On each node, Elasticsearch has a logs folder whose location is set by its configuration. Inside, you will find the normal cluster log and, once slow logging produces something, the slow log files: one for searches and one for indexing, named after the cluster. Modern versions write them as JSON files, while older versions also wrote plain text versions. Files are rotated, meaning that when they get large or old, they are closed, renamed, and possibly compressed, and a new file begins.

The problem of many nodes

Because each shard copy writes to its own node’s file, the evidence for a single slow search may be spread across several machines. Logging into each node and searching files by hand does not scale. A cluster with thirty data nodes would need thirty separate visits. The answer is to ship the logs to a central place.

flowchart LR
    A["Data node 1: slow log files"] --> D["Log shipper such as Filebeat or Elastic Agent"]
    B["Data node 2: slow log files"] --> D
    C["Data node 3: slow log files"] --> D
    D --> E["Monitoring cluster or log store"]
    E --> F["Dashboards and searches in Kibana"]
    E --> G["Alert rules"]

Common ways to collect

• Filebeat with its Elasticsearch module. Filebeat is a light program that reads log files and sends the lines onward. Its Elasticsearch module knows about the slow log files and parses the fields for you.
• Elastic Agent with the Elasticsearch integration. The newer, unified collector offers the same idea with central management.
• Managed cloud logging. On managed Elasticsearch offerings, slow logs can often be delivered through the provider’s log features once they are enabled in the settings. The exact steps depend on the provider.
• Any general log platform. Because the files are structured JSON, most log platforms can ingest them with a small amount of setup.

Keep the monitoring separate

Many teams send slow logs to a separate monitoring cluster instead of the production cluster being watched. If the production cluster is struggling, you still want to read the evidence. A separate destination also avoids the odd loop in which the act of storing slow logs creates more load on the very cluster that is slow.

Retention

Decide how long to keep the lines. A few weeks is common for detailed review. Some teams keep summaries longer, such as counts per query shape per day, because trends over months help with capacity planning. Retention should match your needs, and privacy rules may set an upper limit, as discussed in Chapter 17.

Reading logs without a plan is tiring. This simple loop turns raw lines into fixes and proves that the fixes worked.

flowchart TD
    A["Set thresholds"] --> B["Collect lines for a full business cycle"]
    B --> C["Group by index, query shape, time, and caller"]
    C --> D["Rank by total time spent"]
    D --> E["Reproduce with the profile option"]
    E --> F["Fix the query, mapping, or layout"]
    F --> G["Compare slow log volume before and after"]
    G --> C

Rank by total cost, not by worst case

New readers of the slow log often chase the single slowest line. That feels natural, but it can mislead. A query that took 40 seconds once might have hit a garbage collection pause. A query that took 1.2 seconds but appeared 50,000 times today is the real drain. The best measure is usually frequency multiplied by time, which tells you how much of the cluster’s day each shape consumed.

Capture a baseline first

Before making changes, save a copy of the counts and the average times of the top groups. Without a baseline, you cannot show that a fix worked, and you cannot notice that a later release made things worse.

After a few weeks of reading slow logs, the same handful of culprits appear again and again. Here are the usual suspects and how to recognize them.

Filters versus scoring

Many searches mix two ideas: matching documents, and ranking them by relevance. When you only need a yes-or-no test, such as “status is published,” it is cheaper to place that test in a filter, because filters skip the scoring step and can be reused from a cache. If the slow log shows large queries where most conditions are yes-or-no tests inside scored parts, moving them into filters is often an easy win.

The chatty client

Sometimes no single query is slow, but a client sends the same medium-slow query many times per second, perhaps because a page refreshes too often. The slow log shows this as a long list of near-identical lines. The fix lives in the application: fewer refreshes, shared results, or a cache in front of the cluster.

The hot shard

If most slow lines mention the same shard or the same node, then the problem may be that shard’s data, its node’s hardware, or its uneven share of the traffic. Compare it with its siblings. A shard that is much larger than its peers, or a node with a slower disk, often explains the whole pattern.

Writes are slowed by a different set of problems. The indexing slow log helps you separate a bad document from a bad disk.

Do not forget what is not measured

If your data passes through an ingest pipeline before it is stored, the time spent there is not part of the indexing slow log. A team can spend days tuning the shard layout while the real delay sits in a pipeline processor. Check the pipeline statistics as a separate step. Similarly, time spent by your own application preparing documents or waiting on the network is invisible here.

Bulk sizes and concurrency

The way clients send data also matters. Very small bulk requests waste trips. Very large ones hold memory and can cause pauses and rejections. There is no universal best size. Teams find it by measuring: try a range, watch throughput and slow lines, and choose a value on the plateau. Sending from too many parallel clients can overload the write queues, so control concurrency as well.

The slow log itself costs very little when its limits are sensible. It becomes costly only when the limits are so low that it writes far too much.

The cost of checking

Every search phase and every document write already measures its own duration for normal statistics. The slow log adds a comparison against a few numbers, which is a trivial amount of work. The real cost appears only when a line is written: building the text, copying the query or document source, and writing to disk. A cluster that writes a handful of lines per minute will never notice. A cluster that writes thousands of lines per second may spend measurable CPU and disk time on logging alone.

How volume grows

The volume of slow log data depends on three things: how many operations you run, how many of them cross the limits, and how big each line is. Line size is often dominated by the source. A search body with a long list of filters, or a document with big text fields, can make each line several kilobytes. Multiply by the number of lines per day, and you can estimate disk use easily. Chapter 19 shows a worked example.

Shards multiply the lines

Because entries are written per shard, an index with more shards can produce more lines for the same search. A slow search across 40 shards, where 25 of them cross the limit, writes 25 lines. This is a good reason to keep shard counts reasonable, and also a reason to choose limits with the shard count in mind. A limit that gives a tidy log on a small index can flood the log on a large one.

Protecting the disks

Log rotation limits the size of individual files and, when configured, the total history. Still, a sudden flood can fill a disk faster than rotation can react. Make sure the log folder has enough space, set alerts on disk usage, and consider placing logs on a different volume than your index data so that a log flood cannot starve the shards.

Scaling with the cluster

As the cluster grows, the slow log scales naturally, because each node writes only about its own shards. There is no central bottleneck in Elasticsearch itself. The pressure moves to the collection pipeline and the storage where you keep the lines. Plan those to handle the busiest day, not the average.

No tool is perfect. Here is an honest scorecard so you can decide when to lean on the slow log and when to look elsewhere.

The central trade-off: detail against noise

Lower limits capture more and give finer detail, but they also produce more lines and more cost. Higher limits give a quiet log that shows only the disasters. The best setting is the one where the log is short enough to read regularly but sensitive enough to reveal trends. That balance moves over time as your cluster and your goals change.

The second trade-off: insight against privacy

Including the full query or document source makes lines easy to understand, but it also copies possibly sensitive data into log files. Shortening or removing the source protects privacy but makes each line harder to interpret. Many teams keep the source for searches, which usually hold filters rather than personal data, and cut or remove it for indexing.

The third trade-off: local files against central collection

Leaving the files on each node is simple, but hard to use. Shipping them centrally is more work to build, but far more powerful. The added effort pays back the first time you need to answer “what was slow last Tuesday across the whole cluster?”

When the slow log is the wrong tool

If the problem is a request stuck right now, use the tasks operations. If the problem is one query you already know about, use the profile option. If the problem is queueing and rejections, look at thread pool statistics. If the problem is that the coordinating node is overloaded merging results, look at that node’s CPU and memory. The slow log is best for the question, “over the last days, which shard-level work was slower than expected?”

Good slow logging rarely happens by accident. These six habits make the log easy to collect, easy to read, and easy to act on.

Pattern 2 in action

Suppose a company has three families of indices: customer-facing product search, internal analytics, and audit archives. The customer-facing family gets tight limits, with warn at one second and info at 500 milliseconds. The analytics family gets warn at ten seconds. The archive family gets warn at sixty seconds, because slow queries there are normal. Each family lives in its own template. The result is a log whose lines almost always deserve a look.

Pattern 4 in action

A team ships slow logs from all nodes to a small monitoring cluster. They build a dashboard with four charts: lines per hour by level, top indices by slow lines, top query shapes by total time, and top callers by opaque id. On Monday mornings, the on-call engineer opens the dashboard and knows within five minutes what deserves attention this week.

A pattern for a healthy culture

Make slow log findings visible and blameless. When the dashboard shows that a new feature made search slower, share the finding as information for improvement, not as a complaint. Teams that see the slow log as a friendly mirror fix problems faster than teams that see it as a source of blame.

These are the traps that make the slow log useless, noisy, or risky. Each one is common in real projects and easy to avoid once you know it.

Other frequent mistakes

• Choosing limits once and never revisiting them as the data and traffic grow.
• Chasing the single slowest line instead of the biggest total cost.
• Fixing the query when the real delay was in an ingest pipeline or the network.
• Forgetting to reset a temporary low threshold after an incident.
• Leaving out the fetch phase limits and missing slowness caused by large documents.

The slow log copies your data into new places. Treat those copies with the same care as the data itself.

A search slow log line contains the body of the search, which may include names, email addresses, account numbers, or other values that users typed into search boxes. An indexing slow log line can contain the beginning of the document, which may hold even more. Because these lines are written into ordinary files and then shipped to other systems, they can quietly become a second copy of sensitive data that lives outside your usual protections.

Practical protections

• Limit the source. Use the source setting to log fewer characters or none for indexing. For searches, review whether queries can carry personal values, and consider designing the application so that identifiers, not raw personal data, appear in queries.
• Restrict access. Only people who need the logs should read them. Apply file permissions on the nodes and role-based access in the central store.
• Protect the shipping path. Send logs over encrypted connections and authenticate the collectors.
• Set retention. Delete old lines after the review period ends. Shorter retention means smaller exposure.
• Mind the user fields. Newer versions can record which user made a request. This is very useful for accountability, but it is also personal data in some rules, so decide on purpose whether to turn it on.
• Watch for secrets. If queries or documents can carry tokens or passwords, they can land in the log. Find and remove such practices at the source.

Operational security

Restrict who may change slow log settings. Lowering thresholds can drive a flood of lines, and raising them can hide evidence. Treat setting changes like any other production change: request, review, apply, and record. Also remember that changing log file locations and rotation involves the node’s configuration files, which should be under version control.

Slow logs as a security signal

Interesting side effect: abnormal patterns in the slow log, such as a burst of very heavy queries from an unusual user, can hint at misuse or an attack that tries to overload the cluster. The slow log is not a security tool, but it can be one more source of clues, especially when the user information is recorded.

Once the lines are collected, turn them into a few clear charts and a small number of useful alerts.

Raw lines are for investigation. For day-to-day awareness, you need summaries. A short list of charts covers most needs, and each one answers a simple question.

Alerting with care

Alerts should be rare and meaningful. A good starting set has three rules. First, alert when the count of warn lines in a ten-minute window is above a normal ceiling. Second, alert when the share of slow operations compared with all operations rises well above its usual level. Third, alert when a new query shape appears with a high count within a short time. Avoid alerting on single lines, because a single slow request is normal and will teach people to ignore the alerts.

Combining signals

The best alerts combine the slow log with other signals. A rise in slow lines together with rising CPU and rising search thread pool queues is a strong sign of overload. A rise in slow lines with flat CPU may point to a specific query or a slow disk. Comparing signals helps you choose the right first action.

Linking to a runbook

Each alert should link to a short runbook, a page that says what the alert means, how to check the top offenders, and who to call. The first time an alert fires at night, that page saves many minutes and much confusion.

You can estimate how much space and traffic the slow log will create before you switch it on, using simple arithmetic.

Three numbers decide the volume: how many lines per day, how large each line is, and how many days you keep them. Multiply them together, and add some headroom for growth and for busy days.

A worked example

Ways to reduce the volume without losing insight

• Raise the lowest tier a little, so that trace and debug lines are less common.
• Shorten or remove the logged source when the query shape is already known from other data.
• Apply low limits only to the indices you care about, and higher limits to bulk archives.
• Store detailed lines for a short time and keep only daily summaries for long-term trends.
• Reduce the shard count where it is unnecessarily high, because each shard multiplies the lines.

The slow log can fail in quiet ways. Knowing them in advance keeps surprises small.

Logs vanish when a node is replaced

If a node is replaced or its disk is wiped, its local log files go with it. Any evidence not yet shipped is lost. Central shipping with a short delay is the protection. Also, in containerized setups, remember to place the log folder on a volume that survives restarts, or ship logs out directly.

Shard moves scatter the evidence

Elasticsearch may move shards between nodes for balance or recovery. A slow shard’s history may therefore be split across the old and the new node. Central collection makes this invisible, because you search by index and shard rather than by machine.

The log disk fills up

A flood of lines can fill the disk that holds the logs. If that disk is shared with data, the node can run into disk watermarks and stop accepting writes. Use a separate volume where possible, watch usage, and keep rotation limits sensible.

Shipping falls behind

If the shipper or the central store cannot keep up, lines queue on the nodes or are dropped. You then see gaps in charts exactly at the busiest times. Monitor the shipper’s own health and delay.

Time zones and clocks

Lines carry timestamps written by each node. If node clocks drift, lines from different nodes will not line up correctly, making a single incident look scattered. Keep clocks synchronized with a time service, and prefer a single time zone, usually UTC, in logs and dashboards.

Settings that quietly did not apply

A typing error in a setting name, a wrong index pattern, or a template that was saved after the index was created can leave an index without limits. Always check the live settings of an index after changing them. A simple habit is to read back the settings and confirm that the values you expect are present.

Design for a quiet log

Finally, remember that an empty log is not always good news. It may mean that thresholds are too high, that settings are missing, or that the delay is outside the shard. Confirm your setup by generating a known slow operation in a test index, and check that it shows up. That one test builds trust in everything else.

Here are four common situations, with rough numbers, showing the slow log at work. The companies are imaginary, but the patterns are the ones teams meet every day.

A worked example with numbers

When you expect lines and see none, work through this checklist from the most common cause to the least common.

The slow log is a simple idea with a big payoff when you actually read it. Here is everything to remember.

Keep in mind that details can change between Elasticsearch versions, so confirm exact setting names and options against the documentation of the release you run. With that habit, and the ideas in this guide, you can turn vague complaints about slowness into a clear list of causes and a steady stream of fixes.