Audit logging is a dedicated, structured record of security-relevant activity — a separate stream from ordinary application logs, built specifically to answer “who did what” questions.

The one-sentence answer

Elasticsearch is a search and analytics engine that stores data on servers called nodes. Audit logging is a security feature that records, in a structured and searchable format, events such as authentication attempts, authorization decisions, and administrative changes, so that every meaningful security-relevant action taken against the cluster leaves a permanent, reviewable trace.

It is easy to confuse audit logging with the ordinary Elasticsearch server log, since both are, at the surface level, just files full of timestamped lines. The two exist for entirely different reasons. The standard server log tells you whether the software itself is healthy — whether nodes joined the cluster, whether a shard failed to allocate, whether an error occurred while processing a request. The audit log tells you something else entirely: which identity did what, to which resource, and whether it was permitted. One is about the health of the system; the other is about the behavior of the people and applications using it.

What audit logging is not

• It is not a replacement for role-based access control. RBAC decides what is allowed; audit logging simply records what was attempted and what the decision was, regardless of which way that decision went.
• It is not a performance or slow-query log. A separate feature, the slow log, exists specifically to record requests that took an unusually long time to run. Audit logging cares about identity and authorization, not about speed.
• It is not automatically enabled. Unlike some baseline logging, audit logging is a feature you turn on deliberately, because writing a detailed record of every security event carries a real cost in disk space and, if request bodies are included, potentially in captured sensitive data too.

Without a record of what happened, an organization is left guessing during exactly the moments when guessing is least acceptable.

The problem: memory and assumptions are not evidence

Imagine a sensitive customer index was accessed at 2 a.m. by an account nobody remembers authorizing for that kind of access. Without a written record, answering “who did this, and were they allowed to” depends entirely on asking people to recall events from memory, checking whichever application logs happen to still exist, and hoping the pieces line up. Memory is unreliable, application logs were rarely designed with this exact question in mind, and by the time anyone starts asking, the relevant details may already be gone.

Audit logging removes the guesswork. The record already exists, in the exact structured detail needed, the moment anyone thinks to ask the question. This transforms an investigation from “let’s try to reconstruct what might have happened” into “let’s read what did happen.”

Compliance and accountability

Many industries operate under regulations or standards that specifically require a demonstrable record of who accessed sensitive data and when — financial services, healthcare, and any organization handling personal data under privacy regulations are common examples. These frameworks do not merely ask an organization to have good access controls; they ask the organization to prove, after the fact, that those controls were followed, and to identify who accessed what if something ever goes wrong. Audit logging is very often the specific technical mechanism that satisfies this requirement for an Elasticsearch-based system.

flowchart LR
  A["Sensitive index accessed unexpectedly"] --> B{"Audit logging enabled?"}
  B -->|"No"| C["Rely on memory, guesswork, and incomplete application logs"]
  C --> D["Investigation stalls or reaches an uncertain conclusion"]
  B -->|"Yes"| E["Search the audit log for the exact time window"]
  E --> F["Find the user, the action, and the authorization decision"]
  F --> G["Investigation reaches a confident, evidence-based conclusion"]

Detecting problems before they become disasters

Audit logs are not only useful after something has clearly gone wrong. Reviewed proactively, they can reveal patterns worth investigating long before any actual damage occurs: an account that normally logs in from one office suddenly authenticating from an unfamiliar location, a service account that has started attempting actions far outside its usual, narrow role, or a sudden spike in access-denied events against one particular sensitive index. None of these patterns are visible without a record to examine in the first place.

Verifying that access control is actually working

Designing a careful set of roles, as covered in role-based access control, is only half the job. The other half is confirming those roles behave the way they were intended to, in practice, over time. Audit logs are the evidence that a particular role’s restriction is actually being enforced — showing, for instance, that a support team’s role really is being denied access to a sensitive index it was never meant to touch, rather than merely assuming the role definition is correct and hoping nothing has drifted.

A handful of simple ideas make everything that follows easy to understand.

Why “structured” matters more than it sounds

A plain sentence like “user jdoe was denied access to the payroll index at 2:14 a.m.” is readable by a person, but painful to search across millions of similar lines, and nearly impossible to reliably parse if the wording ever varies slightly between entries. A structured log instead records the same event as a set of clearly named fields — a user field, an action field, an outcome field, a timestamp field — which can be searched, filtered, counted, and correlated with other data instantly, at any scale, regardless of how many million other entries surround it. Elasticsearch’s own audit log is written in exactly this structured way, which is a large part of why it is genuinely useful rather than merely present.

Recording a decision, not making one

It is worth repeating this distinction clearly, since it shapes everything else in this guide: audit logging never decides whether an action is allowed. That decision is made entirely by authentication and role-based access control, described in earlier chapters of this series. Audit logging simply writes down, faithfully and after the fact, what that decision was. This separation matters because it means audit logging cannot be bypassed by “convincing” it of anything — it has no opinion, no judgment, and no power to intervene. It only records.

Audit logging is not a standalone feature bolted on the side. It is woven directly into the same request-handling path that authentication and authorization already use.

The big picture

Every request that reaches a node passes through the security layer before it ever touches actual data. Audit logging is positioned to observe that same path, recording relevant moments as the request flows through, without redirecting, delaying, or altering the request itself in any way.

flowchart TD
  A["Incoming request"] --> B["Authentication layer"]
  B --> C["Audit: authentication_success or authentication_failed"]
  B --> D["Authorization layer (RBAC)"]
  D --> E["Audit: access_granted or access_denied"]
  D --> F["Request proceeds to Elasticsearch's data layer, or is rejected"]
  F --> G["Response returned to the client"]
  C -.->|"written to"| L["Audit log file, one JSON line per event"]
  E -.->|"written to"| L

Notice that audit logging branches off the main path rather than sitting inline within it. Writing an audit entry does not change whether a request succeeds or fails; that outcome was already decided by authentication and authorization. Audit logging simply captures a faithful copy of what happened at each of those decision points, on its own side channel, and writes it to the log.

Per-node operation

Audit logging runs independently on every single node in a cluster, since each node handles its own incoming requests and makes its own authentication and authorization checks for whatever traffic it happens to receive. This means enabling audit logging cluster-wide requires the setting to be applied consistently across every node, and it also means a complete picture of cluster-wide activity requires collecting audit log entries from every node, not just one.

Where the entries go

By default, audit events are written to a dedicated log file on each node, separate from the ordinary Elasticsearch server log, formatted as one JSON object per line. This file lives alongside the node’s other logs and follows similar rotation behavior, so it does not grow without bound on disk. From there, organizations typically ship these entries elsewhere for real analysis, since reading raw log files by hand does not scale much beyond a handful of entries.

Log rotation and file management

The audit log file follows the same general logging framework the rest of Elasticsearch’s own server logs use, which means it rotates on a configurable schedule or size threshold rather than growing into one single, ever-larger file forever. Rotation keeps any individual file at a manageable size for both the operating system and for any tooling that reads it, while older rotated files are retained on disk for a period before eventually being cleaned up, giving a short local buffer even before entries are shipped elsewhere for longer-term storage.

Why a separate file, rather than mixing with the server log

Keeping audit entries in their own dedicated file, distinct from the ordinary server log, is a deliberate design choice rather than an arbitrary one. Mixing security-relevant events into the same stream as routine operational messages — a shard relocating, a node rejoining the cluster, a slow query warning — would make it far harder to apply focused access control to just the security-sensitive portion, and far harder to ship only the audit-relevant entries to a restricted destination without dragging along everything else too. A dedicated file makes both of those things straightforward.

Tracing the journey from “something happened” to “a searchable entry exists” reveals a simple, reliable sequence.

Step by step

Why filtering happens before writing, not after

Filtering policies are applied at the point of deciding whether to write an entry at all, rather than being used to clean up or delete entries afterward. This distinction matters for two reasons. First, it keeps the audit log itself free of noise from the very start, rather than requiring a separate cleanup process later. Second, and more importantly for trustworthiness, it means the log never contains an event that was written and then quietly removed — a filtered event simply never existed in the log to begin with, which is a cleaner and more defensible position than editing a record after the fact.

Common event types worth knowing

While the exact set of event types can be tuned through configuration, a few recur constantly across almost any Elasticsearch deployment with audit logging enabled.

Why security_config_change deserves special attention

Most event types describe what a user did with data. The security_config_change event type describes something more foundational: a change to the rules themselves. A new role being created, an existing role’s privileges being widened, or a role mapping rule being altered are all exactly the kind of change that can quietly open a gap in access control long before anyone notices data being misused through that new gap. Because this event type is comparatively rare in a stable, well-run cluster, many teams treat every single occurrence of it as worth a deliberate, individual look, rather than folding it into routine, high-volume review.

Tracing a single, concrete scenario end to end shows exactly how audit logging captures a real event.

The scenario

Imagine someone attempts to log in with the username jdoe and an incorrect password, then a colleague sitting nearby successfully logs in moments later and tries to delete an index their role does not permit them to touch.

sequenceDiagram
  participant Attempt1 as Login attempt (jdoe, wrong password)
  participant Attempt2 as Login + delete request (colleague)
  participant Auth as Authentication Layer
  participant AuthZ as Authorization Layer
  participant Log as Audit Log File

  Attempt1->>Auth: Submit credentials
  Auth-->>Attempt1: Authentication rejected
  Auth->>Log: Write authentication_failed event

  Attempt2->>Auth: Submit valid credentials
  Auth-->>Attempt2: Authentication confirmed
  Auth->>Log: Write authentication_success event
  Attempt2->>AuthZ: Request to delete an index
  AuthZ->>AuthZ: Check role: delete_index not permitted
  AuthZ-->>Attempt2: Request denied
  AuthZ->>Log: Write access_denied event

Notice that both people’s actions land in the same log, each as its own distinct, self-contained entry. Later, anyone reviewing the log can immediately see that jdoe‘s login failed, that a different, correctly authenticated colleague then attempted an index deletion, and that the deletion was blocked — all without needing to interview anyone or reconstruct events from memory.

What the resulting entries contain

Each audit entry captures a consistent set of details, regardless of which specific event type it represents.

Why capturing both success and failure matters equally

It is tempting to think only failed or denied events matter for security purposes, but successful events are just as important to record. A successful authentication followed by a successful, fully authorized bulk deletion of an important index is not a security failure in the RBAC sense — everything worked exactly as configured — but it is exactly the kind of event an investigation may desperately need to find later, to establish who performed a destructive action that turned out to be a costly mistake rather than malicious intent.

A worked walkthrough: reconstructing an incident from the log

Concepts land better with a concrete story attached. Imagine a Monday morning where a team discovers an important reporting index is missing several days of data, and nobody can immediately recall why. Without an audit log, this becomes a round of asking everyone on the team whether they remember doing anything unusual, with no way to confirm any answer.

With audit logging enabled, the investigation instead starts by searching the shipped audit index for every access_granted event naming that specific index over the affected time window. This search surfaces a single entry: a service account, not a person, successfully authenticated on Friday evening and was granted a delete action against a range of documents matching a filter that, in hindsight, was broader than intended.

The next step is tracing that same service account’s activity a little further back in time, which reveals a security_config_change event from earlier the same week, showing that the account’s role had been widened to include delete privileges on that index as part of an unrelated project, and nobody had narrowed it back afterward once that project’s temporary need had passed.

The resolution that follows is now based on evidence rather than speculation: the role is corrected back to its original, narrower scope, the specific documents are restored from a recent snapshot, and the team adds a reminder to their role-review process to check for exactly this kind of “temporarily widened and never narrowed back” pattern going forward. None of this would have been possible to establish with confidence without the audit trail already sitting there, waiting to be searched.

Audit logging is highly configurable, which matters because logging absolutely everything, for every user, forever, is rarely the right default for a busy production cluster.

Turning it on

Audit logging is disabled by default and must be explicitly enabled on every node where the resulting record is wanted, which in practice usually means every node in the cluster. Enabling it is a deliberate, cluster-configuration-level decision, not something that happens automatically alongside enabling security features in general.

Choosing which event types to include

Rather than logging every single event type at full verbosity everywhere, administrators can specify which event types should actually be written, and which should be deliberately excluded. A cluster might choose to log every authentication failure and every access-denied event, since those are the highest-value signals for spotting trouble, while excluding routine, high-volume, low-value events such as successful health-check requests from a monitoring agent that authenticates constantly, every few seconds, all day long.

Filtering by user, realm, role, or action

Beyond choosing event types broadly, filtering policies allow excluding specific combinations from being logged at all — for instance, ignoring authentication events entirely for one particular internal monitoring account whose constant, expected activity would otherwise flood the log with entries nobody will ever need to review. This kind of targeted filtering is what keeps a busy cluster’s audit log focused on genuinely meaningful activity rather than being dominated by the noise of routine automation.

Deciding whether to capture full request bodies

For certain event types, Elasticsearch can optionally include the full body of the originating request in the audit entry, which is extremely valuable for understanding exactly what a blocked search or write attempt was actually trying to do. This comes with a real trade-off: request bodies can themselves contain sensitive information, such as search terms describing exactly what someone was looking for, so capturing them expands both the audit log’s storage footprint and its own sensitivity as a dataset in its own right.

Balancing verbosity against genuine usefulness

A cluster with dozens of applications, service accounts, and human users authenticating constantly throughout the day faces a real design question: how much detail is enough to reconstruct any incident that might occur, without drowning that same detail in so much routine noise that nobody can find it when it matters. There is no single correct answer that fits every deployment, but a workable general approach is to start narrower than feels comfortable — logging authentication failures, access-denied events, and configuration changes at minimum — and deliberately widen the scope only when a specific, concrete need for more detail becomes clear, rather than starting maximally broad and hoping to narrow it down later once the noise becomes unmanageable.

Coordinating configuration across every node

Because audit logging is a per-node setting, a cluster with inconsistent configuration across its nodes ends up with an incomplete, patchy record — some nodes faithfully logging every relevant event, others silently not logging anything at all, depending on which node happened to handle which request. Managing this configuration through the same centralized deployment and configuration-management process used for every other cluster-wide setting avoids this kind of accidental, partial coverage, ensuring that wherever a request happens to land, the same logging behavior applies.

Audit logging is valuable, but it is not free, and understanding the trade-off helps you configure it thoughtfully rather than as an afterthought.

The trade-off in plain language

Audit logging trades a modest, ongoing cost in storage and configuration effort for something that is very hard to put a price on until the exact moment it is needed: a trustworthy answer to “what actually happened.” Most organizations that have ever needed to answer that question under real pressure, without a log to consult, decide afterward that the modest ongoing cost was worth paying all along.

A record meant to be trusted as evidence has to be protected at least as carefully as the data it describes.

The audit log needs its own access control

An audit log that anyone can freely read defeats part of its own purpose, since the log itself can reveal sensitive details about who accesses what, and potentially the contents of captured request bodies too. Access to the raw audit log file, and to wherever its entries are shipped and stored afterward, should itself be restricted through role-based access control to only the people who genuinely need to review it, such as a security or compliance team.

Protecting against tampering

An audit log that can be silently edited or deleted by anyone with access to the underlying files is far less trustworthy as evidence, since there would be no way to be confident the record reflects everything that actually happened. Shipping audit entries promptly to a separate, more tightly restricted system — rather than leaving them sitting only on the same nodes that generated them — reduces the risk that someone with access to a compromised node could quietly erase the very entries that would reveal that compromise.

The audit log as part of a larger picture

Audit logging works best not as an isolated feature but as one input into a broader security monitoring practice, correlated alongside network logs, application logs, and any dedicated security information and event management platform an organization already runs. A single denied access attempt on its own may mean very little; the same denied attempt, correlated against an unusual login location and a burst of similar attempts against several other systems at the same time, tells a much more complete and urgent story.

A raw log file sitting on a node’s disk is only the starting point. Turning it into something a team actually uses day to day takes a few more deliberate steps.

Shipping logs out of the raw file

The most common next step after the raw audit log file is generated is shipping its entries elsewhere for real analysis, commonly into a dedicated Elasticsearch index built specifically for security data, or into a purpose-built security platform. This is typically done with a lightweight log-shipping agent that tails the audit log file continuously and forwards each new entry as it is written, turning what would otherwise be a growing text file into a live, continuously updated, searchable dataset.

Building dashboards on top of shipped entries

Once audit entries live in a searchable index, they become just as easy to visualize and explore as any other data in Elasticsearch. A simple dashboard might show authentication failures over time, a breakdown of denied actions by which role attempted them, or a map of the network origins requests are arriving from — turning a wall of individual log lines into an at-a-glance picture of overall security activity across the cluster.

Alerting on meaningful patterns

Beyond passive dashboards, shipped audit data can drive active alerting: a sudden spike in authentication failures against one account, a burst of access-denied events against a particularly sensitive index, or any authentication success from a geographic region the organization does not normally operate in. Alerting turns the audit log from something a person has to remember to check into something that proactively raises a hand the moment a genuinely unusual pattern appears.

Deciding on retention

How long audit entries should be kept is a deliberate policy decision, not a technical default to accept blindly. Some organizations are bound by regulatory requirements that specify a minimum retention period, sometimes measured in years. Others balance the value of a longer historical record against the ongoing storage cost of keeping it, often keeping recent entries in fast, easily searchable storage while moving older entries into cheaper, longer-term storage that is still retrievable if an investigation ever needs to reach that far back.

Seeing audit logging applied to genuine situations makes the concept concrete rather than abstract.

Practical guidance for building an audit logging setup that is actually useful, not merely present.

A useful design pattern: the two-tier alert

Rather than treating every audit event as equally worth an immediate alert, many teams settle on a two-tier approach. A small set of especially significant event types — a security configuration change, an authentication success from an unexpected geography, or an access-denied event on a small handful of explicitly named, highly sensitive indices — triggers an immediate, individual notification. Everything else feeds into aggregate dashboards and periodic summary reviews instead, reserving urgent interruption for the handful of signals that genuinely warrant it, while still keeping the full record available for the less urgent, more exploratory kind of review.

A useful design pattern: baseline-and-deviation review

Rather than reviewing raw audit events one by one, a useful pattern is establishing what “normal” looks like for a given account or role — roughly how often it authenticates, from where, and what it typically does — and then watching specifically for deviations from that established baseline. A service account that has authenticated from the same two IP addresses every day for a year suddenly authenticating from a third, unfamiliar one is a far more actionable signal than any single event viewed without that context, and this pattern turns a large volume of routine-looking events into a small, manageable number of genuinely worth-investigating deviations.

Common mistakes to avoid