Lag only makes sense once you know what is being produced and consumed, and where the numbers live.

The moving parts

Apache Kafka is a system for moving streams of events between applications. An event is a small record that says something happened, such as “order 4471 was placed” or “sensor 12 reported 31 degrees”. Programs called producers write events into Kafka. Programs called consumers read those events back out and act on them, such as saving them to a database, updating a dashboard, or sending a notification.

Events are organised into topics, which are named streams such as “orders” or “payments”. Each topic is split into partitions, which are separate ordered logs. Splitting a topic into partitions lets Kafka spread the writing and reading work across many machines and many consumers at once, which is a big part of why Kafka can handle very high volumes.

Words you will see throughout this guide

A single sentence hides several moving numbers. Let us take them apart.

The core formula

For one partition, consumer lag is defined as the log end offset minus the consumer group’s committed offset for that partition. The log end offset is the position where the next event a producer writes will land. The committed offset is the position the consumer group has confirmed as fully handled. The difference between them counts how many events are sitting in the partition, written but not yet confirmed as processed by that particular group.

flowchart LR
    A["Committed offset - last confirmed position"] --> B["Events waiting to be consumed"]
    B --> C["Log end offset - where the next event lands"]
        

A worked example

Suppose a partition’s log end offset is 10,000, meaning 10,000 events have been written so far. A consumer group has committed offset 9,850, meaning it has confirmed handling of events 0 through 9,849. The lag for that partition is 10,000 minus 9,850, which is 150 events. Those 150 events exist on the broker and are waiting for this group to read and confirm them.

Lag is per group and per partition

Because committed offsets belong to a specific consumer group, lag is always a property of a group reading a particular partition, not a property of the topic itself. Two different groups reading the same topic, such as a billing service and an analytics service, will almost always show different lag numbers, since they read at different speeds and may have started at different times.

Total lag for a group

Because a consumer group typically reads many partitions across one or more topics, people often talk about “total lag” for a group, which is simply the sum of the lag across all its assigned partitions. This total is a useful headline number, but it can hide a serious problem in a single partition behind healthy numbers everywhere else, a point we return to in Chapter 8.

Lag is calculated by combining information from two different places inside Kafka.

The log end offset, from the broker

Each broker that leads a partition tracks the log end offset for it, updating the number every time a producer successfully writes an event. For consumers that only read fully replicated data, the relevant boundary is actually the high watermark, which is the highest offset that has been safely copied to enough replicas. Either way, this number reflects how far producers have gotten.

The committed offset, from the group coordinator

Kafka stores committed offsets in a special internal topic named __consumer_offsets. For each consumer group, one broker acts as the group coordinator, tracking membership and answering questions about committed offsets for that group. When a consumer finishes processing a batch of events, it sends a commit request, and the coordinator records the new committed offset.

sequenceDiagram
    participant P as Producer
    participant B as Broker (partition leader)
    participant C as Consumer
    participant G as Group Coordinator
    P->>B: Write new events
    B-->>B: Log end offset advances
    C->>B: Fetch events from last position
    B-->>C: Batch of events
    C->>C: Process the batch
    C->>G: Commit new offset
    G-->>G: Committed offset advances
        

Who actually computes the subtraction

Kafka itself does not maintain a running “lag” field somewhere; it stores the two raw numbers and lets tools calculate the difference on demand. Several sources can compute lag for you:

Lag is the visible result of a simple race between two speeds: how fast events arrive, and how fast they are handled.

The basic race

If the rate at which a consumer processes events matches or exceeds the rate at which producers write them, lag stays flat or shrinks. If processing falls behind production, lag grows. Momentary spikes in production, or momentary slowdowns in processing, are normal and usually self-correct once the surge passes or the slowdown ends.

flowchart TD
    A["Production rate vs consumption rate"] --> B{"Consumption rate greater or equal?"}
    B -- "Yes" --> C["Lag stays flat or shrinks"]
    B -- "No" --> D["Lag grows over time"]
        

Reasons production speeds up

Reasons consumption slows down

The same underlying numbers can be expressed in a few different useful shapes.

Record lag versus time lag

Record lag is easy to compute and always available, but its meaning depends entirely on the topic’s throughput. A record lag of one million events is unremarkable on a topic that handles two million events per second, and severe on a topic that handles ten events per minute. Time lag fixes this by expressing the same gap in minutes or hours, which is usually the number that matters to the business, since “we are forty minutes behind” is meaningful regardless of the topic.

Two practical approaches convert an event count into a duration people can reason about.

Approach one: divide by throughput

If you know the typical consumption rate, in events processed per second, you can estimate time lag by dividing record lag by that rate. A lag of 3,000 events on a consumer that processes 100 events per second suggests roughly 30 seconds to catch up, assuming production does not speed up further in the meantime. This approach is simple but only as accurate as your throughput estimate, and it ignores the possibility that production keeps growing while the consumer catches up.

Approach two: read the timestamp of the next unread event

Every Kafka event carries a timestamp, usually set when the producer wrote it. A monitoring tool can look up the timestamp of the event sitting right at the committed offset, which is the oldest event the consumer has not yet finished, and compare it to the current time. The difference is a direct measurement of how stale the data being processed is, without needing to guess a processing rate.

flowchart LR
    A["Committed offset event's timestamp"] --> B["Compare to current time"]
    B --> C["Result: time lag in seconds or minutes"]
        

A worked comparison

From a quick manual check to a full production dashboard, several layers of tooling exist.

Follow one measurement from raw numbers to a decision, and see where things can quietly go wrong.

flowchart TD
    A["Monitoring tool polls broker and coordinator"] --> B["Read log end offset per partition"]
    A --> C["Read committed offset per partition"]
    B --> D["Compute record lag per partition"]
    C --> D
    D --> E["Sum into total group lag"]
    D --> F["Optionally convert to time lag using timestamps"]
    E --> G{"Above alert threshold?"}
    F --> G
    G -- "Yes" --> H["Alert fires, team investigates"]
    G -- "No" --> I["Value stored for trend graphs"]
        

Why summing into a total can hide problems

A group with twelve partitions, eleven of which are perfectly caught up and one of which is badly stuck, can still show a total lag number that looks only moderately elevated once averaged across all twelve. Meanwhile every event destined for that one stuck partition is stale. This is why serious monitoring setups track per-partition lag, not only the group total, and specifically watch for the maximum lag across partitions rather than only the sum or average.

A subtlety with idle partitions

A partition that simply receives no new traffic for a while will show its lag correctly staying at zero, since both offsets stop moving together. But a partition that is not currently assigned to any active consumer, for example right after a member left the group and before rebalancing finishes, can show a growing lag that has nothing to do with processing speed and everything to do with nobody currently reading it. Good monitoring distinguishes “slow consumer” from “no consumer assigned” where possible.

A threshold that never fires is useless, and one that fires constantly gets ignored. Aim for the middle.

Static thresholds

The simplest approach sets a fixed number, such as “alert if lag exceeds 10,000 events” or “alert if time lag exceeds five minutes”. Static thresholds are easy to understand and quick to set up, but as shown in Chapter 6, a single record-lag number rarely fits every topic equally well, so static thresholds usually need to be set per topic or per group rather than globally.

Rate-of-change thresholds

Instead of, or alongside, a fixed ceiling, some teams alert on how fast lag is growing. A group whose lag has been climbing steadily for ten straight minutes is a stronger signal of trouble than a single moment where lag happens to be somewhat high, especially on topics with naturally bursty traffic.

Business-driven thresholds

The most reliable thresholds are set by asking what delay the business can actually tolerate for that specific data. A fraud alerting topic might need action within seconds. A daily reporting pipeline might tolerate being twenty minutes behind without anyone caring. Setting the threshold from the business requirement, rather than an arbitrary round number, keeps alerts meaningful.

Once an alert fires, a structured set of questions narrows the cause quickly.

flowchart TD
    A["Lag alert fires"] --> B{"All partitions affected?"}
    B -- "Yes, whole group" --> C{"All groups on topic affected?"}
    C -- "Yes" --> D["Suspect producer surge, broker or network issue"]
    C -- "No, just this group" --> E["Suspect shared downstream dependency or resource limit"]
    B -- "No, one or two partitions" --> F{"Offset moving at all?"}
    F -- "Not moving" --> G["Suspect a poison message or unassigned partition"]
    F -- "Moving slowly" --> H["Suspect uneven load or a slow key"]
        

Once the cause is known, the fix usually falls into one of a few well-worn categories.

Recovering once the cause is fixed

After the root cause is resolved, a healthy consumer naturally works through its backlog because its processing rate again exceeds the production rate, and lag falls back toward zero over time. Watching the lag trend during recovery tells you whether the fix actually worked, since a fix that only stops lag from growing further, without it also shrinking, has not fully solved the problem.

flowchart LR
    A["Cause identified and fixed"] --> B["Processing rate exceeds production rate again"]
    B --> C["Backlog of unread events shrinks"]
    C --> D["Lag trends back toward its normal baseline"]
        

Several other Kafka ideas sound similar to lag but measure something different.

Consumer lag and end-to-end latency

Consumer lag measures only the wait inside Kafka before a consumer starts and finishes handling an event. It does not include time spent earlier, such as how long it took the producer to send the event after the real-world action happened, nor time spent later, such as writing a result to a downstream database. Teams that care about the full journey of data, from the original action to the final visible result, track end-to-end latency as a separate, broader measurement that uses consumer lag as one input among several.

Frameworks built on top of the plain Kafka consumer track and expose lag in their own way.

Lag is not only an alerting signal; it is also a popular input for automatically adjusting capacity.

The basic idea

Instead of running a fixed number of consumer instances all the time, an autoscaling system watches lag and adds more consumer instances when lag climbs, then removes instances again once lag falls back to a healthy level. Because a single partition can only be actively read by one consumer in a group at a time, scaling stops helping once the number of instances reaches the number of partitions.

flowchart TD
    A["Autoscaler watches group lag"] --> B{"Lag above target?"}
    B -- "Yes" --> C{"Instances less than partition count?"}
    C -- "Yes" --> D["Add consumer instances"]
    C -- "No" --> E["Cannot help further by scaling; investigate other causes"]
    B -- "No, at or below target" --> F{"More instances than needed?"}
    F -- "Yes" --> G["Remove idle instances"]
        

Practical cautions

Reading lag information is usually low-risk, but it still touches real access controls.

What monitoring actually needs to read

Computing lag requires read access to the partition metadata that exposes the log end offset, and read access to the consumer group’s committed offsets. Neither requires reading the actual event contents, so a monitoring tool can usually be granted narrow permissions that see offsets and group metadata without ever being able to read the sensitive data flowing through the topics themselves.

Separating monitoring identity from application identity

Good practice gives lag monitoring tools their own dedicated credentials, separate from the credentials used by the actual producing and consuming applications. This way, a compromised monitoring tool cannot read or write application data, and a change in application credentials does not silently break monitoring, or vice versa.

Sensitivity of lag data itself

Lag numbers reveal operational information about a system, such as how much traffic a topic handles and how healthy its consumers are. In most organisations this is treated as ordinary operational telemetry, but in a multi-tenant platform where different customers or teams run isolated workloads, access to another team’s lag dashboards may reveal more about their business volume than intended, so dashboard access itself deserves the same access review as any other internal metric.

Habits that make lag monitoring genuinely useful, and habits that quietly undermine it.

Patterns worth adopting

Anti-patterns to avoid

A practical checklist for building lag monitoring that people actually trust.

How to set up monitoring well

How different kinds of systems watch and respond to lag in practice.

Kafka is used widely across industries such as retail, finance, telecommunications, travel and media for moving operational data, events and analytics at scale. The examples below are illustrative patterns of how lag monitoring is commonly applied, rather than descriptions of any specific company’s internal setup.

Not necessarily. Momentary nonzero lag is normal, since producing and consuming are separate processes. The real goal is bounded, stable lag that matches what the business can tolerate, not a permanent reading of exactly zero.

Deploying typically restarts consumer instances, which triggers a rebalance as the group reassigns partitions. Consumption pauses briefly during this process, causing a short lag spike that usually recovers quickly once the new instances are assigned and start fetching.

No, under normal operation lag cannot go below zero, since the committed offset cannot legitimately exceed the log end offset. If tooling shows a negative number, it usually indicates a timing artifact between two separate reads of the two offsets, or a bug in the calculation.

Not directly. Broker count affects storage capacity and fault tolerance, not how fast a specific consumer group processes events. Reducing lag usually means adding consumer instances up to the partition count, increasing partitions, or speeding up per-event processing.

Common setups poll every ten to sixty seconds for dashboards and alerting. Polling much more frequently adds unnecessary load for little extra insight, while polling only every several minutes can delay detection of fast-developing problems.

This often points to uneven load across partitions, for example if events are keyed in a way that sends a disproportionate share of traffic to one partition, or to a consumer instance handling that partition being under-resourced compared to its peers.

No. Lag means events are waiting to be processed, not that they are gone. Data loss becomes a real risk only if lag grows so large that unprocessed events fall outside the topic’s retention window and are deleted before the consumer reaches them.

No. Thresholds should reflect each topic’s throughput and each use case’s tolerance for delay, which is why business-driven, per-topic thresholds are recommended over one uniform number across the cluster.

Yes. Monitoring tools read offsets and metadata independently of the consumer’s own fetch and commit path, so routine polling for monitoring has only a very small, usually negligible, impact on cluster load.

Throughput tells you how fast events are being produced or consumed right now. Lag tells you the accumulated result of the relationship between those two speeds over time. Both are useful together: throughput helps explain why lag is changing.

Consumer lag monitoring measures the gap between how far producers have written and how far a specific consumer group has confirmed processing, for every partition it reads. It is calculated from two independent numbers, the log end offset and the committed offset, and it rises whenever processing falls behind production for any reason, from a slow downstream call to a stuck poison message to a group-wide outage. Turning raw record counts into time-based figures, watching per-partition detail alongside group totals, and setting thresholds around real business tolerance are what separate lag monitoring that actually catches problems from a dashboard nobody trusts.