An offset is simply the position number of one event inside one partition, and nothing more complicated than that.

In Kafka, data is organized into named streams called topics, and each topic is split into one or more partitions for scale and parallelism. Every partition is really just an ordered, append-only list, a log. As new events arrive, they are written to the end of this list, one after another, and each one is given a number: 0, 1, 2, 3, and so on, counting upward forever. That number is the offset.

An offset always belongs to exactly one partition. The tenth event in Partition 0 has offset 9 in Partition 0’s own numbering, completely separate from whatever number the tenth event in Partition 1 might have. Offsets are never shared or compared across different partitions; each partition keeps its own private counter.

This single idea, a simple, ever-increasing counter per partition, is what allows Kafka to answer a question that matters enormously in real systems: “exactly how far has this reader gotten?”

Without a reliable way to mark “how far I have read,” a streaming system quickly becomes chaotic.

Picture a topic receiving a thousand new events every second, read by several completely different services: a billing system, a fraud checker, and an analytics dashboard. Each of these services reads at its own pace. The billing system might be caught up to the very latest event, while the analytics dashboard, doing heavier processing, might be a few minutes behind. If Kafka had no way to track each reader’s exact position, there would be no way to guarantee no event was skipped, and no way to avoid processing the same event over and over.

The offset solves this cleanly. Each independent group of readers, called a consumer group, remembers its own offset for each partition it reads. This is often described as a bookmark: a private marker that says “I have processed everything up to and including this position; the next thing I read should start right after it.”

A handful of closely related terms appear whenever offsets are discussed. Learning them together avoids a lot of confusion.

Every partition is, underneath everything else, just a simple growing list. Offsets are the addresses into that list.

When a producer sends a new event to a partition, Kafka appends it to the end of that partition’s log and assigns it the next available offset, one higher than the previous event. This numbering never resets and never skips backward under normal operation; it only ever grows, event after event, for as long as the partition exists.

flowchart LR
    O0["Offset 0
order placed"] --> O1["Offset 1
payment made"]
    O1 --> O2["Offset 2
item shipped"]
    O2 --> O3["Offset 3
item delivered"]
    O3 --> O4["Offset 4
next new event"]
        

A consumer reading this partition simply walks through these offsets in order: it reads offset 0, processes it, then reads offset 1, and so on. At any point, if the consumer needs to pause, stop, or restart, all it needs to remember is the single number of the last offset it successfully finished, and it can pick up exactly where it left off.

Reading an event and committing its offset are two separate steps, and the gap between them matters a great deal.

When a consumer finishes handling an event, it does not automatically save its progress. It must explicitly tell Kafka, “commit my offset here,” which is Kafka’s way of durably recording “this consumer group has confirmed everything up to this point.” This committed offset is stored by Kafka itself, in a special internal topic, so it survives even if the consumer restarts or a different machine takes over the job.

The timing of this commit creates an important trade-off. If a consumer commits its offset before finishing the actual work, a crash partway through could mean the event is marked as done even though the work never completed. If a consumer commits after finishing the work, a crash before the commit could mean the same event gets processed again later. Kafka gives engineers the choice of how to handle this.

Different teams reading the same topic never step on each other’s toes, because each consumer group keeps its own set of offsets.

A single partition can be read by many different consumer groups at the same time, each completely unaware of the others. The billing service, the analytics pipeline, and the fraud detector might all read the exact same “orders” partition, yet each one tracks its own offset independently. One group might be sitting at offset 950, while another, doing slower work, is only at offset 400.

flowchart LR
    T[Partition: orders] --> A[Consumer Group: Billing
offset 950]
    T --> B[Consumer Group: Analytics
offset 400]
    T --> C[Consumer Group: Fraud Check
offset 875]
        

Inside one consumer group, the situation is a little different: the partitions of a topic are divided up among the group’s members, so that each partition is read by only one consumer within that group at any given time. This avoids two members of the same group accidentally processing the same event twice.

Because offsets are just numbers, and Kafka keeps history, a consumer can deliberately move its position around, not just forward.

This ability is called seeking, and it is one of Kafka’s most powerful features. A consumer is not limited to always reading forward from wherever it last stopped. It can be told to jump to a specific offset, to the very beginning of the partition, or to the very latest available offset, skipping everything older.

Offsets give us a precise, numerical way to answer a question every team eventually asks: “are we keeping up?”

Consumer lag is simply the difference between the newest offset written to a partition, the log end offset, and the offset a consumer group has actually committed. If the log end offset is 10,000 and a consumer group’s committed offset is 9,850, that group’s lag is 150, meaning it is 150 events behind the very latest data.

flowchart LR
    A["Committed Offset: 9850"] -->|Lag = 150| B["Log End Offset: 10000"]
        

Lag is one of the single most useful health signals in any Kafka-based system. A small, steady lag is completely normal, since consumers rarely process events the exact instant they are written. But a lag that keeps growing over time is a warning sign: the consumer cannot keep up with incoming traffic, and something needs attention, whether that is slow processing logic, insufficient workers, or a genuine spike in traffic.

Offsets are the reason Kafka can make a strong ordering promise, but only within a certain boundary.

Because a partition’s offsets always increase in the exact order events were written, reading a partition in offset order guarantees you see those events in the exact order they happened. This is a strong, reliable promise, but it applies strictly within one partition. Kafka makes no promise about the relative order of events sitting in two different partitions, since each partition has its own independent offset sequence running at its own pace.

This is precisely why related events, all actions belonging to the same customer, for example, are usually sent to the same partition using a shared key. Doing so guarantees they will always be read back in the correct order, since they share one offset sequence.

Kafka does not keep every event forever by default, and this interacts with offsets in a way worth understanding clearly.

Each topic has a retention policy, a rule such as “keep events for seven days” or “keep up to a certain total size.” Once an event’s retention period expires, Kafka deletes it from disk to free up space. Crucially, offset numbers themselves are never reused or renumbered when old events are deleted; the earliest offset still present in the partition simply moves forward.

Offsets are working quietly behind the scenes anywhere Kafka is used at scale.

The offset model is simple and powerful, but it is not entirely free of complexity.

Offsets themselves are just numbers, but controlling who can move them matters for a system’s integrity.

Because offsets are just numbers, they are unusually easy to monitor, chart, and alert on compared to many other system metrics.

A few habits separate teams who use offsets confidently from teams who get quietly bitten by them.

Helpful patterns

• Commit after processing — save the offset only once the actual work has succeeded, not before.
• Idempotent processing — design consumers so reprocessing the same offset twice causes no harm.
• Lag-based alerting — set alerts on consumer lag trends to catch slowdowns before they become outages.
• Deliberate seeking — use seek-to-beginning or seek-to-timestamp intentionally for backfills and investigations.

A short checklist that keeps offset-related surprises to a minimum.

Every message gets a unique offset within its own partition, but not across the whole topic. Two different partitions can each have an event at offset 5, and these are two entirely separate events.

No, the numbering itself only ever increases as new events are appended. What can move is a consumer’s committed offset, which can be deliberately reset backward through seeking.

Each consumer group maintains its own separate committed offset for that partition. They do not affect or interfere with each other in any way.

Not necessarily. A small, stable lag is completely normal. A lag that keeps growing steadily over time is the real warning sign that a consumer cannot keep pace with incoming data.

Yes, by seeking directly to any specific offset, or to a timestamp that Kafka translates into the nearest offset, a consumer can begin reading from any point in the partition’s available history.

The offset numbers themselves are not reused or renumbered. The log start offset simply moves forward to reflect the earliest data that is still actually available.

No. The offset is purely a position counter, while a timestamp records when the event was created or written. Kafka can convert between the two, but they measure different things.

We began with a simple bookmark in a novel and ended inside one of Kafka’s most consequential design decisions. A partition offset is nothing more than the position number of one event inside one partition, an ever-increasing counter that starts at zero. Yet this simple number is what lets many independent readers track their own progress, resume precisely after a crash, replay history on demand, and give engineers a clear, measurable way to know whether a system is keeping up. Understanding offsets deeply is one of the fastest ways to truly understand how Kafka behaves.