Before meeting at-most-once, we need to see the problem that delivery guarantees solve.

Kafka in a nutshell

Apache Kafka stores streams of events, such as orders, payments, clicks, or sensor readings, in topics. Producers are the applications that write events, and consumers are the applications that read them. Between them sit the Kafka servers, called brokers, which store and copy the events safely.

Failures are normal

In any real system, things go wrong. A network cable is unplugged. A broker restarts. A producer crashes in the middle of sending. A consumer is killed by its host after reading an event but before finishing its work. When something fails at the wrong moment, the two sides may disagree about what has happened. Did the broker receive the event? Did the consumer finish processing it? Nobody knows for sure.

Two ways to react to doubt

When in doubt, a system can choose between two natural reactions. It can try again, which risks doing the same thing twice. Or it can give up, which risks doing the thing zero times. Delivery guarantees are the names for these choices, and Kafka lets you choose on purpose.

At-most-once means each event is delivered zero times or one time, never more than once.

The simple definition

Under at-most-once delivery, the system never repeats an event. If everything goes well, the event is delivered once. If something goes wrong at the wrong moment, the event may not be delivered at all, and the system does not try again. Duplicates are impossible, but loss is possible.

The three guarantees side by side

What the name really says

The words are literal. “At most once” means the number of deliveries is at most one: it could be one, and it could be zero, but it can never be two. Compare this with “at least once”, where the count is one or more, and “exactly once”, where the count is precisely one.

An event’s journey has two legs, and at-most-once behaviour can appear on either.

Leg one: producer to Kafka

The producer sends an event to the broker. If the send fails or the confirmation is lost, the producer must decide whether to retry. A producer that never retries, or that never waits for confirmation, behaves at-most-once on this leg.

Leg two: Kafka to consumer

The consumer reads the event and processes it, and saves its bookmark, called the committed offset, which records how far it has read. The order of “process” and “save the bookmark” decides the guarantee on this leg. Saving the bookmark first and processing afterwards behaves at-most-once.

flowchart LR
  P["Producer"] -->|"Leg 1: send and confirm"| K["Kafka Cluster"]
  K -->|"Leg 2: fetch, process, commit bookmark"| C["Consumer"]
        

The overall guarantee is the weakest leg

The guarantee of the whole journey is only as strong as its weakest leg. If the producer leg is at-least-once but the consumer leg is at-most-once, then events may still be lost between reading and processing. Likewise, if the producer loses events, no consumer setting can bring them back. To claim a guarantee for a pipeline, both legs must support it.

A producer becomes at-most-once when it sends an event and never checks or never repeats.

Fire and forget with no confirmation

A producer can be set to wait for no confirmation at all. It hands the event to the network and moves on immediately. It never learns whether the broker received it. If the broker was down, the network dropped the packet, or the leader changed at that moment, the event is gone, and the producer has no idea. This is the fastest but riskiest setting.

Confirmation from the leader only

A producer can wait until the leader broker has written the event, but not until the followers have copied it. This is safer, but if the leader fails just after confirming and before the followers copy the event, the event can be lost when a follower becomes the new leader. Loss is possible, though less likely than with no confirmation.

Retries turned off

Even with strong confirmations, a producer that is told never to retry will give up on the first failure. Because it never resends, it cannot create duplicates, but any temporary hiccup, such as a leader election lasting a second, causes the event to be dropped and reported as failed. If the application ignores that failure report, the event is lost silently.

sequenceDiagram
  participant P as Producer
  participant L as Leader Broker
  P->>L: Send event
  Note over P,L: Network drops the event
  Note over P: Producer expects no confirmation, does not retry
  Note over L: Event was never stored
        

On the reading side, the order of two small actions decides everything.

The two actions

A consumer reads a batch of events, then does two things: it processes the events, for example by saving them to a database or sending an email, and it commits its offset, which saves the bookmark in Kafka. If the consumer crashes, a replacement starts from the last committed bookmark.

Commit first, process later

If the consumer commits the bookmark first and processes afterwards, a crash between the two steps leaves Kafka believing the events are done, even though they were never processed. The replacement consumer starts after them and never sees them again. The events are lost as far as this application is concerned. This is at-most-once on the consumer side.

sequenceDiagram
  participant C as Consumer
  participant K as Kafka
  participant D as Database
  K-->>C: Events 100 to 109
  C->>K: Commit bookmark 110
  C->>D: Save event 100
  C->>D: Save event 104
  Note over C: Consumer crashes here
  Note over K: Bookmark says 110, events 105 to 109 never processed
        

Many teams end up with at-most-once behaviour without ever choosing it. Knowing the sneaky routes helps you avoid them.

Automatic commits with handed-off work

Consumers can save their bookmarks automatically every few seconds. When processing happens fully inside the reading loop, automatic commits normally give at-least-once behaviour, because the bookmark for a batch is saved only when the consumer comes back for more. But if the application hands events to other threads, queues, or asynchronous tasks and immediately returns to read more, the bookmark can move ahead while the real work is still waiting or in progress. A crash then loses everything queued but unfinished. The behaviour is at-most-once, though nobody meant it.

Committing at the wrong moment

Code that commits immediately after fetching, or commits in a background timer regardless of progress, moves the bookmark ahead of the actual work.

Skipping failed events

A consumer that catches an error for a bad event, logs it, commits, and moves on, deliberately drops that event. Sometimes this is right, but if done broadly, it silently turns the pipeline into at-most-once for every failure.

Lagging beyond retention

A consumer that falls so far behind that unread events expire from Kafka loses them, and its reset rule may skip forward. The application experiences at-most-once loss, caused by slowness and retention rather than by the commit order.

Skipping ahead during recovery

Operators sometimes reset a group to the latest position to “clear a backlog”. Every event in the skipped range is lost to that group.

There are honest, sensible reasons to accept possible loss. The key is to choose it deliberately and only for the right data.

Why some teams choose it

Data that suits at-most-once

Data that does not

Seeing the neighbours makes the choice clear.

At-least-once: retry until sure

The producer retries until it gets a confirmation. The consumer processes first and commits afterwards. Nothing is lost, but a crash or a lost confirmation can cause events to be handled again. Applications must therefore tolerate repeats, usually by making the processing safe to repeat.

Exactly-once: retry safely

Kafka offers tools so that retries do not create visible duplicates: duplicate protection in the producer, and transactions that tie reading, processing, and writing together inside Kafka. The effect appears once even though internal steps may be repeated. It costs some extra delay and complexity, and it fully applies to processing within Kafka, while effects on outside systems need extra care.

flowchart TD
  Q1{"Is losing an event acceptable?"} -->|"Yes"| A["At-most-once: fastest and simplest"]
  Q1 -->|"No"| Q2{"Can consumers safely handle repeats?"}
  Q2 -->|"Yes"| B["At-least-once: safe and common"]
  Q2 -->|"No"| C["Exactly-once: transactions and duplicate protection"]
        

Consumer groups change over time, and each change is a moment when events can slip through.

What a rebalance does

When a consumer joins or leaves a group, or crashes, the partitions are redistributed among the members. The new owner of a partition starts from the last committed bookmark. If the old owner had already committed a bookmark ahead of its real work, the new owner starts after events that were never processed.

The risky window

The dangerous period is the gap between the bookmark moving forward and the work finishing. The longer the work takes after the bookmark is saved, the wider the window in which a crash or a rebalance causes loss. Large batches and slow processing make the window bigger.

The cruelty of at-most-once is that losses are silent. Good teams find ways to measure them anyway.

Compare counts at both ends

The simplest method is to count events sent and events received over the same period and compare. If a producer reports having sent one million events and consumers processed 999,000, a small loss is happening. Even a rough comparison reveals whether the loss rate is what you expected and accepted.

Sequence numbers in events

Some producers stamp each event with a running number per source. A consumer that notices a gap in the sequence, such as receiving 41 and then 43, knows that 42 is missing. This gives an exact count of losses and their timing.

Watch the producer’s error signals

Even at-most-once producers can report errors through callbacks or metrics. Counting these errors reveals loss on the sending side, provided the confirmation setting is not so low that the producer learns nothing.

Watch the consumer’s lag and resets

Unexpected jumps in a group’s position, out-of-range errors, and manual resets all signal potential loss on the reading side.

Many systems start as at-most-once and later need something stronger. The path upward is well known.

At-most-once is a bargain of speed for certainty.

A delivery guarantee may look purely technical, but it touches safety and trust too.

Lost security events

Security and access events, such as failed logins or permission changes, are exactly the events an attacker hopes will go missing. Sending them with at-most-once behaviour can leave gaps in the very records that investigators rely on. Treat security telemetry as data that needs at-least-once handling.

Lost deletions and consent changes

Events that record a customer’s request to delete their data or withdraw consent must not be dropped. A lost event of this kind can mean that personal data remains in some downstream system when it should have been removed.

Audit trails

Audit records are evidence. A gap in an audit trail may itself be treated as a failure of control. If some data is both noisy and audited, split it into two topics with different guarantees.

Most trouble comes from the same few mistakes.

Best practices at a glance

Let us see how a delivery company might decide which topics can be at-most-once.

flowchart LR
  V["Vehicle location pings"] -->|"At-most-once: fast, no retries"| T1["Locations topic"]
  E["Engine health readings"] -->|"At-most-once with sequence numbers"| T2["Health topic"]
  D["Delivery confirmations"] -->|"At-least-once with repeat-safe consumers"| T3["Deliveries topic"]
  Y["Customer payments"] -->|"At-least-once or exactly-once"| T4["Payments topic"]
  S["Security logs"] -->|"At-least-once with dead letter topic"| T5["Security topic"]
        

It is a delivery guarantee under which each event is delivered zero times or one time, never repeated, so duplicates are impossible but loss is possible.

No. Modern Kafka clients default to safer producer settings, and consumers that process before committing behave at-least-once. At-most-once appears when settings or code are chosen or written in a way that skips confirmations, retries, or the safe commit order.

By not waiting for confirmations or by not retrying after failure. The producer sends and moves on, so an event that fails to arrive is never resent.

By saving its bookmark before finishing the work, or by letting bookmarks advance while work is still queued elsewhere. A crash then leaves those events unprocessed and unrepeated.

Not through the delivery mechanism itself when configured correctly. You can still lose events through other causes, such as falling behind the retention period, so retention and monitoring remain important.

For data where a newer event replaces an older one or where occasional gaps do not matter, such as frequent sensor readings, metrics, and rough analytics, especially when speed and low delay matter most.

Compare the number of events sent with the number processed, add running sequence numbers to spot gaps, and watch producer errors and consumer out-of-range or reset events.

Fire and forget is the most extreme producer form of it. At-most-once is broader and also covers consumers that commit before processing, and producers that report failures but never retry.

Not literally. Exactly-once means the effect appears once, even though internal retries may occur. Duplicate protection and transactions make the repeats invisible.

Yes, and they should. Match each topic’s guarantee to how costly a loss or a duplicate would be for that data.

We began with a postman who ticks each letter off his list before delivering it, so that no letter is ever delivered twice, at the price of losing any letter he drops on the way. At-most-once delivery in Kafka is that rule. On the producer side, it appears when events are sent without waiting for confirmation or without retrying. On the consumer side, it appears when the bookmark is committed before the work is done, or when work is handed to other threads while bookmarks race ahead. It never repeats an event, but it can silently lose one, and the loss is invisible unless you count or number your events. It is a sensible choice for disposable data such as frequent sensor readings, metrics, and rough analytics, where speed and simplicity matter more than completeness, and a dangerous choice for payments, orders, security events, and audit records. It can also arise by accident through careless settings, so review commit order, retries, confirmations, and error handling. Choose the guarantee topic by topic, measure the loss you accept, document the decision, and move to at-least-once or exactly-once as soon as the data becomes too valuable to lose.