Retries exist because networks and machines are not perfectly reliable.

A message can fail in more than one way

A Kafka producer is a program that sends messages, called records, to a Kafka broker, which is a server that stores and manages the data. Between the producer and the broker sits a network, and networks are never perfectly dependable. A cable can be briefly unplugged, a broker can be too busy to answer immediately, or a leader broker might be in the middle of restarting.

When a send fails, or when no reply comes back in time, the producer has a choice. It can give up and report an error to your application, or it can try sending the exact same data again. This second option is called a retry, and it is one of the simplest and most valuable safety features a messaging system can offer.

What Kafka considers a retriable error

Not every failure is worth retrying. Some errors are temporary, such as the broker being briefly unavailable during a leader election. Kafka calls these retriable errors, and the producer is designed to try again automatically. Other errors are permanent, such as sending a message that is too large for the broker to accept, and retrying those would be pointless, so the producer reports them immediately instead.

flowchart TD
    A[Producer sends a batch] --> B{Reply received in time?}
    B -- Yes, success --> C[Batch is safely stored]
    B -- No, or an error came back --> D{Is the error retriable?}
    D -- Yes --> E[Wait briefly, then resend the same batch]
    E --> B
    D -- No --> F[Report the failure to the application]
        

A single configuration value controls how persistent the producer is allowed to be.

What the number means

The retries setting tells the producer the maximum number of times it may resend a batch after a retriable failure, before it gives up and reports an error. In older Kafka client versions, this defaulted to a small number, such as zero or a handful of attempts. In modern Kafka client versions, it defaults to a very large number, which in practice means “keep trying until the request simply times out.”

The setting that really controls the clock

A high retries number alone would not be very useful if the producer could retry forever without limit. Kafka solves this with a companion setting called delivery.timeout.ms, which sets the total time budget for a message, covering every attempt, wait, and retry combined. Once this overall time budget runs out, the producer stops trying and reports failure, no matter what the retries number says.

A short pause between attempts

The producer does not retry instantly, back to back, because that could make a struggling broker even more overloaded. A setting called retry.backoff.ms controls how long the producer waits before trying again, giving the broker or the network a small chance to recover first.

A handful of terms carry the entire rest of this guide, so let us define each one clearly.

Why producers do not wait for one reply before sending the next

Waiting for a full round trip, request out and reply back, before sending the next request would be safe but very slow, especially across a network with any real distance or congestion. Kafka producers instead allow several requests to be outstanding on the same connection at once, which keeps the pipe full and throughput high. This is exactly what max.in.flight.requests.per.connection controls, and it is the subject of the next chapter.

This setting is the speed-versus-order dial for a Kafka producer.

The plain meaning

Max.in.flight.requests.per.connection sets the maximum number of produce requests a producer is allowed to have outstanding, meaning sent but not yet acknowledged, on a single connection to a given broker at the same time. If the limit is five, the producer can have up to five unanswered requests traveling to that broker before it must pause and wait for at least one reply.

Why more than one in-flight request helps

If the producer had to wait for each request to be fully acknowledged before sending the next one, every network round trip would sit idle while nothing new is being sent. Allowing several requests in flight means the producer keeps pushing new batches out while still waiting on earlier ones, which uses the network connection far more efficiently.

sequenceDiagram
    participant P as Producer
    participant B as Broker
    P->>B: Request 1
    P->>B: Request 2
    P->>B: Request 3
    B-->>P: Reply to Request 1
    P->>B: Request 4
    B-->>P: Reply to Request 2
    B-->>P: Reply to Request 3
    Note over P,B: Up to 3 requests in flight at once
        

This is the single most important interaction in this whole topic, so we slow down here.

How order can break

Imagine three batches, A, B and C, sent one after another to the same partition, with up to three requests allowed in flight. All three are sent quickly. Batch A happens to fail, perhaps because of a brief network blip, while batches B and C succeed and are written to the log. The producer retries batch A, and it now arrives and is written after B and C, even though your application sent it first.

sequenceDiagram
    participant P as Producer
    participant B as Broker Leader
    P->>B: Batch A
    P->>B: Batch B
    P->>B: Batch C
    B--x P: Reply to A is lost
    B->>P: Reply to B, written
    B->>P: Reply to C, written
    Note over P: A times out, producer retries it
    P->>B: Batch A again
    B->>P: Reply to A, written
    Note over B: Log order is now B, C, A instead of A, B, C
        

Why this matters

Kafka only promises that messages within one partition stay in the order they were written to the log, and this ordering problem is exactly what can break that promise if retries and multiple in-flight requests are combined carelessly. For data such as a sequence of state changes on an order, or a series of sensor readings, reading them out of order can lead to wrong conclusions, such as thinking an order was cancelled before it was ever placed, which is a much harder bug to trace than a simple crash.

Modern Kafka solves the ordering problem a smarter way, without forcing everyone back to one request at a time.

Stamping every batch with a sequence number

When the idempotent producer feature is enabled, which is the default in modern Kafka clients, every batch sent to a partition carries a producer ID and a strictly increasing sequence number, stamped automatically by the client library without any extra work from your application code. The broker checks this number before writing a batch to the log.

If a batch’s sequence number is not the next number the broker expects for that producer and partition, the broker holds off on writing it and waits, rather than writing it out of order. This means even if batch A’s retry physically arrives after batch C, the broker recognises that A must come first in the log and places it correctly, or rejects an attempt that would break order.

Why up to five in-flight requests is now considered safe

With idempotence switched on, Kafka allows max.in.flight.requests.per.connection to be set as high as five while still guaranteeing correct ordering, because the sequence-number check does the ordering work that used to depend entirely on sending one request at a time. This is why five, not one, is the modern default, giving you most of the throughput benefit of parallel requests without the old ordering risk.

It helps to see the older, riskier behaviour clearly, since some systems and older configurations still run this way.

Three delivery styles, side by side

The combination of retries, in-flight requests and idempotence produces a few distinct outcomes depending on how each setting is configured. Understanding these outcomes is the fastest way to reason correctly about a specific application’s needs.

Why this history still matters today

You may encounter older systems, tutorials, or configuration files written before idempotence became the default, where max.in.flight.requests.per.connection is explicitly set to 1. Recognising this pattern helps you understand that the original author was protecting message order the hard way, and it is often safe, and faster, to modernise that configuration once idempotence is confirmed to be enabled.

Let us trace one batch from creation to final confirmation, including a retry along the way.

A short, practical reference for setting these values sensibly.

These settings sit alongside other Kafka features that people sometimes confuse them with.

A balanced look at what these settings buy you, and what they cost.

Numbers you cannot see are numbers you cannot trust in production.

Patterns that work well with retries and in-flight requests, and habits that undermine them.

Seeing where these settings matter in practice makes the idea concrete.

Let us tie every idea together with one end-to-end story.

A ride-hailing company streams trip-status events, such as requested, driver assigned, and trip started, into a Kafka partition keyed by trip ID. During a period of mobile network instability in one city, several producer requests begin failing intermittently, and the on-call engineer needs the pipeline to stay both fast and correct without any manual intervention.

A short checklist of the errors that show up most often in real projects.

Quick, direct answers to the questions that come up most often.

It limits how many produce requests can be sent to a specific broker on one connection before at least one of them must be acknowledged. It does not limit the total number of messages a producer can be working with across all brokers and partitions.

It is a sensible default for most workloads when idempotence is enabled. Extremely latency-sensitive or ordering-critical pipelines occasionally choose 1 for extra caution, and very high-throughput pipelines rarely need to go above 5, since the setting is capped there for idempotent producers in any case.

Retries themselves do not cause loss; they exist to prevent it. Data loss happens when retries are disabled, or when the delivery.timeout.ms budget runs out before a message can be confirmed, in both of which cases the producer reports a failure your application can act on.

Retries caps the number of attempts. Delivery.timeout.ms caps the total elapsed time across all attempts combined. In modern Kafka, the time-based limit is usually what actually determines when a producer gives up.

Yes, and this remains a perfectly valid, safe combination. It guarantees strict ordering the traditional way, at the cost of some throughput, and works whether or not idempotence is enabled.

Not when idempotence is enabled, since the broker’s sequence-number check removes duplicates regardless of how many requests are in flight. Without idempotence, a higher in-flight limit does increase the practical chance of both duplicates and reordering during retries.

No, it is purely a producer setting governing how many produce requests can be outstanding on a connection. Consumers have their own separate settings for how much data they fetch at once and how they manage parallel processing.

Yes. Even with a single partition, transient network failures can still trigger retries, and multiple in-flight requests can still arrive out of order relative to each other, so the same reasoning about idempotence and the in-flight limit still applies.

Individual messages that hit a retriable failure will take longer to be confirmed, since they must wait through the retry backoff and a second attempt. Overall producer throughput usually stays healthy as long as failures remain occasional, but a sustained high retry rate is a sign of a deeper problem worth investigating, as covered in the monitoring chapter.

Knowing which errors the producer will retry, and which it will not, helps you reason about failure behaviour with confidence.

Why the distinction exists

Not every failure means the same thing. Some failures are about timing and temporary conditions, and trying again a moment later is likely to succeed. Other failures are about the request itself being fundamentally wrong, and no amount of waiting will fix that. Kafka’s client library classifies errors into these two buckets so that it can behave sensibly without your application having to inspect every failure by hand, sparing every team from having to reinvent this classification logic on their own.

Max.in.flight.requests.per.connection is scoped more narrowly than many people first assume.

Per connection, not per producer

The limit applies separately to each connection a producer holds. A producer that is actively sending to three different broker leaders, because its messages span several partitions led by different brokers, can have up to the configured limit of in-flight requests on each of those three connections independently. The total number of in-flight requests across the whole producer can therefore be higher than the configured number itself, since that number is a per-connection ceiling, not a producer-wide one.

flowchart LR
    P[Producer] -->|up to 5 in flight| B1[Broker 1, Partition Leader]
    P -->|up to 5 in flight| B2[Broker 2, Partition Leader]
    P -->|up to 5 in flight| B3[Broker 3, Partition Leader]
        

Why this matters for cluster-wide throughput

Because the limit is per connection, a producer that spreads its writes across many partitions on many different brokers naturally achieves high overall throughput, even with a conservative in-flight limit like 5 on each individual connection. This is one reason Kafka’s designers felt comfortable keeping the default modest rather than raising it further, since real throughput usually comes from spreading load across brokers and partitions, not from stacking an enormous number of requests onto one connection.

This also explains why doubling a topic’s partition count, and letting those partitions spread across more brokers, often improves producer throughput more reliably than raising max.in.flight.requests.per.connection ever could. The in-flight setting optimises how busy one connection stays, while partition count and distribution decide how many connections, and therefore how much total in-flight capacity, a producer has available in the first place.

Connection pooling and reconnection behaviour

Producers do not open and close a connection for every batch. Instead, a connection to a given broker is opened once and reused for as long as it stays healthy, which is part of why in-flight requests are tracked per connection rather than per batch or per request. If a connection is lost entirely, perhaps because a broker restarted, the producer reconnects and its in-flight tracking for that connection resets, with any batches that were genuinely unacknowledged at that moment becoming candidates for the normal retry process described earlier in this guide.

This reconnection behaviour is also why a brief broker restart, such as during a routine rolling upgrade of a Kafka cluster, is usually invisible to a well-configured producer. The connection drops, a small number of in-flight batches become candidates for retry, the producer reconnects automatically, and those batches are resent and correctly sequenced once the broker is back, all without any error surfacing to your application code.

Retries and in-flight limits do not act alone. They sit alongside a memory buffer that decides what happens when a producer cannot keep up.

Where messages wait before they are even batched

Before a message becomes part of a batch heading toward a broker, it sits briefly in an in-memory buffer inside the producer, sized by a setting called buffer.memory. This buffer absorbs short bursts of messages, smoothing out the difference between how fast your application produces data and how fast the network and broker can currently accept it.

When retries and a full in-flight queue combine during a period of broker trouble, this buffer can start filling up faster than it empties, since batches that would normally be cleared out by a quick acknowledgement are instead stuck waiting on retries. If the buffer fills completely, the producer applies backpressure, pausing your application’s calls to send until space frees up, rather than silently dropping data or growing memory without bound.

flowchart TD
    A[Application calls send] --> B[Message enters the buffer]
    B --> C[Batched and sent, becomes in flight]
    C --> D{Acknowledged quickly?}
    D -- Yes --> E[Batch leaves the buffer, space freed]
    D -- No, retry needed --> F[Batch stays in buffer longer]
    F --> G{Buffer full?}
    G -- Yes --> H[New send calls block briefly, backpressure applied]
    G -- No --> A
        

A setting that controls how long the application waits

The max.block.ms setting controls how long a call to send is allowed to pause waiting for buffer space before it gives up and raises an exception to your application. This gives you a predictable upper bound on how long your own code can be blocked, even during a sustained period of broker trouble combined with heavy retrying.

Sizing the buffer with retries in mind

Teams that tune buffer.memory often think only about steady-state throughput, sizing it just large enough to hold a normal burst of messages. It is worth also thinking about a degraded scenario: if retries are keeping batches in flight for longer than usual because of a struggling broker, the buffer needs enough headroom to absorb that extra dwell time without immediately hitting its limit and blocking the application. A buffer sized only for the happy path can turn an ordinary transient failure into a much more visible slowdown across your whole application.

Seeing how other systems handle retries and pipelining sharpens the picture of what makes Kafka’s approach distinctive.

Retries give a Kafka producer the resilience to survive transient network and broker failures automatically, and max.in.flight.requests.per.connection gives it the throughput to keep a connection busy by sending several requests at once. On their own, combining a high in-flight limit with retries opens a real risk of reordered messages within a partition. Idempotence closes that gap by stamping every batch with a sequence number the broker can use to keep the log correctly ordered and duplicate-free, which is why modern Kafka can safely default to five in-flight requests instead of the older, slower value of one.