Before learning the solution, it helps to feel the risk it protects against.

Machines fail, and it is not rare

Kafka stores data in partitions, which are ordered, append-only sequences of events that live on individual machines called brokers. Every broker is, at the end of the day, a physical or virtual computer, and computers fail. Disks wear out, power supplies die, network cables get unplugged, and software occasionally crashes. In a large cluster with many brokers, some kind of failure is not a rare, unlucky event, it is something that will eventually happen as a matter of routine operation.

If a partition’s data existed on only one broker, a single failure would mean that data is gone, along with every producer trying to write to it and every consumer trying to read from it. For a system that businesses rely on to move orders, payments, and critical alerts, this level of fragility is unacceptable.

The answer: keep copies, and organise them

Kafka’s answer to this risk is to keep multiple copies of every partition on different brokers, and to organise those copies so that everyone, including producers, consumers, and the brokers themselves, always agrees on which copy is currently in charge. This is where the ideas of leader and follower partitions come in.

The core idea in one sentence, then in plain words.

A short definition

When a Kafka partition is copied onto several brokers, each copy is called a replica. Among these replicas, exactly one is chosen to be the leader, and the rest are called followers. The leader is the only replica that handles requests from producers and consumers directly. The followers do nothing but quietly copy every new event the leader receives, staying ready to take over if they are ever needed.

In everyday terms, the leader is the active, working copy of a partition, and the followers are standby copies, always watching and updating themselves, but never doing any of the actual work unless called upon.

Leadership belongs to the partition, not the broker

It is important to notice that “leader” and “follower” describe roles for a specific partition, not a permanent label for an entire broker. A single broker might be the leader for some partitions while simultaneously being a follower for other partitions belonging to different topics. Leadership is spread out across the cluster so that no one broker carries the entire workload alone.

A closer look at what makes the leader special.

Handling every read and write

The leader replica is the single point of contact for a partition. When a producer sends a new event, it goes to the leader. When a consumer wants to read events, it also asks the leader. This might sound like it creates a bottleneck, but remember that leadership is spread across many partitions and many brokers, so the overall cluster can still handle enormous amounts of traffic in parallel, even though any single partition only has one active leader at a time.

Deciding the official order of events

Because all writes to a partition go through one leader, the leader is also what decides the exact order in which new events are appended. This is precisely why Kafka can guarantee strict ordering within a single partition. If multiple replicas could all accept writes at once, there would be no single, agreed-upon sequence, and different readers might see events in different orders, which would break many of the guarantees Kafka is built to provide.

Quiet, essential work that most people never see happening.

Constantly copying the leader

A follower’s only job is to continuously fetch new events from the leader and append them to its own copy of the partition, in the exact same order the leader received them. This process is called replication. A healthy follower is always trying to stay as close as possible to whatever the leader currently has, so that it could take over with almost no missing data if it ever needed to.

Followers normally stay out of the spotlight

In the most common Kafka setup, followers do not respond to ordinary producer or consumer requests at all. They exist purely as a safety net, quietly replicating in the background. Some Kafka configurations do allow certain read requests to be served by nearby followers for performance reasons in specific situations, but the default and most common behaviour is that all reads and writes go through the leader alone.

sequenceDiagram
  participant P as Producer
  participant L as Leader Replica
  participant F1 as Follower Replica 1
  participant F2 as Follower Replica 2
  P->>L: Send new event
  L->>F1: Replicate event
  L->>F2: Replicate event
  F1-->>L: Copy saved
  F2-->>L: Copy saved
  L-->>P: Acknowledge success
        

Not every follower is equally ready to take over at any given moment.

What “in-sync” means

A follower that has recently and reliably kept up with the leader’s latest events is called an in-sync replica. Kafka keeps a running list of which replicas currently qualify as in-sync for each partition, often called the in-sync replica set, or ISR for short. A follower can temporarily fall out of this list if it becomes slow, disconnected, or otherwise unable to keep pace with the leader.

Why this list matters so much

Only replicas in the in-sync list are considered safe candidates to become the new leader if the current leader fails. Promoting a replica that has fallen far behind would risk silently losing recent events, since that replica’s copy of the partition would be missing data the old leader already had. By only promoting from the in-sync list, Kafka protects against this kind of silent data loss during a leadership change.

The moment where all this preparation actually pays off.

flowchart LR
  subgraph Before[Before Failure]
    L1[Leader: Broker 1]
    F1a[Follower: Broker 2]
    F1b[Follower: Broker 3]
  end
  subgraph After[After Failure]
    L2[New Leader: Broker 2]
    F2a[Follower: Broker 3]
    X[Broker 1 unreachable]
  end
  Before -.failure & promotion.-> After
        

Leadership decisions are not made randomly; they follow a coordinated process.

The role of the controller

Every Kafka cluster has a small group of nodes, called controllers, responsible for tracking the overall state of the cluster, including which broker leads which partition. When a broker fails, the controllers are responsible for noticing this and choosing a replacement leader from the eligible in-sync replicas, then informing every other broker of the change.

In modern Kafka clusters, this controller responsibility is handled by Kafka’s own built-in coordination system, using a consensus approach that lets a small group of controller nodes reliably agree on cluster-wide facts like partition leadership, even if some of them are temporarily unavailable.

Preferred leaders and balance

When a topic’s partitions are first created, Kafka assigns a preferred leader for each one, aiming to spread leadership duties evenly across all the brokers in the cluster. Over time, as failures and recoveries happen, leadership can drift away from this ideal, evenly balanced arrangement, with some brokers quietly ending up as leader for more partitions than others. Kafka provides tools to rebalance leadership back toward this preferred, even distribution when needed.

The leader-follower system directly shapes how safely a producer’s data is saved.

Three common levels of care

When a producer sends an event to the leader, it can choose how much confirmation it wants before considering the send successful. This setting is often called the acknowledgement level. At the lightest level, the producer does not wait for any confirmation at all, which is fast but risky if the leader fails immediately afterward. At a middle level, the producer waits until the leader itself has saved the event. At the strictest level, the producer waits until enough in-sync followers have also confirmed they have saved a copy.

Why this connects back to leaders and followers

This whole system of confirmation only works because followers are constantly replicating from the leader. If a producer asks for the strictest level of safety, it is really asking Kafka to confirm that the leader-follower replication has already happened for that specific event, before telling the producer it can move on.

A setting that gives teams direct control over the safety versus availability trade-off.

Setting a floor for safety

A topic can be configured with a minimum in-sync replicas setting, which defines how many replicas, including the leader, must be in-sync for the leader to keep accepting writes with the strictest acknowledgement level. If the number of in-sync replicas ever drops below this floor, perhaps because too many followers have fallen behind or failed, the leader will refuse writes at that safety level rather than risk silently under-protecting the data.

Balancing safety and availability

Setting this minimum too low weakens the safety Kafka can guarantee, since fewer confirmed copies exist before a write is accepted. Setting it too high, close to or equal to the full replication factor, can make the partition unavailable for strict writes more often, since it takes very little trouble for the in-sync count to dip below a high threshold. Most teams settle on a middle ground, commonly requiring two in-sync replicas out of three total.

Deciding how many total copies of a partition to keep.

Common choices

A replication factor of one means no protection at all, since there is only a single copy with no followers. This is sometimes used for quick local testing but is considered unsafe for any real production use. A replication factor of three, meaning one leader and two followers, is the most common choice in production, offering solid protection while keeping storage and network costs reasonable. Some highly critical systems may choose even higher replication factors for extra safety, at the cost of additional storage and network traffic.

No design is free of cost, even one this well proven.

A healthy-looking cluster can still be quietly at risk if replication is not watched closely.

Practical lessons for keeping leaders and followers working safely.

Best practices

Mistakes beginners often make

No. In standard Kafka behaviour, only the leader accepts writes. Followers exist to replicate what the leader has already accepted, not to accept new writes themselves.

Yes. Leadership can shift due to failures, maintenance, or rebalancing, and Kafka is designed to handle this smoothly and automatically, without requiring manual action for every single change.

If every replica, including all followers, becomes unavailable at the same time, the partition has no leader and cannot serve reads or writes until at least one replica comes back online, since there is no remaining copy to promote.

It adds some extra network and storage overhead, since more copies must be maintained, but Kafka is designed to handle this efficiently, and the added safety is generally well worth the modest cost for important data.

Under normal, safe configurations, only in-sync replicas are eligible to become leader, precisely to avoid promoting a replica that is missing recent data.

This typically happens within seconds, since the coordination system is specifically designed to detect failures and promote a new leader quickly, minimising disruption to producers and consumers.

No, not with standard Kafka client libraries. They are built to automatically detect when leadership has changed and redirect their requests to the new leader without any manual reconfiguration.

Leader and follower partitions are how Kafka protects data against the everyday reality that machines eventually fail. Every partition is copied across several brokers as replicas, with exactly one replica chosen as the leader to handle all reads and writes, while the rest quietly replicate as followers, ready to step in. Kafka tracks which followers are truly caught up as in-sync replicas, and only promotes from that trusted group if the leader ever fails, protecting against silent data loss during the handover. Settings like acknowledgement level, replication factor, and minimum in-sync replicas let teams tune exactly how much safety they want, at the cost of a little extra time or resources. Together, this system lets Kafka keep working smoothly and automatically, even as individual machines come and go.