Every shared system needs one clear leader and a fair way to choose that leader.

Elasticsearch is a distributed search and analytics engine. “Distributed” means the work is spread across many computers, each called a node, and the group of nodes working together is called a cluster. Spreading work makes the system faster and safer, because no single computer failure has to stop everything. But it creates a new question: when many nodes hold pieces of the same picture, who keeps the master copy of the plan?

The plan is called the cluster state. It records which indices exist, what their fields look like, which nodes are members, and which node holds which shard. If two nodes each believed they were in charge and each changed the plan differently, the cluster would fall into what engineers call split brain. Different parts of the cluster would accept different writes, and later it would be nearly impossible to merge them back without losing data.

Elasticsearch prevents split brain by using a majority vote. A node can act as the leader only while more than half of a specially chosen group of nodes agree that it is the leader. Two groups cannot both hold a majority of the same set, so two leaders can never exist at once. The voting-only node is a clever helper inside this majority system.

The definition, the vocabulary, and the single idea that ties them together.

A voting-only node is a master-eligible node that takes part in master elections and in confirming cluster changes, but is never allowed to become the elected master itself. In Elasticsearch it is created by giving a node both the master role and the voting_only role.

Read that sentence slowly. The node is “master-eligible”, so it sits in the club of nodes whose votes count. The node is “voting-only”, so it has promised never to run for leader. Its whole value is its vote. It can be very small and cheap, yet it can be the vote that gives a majority to one side of a broken network.

Words you need to know

A voting-only node only makes sense once you see the other jobs a node can hold.

In Elasticsearch a node’s job is decided by its roles. A role is a label, set in the node’s configuration under the setting node.roles, that tells the node what work to accept. One node can hold many roles, or just one. Small clusters often give every node many roles. Large production clusters usually separate them so each machine can be sized for one kind of work.

Reading roles in the node list

Elasticsearch’s cat nodes API prints a short letter code for each node’s roles. The letter m stands for master-eligible, d for data, i for ingest, and v for voting-only. An asterisk in the master column marks the node currently elected as leader. If you see mv next to a small node, you are looking at a dedicated voting-only node.

To understand why a node might promise not to lead, see what leading involves.

The elected master is the one node that is allowed to change the cluster state. Everyone else may read the plan, but only the master writes it. That is what keeps the plan consistent. The master’s work is light compared with a data node’s, yet it is critical, because the whole cluster waits for it.

What happens when there is no master

Without an elected master, the cluster cannot change its plan. Searches and indexing on shards that are already in place may continue for a while, because data nodes know where their shards are. But you cannot create an index, cannot recover a lost shard copy, and cannot change a mapping. The cluster reports that no master is available, and many operations fail with a master-not-discovered error. A cluster without a master is technically alive but cannot heal itself.

Why some nodes should not lead

The master must stay responsive because every cluster-state change goes through it. A leader that is overloaded, far away on a slow link, or running on a tiny machine will slow down the entire cluster. So you want a leader on a healthy machine near the other nodes. Yet you may still want votes from machines that cannot meet that standard, such as a small machine in a distant third location. Voting-only nodes let you separate “counts as a vote” from “can be the leader”.

The whole design rests on one small piece of arithmetic.

A quorum is more than half of the voters. If there are three voters, the quorum is two. If there are five voters, the quorum is three. The rule “always need more than half” guarantees that any two quorums must share at least one voter. Because that shared voter would never vote for two different leaders at the same term, two leaders cannot both be confirmed at once.

Why two voters are no better than one

Look at the table again. With two voters you need both of them, so losing either one freezes the decision-making. Two voters are actually worse than one, because you have doubled the chances that something fails while gaining no extra tolerance. The same is true of four compared with three. Even numbers waste a voter. This is why Elasticsearch keeps the count of voters odd, and may quietly leave one master-eligible node out of the voting configuration when the total would be even.

Where the voting-only node fits into the arithmetic

A voting-only node counts as a full voter in this arithmetic. Its vote is worth exactly as much as any other master-eligible node’s vote. The only difference is that it never asks for votes for itself. So in a cluster with two electable nodes and one voting-only node, there are three voters, the quorum is two, and the cluster can lose any one of the three and carry on.

The voting-only node exists because earlier designs were hard to run safely.

Early Elasticsearch versions used a discovery system that relied on a human to tell every node how many master-eligible nodes existed. The administrator had to set a number called the minimum master nodes. If the number was wrong, or if the cluster grew and the number was not updated, split brain could occur and data could be lost. Many production incidents came from this single setting.

The new system solved the counting problem, but a physical problem remained. Real companies often run in two data centres or two cloud zones, and adding a full third master-eligible node in a third place can be expensive or awkward. The voting-only role answers that exact situation: the third site needs to hold a vote, not a heavy workload.

Inside the cluster, elections follow a careful, polite protocol.

When a cluster starts, or when the current master disappears, master-eligible nodes begin looking for a leader. A node that finds no leader waits a short random time, then decides whether to try to become one. The random delay is important: it stops every node from running for leader at the same instant and splitting the vote again and again.

flowchart TD
    A["Node notices no leader is available"] --> B["Node is electable, meaning master role without voting_only"]
    B -->|Yes| C["Wait a short random delay"]
    B -->|No, voting-only| V["Stay quiet and wait to be asked for a vote"]
    C --> D["Ask other master-eligible nodes if they would vote for me"]
    D --> E{"Did a majority of the voting configuration say yes?"}
    E -->|Yes| F["Become elected master and publish first cluster state"]
    E -->|No| G["Back off and try again later"]
    V --> H["Voting-only node answers requests using the same rules as everyone else"]
    H --> D
        

Step by step

First, an electable node sends a lightweight check to peers, sometimes described as a pre-vote, to learn whether an election could succeed before it disturbs the cluster. If enough peers answer positively, it starts a real election in a new numbered round called a term. Each voter gives at most one vote in each term. The candidate that collects a majority of the voting configuration wins.

A voter only supports a candidate whose knowledge of the cluster is at least as fresh as its own. This protects committed changes from being forgotten. A voting-only node applies precisely the same tests as any other voter. It stores the same information about the last committed cluster state and refuses candidates who would roll history backwards.

Timing settings you may meet

A handful of settings control how patient nodes are. The election initial timeout, back-off time, maximum timeout and election duration decide how long a node waits and how quickly it retries. Their defaults are tuned for ordinary networks and rarely need change. The documentation warns that lowering them on a slow or unreliable network can cause repeated failed elections, so leave them alone unless you have measured a real problem.

A clear list of duties makes the design easy to remember.

It still holds a copy of the plan

A voting-only node is not an empty shell. It receives every cluster state update and writes it to its own disk, just like any other master-eligible node. It has to, because a voter that cannot remember the last committed decision cannot safely vote. This is why even the smallest tiebreaker needs a reliable local disk that survives restarts, and enough memory to hold the cluster state comfortably.

It carries the same fault detection duties

Nodes in the cluster constantly check each other. The elected master checks that followers are alive, and followers check that the master is alive. A voting-only node participates in this checking as a follower. If it stops hearing from the master, it helps the cluster notice that the master has gone.

Elections are rare, but confirming changes happens constantly, and it needs votes every time.

Elections get most of the attention, but they are rare events. Most of the time the cluster has a stable master, and the daily work of the voting configuration is confirming changes. Every time the master wants to change the plan, for example after an index is created, it must publish the new cluster state and wait until a quorum of voters has accepted it.

sequenceDiagram
    participant M as Elected Master
    participant A as Electable Node
    participant T as Voting-only Node
    participant D as Data Nodes
    M->>A: Publish new cluster state
    M->>T: Publish new cluster state
    M->>D: Publish new cluster state
    A-->>M: Accepted and saved
    T-->>M: Accepted and saved
    M->>M: Quorum reached, change is committed
    M->>A: Commit
    M->>T: Commit
    M->>D: Commit
    D-->>M: Applied
        

Two phases: accept, then commit

In the first phase the master sends the proposed new plan. Each voter writes it to disk and replies that it has accepted it. Only when a majority of the voting configuration has replied does the master send a commit message, telling everyone the change is now official. This two-phase approach ensures that a change is never considered official unless a majority already has a durable copy.

Why this makes the tiebreaker important every day

Suppose a two-zone cluster has one electable node in each zone and a voting-only tiebreaker in a third place. When the second zone goes offline, the first zone plus the tiebreaker still form a majority of three, so cluster changes can continue. But note that every change must now be accepted by the tiebreaker as well. If the link to the tiebreaker is slow, cluster state changes become slow. A tiebreaker is small in size, but not unimportant in networking terms.

The configuration is short, but the rules around it matter.

You create a voting-only node by editing its configuration file and setting its roles list. A dedicated voting-only node carries exactly two roles: master and voting_only. Nothing else. Because it holds no data and serves no searches, it can run on a small machine.

The role must be paired with master

The voting-only role has no meaning on its own. It restricts a master-eligible node, so it must always appear together with the master role. If you write voting_only without master, Elasticsearch stops with a configuration error instead of guessing what you meant, which is a helpful safety feature.

Dedicated versus mixed

Mixed nodes, which also hold data, are sometimes used in small clusters to save cost. For example, a small cluster may have two electable master and data nodes, plus one data node that also carries the voting-only flag. That gives three voters with only three machines. However, a busy data node can become slow under heavy indexing or long garbage collection, and a slow voter can delay every cluster state change. In serious production clusters, most teams prefer a dedicated voting-only node for the tiebreaker.

The most famous use of a voting-only node is protecting a two-zone cluster.

Many organisations run in two places, such as two data centres or two availability zones. They want the cluster to survive the loss of either place. This is a hard problem for any voting system. If each zone holds one master-eligible node, there are two voters, and a quorum needs both. If one zone vanishes, the other zone has only half the votes, and no majority.

One workaround is to give one zone more master-eligible nodes than the other. That zone can then survive alone, but the cluster cannot survive the loss of that bigger zone. The complete solution is a third location that holds a single voting-only node.

graph TD
    subgraph ZA["Zone A"]
        A1["Electable master-eligible node"]
        A2["Data nodes"]
    end
    subgraph ZB["Zone B"]
        B1["Electable master-eligible node"]
        B2["Data nodes"]
    end
    subgraph ZC["Third location"]
        T["Dedicated voting-only tiebreaker"]
    end
    A1 --- B1
    A1 --- T
    B1 --- T
        

What each failure looks like

The fifth row is the most beautiful. During a network partition, both zones are alive and think the other is dead. In a system without a tiebreaker, both might try to lead. With the tiebreaker, only one side can gain the third vote, so only one side is allowed to continue changing the cluster state. The other side pauses safely instead of creating a second version of history.

Following a real outage makes the design stick in memory.

Let us follow a cluster called Orders that runs in two cloud zones, Zone A and Zone B, with a tiny voting-only tiebreaker in Zone C. Zone A currently holds the elected master.

Small does not mean careless.

A dedicated voting-only node does not index documents, run searches, or hold shards. Its main memory use is the cluster state and the normal overhead of running the Java virtual machine. So its CPU needs are modest and its disk needs are small. The exact figures depend on how big your cluster state is, which grows with the number of indices, shards and fields, so size the machine from your real cluster state rather than from a fixed rule.

Why the disk must be durable

A voter must remember its last commitments. If a tiebreaker restarts on a fresh empty disk, it has forgotten what it promised. Elasticsearch has protections, but losing the data path of master-eligible nodes can lead to an unrecoverable cluster in the worst cases. In containers and cloud instances, mount a persistent volume for the data path of every master-eligible node, including voting-only nodes.

Where you put the tiebreaker matters as much as how you configure it.

A good tiebreaker lives in a separate failure domain. That means it should not share power, network switches, or a cloud zone with either of the main zones. If the tiebreaker sits in Zone A next to an electable node, then losing Zone A removes two of three votes, and the design has protected nothing.

Latency limits

Elasticsearch documentation assumes master-eligible nodes can talk to each other reliably and quickly. When a leader publishes a change, it waits for a quorum of voters. If the tiebreaker is far away, every change waits for round trips to it in the one scenario where its vote is needed. A tiebreaker in a different city may be acceptable on a stable private link, but a tiebreaker on an unreliable internet path is dangerous.

Firewalls and discovery

The tiebreaker must be reachable on the cluster’s transport port from every other master-eligible node. New nodes find the cluster through a list of seed hosts, and the tiebreaker should appear there if it is one of the stable master-eligible nodes. Firewall mistakes are a frequent cause of one-directional connectivity, where the tiebreaker can reach others but they cannot reach it, producing confusing election failures.

Clusters change. Nodes join, leave and get replaced, and the voting list must follow safely.

The voting configuration is the list of nodes whose votes count. Elasticsearch adjusts it automatically as master-eligible nodes join and leave, so long as you make changes gently. It aims to keep the number of voters odd, and by default it can shrink the list when nodes depart, through a setting called auto shrink voting configuration. Because of this automation, you rarely edit the list by hand.

Adding a tiebreaker to a running cluster

To add a new voting-only node, start it with the master and voting_only roles and point it at the cluster’s seed hosts. When it joins, the cluster notices a new master-eligible node and can include it in the voting configuration. Then check the node list to confirm the roles, and check the cluster state’s coordination information to confirm it is a voter.

Removing or replacing master-eligible nodes

Removing master-eligible nodes carelessly is dangerous, because too many departures at once can leave the survivors without a majority of the old configuration. Elasticsearch offers the voting configuration exclusions API. You tell the cluster which node you are about to remove, the cluster moves the node out of the voting list first, and only then do you shut it down. After the removal, you clear the exclusion list.

The same small idea appears in very different environments.

A voting-only node is one option among several, and each layout has its place.

When a full third master is better

If you truly have three healthy zones with good connectivity, three full master-eligible nodes are simpler. Any of them can lead, and there is no special role to explain to new colleagues. The voting-only node shines when the third location is weaker, remote or expensive, so that you want a vote there but not a leader.

When a voting-only node is the wrong tool

A voting-only node will not add search capacity, will not add storage, and will not help a cluster that has only one location. It also does not replace replicas. Think of it as a specialist for coordination, not a general performance upgrade.

A tiebreaker holds a vote, and a vote can be abused, so protect it.

Because every voter influences which node becomes the leader, a compromised voting-only node is a real risk, even though it stores no index data. An attacker who controls a voter could disturb elections, delay cluster state changes or read the cluster state, which contains index names, mappings and node addresses. Treat the tiebreaker with the same discipline as any other master-eligible node.

Disk encryption on the tiebreaker is also sensible, since its data path contains cluster metadata. Finally, keep clear records of who may change the roles of master-eligible nodes. Changing a node’s role from voting-only to a full master, or the reverse, changes the election rules of the entire cluster.

You should know a voter is missing before the day you need it.

The tiebreaker’s value lies in being available at the worst moment, so you need to know quietly, in advance, whether it is healthy. Combine simple node-level monitoring with cluster-level checks.

Alerts worth having

Habits that keep clusters safe, and traps that catch careful teams.

Patterns worth copying

Anti-patterns to avoid

A fair look at what the voting-only role gives you and what it costs.

A checklist worth pinning near the operations desk.

Best practices

Common mistakes

When the cluster says it has no master, work through causes in a calm order.

A calm routine

Start with the simplest question: which master-eligible nodes can talk to each other right now? Draw the picture on paper, marking zones, links and roles. Then count votes against the voting configuration. Only after you understand the vote count should you look at timeouts and performance. Most “mystery” election problems turn out to be either a missing voter, a blocked network path, or a wrong role.

Short answers to doubts that come up again and again.

No. It votes in elections and helps confirm cluster changes, but it never stands as a candidate, so it cannot be elected.

Not when it is dedicated. It stores the cluster state, which describes the cluster. If you give it a data role as well, it will store shards, but then it is no longer a light tiebreaker.

No. At least one electable master-eligible node is needed for any leader to exist, and for resilience you should keep at least two.

It can help keep voter counts odd, but it does not protect against a zone failure since there is only one zone. Its real power appears in multi-location layouts.

It can be much smaller than a data node, but it must hold the whole cluster state in memory and keep a durable disk. Size it from your real cluster state, and revisit the size as the cluster grows.

The cluster continues as long as the remaining voters still form a majority. But you have lost your protection against the next failure, so restore it quickly.

A coordinating-only node routes search and indexing requests and has no vote. A voting-only node takes part in elections and cluster state commits and does not handle client requests unless you give it other roles.

Not in modern versions. Elasticsearch manages the voting configuration itself. Your job is to choose sensible numbers and locations of master-eligible nodes.

Yes. Larger clusters may use several, for example in a five-voter layout. Just remember that enough electable nodes must always remain.

Yes. It participates in cluster coordination, so it needs the same transport encryption, certificates, network rules and patching as any other master-eligible node.

Check the cat nodes API. Master-eligible nodes show the letter for master, and voting-only nodes also show the letter v in their role column.

Many managed services add tiebreaker nodes automatically for multi-zone deployments. You should still learn the concept, so you understand what the provider is protecting and what your own responsibilities are, such as replicas and allocation.

A voting-only node is a master-eligible node that promises never to lead. It exists so that a cluster can gain the safety of an extra vote, especially in a third failure domain, without paying for a full-sized master. It votes in elections, accepts and commits cluster state changes, and keeps a copy of the cluster state, yet it never asks to be the boss. Combined with two electable master-eligible nodes in two main zones, it forms the standard layout that survives the loss of any one location and prevents split brain. Its success depends on good placement, a stable network, a durable disk, tight security and regular rehearsal.