To understand kube-proxy, first understand the headache it cures.

A quick refresher on pods

Kubernetes is a system that runs your applications across many computers. A node is one of those computers. A pod is the smallest unit Kubernetes manages, a small wrapper around one or more containers, and each pod receives its own network address, called an IP address.

Pods are disposable by design. If a pod crashes, is updated, or its node fails, Kubernetes does not repair it. It creates a fresh replacement, and the replacement gets a brand new address.

Why this breaks simple connections

Suppose your website needs to call a payment program. The payment program runs as three pods, so there are three addresses. If your website wrote those addresses down, they would go out of date within hours, maybe minutes. Every deployment, every crash and every scaling event would make the notes wrong.

There is a second problem. With three copies running, who decides which copy receives each request? Somebody must spread the work fairly and avoid sending visitors to a pod that is broken or still starting.

Kubernetes gives a two-part answer

The first part is the Service: a steady name and a steady virtual address placed in front of a group of pods. The second part is the machinery that makes that steady address actually work on every machine. That machinery is kube-proxy.

kube-proxy is a network agent that runs on each node and keeps that node’s traffic rules in sync with Kubernetes Services.

Let us unpack the sentence. An agent is a small helper program that does a job on behalf of a bigger system. Traffic rules are instructions stored inside the machine’s operating system that say “packets going to this address should be redirected to that address.” In sync means the rules always match the current list of Services and healthy pods.

So kube-proxy does not carry your data itself in the common modes. Instead, it writes a set of rules into the Linux kernel, the core of the operating system, and the kernel does the fast work of redirecting packets. It is more like a traffic planner who paints the road signs than a police officer who stands in the road directing every car.

The three things kube-proxy always does

Why it is called a proxy

A proxy is a middleman that stands between a caller and the real destination. In the earliest versions of Kubernetes, kube-proxy really did sit in the middle, receiving each connection and forwarding it by itself. Later designs moved the forwarding into the kernel for speed, but the original name stuck. Today the name is slightly misleading, because in the modern modes kube-proxy mostly programs rules rather than proxying traffic.

Half of the confusion about kube-proxy comes from mixing it up with its neighbours.

Kubernetes networking involves many cooperating parts, and beginners often blame or credit the wrong one. Here is a clear map of who does what.

The relay race view

Think of a request as a baton in a relay race. CoreDNS runs the first leg by converting a name into a Service address. kube-proxy runs the second leg, using node rules to swap that Service address for a real pod address. The CNI plugin runs the final leg, carrying the packet across the network to that pod. Each runner is essential, and none replaces the others.

flowchart LR
  A[App asks for name payments] --> B[CoreDNS returns Service address]
  B --> C[Packet leaves the pod]
  C --> D[kube-proxy rules swap Service address for pod address]
  D --> E[CNI plugin carries packet to the target pod]
  E --> F[Payment pod answers]
        

A cluster has two halves: a control plane that decides and worker nodes that work. kube-proxy sits with the workers.

The control plane is the brain of the cluster. It includes the API server, the front desk that every request passes through, and etcd, the reliable database that remembers the state of everything. The worker nodes are the machines that run your pods.

kube-proxy runs on every node. In most clusters it is deployed as a DaemonSet, a Kubernetes object that places exactly one copy of a pod on each node and automatically adds one to any new node. In some setups it runs directly as a system service instead. Either way, the goal is identical: one kube-proxy per machine.

flowchart TB
  subgraph CP[Control Plane]
    API[API Server]
    ETCD[(etcd)]
  end
  subgraph N1[Worker Node 1]
    KP1[kube-proxy]
    RULES1[Kernel network rules]
    PODA[Pods]
  end
  subgraph N2[Worker Node 2]
    KP2[kube-proxy]
    RULES2[Kernel network rules]
    PODB[Pods]
  end
  API  ETCD
  API -->|Service and endpoint updates| KP1
  API -->|Service and endpoint updates| KP2
  KP1 --> RULES1
  KP2 --> RULES2
        

Why every node needs its own copy

A pod on any node may call any Service at any moment. The redirect must happen on the very node where the call starts, because that is the first place the packet appears. So every node must know about every Service. That is why kube-proxy is distributed rather than centralised, and why a problem in one node’s kube-proxy usually affects only the pods on that node.

Eight simple terms make the rest of the story easy to follow.

The virtual address mystery

A ClusterIP is a strange thing. No machine has it, no network card answers for it, and you cannot find it by looking at any pod. It exists only as a promise written into rules. When a packet is addressed to it, the rules on the sending node rewrite the destination to a real pod before the packet goes anywhere. If the rules were missing, the packet would simply wander off and be dropped.

Why EndpointSlices matter

Older Kubernetes stored all endpoints of a Service in one big record. With thousands of pods, every small change forced the whole giant record to be sent to every node. EndpointSlices split the list into small chunks, so a change to one pod triggers only a tiny update. kube-proxy watches these slices, and this is one reason modern clusters scale so much better.

Watch, translate, program, and repeat forever. That is the entire heartbeat.

kube-proxy follows the same idea that drives all of Kubernetes: desired state versus actual state. The desired state says “Service payments should reach these three healthy pods.” The actual state is whatever rules currently exist on the node. kube-proxy keeps comparing the two and fixes any gap, again and again.

sequenceDiagram
  participant U as You
  participant A as API Server
  participant K as kube-proxy on Node
  participant N as Node Kernel
  U->>A: Create Service and pods
  A-->>K: Notify of new Service
  A-->>K: Notify of endpoint list
  K->>K: Build rules from Services and endpoints
  K->>N: Install or update rules
  Note over K,N: A pod later crashes
  A-->>K: Endpoint list changed
  K->>N: Remove dead pod from rules
        

Why only ready endpoints are used

A pod that has just started may not be ready to serve. Kubernetes uses a readiness check to tell whether a pod can take visitors. kube-proxy includes only ready pods in its rules, so a pod that is still warming up, or one that is unhealthy, receives no traffic until it recovers.

Let us follow one message from a website pod to a payment pod, step by step.

flowchart LR
  W[Website pod] -->|to Service address| R[Node kernel rules]
  R -->|rewrite to pod A| PA[Payment pod A]
  R -.->|other calls| PB[Payment pod B]
  R -.->|other calls| PC[Payment pod C]
  PA -->|reply| R
  R -->|reply appears from Service| W
        

Different Service types open different doors, and kube-proxy builds the rules behind each door.

flowchart TB
  EXT[Outside user] --> LB[Cloud load balancer]
  LB --> NP[NodePort on a node]
  NP --> CIP[ClusterIP rules by kube-proxy]
  INT[Pod inside cluster] --> CIP
  CIP --> P1[Pod 1]
  CIP --> P2[Pod 2]
        

kube-proxy can speak more than one dialect of Linux networking. Each dialect is called a mode.

A mode decides which kernel feature kube-proxy uses to hold its rules. The end result, redirecting Service traffic to pods, is the same. What differs is speed, scale and complexity. Choosing the mode is a decision made when the cluster is set up or upgraded.

The mode most people meet first, and the easiest one to picture.

iptables is a Linux feature that lets an administrator define lists of rules for network packets. Rules are grouped into chains, which work like checklists. A packet is compared against the entries of a chain, and the first matching entry decides what happens.

How kube-proxy arranges the checklists

In this mode, kube-proxy builds a small hierarchy. A main entry chain lists every Service address. Each Service then has its own chain that chooses among its pods, and each pod has a tiny chain that performs the actual address rewrite. So a packet first finds its Service, then its pod choice, then its rewrite.

flowchart TB
  IN[Packet to Service address] --> ENTRY[Main entry chain lists all Services]
  ENTRY --> SVC[Chain for this Service]
  SVC -->|one third chance| E1[Pod A rewrite]
  SVC -->|one half of the rest| E2[Pod B rewrite]
  SVC -->|remaining| E3[Pod C rewrite]
        

How it spreads the load

iptables has no built-in idea of round robin. Instead, kube-proxy uses probabilities. With three pods, the first rule says “take this pod one time in three.” If that roll fails, the second rule says “take this one half of the time,” and the last rule takes whatever remains. The mathematics works out so that each pod receives an equal share over many connections.

A mode built on a real load balancer that lives inside the Linux kernel.

IPVS stands for IP Virtual Server. It is a load balancing feature in the Linux kernel that was designed from the start to spread connections across a group of servers. Instead of long checklists, it stores Services in a hash table, a structure that finds an entry almost instantly no matter how many entries exist.

How kube-proxy uses it

kube-proxy creates one virtual server in IPVS for each Service address and port, and adds each ready pod as a real server behind it. When a packet arrives for the Service, the kernel finds the virtual server by direct lookup and picks a pod using the chosen scheduling method. A few helper rules still exist in iptables or in address sets for supporting jobs such as masquerading.

Linux networking is moving forward, and Kubernetes is following it.

nftables: the successor to iptables

nftables is the newer Linux packet filtering framework meant to replace iptables. It stores rules in structures such as maps and sets, which allow a lookup without walking a long checklist. The kube-proxy nftables mode brings this design to Services, giving good speed at large scale while keeping the familiar random-spread balancing approach. It became generally available in the Kubernetes 1.33 release, and many distributions are moving toward it over time.

Replacing kube-proxy altogether

Some network plugins can take over the job of kube-proxy entirely. The best-known example is Cilium, which uses eBPF, a technology that lets small safe programs run inside the Linux kernel. With eBPF, Service lookups can happen at a very early point in packet handling, skipping much of the traditional rule machinery. Clusters that do this are said to run in kube-proxy replacement mode, and kube-proxy is simply not installed.

flowchart LR
  A[Standard path] --> B[kube-proxy writes rules]
  B --> C[Kernel rules redirect packets]
  D[Replacement path] --> E[CNI plugin with eBPF]
  E --> F[Kernel programs redirect packets]
        

Sometimes “spread traffic evenly everywhere” is not what you want. Kubernetes offers a few dials.

Session affinity: sticking to one pod

Some programs remember a visitor between requests, for example a shopping cart kept in a pod’s memory. If the next request lands on a different pod, the cart is gone. Session affinity, set to client address, tells kube-proxy to keep sending the same caller to the same pod for a limited time. It is a workaround, and the healthier long-term design is to keep such memory outside the pod.

External traffic policy

When outside traffic enters through a NodePort or load balancer, the node that receives it may not host a pod of the Service. Two choices exist.

With the Local choice, the cloud load balancer needs to know which nodes actually hold a pod. Kubernetes assigns a special health check port for this, and kube-proxy answers on it, saying “yes, I have a local pod” or “no, skip me.” That is one more small job of kube-proxy that beginners rarely notice.

Internal traffic policy and topology hints

Similar ideas exist for traffic inside the cluster. An internal policy can keep calls on the same node when possible. Topology-aware routing can prefer pods in the same availability zone as the caller. Staying close reduces delay and, on many clouds, cuts the cost of data crossing zones.

A few quirks explain most of the strange bugs people report.

Conntrack: the kernel’s notebook

After the kernel rewrites a packet’s destination, it must remember that translation so replies can be reversed. It writes an entry in its connection tracking table, called conntrack. The table has a size limit. On busy nodes handling huge numbers of short connections, it can fill up, and new connections may then fail in confusing ways.

flowchart TB
  A[New connection arrives] --> B{Conntrack has room?}
  B -->|Yes| C[Entry created and packet forwarded]
  B -->|No| D[Packet dropped]
  D --> E[User sees timeouts]
        

How kube-proxy is set up, kept alive and upgraded.

How it is configured

kube-proxy reads a configuration document, usually stored in a ConfigMap, which is a Kubernetes object for holding settings separately from programs. Typical choices include the proxy mode, how often rules are refreshed, the range of pod addresses in the cluster and the balancing method for IPVS. Changing these settings normally requires restarting the kube-proxy pods so that they pick up the new values.

Health and metrics endpoints

Each kube-proxy exposes two small network endpoints for operators. A health endpoint, by default on port 10256, reports whether the agent is working and up to date. A metrics endpoint, by default on port 10249, publishes numbers about its activity in a format that monitoring tools such as Prometheus can collect.

Logging

kube-proxy writes logs describing which mode it selected, which Services it processed and any errors while applying rules. Because pods are replaced, these logs should be shipped to a central logging system so that they survive restarts. Raising the log detail level temporarily can help during investigations, but it should be lowered afterwards because verbose logs on large clusters are noisy.

Upgrades and version matching

kube-proxy is released together with Kubernetes and is expected to stay close in version to the rest of the cluster. Nodes are commonly allowed to run a slightly older version than the control plane, but not a newer one. Always follow the upgrade order recommended for your Kubernetes release, and upgrade kube-proxy along with the nodes.

A component that edits network rules on every node deserves careful protection.

kube-proxy needs strong permissions on the node, because writing kernel network rules is a privileged action. It therefore typically runs with elevated rights, usually as a privileged container or a system service. On the cluster side, it needs only permission to read Services and endpoints, which Kubernetes grants through a narrow role.

kube-proxy is not a firewall

It is easy to assume that kube-proxy decides who may talk to whom. It does not. It only decides where Service traffic goes, not whether it is allowed. Rules that permit or deny communication between pods are written as Network Policies, and they are enforced by the network plugin. Relying on kube-proxy for access control would leave the cluster wide open.

An honest look at what the kube-proxy design gives you and what it costs.

When the default is enough

For small and medium clusters, the standard kube-proxy in iptables mode is dependable and requires almost no attention. Spend effort on application design first, and revisit networking when measurements show a problem.

When to look further

Clusters with thousands of Services, very high connection rates, need for detailed traffic visibility or strict latency targets may benefit from IPVS, nftables or an eBPF-based replacement. Such choices should follow careful testing, since the network layer touches everything.

Habits that help kube-proxy do its job, and traps that make its life hard.

Helpful patterns

Traps to avoid

When a Service does not work, follow the chain from name to pod, and check each link.

You rarely notice kube-proxy, and that is exactly the point.

In the modern modes, no. kube-proxy writes rules and the kernel forwards packets. The program itself is not in the path of your data, which is why traffic keeps flowing during a brief kube-proxy restart.

Usually yes. An Ingress controller itself is reached through a Service, and traffic from the controller to your application pods commonly uses Service rules too. Ingress handles web routing, while kube-proxy handles Service-to-pod redirection.

Existing rules remain in the kernel, so current routing still works for a while. But changes such as new pods, removed pods or new Services will not reach that node, and its rules slowly become out of date, leading to traffic sent to dead pods or missing new ones.

No. A ClusterIP is virtual and lives only in the rules. In IPVS mode, kube-proxy may add the address to a special dummy network interface for its own use, but it is still not a real machine address.

Start with the default for your platform. Consider IPVS or nftables if you have very many Services and measure slow updates. Choose an eBPF replacement when you have deeper performance or visibility goals and the team can support it.

No. A service mesh works at the request level with features such as retries, encryption between services and detailed request metrics. kube-proxy works at the connection level and offers only basic redirection and spreading.

No. Name resolution is handled by CoreDNS. kube-proxy acts after DNS has returned the Service address.

Because each node has its own rules. If kube-proxy is unhealthy or out of date on one node, only pods on that node are affected. Comparing kube-proxy health and logs across nodes usually reveals the odd one out.

No. It spreads connections by simple methods such as random choice, and in IPVS mode by chosen schedulers like round robin or least connections. It does not know how busy a pod’s application really is.

Yes, if another component such as an eBPF-based network plugin implements Service routing. The function is still required, but a different program performs it.

Not in the usual sense. There is one per node, each independent, and the kernel keeps rules working even if it briefly stops. Failure affects updates on that node rather than the entire cluster.

Pods in Kubernetes are short-lived and their addresses keep changing, so applications need a steady way to reach each other. A Service provides the stable name and virtual address, and kube-proxy makes that address real on every node. It watches the control plane for changes to Services and endpoints, translates them into network rules, and installs those rules in the operating system so that the kernel can redirect Service traffic to healthy pods.

The mode you choose, whether iptables, IPVS or nftables, changes how those rules are stored and how well they scale, but not what they achieve. Around this core sit useful details: traffic policies, session affinity, connection tracking, health and metrics endpoints, and security choices. And for special needs, eBPF-based network plugins can replace kube-proxy while carrying out the same responsibility.

Keep the receptionist picture in mind. kube-proxy does not carry visitors around the building. It makes sure every entrance has an up-to-date directory, so that each visitor is quietly pointed to someone who is present and ready to help.