Before we meet the solution, let us feel the problem. A tool only makes sense when you understand the pain it removes.

What is Kubernetes, in one minute?

A container is a small, sealed box that holds a program and everything the program needs to run. Running one container on one computer is easy. Running thousands of them across hundreds of computers is very hard. Kubernetes is the free, open-source system that does this hard work for you. It started life inside Google, which had been running containers at giant scale for years, and it was shared with the world in 2014. Today it is looked after by the Cloud Native Computing Foundation and is used by companies of every size.

A group of computers managed by Kubernetes is called a cluster. Each computer in the cluster is called a node. Your containers run inside small wrappers called Pods. A Pod is the smallest thing Kubernetes runs, and it usually holds one container.

Why Pods cannot simply be “visited”

Every Pod receives its own private address, called an IP address, from the cluster’s internal network. This sounds helpful, but there are three catches.

• The address is private. It only works inside the cluster. Someone on the internet cannot type it into a browser.
• The address is temporary. When a Pod crashes, gets updated, or is moved to another node, Kubernetes creates a new Pod with a brand new address. The old one is gone forever.
• There are many Pods. A busy app might have twenty copies running at once. A visitor should not have to choose one.

Eight small ideas carry the whole story. Learn these and everything later will feel natural.

Kubernetes offers several kinds of Service. Each one builds on the one before it, like floors of a building.

The floors of the building

A LoadBalancer Service is not a separate invention. It is built on top of the others. When you ask for a LoadBalancer, Kubernetes first gives you a ClusterIP (the internal receptionist). It then opens a NodePort on each node (a side door on every floor). Finally, it asks the cloud to build a load balancer outside the cluster that sends visitors to those side doors. Each layer depends on the one beneath it.

One detail for the curious: newer Kubernetes versions let you switch off the NodePort step for certain cloud setups that can send traffic straight to Pods. This is a special case. For learning, remember the three floors.

Let us define it in plain words, then look at exactly what it promises and what it does not.

A load balancer is a traffic manager. Many visitors arrive at one address, and the load balancer spreads them across several workers so that no single worker is overwhelmed. If a worker falls ill, the load balancer notices and stops sending visitors there.

A LoadBalancer Service in Kubernetes is a request written down in your cluster’s records. It says: “Please give this group of Pods a public front door that shares visitors fairly.” Kubernetes itself does not contain a full-strength load balancer machine. Instead, it asks the environment it lives in, usually a cloud company, to build one and then connects everything together.

What you get

• A stable external address. It is either a number (an IP address) or a DNS name, depending on the cloud. It does not change when Pods change.
• Fair sharing of visitors across healthy nodes and Pods.
• Health checking, so broken targets are skipped.
• Hands-off setup. You describe the Service once; the cloud pieces are created and updated for you.

What it does not do

• It does not read web addresses like “/cart” or “/login” and route by them. It works at the level of connections (TCP and UDP), not web pages. That smarter routing is the job of Ingress or Gateway API, which we compare later.
• It does not work everywhere out of the box. On a laptop cluster with no cloud behind it, the external address can stay stuck on “pending” forever until you add a helper.
• It does not encrypt anything by itself. Security is a separate design choice.

Here is the full picture. Follow the arrows from the user on the left to the Pods on the right.

flowchart LR
    U["User on the Internet"] --> DNS["DNS Name"]
    DNS --> CLB["Cloud Load Balancer with Public Address"]
    subgraph K8S["Kubernetes Cluster"]
        direction LR
        subgraph N1["Worker Node 1"]
            NP1["NodePort"] --> KP1["kube-proxy Rules"]
            KP1 --> P1["Pod A"]
        end
        subgraph N2["Worker Node 2"]
            NP2["NodePort"] --> KP2["kube-proxy Rules"]
            KP2 --> P2["Pod B"]
            KP2 --> P3["Pod C"]
        end
        SVC["LoadBalancer Service"]
        CCM["Cloud Controller Manager"]
        API["API Server"]
        API --> SVC
        API --> CCM
    end
    CLB --> NP1
    CLB --> NP2
    CCM -. creates and updates .-> CLB
    

Reading the diagram

There are two separate flows here, and mixing them up is a common beginner mistake.

The traffic flow is the path a visitor’s request takes: from the user, to a DNS name, to the cloud load balancer, to a NodePort on one of the nodes, through the kube-proxy rules, and finally to a Pod. This path is used thousands of times per second.

The control flow is the dotted line. It is how the load balancer got created in the first place. You submit a Service to the API server, the Cloud Controller Manager notices it, and it talks to the cloud provider to build and maintain the load balancer. This path is used only when something changes, like a new Service or an updated setting.

Meet each component

What happens between “I wrote the Service” and “my app has a public address”? Let us walk through it slowly.

sequenceDiagram
    participant Dev as Developer
    participant API as API Server
    participant CCM as Cloud Controller Manager
    participant Cloud as Cloud Provider
    Dev->>API: Create a Service of type LoadBalancer
    API-->>CCM: A new Service was added
    CCM->>Cloud: Please build a load balancer
    Cloud-->>CCM: Done, here is the public address
    CCM->>API: Save the address in the Service status
    Dev->>API: Read the Service
    API-->>Dev: External address is ready
    

Let us follow a single visitor, from typing a web address to seeing a page, and watch how each layer hands the request forward.

How kube-proxy writes its rules

kube-proxy has a few ways of doing its job. The classic method uses a long list of packet-matching rules in the Linux kernel, known as iptables. A second method, called IPVS, uses a table built for load balancing, which handles very large numbers of Services more efficiently. Newer versions also offer a mode based on nftables, the modern successor to iptables. Some clusters skip kube-proxy completely and let a networking add-on such as Cilium do the same work with a technology called eBPF. You do not need to master any of these to use a LoadBalancer Service. Just know that a small, fast rule table on every node is what turns “Service address” into “real Pod”.

A data flow with an important detail: the lifecycle of a connection

A connection has a beginning, a middle and an end. The load balancer picks the target when the connection starts. It does not move the connection midway. This is why, when you remove a Pod, existing long connections to it can be cut, and why graceful shutdown settings matter. We return to this in the best practices chapter.

One small setting decides whether traffic takes an extra hop and whether your app can see who the real visitor is.

Every Service has a setting called externalTrafficPolicy. It has two possible values, and understanding them will save you many hours of confusion.

flowchart LR
    subgraph CL["Policy: Cluster (default)"]
        LB1["Load Balancer"] --> NA["Node A with no local Pod"]
        NA -->|"extra hop"| PB["Pod on Node B"]
        LB1 --> NB["Node B with a Pod"]
        NB --> PB
    end
    subgraph LO["Policy: Local"]
        LB2["Load Balancer"] -. health check fails .-> NC["Node C with no local Pod"]
        LB2 --> ND["Node D with a Pod"]
        ND --> PD["Pod on Node D"]
    end
    

Why the visitor’s address matters

Many apps want to know who is calling. A bank may want to block certain countries. A security tool may want to count failed logins per visitor. If the address is replaced along the way, every visitor looks like it came from one of your own nodes. With the Local policy, or with a load balancer that passes the original address along using a small extra label called the PROXY protocol, your app can see the truth.

How health checks fit in

The cloud load balancer checks each node regularly, like a teacher calling the register. With the Local policy, Kubernetes opens a special health-check port on each node that answers “yes” only if a healthy local Pod exists. With the Cluster policy, the check is simpler and mostly tells the load balancer that the node itself is alive.

The Service looks the same everywhere, but what gets built behind it depends on where your cluster lives.

Annotations: the cloud-specific notes

The Service object has a notes field called annotations. Cloud providers read special notes there to unlock extra features that Kubernetes itself does not define, such as choosing a private load balancer, picking a certificate, or setting idle timeouts. This is powerful but has a cost: a Service with AWS-style notes will not automatically behave the same on Azure. Keep this in mind if you plan to run in more than one cloud.

Public versus internal

Not every LoadBalancer must face the internet. Teams often create internal load balancers that only answer inside their private cloud network. This is common for services shared between departments, or for apps that a company’s other systems call but customers never do. The Service is the same; only a cloud-specific note changes.

What if there is no big cloud company to build the front door for you? Then you need to bring your own doorman.

Remember that Kubernetes does not ship with a load balancer machine. It only knows how to ask for one. On a cloud, the cloud company answers. On your own servers in a data center, or on a laptop, nobody answers, and the Service waits forever with its address shown as “pending”. This surprises almost every beginner. The fix is to install a small add-on that plays the role of the cloud.

Two ways MetalLB announces an address

In Layer 2 mode, one node in the cluster answers the network’s question “who owns this address?”. All traffic for that address goes to that single node first, and then Kubernetes spreads it onward. It is very simple and works on almost any network. If that node dies, another node takes over after a short pause. The catch is that one node handles all the incoming traffic for that address at any moment.

In BGP mode, the cluster talks to your network routers using the same language the internet itself uses to find routes. The routers learn that several nodes can reach the address, so they spread incoming traffic across them. It scales better, but you need routers that support it and someone who understands your network.

Three tools, three different jobs. Choosing the right one saves money and headaches.

Ingress is a set of routing rules for web traffic. It can look at the website name and the page path, then send the visitor to the right Service. It is carried out by a program called an Ingress controller. Gateway API is the newer, more flexible family of objects designed to replace and extend Ingress, and it reached its stable 1.0 release in late 2023. Both work mostly at layer 7, which means they understand web requests, not only raw connections.

The surprising link between them

Here is a fact that clears up a lot of confusion: the Ingress controller or Gateway itself is just a set of Pods, and those Pods need to be reached from outside too. In most clusters, they are exposed using a single LoadBalancer Service. So these tools are not rivals. A common design is one LoadBalancer Service in front of a shared Ingress or Gateway, which then routes to dozens of apps behind it.

No tool is perfect. Knowing the honest strengths and weaknesses helps you pick well.

The trade-offs behind the choices

Simplicity versus cost. One LoadBalancer per app is the easiest to reason about because each app owns its door. The price is a growing pile of load balancers. Sharing one entrance through Ingress or Gateway is cheaper but adds a layer that someone must run and understand.

Even spread versus honest addresses. The Cluster traffic policy shares load more evenly but may hide the visitor’s address. The Local policy keeps the address but can create hot spots.

Portability versus power. Cloud-specific notes unlock great features but tie your setup to one provider. Teams that may move between clouds keep those notes in one clearly documented place.

Speed versus smartness. A layer 4 load balancer only moves connections, so it is fast and light. A layer 7 tool reads each request, so it can do smart routing, but it costs a bit more time and computing power per request.

A public door invites customers, but it also invites troublemakers. Good security is layered, like a castle with a moat, a wall and locked rooms.

Protecting the controller’s keys

The Cloud Controller Manager needs permission to create and edit load balancers in your cloud account. Give it only the permissions it needs and nothing more. Keep an eye on cloud audit logs to see who created which load balancer and when.

You cannot fix what you cannot see. A healthy front door needs a few dashboards and a few alarms.

Think of your front door like a shop entrance with a people counter, a camera and a fire alarm. You want to know how many people come in, whether the door is stuck, and whether anything is going wrong.

Three places to look

Logs that help

Many clouds can save access logs and network flow logs from the load balancer to storage, showing who connected and when. Keep them for security reviews and for solving strange incidents. Inside the cluster, gather Pod logs in one central place so you can follow one request from the door to the answer.

Alarms worth having

• Any Service whose external address stays “pending” for longer than a few minutes.
• The number of healthy targets falling below a safe minimum.
• Sudden spikes in failed connections or in total traffic.
• Latency rising above what your users tolerate.
• A load balancer that exists in the cloud but no longer belongs to any Service, since this means you are paying for nothing.

A pattern is a solution that many teams found works well. An anti-pattern is a trap many teams fell into.

Patterns that work

Traps to avoid

These habits separate a shaky setup from a calm, dependable one.

Habits that pay off

• Use readiness checks. A readiness check tells Kubernetes whether a Pod is truly prepared to serve. Pods that are not ready are removed from the Service’s list, so visitors never reach a Pod that is still starting up.
• Shut down gracefully. When a Pod is being removed, give it time to finish its current work and let the load balancer notice before it disappears. Without this, users see random errors during every update.
• Run enough Pods, in more than one zone. A zone is a separate building within a cloud region. Spreading Pods across zones lets you survive the loss of one.
• Protect against too many removals at once. A disruption budget tells Kubernetes the minimum number of Pods that must stay alive during maintenance.
• Pick the traffic policy on purpose. Decide whether you need the real visitor address or the smoothest spread, and test it.
• Restrict access by default. Start with a short allowed-range list and widen it only if needed.
• Label and track cost. Add notes so you can tell which team owns which load balancer, and review the list regularly.
• Delete what you no longer use. Unused Services keep their load balancers alive and their bills running.
• Test failure on purpose. Turn off a node in a test setting and watch what your users would see.

Mistakes beginners make

Seeing how real companies and everyday setups use these ideas makes them stick.

Quick answers to the questions beginners ask most often.

You started with a shop nobody could enter. Now you know how to build it a proper front door.