A service mesh solves a problem that only appears once you have many small services. To understand it, we first need to see that problem clearly.

From one big program to many small ones

Older applications were often built as a single large program, called a monolith. All the features lived in one place and talked to each other through ordinary function calls inside the same program. Those calls are fast and almost never fail.

Modern teams often split an application into many small programs called microservices. One service handles logins, another handles payments, another handles search. They run separately, often in different containers, and they talk to each other over the network. A container is a sealed package holding a program and everything it needs, and Kubernetes is the system that runs and manages all those containers across many computers.

The network is not reliable

Network calls are very different from function calls. They can be slow, they can fail halfway, and they can be intercepted. When one service calls another, many things can go wrong: the other service may be starting up, overloaded, or simply gone. Someone on the network could also read or change the message if it is not encrypted.

Every team that builds microservices must therefore deal with the same list of chores. Calls should be retried when they fail, but not forever. Slow calls need time limits. Traffic should be encrypted. Each service should prove who it is. And operators need to see which service is calling which, and how healthy each conversation is.

The old way: put it in a library

Before service meshes, teams solved these chores by adding code libraries to every application. A library might handle retries, another might handle encryption, another might collect metrics. This works, but it has serious costs. Every language needs its own version of every library. Every update requires rebuilding and redeploying each application. And if one team forgets to add the library, or configures it differently, the whole system has a hole in it.

A service mesh moves the networking chores out of your code and into a layer of small proxies that sit beside every service.

The idea in plain words

A service mesh is a dedicated layer of infrastructure that manages communication between services. It works by placing a small network program, called a proxy, next to each service. A proxy is a middleman that receives a message on behalf of someone else and passes it along. In a mesh, all traffic going into or out of a service passes through its proxy first.

Because every message flows through a proxy, the proxy can do useful work on it: encrypt it, retry it, time it, count it, or refuse it. The application itself knows nothing about this. It sends a plain request as if it were talking directly to the other service, and the proxies take care of everything else.

The two halves of a mesh

Every service mesh has two parts. The proxies that carry traffic are called the data plane, because the data flows through them. The central program that tells all the proxies what to do is called the control plane. The control plane does not carry any user traffic. It only sends instructions, such as which services exist, which rules to apply, and which security certificates to use. We look at both in detail in Chapter 4.

The vocabulary you will use

A mesh is built on top of ordinary Kubernetes objects. Knowing them makes the rest of the tutorial feel natural.

What Kubernetes already does for traffic

Kubernetes already gives you basic networking. A Service provides a stable address, and kube-proxy spreads connections across the pods behind it. This is enough for many simple applications, but it works at the level of connections, not individual requests. It does not understand HTTP, does not encrypt anything by itself, and cannot retry, time out, or split traffic by percentage.

This gap is exactly what a mesh fills. The mesh does not replace Kubernetes networking. It builds a smarter layer on top of it, using the pods, Services, and labels that already exist.

Why per-request matters

Modern services often use long-lived connections, especially with the gRPC protocol, which sends many requests over one connection. Because kube-proxy balances connections rather than requests, all the requests on one long connection may end up at a single pod, while other pods sit idle. A mesh proxy understands each request separately, so it can spread them evenly across all pods.

Every mesh is split into workers that handle traffic and managers that give instructions. Keeping these two roles apart is the secret of its design.

The data plane: the proxies

The data plane is made of the proxies that sit beside your services. They are the busy workers. Every request goes through them, so they must be fast, small, and reliable. Each proxy knows how to find other services, balance load, encrypt and decrypt traffic, apply retries and timeouts, and record metrics.

Istio uses a well-known open source proxy called Envoy. Envoy is powerful and highly configurable, which is why many other tools use it too. Linkerd takes a different route and uses its own tiny proxy, written in the Rust language and built only for Linkerd. It does fewer things, but it is designed to be lean and simple.

The control plane: the managers

The control plane is the brain. It does not touch user traffic. Instead, it watches the cluster, learns which services and pods exist, and pushes that knowledge, along with your rules, to every proxy. It also acts as a certificate authority, meaning it issues the digital identity documents that let services prove who they are.

Because the control plane is separate, a short outage in it does not stop traffic. The proxies already have their instructions and keep working. They just cannot receive new instructions until the control plane returns.

flowchart LR
    CP["Control plane: gives config, identities, and service lists"] -->|"instructions"| PA["Proxy beside Service A"]
    CP -->|"instructions"| PB["Proxy beside Service B"]
    AA["Service A"] --- PA
    PA -->|"encrypted traffic"| PB
    PB --- BB["Service B"]
        

Read the figure from the top. The control plane sends instructions to each proxy, shown by the arrows leaving it. The services themselves, drawn as A and B, only talk to their own proxy, shown by the plain lines. The real traffic goes from proxy to proxy along the encrypted arrow. The control plane never appears on that path.

The magic of a mesh is that developers do nothing. This chapter reveals the trick behind that.

Automatic injection

When you want a namespace to be part of the mesh, you mark it with a label or annotation. From then on, whenever a new pod is created in that namespace, Kubernetes sends the pod definition through an admission webhook belonging to the mesh. The webhook quietly adds the proxy container to the pod before the pod is saved. This is called sidecar injection.

Developers write their normal deployment as usual. They never mention the proxy. The mesh adds it automatically, which is why teams can adopt a mesh gradually, one namespace at a time. Pods that already exist do not change until they are restarted, so enabling injection usually needs a rolling restart of the applications.

flowchart TD
    A["Namespace is labelled for the mesh"] --> B["A new pod is requested"]
    B --> C["API server calls the mesh admission webhook"]
    C --> D["Webhook adds the proxy container and network setup"]
    D --> E["Pod is scheduled and started with its proxy"]
    E --> F["Proxy contacts the control plane for identity and settings"]
        

Catching the traffic

Adding a proxy is only half of the trick. The application must be forced to send its traffic through that proxy, even though the application does not know the proxy exists. The mesh does this by changing the pod’s network rules so that all outgoing traffic is redirected to the proxy, and all incoming traffic arrives at the proxy first.

This redirection is usually set up by a short init container that runs before the application starts. It needs elevated network permissions to do its job, which some organisations dislike. As an alternative, both Istio and Linkerd offer a network plugin approach that does the redirection at the node level, so pods do not need those permissions.

Startup, shutdown, and other lifecycle wrinkles

Because the proxy and the application are separate containers, their starting order matters. If the application starts first and immediately makes a call, the proxy may not be ready yet, and the call fails. Meshes offer settings to hold the application until the proxy is ready. Newer Kubernetes versions also support sidecar containers as a first-class idea, which have a clear start and stop order relative to the main container.

The same issue appears at shutdown. If the proxy stops before the application has finished its last requests, those requests fail. It is also a classic problem with batch jobs, where the job finishes but the proxy keeps running, so the pod never completes. Good mesh setups handle both cases explicitly.

Both meshes follow the same two-part design, but they package the parts differently. Knowing the pieces helps you read logs and diagrams later.

Istio’s parts

Istio’s control plane is delivered mainly as a single program called istiod. It watches Kubernetes, converts your rules into proxy configuration, sends that configuration to every Envoy proxy, and acts as the certificate authority that issues identities. Older versions split these jobs across several programs, which were merged into one to make Istio simpler to run.

Istio’s data plane is a set of Envoy proxies, one per pod in the classic sidecar mode. Istio also provides gateways, which are Envoy proxies running at the edge of the mesh, handling traffic entering and leaving the cluster. Istio’s rules are written as Kubernetes custom resources, such as routes, destination settings, gateways, and security policies.

Linkerd’s parts

Linkerd’s control plane is made of a few small components. One keeps track of destinations, meaning where each service’s pods are. One acts as the identity authority, issuing certificates. One is the injector, which adds the proxy to new pods. Extension components add features such as a metrics dashboard.

Linkerd’s data plane is its purpose-built Rust proxy, added to every meshed pod. Linkerd deliberately keeps its configuration small. It leans on the newer Kubernetes Gateway API for routing rules and on a small set of its own resources for security policy.

Let us follow a single request from one service to another and see exactly what each proxy does along the way.

Step by step through the mesh

Suppose a checkout service wants to call a payment service. The checkout application sends an ordinary request to the payment service’s name, as it always did. It has no idea a mesh exists. What happens next is a chain of events inside two proxies, shown in the diagram below.

sequenceDiagram
    participant C as Checkout App
    participant CP as Checkout Proxy
    participant SP as Payment Proxy
    participant S as Payment App
    C->>CP: Plain request to the payment service name
    CP->>CP: Pick a healthy pod, apply retry and timeout rules
    CP->>SP: Encrypted request using mutual TLS
    SP->>SP: Verify caller identity and check access rules
    SP->>S: Plain request on the local connection
    S-->>SP: Response
    SP-->>CP: Encrypted response
    CP-->>C: Response, metrics recorded on both sides
        

How proxies know where to send traffic

Proxies do not search for services on their own. The control plane watches Kubernetes for Services and their pods, and continually sends each proxy the current list of addresses for every service. When a pod starts, stops, or becomes unhealthy, the change flows to the proxies within moments. This means the proxy always has a fresh map and never needs to ask a central directory on each request.

Smarter load balancing

Because a mesh proxy understands individual requests, it can balance them far better than the default. Instead of sending each connection to a random pod, it can favour the pod that has been answering fastest. Linkerd, for example, uses a latency-aware method that weighs recent response times. This steers work away from slow or struggling pods before they cause problems.

Because every request passes through a proxy, you can shape traffic with rules instead of code. Here are the main tools.

flowchart LR
    U["Caller"] --> P["Caller proxy applies the route rule"]
    P -->|"90 percent"| V1["Checkout version 1"]
    P -->|"10 percent"| V2["Checkout version 2"]
        

The figure shows a canary release. The caller’s proxy applies a rule that sends ninety percent of requests to version one and ten percent to version two. To roll forward, you change the numbers. To roll back, you set the new version to zero. No pods are added or removed, and no application changes are needed. This is far finer control than Kubernetes offers on its own, where traffic share is tied to the number of pods.

For many teams, security is the main reason to adopt a mesh. The mesh can encrypt every internal call and decide who may make it.

Identity: proving who you are

In a mesh, every service gets a cryptographic identity. This is a certificate, a digital document that says “this proxy belongs to this service account in this namespace”, signed by the mesh’s certificate authority. The identity is tied to the Kubernetes ServiceAccount of the pod, so it reflects something real rather than a name someone typed in.

Certificates are short-lived, typically valid for about a day, and are rotated automatically by the control plane. If one is stolen, it becomes useless quickly. Nobody needs to copy files, set reminders, or restart applications to renew them.

mTLS: encryption where both sides prove themselves

You may know that ordinary secure web browsing uses TLS, where the website proves its identity to the browser. Mutual TLS, or mTLS, goes one step further: both sides present certificates. The client proxy proves who it is to the server, and the server proxy proves who it is to the client. Everything sent between them is encrypted.

The result is that traffic between meshed services cannot be read or altered by someone on the network, and neither side can pretend to be someone else. All of this happens between the proxies, so the applications need no certificate code at all.

Authorization: deciding who may call whom

Knowing who is calling is only useful if you act on it. Authorization policies let you say things like “only the checkout service may call the payment service”, or “only requests using this method on this path are allowed”. The server proxy checks each request against these rules and rejects anything that does not match.

A strong starting position is to deny everything by default and then allow only the specific calls that are needed. This limits the damage if any one service is compromised, because the attacker can only reach what that service was already permitted to call. This idea is often called zero trust.

Hardening the mesh itself

Because every proxy sees every request, a mesh gives you detailed visibility into your system almost for free.

Metrics: the golden signals for every service

Each proxy records, for every service pair, the four golden signals. These are enough to describe the health of almost any service, and the mesh provides them uniformly, whatever language the application uses.

These metrics are exposed in a format that Prometheus, a popular monitoring system, can collect. They can then be drawn as charts in Grafana. Istio has a companion tool called Kiali that draws a live map of which services talk to which, and Linkerd ships a lightweight dashboard that shows success rates and latency for each service and each route.

Logs and access records

Proxies can write an access record for every request, showing who called whom, the method, the path, the response code, and the time taken. These records are very useful when investigating an incident, but they are also very numerous. Most teams turn them on selectively, for specific services or only for failed requests, to control storage cost.

Distributed traces

A trace follows a single user action across many services, showing the time spent in each step. Proxies can create trace records for each hop, but they cannot connect the hops on their own. To link the incoming request to the outgoing calls that it caused, the application must copy certain tracing headers from the request it received into the requests it sends.

Both are excellent and mature. The right choice depends on how much you need, and how much complexity you can carry.

How to choose

If your needs are mainly automatic encryption, good metrics, and reliable request handling, Linkerd’s smaller design often gets you there faster and with less to operate. If you need advanced traffic control, edge gateways, extensive policy options, or integration with workloads outside Kubernetes, Istio’s wider toolbox may be worth its extra complexity.

Whichever you pick, judge by your team’s real needs rather than by feature lists. A mesh with a hundred features that nobody understands is worse than a mesh with ten features that everyone can debug at night.

The sidecar model is popular but not perfect. Newer designs try to keep the benefits while removing some of the costs.

Why people look beyond sidecars

Adding a proxy to every pod has costs. Each proxy uses memory and processor time, so a cluster with thousands of pods pays that price thousands of times. Pods must be restarted to receive proxy updates. Startup and shutdown ordering causes subtle problems. And the proxy lives inside the application’s own pod, which some security teams dislike.

Istio ambient mode

Istio’s ambient mode removes the per-pod sidecar. Instead, a small shared agent on each node, called ztunnel, handles the basic jobs: encrypting traffic with mTLS and enforcing simple identity-based rules. When an application needs richer features, such as detailed routing by path or header, an optional proxy called a waypoint is added for a namespace or a service. Apps that need only encryption and identity pay a much smaller cost.

Kernel-level approaches

Some networking projects, such as Cilium, use a Linux kernel technology called eBPF to handle much of the work inside the operating system, with shared per-node proxies for the more complex web-level features. These designs also avoid a proxy in every pod, and blur the line between the network layer of the cluster and the mesh.

A mesh is not installed once and forgotten. It has a lifecycle, from careful first steps to regular upgrades, and each stage has its own traps.

A safe path to adoption

The safest way to adopt a mesh is slowly. Start with a small, low-risk namespace, watch what happens, and expand only when you understand the results. Turning on everything for every team on the first day is the most common way to turn a promising tool into a painful outage.

flowchart TD
    A["Install the control plane"] --> B["Mesh one low-risk namespace"]
    B --> C["Observe metrics with permissive mTLS"]
    C --> D["Enable strict mTLS"]
    D --> E["Add authorization policies"]
    E --> F["Add traffic rules: timeouts, retries, canaries"]
    F --> G["Expand to more namespaces"]
    G --> H["Upgrade and review regularly"]
        

Monitoring the mesh itself

A mesh gives a lot, and it costs a lot. An honest comparison helps you decide whether it is right for you, and when.

Thinking about cost honestly

The cost of a mesh has three parts. The first is machine cost: the proxies use resources, and in a cluster with thousands of pods that becomes real money. The second is people cost: someone must learn, run, and upgrade the mesh, which takes real time. The third is complexity cost: every incident now has one more place where the cause might hide.

These costs are worth paying when the benefits are large. A company with hundreds of services and strict security requirements gains far more than it spends. A team with five services and a simple architecture may gain almost nothing. The key is to ask which specific problems you have today, and whether a mesh is the smallest tool that solves them.

When a mesh may not be the answer

Some habits make a mesh calm and useful. Others turn it into an expensive source of surprise.

Patterns that work well

Anti-patterns to avoid

A short field guide collected from teams that run meshes in production, and from the mistakes newcomers make.

Best practices

• Start with a clear problem — name the specific need, such as encryption or metrics, before installing anything.
• Pilot in a small namespace — learn how the mesh behaves with real workloads before widening the rollout.
• Measure the overhead — record proxy resource use and latency so decisions rest on numbers.
• Use your own certificate authority — anchor trust in a system your organisation controls and rotates.
• Move to strict mTLS — treat permissive mode as temporary and confirm that no plain traffic remains.
• Set timeouts on every route — no call should be allowed to wait for ever.
• Limit and coordinate retries — one layer, a small cap, and only for safe requests.
• Handle startup and shutdown order — make sure applications wait for their proxy and finish their work before it stops.
• Propagate tracing headers — update applications so traces link across services.
• Keep network policies — they still protect anything that bypasses the mesh.
• Upgrade regularly — plan control plane and proxy upgrades, and test them on non-critical namespaces first.
• Document conventions — publish defaults and templates so teams do not each invent their own rules.

Common mistakes

Service meshes appear wherever many services must talk safely, and each industry leans on a different part of the toolbox.

Quick answers to the questions that come up most often when teams first meet a service mesh.

Not for the basic features such as encryption, metrics, and load balancing. The proxies handle them. For full distributed tracing, applications do need to pass along tracing headers.

No. A gateway mainly handles traffic entering the system from outside, while a mesh mainly handles traffic between services inside. Meshes often include their own gateways so that both work together.

No. It builds on them. Services and pods provide the basic map of the cluster, and the mesh adds smarter behaviour on top.

Each proxy uses some memory and processor time, and each request passes through two extra hops that add a small delay. The exact numbers depend on the mesh, the traffic, and the features used, so measure in your own environment.

Existing traffic keeps flowing, because proxies keep their last configuration. New pods cannot get identities or settings, and certificates cannot be renewed, so a long outage becomes a real problem.

Both meshes enable mTLS between meshed pods automatically. Istio often starts in a permissive mode that also accepts plain traffic, so you should move to strict mode when your services are ready.

Running two meshes over the same pods is generally a bad idea, because both try to capture the same traffic. Choose one for each set of workloads.

Usually not. With a few services, simpler tools such as network policies, a good client library, and application metrics are often enough. A mesh pays off as the number of services and teams grows.

The proxy container keeps running after the job’s own container exits, so the pod never completes. Meshes and newer Kubernetes sidecar support offer ways to stop the proxy when the main work is done.

Yes, to a degree. Proxies can encrypt and balance plain TCP traffic, and can report basic metrics. Rich features such as path routing and per-request retries apply mainly to protocols the proxy understands, such as HTTP and gRPC.

Remove injection labels from namespaces, restart pods so they lose their proxies, and then uninstall the control plane. Doing it namespace by namespace, in the reverse order of adoption, keeps the risk low.

Not soon. Sidecars remain widely used and well supported. Alternatives such as ambient mode are growing, and many teams will use a mix depending on their needs.

A service mesh moves the repeated work of service-to-service communication out of your applications and into a shared layer of proxies, managed by a central control plane. In return for a modest cost in resources and complexity, you gain encryption, identity, smarter load balancing, fine traffic control, and detailed metrics, all without rewriting your code. Istio offers a very wide toolbox, and Linkerd offers a smaller, simpler design, and newer models such as ambient mode try to reduce the cost of the sidecar approach.

The mesh is a powerful tool, not a magic one. It needs careful adoption, regular upgrades, and honest measurement. It works best when it solves a problem you actually have, and when it is run as a well-documented platform rather than a surprise. Teams that follow those habits find that the mesh quietly makes their systems safer and easier to understand.