Start with the big picture: what it is, what it is not, and where it fits.

A service is a small program that does one job and is reached over the network. A microservices application is built from many such services. Traffic between services inside your application is often called east-west traffic, because on a diagram it flows sideways between boxes. Traffic coming in from users on the internet is called north-south traffic, because it flows down from the outside.

A service mesh is a layer that manages east-west traffic. It gives every service the same helpful abilities, such as finding other services, spreading load evenly, retrying failed calls, encrypting conversations and recording what happened, without each development team having to write that code again and again.

A service mesh has two parts. The data plane is the set of small programs, called proxies, that actually carry the requests. A proxy is a middleman that receives a request and forwards it to the right place. The most common proxy is Envoy, a popular open-source proxy. The control plane is the brain that tells all those proxies how to behave.

Traffic Director is Google Cloud’s fully managed control plane for a service mesh. “Fully managed” means Google runs it, scales it, keeps it available and updates it. You describe the rules you want, and Traffic Director sends the right instructions to every proxy in your system. It uses open standard APIs known as xDS, which are the same discovery APIs that Envoy understands, so it works with Envoy and with gRPC applications that can speak xDS directly.

An important note about names

Google’s product names change over time. Traffic Director has been presented together with other Google service mesh offerings under the broader name Cloud Service Mesh, and some features are configured through newer routing APIs. The ideas in this guide stay the same, but always check the current Google Cloud documentation for the latest names, supported features and recommended setup.

Every good tool begins with a pain. Here are the pains of running many services.

Pain 1: Finding the other service

Services come and go. Machines are replaced, containers restart and copies are added during busy hours. Their network addresses change all the time. A service that wants to call another must always know the current, healthy addresses. This job is called service discovery. Doing it by hand, or with a hard-coded list, breaks quickly.

Pain 2: Sharing the load fairly

If one service has ten copies, the caller must decide which one gets each request. A simple rotation is not always best. One copy may be slower, another may be overloaded. Smart load balancing needs live information.

Pain 3: Failures spread like dominoes

When one service becomes slow, callers keep waiting and pile up more requests. Soon they too become slow, and the problem spreads across the whole application. This is called a cascading failure. Good systems retry sensibly, give up quickly when needed, and stop sending traffic to sick copies.

Pain 4: Releases are scary

Releasing a new version of a service to everyone at once is a gamble. Teams want to send just a small share of traffic to the new version first, watch for problems, and then slowly increase the share. This is a canary release. Doing this needs careful traffic control.

Pain 5: Every team writes the same code

Without a mesh, each team writes its own retry logic, timeouts, encryption and logging inside every service, often in different programming languages and different styles. The result is inconsistent and hard to update. If a rule changes, hundreds of services must be rebuilt.

Pain 6: Mixed environments

Real companies often have services on virtual machines, in containers, in several regions, and even in other clouds or data centers. Treating them as one connected application is difficult without a common layer of control.

Eight building blocks explain almost everything you will see in the settings and documents.

Handy words

See how the brain, the proxies and your services fit together.

The big idea is the clean split between the control plane and the data plane. Traffic Director lives in the control plane and runs as a global service on Google’s side. Your proxies, which sit in the data plane, open a long-lived connection to Traffic Director and ask for their instructions. Your requests never pass through Traffic Director. They flow directly from one service’s proxy to another service’s proxy.

flowchart TB
  Admin["Platform Engineer Defines Routing Rules"] --> TD["Traffic Director Managed Control Plane"]
  Health["Health Checks and Endpoint Lists"] --> TD
  TD -->|"xDS configuration"| P1["Envoy Sidecar Beside Service A"]
  TD -->|"xDS configuration"| P2["Envoy Sidecar Beside Service B"]
  TD -->|"xDS configuration"| G1["Proxyless gRPC Client in Service C"]
  SA["Service A"] --> P1
  P1 -->|"Direct request chosen by rules"| P2
  P2 --> SB["Service B"]
  G1 -->|"Direct request"| P2
    

Walking through the picture

A platform engineer, or an automated pipeline, writes the routing rules. Traffic Director also gathers live facts, such as which service copies exist and which ones passed health checks. It combines the rules and the facts into a complete configuration for each proxy and pushes it out. When Service A wants to call Service B, its sidecar already knows where all the healthy copies of B are and which rules apply, so it can choose and send the request immediately.

The main components

• Traffic Director control plane: the global, managed brain that stores configuration and serves it using xDS.
• Configuration resources: rules and settings you create, such as backend services, health checks, routes and policies.
• Endpoint sources: NEGs and instance groups that tell Traffic Director where service copies are running, including on virtual machines, in Google Kubernetes Engine containers and in hybrid locations.
• Data plane proxies: Envoy sidecars, Envoy running as gateways or middle proxies, and proxyless gRPC clients.
• Security services: certificate authority integration, so proxies can get identities for encrypted conversations.
• Telemetry destinations: Cloud Monitoring, Cloud Logging and Cloud Trace, where proxies report what they see.

Understand the quiet, constant conversation between proxies and the control plane.

When an Envoy proxy starts, it reads a small startup file that tells it who it is and where to find Traffic Director. It then opens a long-running connection to Traffic Director and says, in effect, “Hello, I am this proxy, in this network, serving this service. What are my instructions?” Traffic Director replies with a complete setup, and then keeps the connection open to push changes as soon as they happen.

sequenceDiagram
  participant E as Envoy Proxy
  participant T as Traffic Director
  participant S as Backend Endpoint
  E->>T: Connect and send node identity
  T-->>E: Send listeners and routes
  T-->>E: Send clusters and settings
  T-->>E: Send endpoint addresses
  E->>S: Forward application request using received rules
  T-->>E: Push updated endpoints after a change
    

The four kinds of information

The xDS family splits the configuration into pieces. You do not need to memorize the short names, but the ideas help a lot.

• Listeners: where the proxy should listen for traffic, for example “listen for requests meant for the payments service”.
• Routes: how to match a request, using host name, path or headers, and which destination group it should go to.
• Clusters: the destination groups, with settings such as load balancing method, timeouts, circuit breaking and encryption.
• Endpoints: the actual addresses of the healthy copies in each group, including which zone or region each one lives in.

Why pushing beats polling

Because the connection stays open and Traffic Director pushes changes, proxies learn about a new or failing service copy within moments. A system that asks for updates only every few minutes would keep sending requests to dead copies for a long time. The push model keeps the picture current.

What happens if the control plane is unreachable

Proxies keep working with the last good configuration they received. They continue to route requests, apply load balancing and enforce rules. What they cannot do during such a gap is learn about new changes. This is why the data plane keeps carrying traffic even when the control plane is briefly unavailable, but also why you should not rely on instant updates during an outage.

Follow one request from Service A to Service B, step by step.

The slow lane: configuration lifecycle

Next to the fast lane of requests, there is a slow lane of configuration. You change a rule. Traffic Director validates it and prepares updated configuration for the affected proxies. The proxies receive the update over their open connections and apply it to new requests, usually without restarting. A configuration change therefore takes effect within a short time across the whole mesh, though not at the exact same instant on every proxy.

How you describe what you want to Traffic Director.

Traffic Director is configured with resources, which are named objects that you create through the console, command line, automation tools or the APIs. There are two styles you will meet in documentation. One is built on Google Cloud’s load balancing resources, and the other uses newer service routing resources modeled on ideas from the Kubernetes Gateway world. Both aim to answer the same questions, so learning the concepts first makes either style easier.

flowchart LR
  FR["Forwarding Rule - Internal Self Managed"] --> TP["Target Proxy"]
  TP --> UM["URL Map - Routing Rules"]
  UM --> BS1["Backend Service Version 1"]
  UM --> BS2["Backend Service Version 2"]
  BS1 --> NEG1["Network Endpoint Group One"]
  BS2 --> NEG2["Network Endpoint Group Two"]
  BS1 --> HC["Health Check"]
  BS2 --> HC
    

The load-balancing style, explained

The service-routing style, explained

The newer style uses simpler building blocks. A Mesh resource represents the mesh itself. Route resources, such as HTTP routes and gRPC routes, describe how requests are matched and where they go. A Gateway resource represents an entry or exit point managed by proxies. Services are attached to the mesh by binding them to routes. If you already know Kubernetes Gateway API ideas, this will feel familiar. Which style is recommended for your case can change, so check the current documentation.

Matching requests

A route can match on many things:

• Host: which service name was asked for.
• Path: the exact path, a prefix, or a pattern, such as everything under “/checkout”.
• Headers: a request label, for example “x-user-group is beta-testers”.
• Query parameters: values in the address after the question mark.

Actions that a route can take

• Send traffic to one backend, or split it between several with weights.
• Rewrite the host or path before forwarding.
• Set a timeout and retry policy.
• Mirror a copy of the traffic to another backend for testing.
• Inject delays or errors on purpose, to test resilience.
• Redirect the caller to another address.

The most loved feature: control exactly who gets which version.

Weighted traffic splitting

With weights, you decide the share of traffic each version receives. You might begin with ninety-five percent to the stable version and five percent to the new one. If the new version behaves well, you raise it to twenty-five, then fifty, then one hundred. If it misbehaves, you set it back to zero in seconds, without redeploying anything.

Header-based routing

You can send only certain requests to the new version, for example requests from your own testers who carry a special header. Real customers still see the stable version. This lets teams test in production safely.

flowchart TD
  Req["Request From a Service Proxy"] --> Rule{"Does the Tester Header Match?"}
  Rule -->|"Yes"| V2["Send to Version 2"]
  Rule -->|"No"| Split{"Apply Weighted Split"}
  Split -->|"95 percent"| V1["Version 1 Stable"]
  Split -->|"5 percent"| V2
    

Traffic mirroring

Mirroring sends a copy of real requests to a new version, but throws away its answers. Real users only get answers from the stable version. The team can watch how the new version handles real traffic with zero risk to customers. Be careful with requests that change data, because a mirrored copy could still cause side effects unless the test system is isolated.

Timeouts and retries

A timeout sets the longest a proxy will wait. A retry policy says which failures are worth trying again, how many times, and how long to wait between tries. These rules, set centrally, give every service the same good behavior. Use retries only for requests that are safe to repeat, since a repeated payment request could charge a customer twice.

Fault injection

Fault injection means breaking things on purpose, in a controlled way. You can add delay to a share of requests or return an error for some of them, to learn whether callers cope. It is a safe way to practice for real failures, and a core tool of chaos engineering.

Rewrites and redirects

Routes can rewrite the host or path, which helps when you are renaming services or moving them while keeping old callers working. This makes migrations gradual rather than sudden.

How the mesh keeps working when parts of the system struggle.

Choosing a load balancing method

Traffic Director can tell proxies how to choose among healthy copies. Common methods include:

• Round robin: take turns, one copy after another. Simple and fair when copies are equal.
• Least request: prefer the copy with the fewest requests in progress. Helps when requests take different times.
• Random: pick any healthy copy.
• Consistent hashing methods: send the same kind of request, such as the same user, to the same copy. This is useful when copies keep helpful temporary data in memory.

The exact list of supported methods can differ by proxy type, so check the current documentation.

Locality-aware routing

Google Cloud is divided into regions and zones. A call that stays in the same zone is faster and cheaper than one that crosses to another region. With locality-aware load balancing, proxies prefer the nearest healthy copies and only move to farther ones when nearby copies are unhealthy or overloaded. This lowers delay and network cost.

Failover

If every copy in a zone fails, traffic spills over to the next-nearest healthy location. You can design multi-zone and multi-region setups so that a whole zone can be lost without users noticing much.

Circuit breaking

A circuit breaker limits how many connections or requests may be waiting for a service at once. When the limit is hit, extra requests are rejected quickly instead of piling up. This protects both the caller and the struggling service, stopping the domino effect.

Outlier detection

Outlier detection watches for copies that keep failing compared to their peers. A copy that returns too many errors is temporarily removed from the pool, then tried again later. This is like benching a player who is injured and letting them return when they have recovered.

Health checks and draining

Health checks tell Traffic Director which copies are well. When you remove or replace a copy, connection draining lets existing requests finish while new ones go elsewhere, so deployments do not cut people off mid-request.

Several ways to put proxies and services together, and several kinds of places to run them.

Model 1: Sidecar proxies

An Envoy proxy runs next to each service copy, in the same virtual machine or in the same pod, which is the smallest deployable unit in Kubernetes. All of the service’s network traffic goes through its sidecar. This model works with any programming language, because the application does not need to know about the mesh. The cost is extra memory and CPU for each proxy, and a small added delay per hop.

Model 2: Proxyless gRPC

If your services use gRPC, the library inside the application can understand xDS and talk to Traffic Director directly. There is no sidecar. This reduces resource use and delay, and removes one moving part. The trade-offs are that you depend on supported gRPC versions and on the features that the library implements, and not every advanced capability is available in this style.

Model 3: Gateways and middle proxies

Sometimes you want Envoy proxies at the edge of the mesh or in the middle of it, acting as a shared entry point or exit point instead of one per service. Traffic Director can configure these as well. They are useful for controlling how outside systems reach the mesh, or for routing traffic between groups of services.

Model 4: Mixed and hybrid environments

Real systems are rarely tidy. Traffic Director can manage services on Compute Engine virtual machines, in containers on Google Kubernetes Engine, and, through hybrid connectivity options, services running in other clouds or on-premises. A hybrid NEG is a list of endpoints outside Google Cloud that Traffic Director can include in its routing, reachable over private connections such as VPN or Interconnect. This lets you move services to Google Cloud gradually while keeping a single set of routing rules.

Inside a network, “trusted” is not a good enough default. Verify everyone.

In older designs, a firewall guarded the outside of a network and everything inside trusted everything else. Today, a single stolen password or one compromised service can let an attacker move around freely. The modern approach is zero trust: every request must prove who it comes from and be checked, even inside the network.

Identity for every workload

Each service gets a verifiable identity, delivered as a certificate that is issued and renewed automatically. The certificates come from a certificate authority service, such as Google’s Certificate Authority Service, so teams do not manage certificate files by hand. The identity tells other services exactly who is calling.

Encryption between services

With mutual TLS, both sides show their certificates, then encrypt the conversation. This protects data from being read by anyone listening on the network, and it blocks a fake service from pretending to be a real one. The proxies do this work, so developers do not need to add encryption code to each service.

Authorization policies

Authentication answers “who are you?”. Authorization answers “are you allowed to do this?”. Policies can say that only the checkout service may call the payment service, or that only certain paths and methods are open. This limits the damage if a single service is compromised, because it can only reach what it truly needs. This idea is called least privilege.

Protecting the control plane

Access to change Traffic Director configuration is controlled by Google Cloud’s Identity and Access Management. Only trusted people and automation should be allowed to edit routing rules, because a wrong rule can redirect traffic anywhere. Proxies also authenticate to Traffic Director with their own workload identities, so impostors cannot ask for the mesh configuration.

Security habits

• Turn on mutual TLS gradually, starting in a permissive mode that accepts both secure and plain traffic, and then tighten to strict mode after you confirm that all callers are ready.
• Write authorization rules as “deny by default, allow what is needed”.
• Keep certificate lifetimes short and make sure renewal is automatic and monitored.
• Use separate identities for separate services, never a shared one.
• Review who can change mesh configuration, and require review for changes.

A mesh makes a hidden world of service calls visible, if you look at the right signals.

Because every request passes through a proxy or a smart client, proxies are perfect places to collect facts about traffic. They can report the same set of signals for every service, regardless of programming language.

The three signals

• Metrics: numbers over time, such as requests per second, error rate and response time.
• Logs: records of individual requests, which show what happened to a particular call.
• Traces: the path of one request through many services, showing where time was spent. Your applications may need to pass along a trace identifier for traces to connect properly across services.

These can be sent to Google Cloud’s Cloud Monitoring, Cloud Logging and Cloud Trace, or to other tools you prefer.

The golden signals

A well-known approach says to watch four things for every service: latency (how long requests take), traffic (how many requests), errors (how many fail) and saturation (how full the resources are).

Checking the control plane’s view

Besides watching traffic, you will sometimes ask a different question: “What configuration does this proxy actually have right now?” Traffic Director provides ways to inspect the status of connected clients and their received configuration. This is very helpful when a rule seems not to apply, because it shows whether the problem is in the rule, the delivery or the proxy.

What grows when your system grows, and what to keep an eye on.

The control plane scales for you

Traffic Director is a managed global service, so Google handles the scaling of the control plane. You do not size servers for it. What you do need to know are the documented limits, such as how many proxies, services or endpoints are supported per configuration. These limits can change, so check the current quotas before designing a very large mesh.

The data plane is your responsibility

Proxies use CPU and memory. The more routes, services and endpoints a proxy has to know about, the more memory its configuration needs. In a very large mesh, giving every proxy the entire picture of every service is wasteful. Where possible, limit each proxy’s view to the services it actually needs to call.

Latency added by proxies

A sidecar adds a small delay on each hop, because the request goes through two proxies (one on each side). For most applications this is tiny compared to the work the services do, but chains of many calls add up. Measure it in your own system. Proxyless gRPC removes this extra hop.

Scaling the services themselves

Traffic Director works nicely with autoscaling. When a managed instance group or a Kubernetes deployment adds copies, the NEG updates, Traffic Director learns of the new endpoints and pushes them to the proxies, so new copies start receiving traffic quickly once they pass health checks.

Tips for large systems

• Use the narrowest configuration scope that works for each group of services.
• Avoid unnecessary changes. Frequent configuration churn makes every proxy work harder.
• Stagger restarts of large groups of proxies so they do not all request their configuration at once.
• Choose sensible health check intervals. Very frequent checks add load and noise.
• Right-size proxy resources after observing real usage.

The cost of a mesh is more than the control plane. Count everything.

Pricing for Traffic Director and related mesh offerings is set by Google and can change, so use Google’s pricing page and calculator for current numbers. What matters for planning is to see the full picture of where money goes.

Ways to keep spending sensible

• Prefer local traffic: keep callers and targets in the same zone when possible.
• Consider proxyless gRPC: where it fits, it saves the cost of running sidecars.
• Sample logs and traces: record all errors, but only a fraction of normal requests.
• Right-size sidecars: measure and set realistic resource requests, instead of copying large defaults.
• Limit configuration scope: smaller configurations mean lighter proxies.
• Label everything: so that billing reports show which team and environment spend what.

A calm, staged path from “no mesh” to “mesh in production”.

How Traffic Director relates to Istio and other options

Istio is a well-known open-source service mesh that includes its own control plane that you or a vendor run. Traffic Director takes a different approach: Google runs the control plane, and it speaks the open xDS APIs, which Envoy and other clients understand. Google has also brought its mesh offerings together under the Cloud Service Mesh name, including options based on Istio, so you may see both styles described together. Which path is right for you depends on your platform, your need for Kubernetes-specific features and your appetite for running your own control plane. Read the current documentation to see which options are recommended for new projects.

No tool is perfect. A good architect weighs both sides before choosing.

How it compares with its neighbors

The main trade-offs, in plain words

• Consistency versus complexity: a mesh gives every service the same behavior, but you must learn and operate the mesh.
• Power versus overhead: sidecars give rich features at a resource cost. Proxyless saves resources but offers fewer features.
• Central control versus blast radius: one place for rules is convenient, but a mistake there can spread widely. Review and staged rollouts reduce the risk.
• Managed versus self-run: a managed control plane saves effort, while a self-run one gives deeper control and portability.

A pattern is a proven way to solve a common problem. An anti-pattern is a tempting mistake that causes pain later.

Helpful patterns

Anti-patterns to avoid

A checklist you can follow from the first day.

Best practices

1. Start small and learn. Begin with a few services in a test environment.
2. Make observability the first step. Turn on metrics, logs and traces before changing routing.
3. Manage configuration as code. Keep it in version control, with reviews and automated checks.
4. Release with canaries. Use weights and header rules, and define clear success signals before you begin.
5. Set explicit timeouts everywhere. A call without a timeout can wait forever.
6. Use retries wisely. Only for safe, repeatable requests, with limits.
7. Use health checks that reflect real readiness. A service that is starting up should not receive traffic yet.
8. Prefer locality. Keep traffic near its source and fail over only when needed.
9. Secure in stages. Permissive first, strict later, with least-privilege policies.
10. Protect who can edit rules. Use tight access control and require review.
11. Run drills. Practice failure with fault injection and test failover.
12. Check the latest documentation. Names, limits and recommended setups change, so verify before final designs.

Common mistakes

Four illustrative stories show where Traffic Director fits. The companies below are imaginary examples to teach the ideas.

When Traffic Director is a strong choice

• You run many services on Google Cloud, and you want consistent routing, resilience and security.
• You want safe releases with traffic splitting and mirroring.
• You have a mix of virtual machines, containers and hybrid locations.
• You prefer Google to run the control plane rather than running your own.

When to think twice

• You have only a few services and simple needs.
• Your team does not yet have the skills or time to operate proxies and troubleshoot them.
• You need a fully self-hosted, cloud-neutral mesh with deep control of the control plane.
• You cannot accept any added resource use or delay from proxies, and proxyless gRPC does not fit.

Quick answers to questions that beginners ask most often.