Traffic Director is Google Cloud’s fully managed control plane for service mesh and application networking.

Modern applications are rarely a single program. They are made of many small services that call each other over the network: a checkout service calls an inventory service, which calls a pricing service, which calls a database. Every one of those calls can fail, slow down, or land on an unhealthy server. Each team could write its own code to handle retries, timeouts, load balancing and encryption, but then every service would solve the same problems differently, in different languages, with different bugs.

A service mesh moves those chores out of application code and into the infrastructure. Small proxies run beside each service and handle every network call on its behalf. But a fleet of proxies needs a brain that tells all of them what the rules are. That brain is called a control plane, and running one yourself is hard work. Traffic Director is that brain, run by Google, so you configure the rules once and the proxies everywhere receive them.

A note on the name

Google’s service mesh offerings have been reorganized over time. Traffic Director now sits inside the broader Cloud Service Mesh product family, where it acts as one of the control planes. The ideas in this guide, proxies receiving configuration from a managed control plane, remain the same, but product names, setup paths and recommended configuration styles change. Always confirm current naming and instructions in the official documentation before building.

A compact vocabulary explains nearly every conversation about Traffic Director.

Each term below is explained by what it is, why it exists and where you meet it. Read them slowly, because mesh discussions use these words constantly.

Configuration versus traffic

Understanding the difference between configuration traffic and application traffic is the single most important mental model. Configuration flows from Traffic Director down to proxies, a small amount of data at moments of change. Application traffic flows between proxies and never passes through Traffic Director. This explains both its scalability and its failure behavior, which later chapters explore.

The design separates deciding from doing, and that separation is what makes it scale.

A Traffic Director system has two halves. The first half is Google’s managed control plane, a global service that stores your configuration, tracks the health and location of backends, and serves configuration to proxies. The second half is the data plane, made of proxies deployed with your workloads on Compute Engine virtual machines, Google Kubernetes Engine pods, or hybrid locations. Each proxy opens a long-lived connection to the control plane, receives its configuration, and keeps receiving updates as things change.

flowchart TD
  CFG["Your configuration: routes, policies, backends"] --> TD["Traffic Director Control Plane"]
  HC["Health information about backends"] --> TD
  TD -- "xDS config streams" --> P1["Proxy beside Service A"]
  TD -- "xDS config streams" --> P2["Proxy beside Service B"]
  TD -- "xDS config streams" --> P3["Proxy beside Service C"]
  P1 -- "Application traffic" --> P2
  P2 -- "Application traffic" --> P3
        

Why separating the planes matters

Because application traffic never passes through the control plane, a surge in requests does not load Traffic Director, and the control plane can serve many thousands of proxies without being in the request path. Equally, if the control plane is briefly unreachable, proxies keep operating with the last configuration they received. They simply cannot receive new updates, and new proxies cannot fetch an initial configuration until the connection returns.

The discovery protocols are the language between the control plane and every proxy.

Envoy and proxyless gRPC clients learn what to do through a family of discovery services, known together as xDS. Instead of reading a static configuration file, the proxy subscribes to updates, and the control plane streams new versions whenever something changes. This is what lets you shift traffic, add backends or change timeouts across a fleet in seconds, with no restarts.

sequenceDiagram
  participant Op as Operator
  participant TD as Traffic Director
  participant PX as Proxy
  participant App as Local Application
  PX->>TD: Connect with bootstrap identity
  TD-->>PX: Listeners, routes, clusters, endpoints
  App->>PX: Outgoing request to a service name
  PX->>PX: Match route, pick healthy endpoint
  Op->>TD: Change traffic split to 90 and 10
  TD-->>PX: Updated route configuration
  Note over PX: New rule applies without restart
        

The bootstrap step

A proxy must know how to find Traffic Director and how to identify itself. This starting information, the bootstrap, carries details such as which project and network the proxy belongs to, and it points to Traffic Director’s address. It also provides an identity so the control plane knows which configuration to send. On Kubernetes this is commonly handled by an injector that adds the proxy and its bootstrap to each pod automatically. On virtual machines, instance templates and startup configuration install and start the proxy.

The control plane can steer several kinds of data plane, and each fits different needs.

Traffic Director is not tied to one way of running proxies. The same control plane can configure sidecar proxies next to applications, gRPC clients that need no proxy at all, and standalone proxies acting as gateways. Choosing the right model per workload is an important design decision.

flowchart LR
  subgraph Sidecar["Sidecar model"]
    A1["App"] --> E1["Envoy sidecar"]
  end
  subgraph Proxyless["Proxyless gRPC model"]
    A2["App with gRPC library speaking xDS"]
  end
  subgraph Gateway["Gateway or middle proxy model"]
    G1["Standalone Envoy"] --> S1["Backend services"]
  end
  TD["Traffic Director"] -.-> E1
  TD -.-> A2
  TD -.-> G1
        

Choosing between sidecar and proxyless

Sidecars are the universal answer. They work for any language and any protocol, and they can add encryption and rich traffic policy to applications that know nothing about the mesh. Their cost is resource overhead and a small added delay, since every call passes through two local proxies. Proxyless gRPC removes that overhead for services that already use gRPC with an xDS-capable library, which suits latency-sensitive internal calls. Many organizations use both: proxyless for high-volume gRPC services and sidecars for everything else.

You can express mesh rules using older load balancing resources or newer service routing resources.

Traffic Director has supported more than one set of APIs for describing routing. Understanding both helps when reading documentation, because examples from different years use different resource names. The recommended path and available features change over time, so confirm which model your current documentation favors.

The load balancing resource model

The original model reused the building blocks of Google Cloud load balancing. A forwarding rule with a special internal self-managed scheme represents the service address. A target proxy and a URL map hold the routing rules. A backend service groups the endpoints and carries policies such as health checks, balancing mode and timeouts. Backends are managed instance groups or network endpoint groups. This model is powerful and familiar to people who know Google Cloud load balancers.

The service routing resource model

The newer approach uses resources shaped after the Kubernetes Gateway API. A mesh resource represents the logical mesh that proxies join. Route resources for HTTP, gRPC, TCP and TLS traffic describe how requests are matched and where they go, and they refer to backend services. This model is more consistent with the Kubernetes world and expresses common routing intent more directly.

Before traffic can be routed, the control plane must know where each service actually lives.

Service discovery answers a simple question: for the name of a service, which servers can currently handle it? In a dynamic cloud, the answer changes constantly as servers are added, removed, replaced and moved. Traffic Director tracks this by watching the backends you attach to each service, and it streams the current list of endpoints to proxies.

Why endpoint groups improve balancing

Balancing at the level of instance groups treats each virtual machine as one unit. Balancing at the level of individual endpoints is more precise, because several containers can run on one machine, and each deserves its own share of traffic. Endpoint-level balancing is also what lets the mesh make good decisions about containers that start and stop frequently.

Routing to a broken server is worse than routing slowly, so health information is central.

A health check is a periodic test that decides whether an endpoint is ready for traffic. If an endpoint fails the test, the control plane marks it unhealthy and tells proxies to stop sending it requests. When it recovers, it is added back. In Traffic Director’s model, health checking is done as a managed, centralized function, and the resulting health states are distributed to all proxies, instead of every proxy independently probing every endpoint.

Beyond the basic check: passive detection

Active health checks run on a schedule, so they can miss a server that starts failing between probes. Proxies can add passive detection, called outlier detection, by noticing real failures in live traffic and temporarily ejecting an endpoint that returns repeated errors. Combining active and passive detection gives fast and accurate removal of bad endpoints.

The depth of the health endpoint also matters. A shallow check that only confirms the process is running can miss a service that is running but cannot reach its database. A deep check that tests every dependency can cause the opposite problem: when a shared dependency hiccups, every instance fails its check at once and the whole service looks dead. A good health endpoint tests what the instance itself needs to serve, without failing for reasons it cannot fix.

Because the control plane sees every backend in every zone, it can steer traffic with global awareness.

Ordinary client-side balancing picks any available server. Traffic Director adds location awareness. Proxies prefer endpoints in the same zone as the client, because keeping traffic local is faster and avoids cross-zone network charges. If local endpoints become unhealthy or overloaded, traffic overflows to the next nearest zone, then to other regions, according to configurable rules.

flowchart TD
  C["Client proxy in Zone A"] --> Q{"Healthy and spare capacity in Zone A?"}
  Q -- Yes --> A["Send to Zone A endpoints"]
  Q -- No --> R{"Capacity in nearby Zone B?"}
  R -- Yes --> B["Spill over to Zone B"]
  R -- No --> F["Fail over to another region"]
        

Balancing algorithms

Within the chosen location, a balancing algorithm picks the specific endpoint. Different algorithms suit different workloads, and the available options depend on configuration, so check current documentation.

Capacity and overflow

For overflow decisions to work, the control plane needs to know how much each group can handle. You describe capacity through settings that express maximum rates or connections per endpoint or group. When a zone nears its stated capacity, the mesh begins shifting traffic away, avoiding overload before errors start. Underestimating capacity wastes resources, and overestimating it delays overflow until the group is already struggling.

Traffic Director lets you steer requests by rules, which makes releases safer and experiments easier.

Because proxies make routing decisions from configuration, you can change where traffic goes without redeploying any application. This capability turns releases from a risky all-or-nothing moment into a gradual, observable process.

flowchart LR
  REQ["Incoming requests"] --> PX["Proxy applies route rules"]
  PX -- "90 percent" --> V1["Service version 1"]
  PX -- "9 percent" --> V2["Service version 2 canary"]
  PX -- "Header marks test users" --> V3["Service version 3 experiment"]
  PX -. "Copy of traffic" .-> M["Shadow mirror for testing"]
        

A safe canary release in steps

Failures between services are normal, so the mesh must contain them instead of spreading them.

Resilience policies are the safety equipment of a mesh. They are easy to configure and easy to misconfigure, and the wrong settings can turn a small problem into an outage. Understanding how they interact is a core advanced skill.

The retry storm

Retries seem harmless, but they multiply load. Imagine service A calls B, which calls C, and each layer retries three times. One failing call to C can become twenty-seven attempts, hitting an already struggling service with many times its normal traffic. This amplification can turn a brief slowdown into a prolonged outage. Defenses include limiting retries to a small number, adding a retry budget that caps retries as a share of total traffic, retrying only safe, repeatable requests, and spacing retries with delays.

Timeout hierarchy

Timeouts should shrink as you go deeper into a call chain. If the outer service allows five seconds, the inner calls should allow less, so that the outer service still has time to respond or retry. Inconsistent timeouts cause wasted work: the caller gives up while the callee keeps working on a result nobody wants.

A mesh can give every service an identity and encrypt every call without changing application code.

Inside a network, traffic is often assumed to be safe, but that assumption fails the moment an attacker gets a foothold. Zero trust thinking says every call must prove who is making it and be checked against a policy. A mesh makes this practical, because the proxies can handle the work consistently for every service.

sequenceDiagram
  participant A as Service A Proxy
  participant B as Service B Proxy
  participant CA as Certificate Authority
  CA-->>A: Short-lived identity certificate
  CA-->>B: Short-lived identity certificate
  A->>B: TLS handshake, both present certificates
  B->>B: Check caller identity against policy
  alt Allowed
    B-->>A: Connection accepted, traffic encrypted
  else Denied
    B-->>A: Connection rejected
  end
        

Why short-lived certificates matter

Certificates that last for years are a liability if stolen. Short-lived certificates, rotated automatically within hours or days, limit the window of misuse. This is only practical because the mesh handles rotation without human involvement. Done manually across hundreds of services, certificate rotation is one of the most common sources of outages from expired certificates.

A gradual path to strict security

Turning on strict encryption everywhere at once can break callers that are not yet part of the mesh. A common path begins in a permissive mode that accepts both encrypted and plain traffic, observes who still uses plain connections, brings them into the mesh, and then switches to strict mode. Authorization policies follow a similar path: first observe, then allow what is needed, then deny the rest.

Because every call passes through a proxy, the mesh is a rich, uniform source of telemetry.

Without a mesh, each service must be instrumented separately, and gaps are common. With proxies in the path of every call, you get consistent data about every service-to-service interaction, whatever language each service is written in. The proxies report metrics, logs and trace information, which flow into Google Cloud’s monitoring, logging and tracing services.

Service level objectives

A service level objective is a measurable promise, such as ninety-nine point nine percent of requests succeeding over a month. Mesh metrics make these objectives easy to define and track per service, and alerts can warn when an error budget is being consumed too fast. This shifts conversations from “the system feels slow” to specific, shared numbers.

Traces need cooperation from the application

Proxies can generate trace spans for each hop, but joining the hops into one end-to-end trace requires applications to pass tracing headers from incoming calls to outgoing calls. This small piece of application cooperation is often forgotten, resulting in traces that break into disconnected fragments.

Real organizations rarely live in one place, and the mesh can span clusters, regions and data centers.

A single control plane can serve proxies in multiple clusters and regions within one network setup, giving one consistent set of rules across them. It can also include backends outside Google Cloud, using private connectivity such as VPN or dedicated interconnect and hybrid endpoint groups. This lets services in different environments find and call each other as if they were neighbors.

flowchart LR
  TD["Traffic Director"] -.-> C1["GKE cluster in Region 1"]
  TD -.-> C2["GKE cluster in Region 2"]
  TD -.-> VM["Compute Engine VMs"]
  TD -.-> OP["On-premises services via hybrid endpoints"]
  C1  C2
  C1  VM
  VM  OP
        

Use cases for spanning environments

Challenges that come with scale

Spanning environments adds real concerns. Latency between regions and data centers is higher, so cross-region calls should be an exception, not the default. Network address ranges must not overlap. Failover into another region needs capacity there, and data stores behind services need their own replication plan, because the mesh routes requests but does not replicate data.

Most early problems come from a short list of setup requirements that are easy to overlook.

A troubleshooting ladder

Introducing a mesh into a running system is a change management project as much as a technical one.

A mesh touches every service and every call. Rolling it out all at once invites outages, while a gradual rollout lets teams learn and fix problems at small scale. The best programs begin with clear goals, such as consistent encryption, safer releases or better visibility, and measure progress against them.

People and ownership

Decide early who owns what. A platform team usually owns the mesh itself, the proxies’ versions and the baseline policies, while application teams own the routing and resilience settings for their own services. Without clear ownership, rules accumulate that nobody dares change, and incidents bounce between teams.

The control plane is only one part of what a mesh costs, and often not the largest part.

Pricing for Traffic Director, and for the Cloud Service Mesh family it belongs to, has been based on usage tied to the number of connected clients or proxies, and the details and discounts have changed over time. Confirm the current pricing model on the official pricing page before budgeting. Beyond the service charge, a mesh adds several real costs that should be counted from the start.

Where a mesh saves money

The savings come from avoided work and avoided outages. Teams no longer write and maintain retry, timeout and encryption code in each service. Locality-aware routing reduces cross-zone traffic charges. Safer releases and faster incident diagnosis reduce the cost of downtime. For organizations with many services, these savings usually exceed the mesh’s direct costs. For a handful of services, a mesh may be more machinery than the problem justifies.

A few patterns recur in healthy meshes, and a few traps recur in troubled ones.

Patterns that work well

Anti-patterns to avoid

Several tools overlap with Traffic Director, and each is strongest in a different place.

How to choose

Start with the shape of your estate. If everything runs in Kubernetes and your team knows Istio, a Kubernetes-focused managed mesh may be the most natural path. If you have virtual machines and containers together, or want proxyless gRPC, or need to bridge on-premises services, Traffic Director’s flexibility is attractive. If your main need is balancing client traffic into your applications, Cloud Load Balancing is the simpler answer. Because product lines have shifted, consult the current Google Cloud guidance on which control plane to choose for new projects.

A short checklist separates calm meshes from chaotic ones.

Best practices

Common mistakes

Concrete stories show how the pieces fit together under real conditions.

Short answers to the questions engineers ask most when evaluating Traffic Director.

No. Traffic Director is a control plane. It sends configuration to proxies, and your requests flow directly between proxies and services.

Existing proxies generally keep working with the last configuration they received. They cannot get updates, and new proxies cannot fetch an initial configuration until the connection returns.

No. Services using supported gRPC libraries can run proxyless, receiving configuration directly. Others use sidecars, and some clients can reach services through shared gateway or middle proxies.

No. Cloud Load Balancing mainly handles traffic coming into your applications from clients. Traffic Director configures service-to-service traffic inside your environment. They are often used together.

Yes, through hybrid connectivity endpoint groups and private connections such as VPN or dedicated interconnect, so on-premises and other cloud services can take part. Check current support and networking requirements.

Deploy the new version beside the old, use weighted routing to send a small share of traffic to it, compare metrics for each version, then increase the weight or set it back to zero. No redeployment is needed to change the split.

The most common cause is a firewall rule that blocks health check traffic from Google’s probe ranges. Other causes are a wrong port or path in the health check, or an endpoint that returns errors to probes.

It can, using mutual TLS with automatically issued and rotated certificates, once you enable and configure the relevant security features. A gradual rollout from permissive to strict mode is recommended.

Traffic Director is now part of the Cloud Service Mesh family, serving as a control plane for proxies across environments. Names and recommended setups change over time, so follow the latest Google Cloud documentation for new projects.

Pick a clear goal, such as safer releases or consistent encryption, choose a small pilot set of low-risk services, bring them into the mesh in observe-only mode, and add policy gradually while measuring results.

Traffic Director is a managed control plane that tells a fleet of proxies, or proxyless gRPC clients, how to route, balance, protect and observe service-to-service traffic. It keeps decision making separate from traffic carrying, which lets it scale and lets existing traffic continue even if the control plane is briefly unreachable. It works across virtual machines, containers and hybrid environments, giving them one consistent set of rules.

Its real power appears in how it changes engineering habits. Releases become gradual dials instead of leaps, security becomes automatic identity and encryption instead of manual certificates, and failures are contained by timeouts, retry budgets and circuit breakers instead of spreading. Used with discipline, a gradual rollout and clear ownership, it removes whole categories of repetitive work and risk. Used carelessly, it adds complexity and amplifies failures. As product names and configuration styles evolve under the Cloud Service Mesh umbrella, keep the current documentation close.