Start with a clear definition, a short history and an honest note about names, because the branding has moved over the years.

Anthos is a platform from Google Cloud for managing applications that run in containers, across many different places, from one central point. A container is a neat package that holds a program together with everything it needs to run. Kubernetes is the open-source system that starts, stops, moves and repairs containers across a group of machines. Anthos builds on Kubernetes. It adds the missing pieces that a large organization needs when it has not one cluster but dozens or hundreds of them, spread over several locations.

In simple words, Kubernetes runs your containers, and Anthos manages your Kubernetes. It gives you a single place to see all your clusters, push the same settings to all of them, enforce the same rules everywhere, connect the programs running in different clusters securely and collect their health information in one view.

A short history

Google announced the platform in 2019, at first under the name Cloud Services Platform and soon after under the name Anthos. The idea came from a plain observation: most big companies would never move everything to one cloud. They had existing data centers, rules about where data may live, deals with several cloud vendors and applications that must run close to factories, shops or hospitals. Google decided to take the technology it used for its own huge systems, packaged around Kubernetes, and let customers run it wherever they wanted, with Google Cloud as the place where everything is managed.

Over time the platform grew to support clusters on Google Cloud, on servers in a customer’s own data center, on bare machines with no virtualization software, on other public clouds, and even on clusters that were created by someone else and merely connected for management.

A note about names

Product names in this area have changed more than once, and it helps to know them so that old articles and new documentation make sense together. Google later grouped the Anthos capabilities into a paid edition of its managed Kubernetes service called GKE Enterprise. The on-premises and edge offerings were gathered under the family name Google Distributed Cloud. Individual pieces were renamed too. The service mesh became Cloud Service Mesh, and several clusters-on-other-clouds features became GKE on AWS and GKE on Azure. In this guide we keep using the word Anthos for the overall idea, and we mention the current names where they matter. Because naming and packaging can change again, always confirm the exact current product names and editions in the official Google Cloud documentation before you plan a purchase.

To understand why a platform exists, first feel the pain that it removes.

Suppose a bank starts with one Kubernetes cluster in the cloud. That is easy. A small team learns the tool, deploys programs and everyone is happy. Then the business grows. A second cluster is added for the test environment. A third is added in another region for disaster recovery. A regulator says customer data must stay in the country, so a fourth cluster appears in the bank’s own data center. A recent acquisition brings applications that already run on another cloud. In two years the bank has twenty clusters in four places.

The trouble with cluster sprawl

Each new cluster brings its own settings, its own users, its own versions and its own rules. Soon nobody can answer simple questions. Which clusters are running an old, unsafe version? Does every cluster block programs from running as an administrator? Who can log in to the cluster in the data center? Why does the same program behave differently in two places? Teams spend more time keeping clusters alike than building software for customers.

The trouble with different tools

Each environment also tends to bring different tools for logging, monitoring, access and deployment. A developer who moves between teams must learn a new set of tools each time. Security teams must check each environment on its own. Auditors must be shown evidence from each place in a different format.

The trouble with moving old applications

Many companies also own older programs that run on virtual machines and were never designed for containers. They want to modernize step by step, without a risky rewrite of everything at once. They need a path that lets old and new live side by side under common management.

Ten words carry almost the entire vocabulary of Anthos. Learn them once and the documentation becomes much easier to read.

The big idea behind all ten

Every concept on the list serves a single purpose: sameness at scale. A fleet gathers the clusters. The Connect agent lets Google Cloud reach them. Configuration sync and the policy controller make them alike and keep them safe. The service mesh makes their programs talk securely. Workload identity lets programs sign in without secrets. Together they let a small team manage a very large estate.

A bird’s-eye view of what sits where, and how Google Cloud reaches clusters that live in other places.

The architecture has two sides. On one side is Google Cloud, which hosts the management layer: the fleet records, the console, the policy and configuration services, the service mesh control services and the logging and monitoring tools. On the other side are the clusters, which may be in Google Cloud, in a data center, on another cloud or at the edge. Between them sits the Connect agent in every cluster, which keeps a secure outgoing connection to Google Cloud.

flowchart TD
  subgraph GC["Google Cloud management layer"]
    F["Fleet and console"]
    CS["Config Sync from Git repository"]
    PC["Policy Controller rules"]
    SM["Cloud Service Mesh"]
    OBS["Cloud Logging and Cloud Monitoring"]
    ID["Identity and access"]
  end
  subgraph C1["Cluster on Google Cloud"]
    A1["Cluster agents"]
  end
  subgraph C2["Cluster in own data center"]
    A2["Cluster agents and Connect agent"]
  end
  subgraph C3["Cluster on another cloud"]
    A3["Cluster agents and Connect agent"]
  end
  F --> A1
  A2 -->|"outbound secure connection"| F
  A3 -->|"outbound secure connection"| F
  CS --> A1
  CS --> A2
  CS --> A3
  PC --> A1
  PC --> A2
  PC --> A3
  SM --> A1
  SM --> A2
  SM --> A3
  A1 --> OBS
  A2 --> OBS
  A3 --> OBS
  ID --> F
    

The management layer

This is the part Google runs for you. It stores which clusters belong to your fleet, shows them in the console and provides the shared services. You do not install servers for it. You turn on the features you need and decide which clusters they apply to.

The clusters

Each cluster is an ordinary Kubernetes cluster with a few extra programs installed. Those programs apply configuration from the central repository, enforce policies, run the mesh helpers and send logs and measurements back.

The outgoing connection

A very important design detail is the direction of the connection. The Connect agent calls out to Google Cloud. Google Cloud does not call into your cluster. This means that a cluster inside a strict company network does not need a hole opened in its firewall for incoming traffic. Many security teams find this a pleasant surprise, because opening incoming holes is exactly what they try to avoid.

The Git repository

The desired state of every cluster lives in a Git repository that your team controls. Git keeps a full history, so you always know who changed what and when. Clusters read from it. This repository is the most important single asset in a well-run fleet.

The promise of Anthos is a choice of location. Here are the main places, and what each one means in practice.

The fleet is the heart of Anthos. Almost every feature is switched on for a fleet, not for a single cluster.

What a fleet is

A fleet is a named group of clusters inside a Google Cloud project, known as the fleet host project. Any registered cluster becomes a member. A fleet does not care where its members live, so a single fleet may contain a cluster on Google Cloud, another in a data center and a third on another cloud.

Why grouping matters

A fleet lets you talk about “all our production clusters” as one thing instead of listing twenty names. You can turn on a feature for the fleet and it applies to every member. You can ask the console a question about the fleet, such as which clusters run an old version, and get one answer. You can give a team access to the fleet and let them see all their clusters in one place.

Sameness within a fleet

Fleets work best when their members are alike in important ways. A common idea is called sameness. Namespaces, which are named sections inside a cluster, with the same name across clusters are treated as the same logical space. A namespace called “payments” in three clusters can be managed as one “payments” area, with the same settings and the same access rules. Services with the same name in the same namespace can be treated as one service spread over several clusters. This makes it natural to run an application across clusters.

Teams and scopes

Large organizations rarely give every team access to every cluster. Fleet features let you divide a fleet into smaller scopes, so that a team sees only its own namespaces across the fleet. This lets central platform teams manage the clusters while application teams manage their own programs.

Registration

Clusters on Google Cloud can be registered when they are created or later. For other clusters, registration installs the Connect agent and creates the membership record. After that, the cluster appears in the console, and fleet features can be switched on for it.

A clear mental picture of the engine explains almost every behavior you will notice.

The reconciliation loop

The deepest idea in Kubernetes, and therefore in Anthos, is the reconciliation loop. You do not tell the system “do these steps.” You tell it “this is how I want things to be.” Programs called controllers constantly compare the wanted state with the actual state and take action to close any gap. If a machine fails, the controller notices that fewer programs are running than wanted and starts replacements. This simple loop is the reason these systems repair themselves.

GitOps: the repository is the truth

Anthos applies the same loop to settings. The configuration repository states the desired state of all the clusters. An agent in each cluster keeps reading the repository, compares it to the cluster and fixes any difference. If someone changes a setting by hand in a cluster, the agent notices that the cluster no longer matches the repository and puts the setting back. This is called drift correction, and it is one of the most valuable features, because manual changes are a classic cause of mysterious problems.

The Connect agent and the management channel

The Connect agent keeps a secure, outgoing channel to Google Cloud. Through that channel, Google Cloud can show the cluster in the console and carry commands for approved users, and the cluster can send back its health. Because only outgoing connections are used, clusters behind strict firewalls can still take part.

Admission control: guarding the door

Kubernetes has a checkpoint called admission control. Whenever someone asks the cluster to create or change something, the request passes through the checkpoint before it is accepted. The policy controller plugs into this checkpoint. If a request breaks a rule, such as running a program with excess privileges, the request is refused before anything happens. This is far better than finding out later.

Sidecars: the mesh at work

The service mesh adds a small helper program, called a proxy, beside each application program. All traffic in and out of the application passes through its helper. The helper encrypts traffic, applies traffic rules and records measurements. The application does not know the helper exists, which is why a mesh can add security and visibility without changing application code.

Everything is an API

Finally, every action in Anthos is available through programming interfaces. The console, the command line and automation tools all use the same interfaces. This allows teams to define their whole fleet as code and rebuild it from scratch if needed.

Follow one change from a developer’s idea to a running application in every cluster.

Suppose a team wants to roll out a new version of an application to all production clusters. In a well-run Anthos setup, no one logs in to a cluster and types commands. Instead, the team proposes a change in the Git repository. Reviewers approve it. Automated checks test it. Once the change is merged, the clusters notice it, apply it and report back.

flowchart TD
  D["Developer proposes change in Git"] --> R["Review and automated tests"]
  R -->|"approved and merged"| G["Git repository holds new desired state"]
  G --> S["Config Sync agent in each cluster reads change"]
  S --> P{"Policy Controller check"}
  P -->|"breaks a rule"| X["Change refused and reported"]
  P -->|"allowed"| K["Kubernetes applies the change"]
  K --> H["Cluster reports status back to fleet"]
  H --> O["Team sees result in console and dashboards"]
    

Why this flow is safer than manual work

Every change leaves a trail in Git: who suggested it, who approved it, and exactly what changed. If a change causes trouble, the team can revert it in Git and the clusters follow. Nobody needs to remember which commands were typed on which machine. The policy guard checks every cluster independently, so a mistake that breaks a security rule is caught at the door.

Status flows back too

The agents also report whether each cluster has reached the desired state. A dashboard can show that twenty-eight of thirty clusters are in sync and two are behind, with the reason. This turns a vague worry, “did the rollout work everywhere?”, into a clear answer.

Rolling out in waves

Careful teams do not push a change to every cluster at the same moment. They send it first to a test fleet, then to a small number of production clusters, and then to the rest. Because each cluster or group of clusters can read from its own branch or folder of the repository, the team can control the order simply by deciding when each branch receives the change.

Config Sync is the tool that keeps every cluster matching the repository.

What it does

Config Sync reads settings from a source that you control, most often a Git repository, though other sources such as packaged images and chart repositories can also be used. It applies those settings to each cluster, and it keeps checking. If the cluster drifts from the source, Config Sync restores the wanted state. What are these settings? Almost anything a cluster can hold: namespaces, access rules, network rules, quotas, and even the applications themselves.

One root, many branches of responsibility

Large organizations usually split the work between a central platform team and many application teams. Config Sync supports this with two kinds of sources. A root source, owned by the platform team, can manage anything in the cluster, including cluster-wide settings. Namespace sources, owned by application teams, can manage things only inside their own namespace. The platform team defines the guardrails once, and each application team works freely inside them.

Organizing the repository

A tidy repository is a gift to your future self. Many teams keep one folder for cluster-wide settings, one folder per namespace and separate folders or branches for each environment. A new cluster is added by pointing it at the right folder. A new team is onboarded by adding one namespace folder. The repository grows in a predictable way.

Handling differences

Not all clusters should be identical. A production cluster may need larger quotas than a test cluster, and a cluster in one country may have different settings from another. Teams handle this with overlays, a technique where a shared base is combined with small environment-specific changes. The shared part is written once, and each environment adds only what is different. This keeps the repository small and avoids copy-and-paste mistakes.

Secrets need special care

Never place passwords or private keys in plain text in the repository. Anyone who can read the repository could read them, and Git remembers old versions forever. Keep secrets in a dedicated secret manager and let the clusters fetch them securely at the time they are needed.

Rules are only useful if they are enforced. Policy Controller makes rules automatic.

What Policy Controller does

Policy Controller is built on a well-known open-source policy engine for Kubernetes. It stands at the admission checkpoint and checks every request against rules you have written. It also scans what already exists in the cluster to find things that broke the rules before the rules existed. This is called auditing. The result is two kinds of protection: prevention of new mistakes and discovery of old ones.

Constraints and templates

Rules are written in two parts. A template describes a kind of rule in general, such as “a required label must be present.” A constraint uses the template for a specific case, such as “every namespace must carry a team label.” Google also supplies ready-made bundles of rules for popular needs, such as basic pod security, commonly used cloud benchmarks and certain industry standards. Teams can start from these and add their own.

Examples of rules companies use

• Containers must not run with administrator-level privileges
• Images may be pulled only from approved registries
• Every workload must declare how much memory and processing power it needs
• Public load balancers may not be created in sensitive namespaces
• Every resource must carry labels for team and cost center

Audit first, enforce second

A sensible habit is to introduce each new rule in audit mode. In this mode the rule only reports violations and does not block anything. Teams can see how many existing workloads would break the rule, fix them and then switch on enforcement. This avoids the unpleasant surprise of a new rule suddenly blocking a critical deployment on a busy day.

Trusting what you run: Binary Authorization

Binary Authorization is a companion control. Its question is “was this container image built and approved by a trusted process?” Build systems sign images when they pass checks, and the cluster refuses to run any image without a valid signature. This blocks a whole class of attacks in which a harmful image is slipped into the system.

When you have many small programs talking to each other across clusters, you need a way to secure, control and observe their conversations.

Why a mesh?

Modern applications are often built from many small services, called microservices. A single customer action may involve a dozen of them calling one another. Each call must be encrypted, authenticated, retried when it fails and measured. If every development team builds these abilities into its own code, the result is inconsistent and expensive. A service mesh moves those abilities out of the code and into the platform.

The managed mesh

Google’s offering, originally named Anthos Service Mesh and now called Cloud Service Mesh, is based on the open-source Istio project. Google runs the control part of the mesh for you, and each application gets a helper proxy. Google also provides ready-made views of service health.

What the mesh gives you

Across clusters

The mesh can span clusters. A service in a data center can call a service running in the cloud as though both were next door, with encryption and identity handled automatically. If a service fails in one cluster, traffic can shift to a healthy copy in another. This is the practical basis for running one application across several locations.

The cost of a mesh

A mesh is not free. Each helper proxy uses some processing power and memory, and each call has a small extra delay. Operating a mesh also adds concepts that teams must learn. Adopt it when you truly need its features, such as consistent encryption and traffic control across many services, not simply because it is available.

Clusters in different places are only useful together if they can reach each other and the outside world.

Connecting locations

For a data center and a cloud to work as one system, they need a good network link between them. Google Cloud offers private connections of several kinds, including encrypted tunnels across the public internet and dedicated physical links with predictable speed. Choose by the amount of traffic, the delay you can tolerate and your reliability needs. Plan for at least two independent paths, because a single cable is a single point of failure.

Traffic coming in from users

Users must be sent to a healthy copy of your application. Google provides ingress and gateway features that work across clusters in a fleet. A single global address can send each user to the closest healthy cluster. If one cluster fails, users are sent to another. This is how a multi-cluster application stays available when a whole location has trouble.

Services talking across clusters

Within a fleet, a service in one cluster can be made available to other clusters under a consistent name. Programs call the name and the platform finds a healthy copy, wherever it runs. Combined with the mesh, such calls are encrypted and measured.

Address planning

A quiet but important task is planning network addresses. Clusters need ranges of addresses for machines, for programs and for services. If two clusters that must talk to each other use overlapping ranges, trouble follows. Plan the ranges for the whole fleet before building the first cluster, because changing them later is painful.

Egress and firewalls

Clusters in restricted networks need controlled paths to reach Google Cloud services and package registries. Work with the network team early to allow the outgoing connections that the Connect agent and other components need, and to monitor them.

Who is allowed to do what, and how do programs prove who they are, in clusters spread over many places?

Access for people

Without a platform, every cluster has its own list of users, and removing a person who has left the company means visiting every cluster. With Anthos, people can sign in using their Google Cloud identity or their company’s identity provider, and access rules can be written once for the fleet. A feature called the connect gateway lets authorized users run their usual Kubernetes tools against clusters anywhere, using the secure channel, without needing network access to each cluster.

Role-based access control

Kubernetes has its own system of roles that say what each user may do, such as view or edit. Anthos encourages storing these roles in the configuration repository so that they are the same in every cluster and changes are reviewed. Give people the least access they need. A developer may edit their own namespace but only view others.

Identity for programs

Programs also need identities. An application that reads files from a cloud storage service must prove who it is. The old way was to place a secret key inside the application, which is easy to leak. The better way is workload identity, in which the platform gives each program a verified identity and cloud services trust it directly. No long-lived secret is stored. Fleet-level workload identity extends this to clusters in many places, so that a program in a data center can safely use a cloud service without a stored key.

Separating duties

Separate who can change policies from who can deploy applications, and separate both from who can read sensitive data. Keeping these duties apart makes a single mistake or a single stolen account less dangerous.

Security is not a single feature. It is many layers working together, each limiting the damage when another fails.

The shared responsibility reminder

For clusters running on Google Cloud, Google secures the control plane. For clusters in your own data center, you are responsible for the machines, the physical security, the operating systems and the network around them, while Google provides tools and software. For attached clusters, the original provider secures the control plane, and you secure what you run. Knowing exactly which party owns which layer prevents dangerous gaps.

Compliance and evidence

Because policies and configuration live in Git and are applied automatically, you can show auditors precisely what rules were in force, when they changed and who approved them. Evidence that once took weeks to collect from many environments becomes a report from one place.

Keeping up with fixes

Security fixes arrive regularly in Kubernetes and its components. A fleet makes it possible to see which clusters have not yet received a fix and to schedule upgrades by priority. Delay in patching is among the most common reasons for real-world incidents.

A platform you cannot see into is a platform you cannot trust. Anthos gathers signals from every location into one place.

Logs

System components and application programs write logs. Agents in the clusters send them to Cloud Logging, where they can be searched across all clusters at once. A single search for an order number can show events from a data center cluster and a cloud cluster side by side.

Metrics

Metrics are numbers measured over time, such as processor use, memory use, request counts and error counts. Google provides managed collection for a popular open-source metrics format, so teams can use familiar tools while Google runs the storage. Dashboards and alerts are built on these numbers.

Traces

A trace follows a single request as it passes through many services, showing how long each step took. With a mesh, much of the tracing information is produced automatically. Traces answer the question every operator eventually asks: where did the time go?

What to watch at the fleet level

• Clusters out of sync with the repository, and for how long
• Policy violations found by auditing
• Cluster versions and how close each is to the end of support
• Connection health between clusters and the management layer
• Error rates and delays of services, especially those spanning clusters

Alerts that matter

Alert on symptoms that users feel, such as rising error rates or slow responses, and on conditions that will soon cause trouble, such as a cluster losing contact with the management layer or running out of capacity. Avoid alerts that fire constantly, because people learn to ignore them.

Most companies do not start with a blank page. They own years of working software, and they want to modernize it without breaking it.

The spectrum of modernization

Moving an old application to containers is not a single choice. It is a ladder with several rungs. Each rung costs more effort and gives more benefit, and teams are free to stop on any rung.

Migrating virtual machines to containers

Google offers a tool, called Migrate to Containers, that examines a virtual machine running in a data center or a cloud and produces a container and the settings needed to run it in a cluster. It works best for server programs that run continuously and do not depend on special hardware. After conversion, the program is managed by Kubernetes with the same fleet tools as every other workload, which lets operations teams treat old and new programs the same way.

The strangler approach

A popular way to modernize large systems without a risky big-bang rewrite is to grow new services around the old system and slowly move features across. Requests are routed to the new service for the moved feature and to the old system for everything else. Over time the old system shrinks until it can be retired. The traffic control of the mesh and the ingress features make this routing straightforward.

Kubernetes moves quickly. Keeping many clusters up to date is a constant chore, and a good platform makes it routine.

Why upgrades matter

New Kubernetes versions arrive a few times a year. Each version is supported for a limited period. After that, the version no longer receives security fixes. A cluster that falls too far behind becomes both unsafe and harder to upgrade, because the gap between its version and the current one keeps growing.

How upgrades differ by location

On Google Cloud, upgrades of the managed control plane are handled by Google, and you can choose release channels that decide how quickly your cluster receives new versions. For clusters in your own data center, you run the upgrade with Google’s tooling, in a planned window. For attached clusters, the upgrade is handled by the cluster’s own provider. A fleet view lets you see all three categories in one place and plan accordingly.

An upgrade routine that works

Maintenance windows and budgets

Applications should declare how many copies must stay available during maintenance, using a setting called a disruption budget. Without it, an upgrade might take down all copies of a service at once. Tell the platform the minimum you can tolerate, and it will respect the limit.

A platform that spans locations is a natural foundation for surviving the loss of any one of them.

Levels of failure

Failures come in sizes. A single program can crash, and Kubernetes restarts it. A machine can fail, and Kubernetes moves its programs elsewhere. A whole rack or zone can fail, and well-spread clusters continue. An entire data center or cloud region can fail, and only designs that span locations survive. Plan for each level with a conscious decision about how much you will pay for protection.

Active-active across locations

In an active-active design, the application runs in two or more clusters at the same time, and traffic is shared between them. If one location fails, the others simply take more traffic. This gives the fastest recovery, but it requires that data be replicated across locations and that the application tolerates data being slightly out of date in places.

Active-passive across locations

In an active-passive design, one location serves traffic and another stands ready. Recovery requires switching traffic and bringing the standby fully into use. It costs less and is simpler for data, but recovery takes longer and must be practiced.

flowchart LR
  U["Users"] --> LB["Global entry point and health checks"]
  LB -->|"normal traffic"| A["Cluster in cloud region"]
  LB -->|"normal traffic"| B["Cluster in own data center"]
  A --> DBA[("Replicated database copy A")]
  B --> DBB[("Replicated database copy B")]
  DBA |"replication"| DBB
  LB -.->|"if one location fails, all traffic shifts to the healthy one"| A
    

The role of the repository in recovery

Because every cluster’s desired state is stored in Git, rebuilding a lost cluster is far easier. A new cluster is created, registered to the fleet and pointed at the repository, and the agents recreate its settings, policies and applications. Application data must be protected separately through backups and replication, since Git holds configuration, not customer data.

Practice, practice, practice

A disaster plan that has never been tested is only a hope. Run planned exercises in which you deliberately shut down a cluster or block a network link and watch what happens. The first exercise almost always reveals a surprise, and finding it in a rehearsal is much cheaper than finding it during a real outage.

Every platform has boundaries and a price. Knowing both early avoids unpleasant surprises.

Performance

Anthos does not change how fast your applications run on their own. It adds management layers around them. The main performance costs are the extra processing for helper proxies if you use a mesh, the delay of calls that cross between locations, and the load of agents running in each cluster. For most applications these costs are small, but latency-sensitive systems deserve careful measurement before and after adoption.

Limits

There are limits on how many clusters a fleet may contain, how many nodes each cluster may have, how many services a mesh can handle and how often configuration can sync. Limits are published in the official documentation and change from time to time. Check the current numbers before designing something very large, and design so that you can split into several fleets or clusters if you approach a limit.

Cost

The pricing model has changed over the years, so confirm the current details with Google Cloud before you plan. In general, the platform fee is based on the amount of computing capacity in the clusters under management, usually counted in virtual processors, with different arrangements for clusters on Google Cloud, on your own hardware and on other clouds. Remember that the platform fee is only part of the bill. You also pay for the machines themselves, for network links between locations, for logging and monitoring volume and for the people who operate it all.

No platform is perfect. An honest list of strengths and weaknesses is the best guide to deciding when to adopt it.

The central trade-off

The core trade-off is consistency against complexity. A fleet platform gives you sameness at scale, but it asks you to adopt its way of working: everything in Git, everything through policy, everything as declared state. Organizations that embrace this gain great leverage. Organizations that keep making manual exceptions end up paying for the platform without getting its benefits.

When it is a good fit

• You have, or expect to have, many clusters in more than one location
• Regulations, latency or existing investments keep some workloads outside the public cloud
• You want one security and compliance story across environments
• You have, or can build, a platform team to run it

When to think twice

• You have one or two clusters in a single cloud, where the managed Kubernetes service alone is enough
• Your team is very small and has no appetite for operating Kubernetes
• You do not intend to adopt Git-based change management

Several products tackle the same hybrid and multi-cloud management problem. Knowing the landscape helps you choose with open eyes.

How to choose

Begin with your center of gravity. If most of your technology already lives in one cloud, the management hub from that cloud tends to feel most natural. If you are strongly committed to a particular vendor-neutral platform, that may decide the matter. If you have deep in-house skill and want total control, assemble open-source parts, and accept that you will build and maintain the glue yourself.

The lock-in question

Anthos is built on open-source Kubernetes and Istio, so your applications remain portable. What you commit to is the management layer. This is a milder form of lock-in than depending on a vendor’s proprietary application services, but it is still a real choice. Keep your applications standard, keep your configuration in plain files and you will always be able to move.

Experienced teams keep returning to the same shapes. Learn the good ones and recognize the bad ones early.

Patterns that work

Anti-patterns to avoid

A short checklist from teams who have run fleets in production for a long time.

Best practices

• Keep all configuration and policy in Git with required reviews and automated checks.
• Separate platform and application ownership through root and namespace sources.
• Introduce policy in audit mode first, then enforce.
• Use workload identity instead of stored keys and review access regularly.
• Keep secrets out of the repository and use a secret manager.
• Plan addresses and network paths early and provide at least two independent links between locations.
• Roll out changes in waves, starting with test clusters.
• Upgrade often and in small steps, and track versions on the fleet dashboard.
• Declare resource requests and disruption budgets for every application.
• Practice disaster recovery at least a few times a year.
• Adopt features gradually, proving value at each step.
• Write a short runbook for the person on call.

Common mistakes

A practical order of work for a company that wants to adopt a fleet platform without getting lost.

Putting every idea together in one realistic design.

Imagine a national retail chain with three kinds of locations. Its online shop and analytics run in Google Cloud. Its inventory and finance systems run in two company data centers because of old contracts and data rules. And each of its four hundred stores has a small cluster that runs the checkout software, which must keep working even when the internet is down.

flowchart TD
  GIT["Git repository with configuration and policy"] --> HUB["Fleet in Google Cloud"]
  HUB --> CL["Cloud clusters - online shop and analytics"]
  HUB --> DC["Data center clusters - inventory and finance"]
  HUB --> ST["Store clusters - checkout in 400 stores"]
  CL |"service mesh across clusters"| DC
  ST -->|"sales data sent when connected"| DC
  OBS["Central logging, monitoring and dashboards"] --- HUB
  CL --> OBS
  DC --> OBS
  ST --> OBS
    

One repository, three kinds of clusters

The platform team keeps one repository with a shared base of rules and settings, plus overlays for each kind of location. Every store cluster is identical except for a few details like its store number. A new store is opened by registering its cluster and pointing it at the store folder. Within minutes the checkout software and its rules are installed, without a technician configuring anything by hand.

Staying up when the link is down

A store cluster keeps running its checkout software even if the connection to the center is cut, because the software and its settings are already in the cluster. When the link returns, the agent catches up with any changes and the store uploads its sales. The design accepts that management is temporarily out of touch while the business keeps running.

Policy and security everywhere

A single set of policies says that only signed images from the company registry may run, that no container may run with administrator privileges and that every workload must have resource limits. These rules apply equally to the cloud, the data centers and the stores. Auditors receive one report for all four hundred and three clusters.

Rolling out a new checkout version

When a new version of the checkout software is ready, the team first sends it to a test fleet of a few lab clusters. Then it goes to ten pilot stores. If error rates stay normal for a day, the change is sent to fifty stores, then to the rest in waves overnight when stores are closed. If anything goes wrong, the team reverts the change in Git, and every store follows.

Across the data center and the cloud

The online shop in the cloud needs stock levels from the inventory system in the data center. The service mesh carries these calls securely between the clusters, encrypting them and measuring their delay. If one data center fails, the mesh sends calls to the second data center, and the shop continues with a short pause.

These scenarios show the patterns above in familiar business settings. They illustrate how large organizations face these problems, and are not statements about any specific company’s tools.

Good engineers imagine failure before it happens. These are the situations you will meet and how a well-built fleet behaves in each.

When the repository itself is the problem

Because clusters obey the repository, a serious mistake there spreads fast. Guard against this with required reviews, automated tests, staged branches for different environments and the ability to roll back quickly. Some teams also keep a break-glass procedure for the rare moment when they must act directly on a cluster, with strong logging and a rule that the same change is recorded in the repository afterwards.

When a team cannot reach a cluster

The connect gateway and outgoing channel usually let people reach clusters even when no direct network route exists. If these fail, teams need a documented backup path, such as a jump host inside the network, and the credentials for it kept safely and tested.

When the cause is unclear

Start with the fleet dashboard. Is the cluster in sync? Is the connection healthy? Are policy violations rising? Then look at logs and traces for the specific service. A habit of asking “what changed recently?” solves a surprising share of problems, and because changes live in Git, the answer is easy to find.

Technology succeeds or fails through people. Here is what a healthy operating model looks like.

The platform team

A small group, often called the platform team, builds and runs the fleet as a service for the rest of the company. Their customers are the application teams. They provide clusters, repositories, policies, dashboards and clear instructions, and they treat the platform as a product: they listen to users, measure satisfaction and improve it steadily.

The application teams

Application teams own their programs from design to operation. They write the configuration for their namespaces, respond to alerts about their services and follow the guardrails set by the platform team. The goal is that they never need to understand every detail of the clusters.

Skills worth building

Communication and documentation

A platform grows trust by being predictable and well explained. Keep a short, current guide for new teams, publish the roadmap, announce upgrades well in advance and hold regular open sessions where teams can ask questions. A platform that people find easy to use will be used. A platform that people find confusing will be bypassed, and the consistency you wanted will disappear.

Governance without bureaucracy

Governance means deciding who may change what, and how decisions are recorded. In a fleet it can stay light, because most rules are written into policy and enforced automatically. A short list of decisions needs a human owner: which clusters exist, which versions are allowed, which registries are trusted, and which exceptions to policy are granted. Write these decisions down, give each an owner and review them a few times a year. Exceptions deserve special care. Every exception should have a reason, an owner and an end date, so temporary permissions do not quietly become permanent holes.

Sharing the bill fairly

When many teams share clusters, someone will ask who is responsible for the cost. Labels on namespaces and workloads, such as team and cost center, make it possible to split the bill by usage. Policy can require these labels, so that nothing runs without an owner. Share a monthly report with each team showing its capacity use and its trend. Teams that can see their own costs tend to right-size their workloads without being asked, and the platform team can focus on improving the platform instead of chasing invoices.

Onboarding a new team

A good platform makes the first day easy. A new team asks for a namespace through a simple request that becomes a change in the repository. After review, the platform creates the namespace, applies the standard access rules and quotas, connects the team to logging and monitoring and gives them a starter template for their first application. The team can deploy a simple program within hours, not weeks. This quick start is the best advertisement the platform can have, and it keeps teams inside the guardrails because the easy path is also the correct path.

Handling feedback and change

Collect feedback regularly and publish what you decide to change, so teams see that their voices matter. When you must make a disruptive change, such as retiring an old feature or tightening a policy, announce it early, explain the reason, offer help with migration and set a clear date. Surprises erode trust, while predictable change builds it.

Short answers to the questions people ask most often.

No. Kubernetes runs containers inside a cluster. Anthos manages many Kubernetes clusters across locations, adding fleet management, configuration sync, policy and a service mesh.

Anthos was the original name of the platform. Google later packaged its capabilities as the GKE Enterprise edition and grouped on-premises and edge offerings under Google Distributed Cloud. Confirm current names in the official documentation.

No. Clusters can run on Google Cloud, on your own servers, on other clouds and at the edge. Google Cloud is the place where the management layer lives.

Not necessarily. Your application data can stay in your own location. Management signals such as cluster status and logs go to Google Cloud unless you choose a fully isolated option or limit what is sent.

No. The Connect agent opens an outgoing connection, so clusters behind strict firewalls need only allow that outgoing path.

Configuration sync notices the difference from the repository and restores the stored state. Make changes through the repository.

Only if you need its features, such as encrypted service-to-service calls, traffic control or detailed service metrics. Many teams adopt fleet management and policy first and add the mesh later.

Yes. Clusters from other providers can be attached to a fleet to gain visibility and policy, though some features may be more limited than on clusters Google builds and manages.

Choose a rung on the modernization ladder: leave it, convert it to a container with migration tools, refactor part of it or rebuild it. Convert only where the benefit is clear.

It is most useful when you have many clusters or several locations. With one or two clusters in a single cloud, the managed Kubernetes service alone is often enough.

Pricing has changed over time and is generally based on the capacity of the clusters under management, plus the cost of the underlying machines and network. Check the current pricing page before planning.

Your applications stay portable because they use open-source Kubernetes and Istio. The management layer is a real commitment, so keep configuration in plain files and avoid unnecessary proprietary dependencies.

No. Limits, names, features and prices change from time to time. Always confirm current details in the official documentation before a major decision.