Before the details, let us understand the problem that Cloud Interconnect solves, using a simple story.

The problem: hybrid needs a good road

Most large companies do not move everything to the cloud on one day. Some systems stay in their own data centers because they are old, because rules require it, or because they are expensive to move. Other systems live in the cloud because it is flexible. A business that uses both is called a hybrid business, and the setup is called a hybrid cloud.

In a hybrid setup, the two sides must exchange a lot of data. A shop’s checkout system in the cloud may need to ask the stock database in the old data center how many items are left. A nightly job may copy many terabytes of data into the cloud for analysis. If the road between the two sides is slow, unsteady or insecure, the whole business suffers.

What Cloud Interconnect is

Cloud Interconnect is a group of services that connect your on-premises network, meaning the network in your own buildings, to Google Cloud through a private connection instead of the public internet. Traffic enters Google’s network at a nearby location, close to your equipment, and then rides Google’s own backbone to your cloud resources. Your private addresses stay private, and the data does not cross the open internet.

What it gives you

Where it fits and where it does not

Cloud Interconnect is about connecting networks. It is not a way for the general public to reach your website, and it is not a content delivery system. Visitors still arrive over the internet through load balancers. Interconnect is for your own systems, partners and offices that need a trusted, fast link into your private cloud network.

Cloud Interconnect uses a few ideas from networking. If you know them, skim this chapter. If not, it gives you everything you need.

IP addresses

An IP address is the number that identifies a device on a network, like a house number on a street. Some addresses are public, meaning they work on the open internet. Others are private, meaning they work only inside a company or a cloud network. Interconnect mostly carries traffic between private addresses.

A group of addresses is written as a range, such as 10.0.0.0/16. The number after the slash says how big the range is. A smaller number after the slash means a larger range. Ranges are also called subnets or prefixes.

Routing

A router is a device that decides where to send data next. It keeps a table of ranges and the next stop for each range, called a route table. When a packet arrives, the router looks up the destination and forwards it. A packet is a small piece of data traveling across a network, like one envelope in a long stream of mail.

Bandwidth and latency

Bandwidth is how much data can flow per second, usually in megabits or gigabits per second. It is the width of the road. Latency is how long one packet takes to travel from one side to the other. It is the length of the road, measured in time. A wide road does not make a long road shorter. Interconnect can give you a wide road and a steady, fairly short one, but it cannot beat the speed of light over distance.

BGP: how networks share their maps

BGP, the Border Gateway Protocol, is the language that networks use to tell each other which address ranges they can reach. Two routers set up a BGP session, which is a conversation over a link, and each says, in effect, “I can deliver traffic for these ranges.” The other side learns those ranges and adds them to its route table. If a link breaks, the advertisements stop and the routes disappear, so traffic can use another path.

Each network in BGP has an autonomous system number, or ASN, which is its unique identification. BGP can also attach hints to routes, such as a priority value, so that one path is preferred over another. Cloud Interconnect depends on BGP.

VLANs

A VLAN, or virtual local area network, is a way to split one physical cable into several separate logical lanes. Each lane has a number, called a VLAN ID, written into the data frames. Cloud Interconnect uses VLANs to carry several private connections, one for each cloud network, over a single physical link.

Layer 2 and Layer 3

Networking is described in layers. Layer 2 is about moving frames between devices on the same local link, using hardware addresses. Layer 3 is about moving packets between networks, using IP addresses and routing. When a provider offers a Layer 2 service, it passes your frames along and you run BGP yourself. When it offers a Layer 3 service, the provider runs BGP with Google for you.

Colocation facilities

A colocation facility, often called a colo, is a big building where many companies rent space for their equipment and connect to each other. Google has a presence in many of them. To use Dedicated Interconnect, your equipment must be in, or connected to, one of those buildings, because that is where Google’s network physically meets yours.

This chapter introduces the main words of Cloud Interconnect. Later chapters use them often, so take a little time here.

VLAN attachment, the heart of the idea

A physical Interconnect is just a pipe. By itself it does not know which cloud network to talk to. A VLAN attachment gives the pipe meaning. It says: “this VLAN lane on this pipe connects to this Cloud Router in this region of this VPC network.” You can have several attachments on one pipe, each leading to a different network or region, and each with its own capacity.

Cloud Router, the translator

A Cloud Router is a fully managed service. It is not a box you hold. It runs BGP with your on-premises router, so cloud and on-premises routes stay in sync without anyone editing tables by hand. When you add a subnet in the cloud, Cloud Router can announce it to your side. When you add a range on-premises, your router announces it and Cloud Router passes it into the VPC network as a learned route.

Dynamic routing mode

A VPC network has a dynamic routing mode, either regional or global. In regional mode, routes learned by a Cloud Router apply only to subnets in that router’s region. In global mode, they apply to all regions of the VPC network. A global setting is often chosen so one Interconnect location can serve workloads in many regions, although the data then crosses Google’s backbone and charges may differ.

Capacity

Each physical circuit has a speed. The attachments you create on top of it have their own capacity values, which cap how much each attachment may carry. Capacity is something you plan and pay for, so choose it from real traffic numbers, not guesses.

Now let us put the parts together and see the whole picture of a typical Cloud Interconnect design.

The big picture

On the left of the picture is your own network: servers, switches and a router at the edge. In the middle is the colocation facility, where your cable meets Google’s equipment. On the right is Google Cloud: the VLAN attachment, the Cloud Router and your VPC network with its virtual machines.

flowchart LR
  subgraph ONPREM["Your Data Center"]
    SRV["Servers and Databases"] --> EDGE["Your Edge Router"]
  end
  subgraph COLO["Colocation Facility"]
    EDGE -->|"physical cable"| GPORT["Google Edge Port"]
  end
  subgraph GCP["Google Cloud"]
    GPORT --> ATT["VLAN Attachment"]
    ATT --> CR["Cloud Router"]
    CR --> VPC["VPC Network"]
    VPC --> VM["Virtual Machines and Services"]
  end
  EDGE |"BGP session"| CR

Reading the diagram

Data from a server in your data center goes first to your edge router. The router sends it over a physical cable to a Google port in the colocation facility. From there, the packet enters the VLAN attachment, which hands it to the right VPC network. The Cloud Router does not carry the data itself. It only keeps the route table up to date through BGP, so the VPC network knows that your on-premises ranges are reachable through this link and your router knows how to reach the cloud ranges.

The main components in detail

Two sides of every link

Every Interconnect has two ends, and each end has its own checklist. On your side you need equipment that supports the right optical type and speed, BGP settings, and firewalls. On Google’s side you need the attachment, Cloud Router, firewall rules in the VPC network and routes. Problems often come from one side believing the other side has done something it has not.

Interconnect and other connection options

Cloud Interconnect is not the only way to join networks. Cloud VPN builds encrypted tunnels over the internet. Peering options connect your network to Google’s edge for public services. A later chapter compares them, so you can choose the right tool for each job.

Dedicated Interconnect is the most direct option. It is a physical connection that belongs to you alone.

What it is

With Dedicated Interconnect, you place or rent equipment in a colocation facility where Google has a presence, and a fiber cable joins your equipment to Google’s. The link is not shared with other customers. You get the whole circuit, and you decide how to divide it among VLAN attachments.

Speeds and sizes

Dedicated Interconnect circuits come in standard sizes, commonly 10 gigabits per second and 100 gigabits per second, and a connection can bundle several circuits together for more capacity. The exact options, limits and the list of supported locations change over time, so always check the current Google documentation before you plan.

What you need to use it

• Equipment in, or a connection to, a supported colocation facility
• A router that supports the optical type, speed and BGP
• A Google Cloud project and a VPC network
• Enough traffic to justify the cost, because these circuits are billed by the hour
• People or partners who can manage the physical cabling and the router

The link bundle

Because a single cable can fail, most designs use several. Each cable is a separate circuit. Circuits can be bundled into a single logical connection, sometimes using a standard called LACP so that several wires act as one fat pipe. Google also offers ways to place circuits in separate failure zones, called edge availability domains, so that planned work on one does not cut the other.

Strengths

• The highest capacity and the most control
• Predictable performance, because the path is private
• Lower data transfer charges than the internet for traffic leaving Google Cloud
• Options for adding encryption on the link itself

Limits

• You must be in, or reach, a supported facility
• Setup takes time, because cables, ports and approvals involve several parties
• It is expensive for small needs
• You are responsible for your side of the equipment

Not everyone can place equipment in a distant colocation facility. Partner Interconnect fills that gap.

What it is

Partner Interconnect lets you reach Google Cloud through a supported service provider, which already has a connection to Google. You connect your network to the provider by whatever means they offer, and the provider carries your traffic onward to Google. You do not need your own presence in Google’s facility.

How you order it

You create a VLAN attachment in Google Cloud and receive a pairing key, a code that identifies the request. You give the key to your provider, who uses it to connect your service to your attachment. After the provider has set things up, you review and activate the attachment on the Google side.

Layer 2 and Layer 3 partners

Sizes

Partner Interconnect attachments come in much smaller sizes than Dedicated circuits, starting in the tens of megabits per second and rising to many gigabits per second. This makes it a good starting point for businesses that do not need a huge pipe, and a flexible way to grow later.

Strengths

• No need for your own equipment in a Google facility
• Small capacity options, so cost can match your needs
• Wide reach through many providers and locations
• Providers may bundle other services such as private lines to your offices

Limits

• You depend on the provider’s quality, support and pricing
• You pay both Google and the provider
• There is one more party in the path, so troubleshooting can take longer
• Redundancy must be planned across providers and locations, not assumed

Many companies use more than one cloud. Cross-Cloud Interconnect joins Google Cloud to another cloud provider privately.

What it is

Cross-Cloud Interconnect is a service where Google sets up and manages a direct physical link between its network and the network of another supported cloud provider, such as another major public cloud. You do not have to place equipment in a colocation facility, and you do not need a go-between. Google arranges the link with the other provider.

Why use it

• You run workloads in two clouds and they exchange a lot of data
• You want to move data between clouds without crossing the public internet
• You want predictable performance for multi-cloud applications
• You want less operational work than building your own link

How it differs from the other types

With Dedicated Interconnect you connect your own network to Google. With Cross-Cloud Interconnect, both ends are cloud networks and Google manages the physical link. Capacity options are usually the larger sizes, such as 10 and 100 gigabits per second. The set of supported providers and locations is limited and changes, so check the current list.

Things to watch

• Plan address ranges so the two clouds do not overlap
• Remember that each cloud has its own firewall and routing model
• Understand charges on both sides, since each provider bills for its part
• Design for failure with redundant links, just as with other types

With several choices, how do you decide? This chapter gives a clear way to compare them.

Interconnect, VPN and peering side by side

Questions to ask

1. How much traffic do I have? Estimate peak and average, in both directions.
2. Can I reach a supported facility? If yes, Dedicated is possible. If no, think Partner.
3. Is the other side a cloud? If so, consider Cross-Cloud.
4. How important is the uptime? The higher the target, the more links and locations you need.
5. Do rules demand encryption? Plan MACsec or VPN over Interconnect.
6. How fast do I need it? A VPN can start in hours, while physical circuits can take weeks.

A common growth path

Many organizations start with Cloud VPN, because it is fast to set up and cheap. When traffic grows and the internet path becomes unreliable, they add a Partner Interconnect. When traffic becomes heavy and steady, and they can reach a facility, they move to Dedicated Interconnect, and keep the VPN as a backup. Each step uses the same Cloud Router and BGP ideas, so skills carry over.

Now we look inside. How does a link become useful traffic? The answer is a sequence of steps ending in a working BGP session.

From cable to conversation

First, the physical layer must be up. Light must pass over fiber between your router and Google’s port, and both sides must agree on the speed. If the optical signals are healthy, the circuit is considered up. Next, the VLAN attachment becomes active, which means Google is ready to accept frames with the agreed VLAN ID. Then your router and the Cloud Router set up a BGP session over addresses given for the attachment. Once the session is established, each side sends its routes. Only then can traffic flow.

sequenceDiagram
  participant R as Your Router
  participant A as VLAN Attachment
  participant C as Cloud Router
  participant V as VPC Network
  R->>A: Light levels good, circuit up
  A->>C: Attachment active
  R->>C: Open BGP session
  C-->>R: Session established
  R->>C: Announce on-premises ranges
  C->>V: Add learned routes to VPC
  C->>R: Announce cloud subnet ranges
  R->>R: Add cloud routes to route table
  Note over R,V: Traffic now flows both ways

Link-local addresses

The BGP session needs addresses for the two ends of the attachment. Google uses special private link-local addresses for this, in a range reserved for such links. They are used only for the BGP conversation and are not part of your normal address plan, so they do not clash with your company ranges.

Advertised routes

By default, Cloud Router announces the subnet ranges of the VPC network it belongs to. You can change this by using custom advertisements, which let you announce extra ranges or limit what is announced. Your on-premises router announces its own ranges. Each side learns only what the other side announces, and each side can filter incoming routes for safety.

Route priority and choosing a path

When there are several links, the same range may be learned over more than one path. BGP uses rules to choose. One tool is the MED value, which is a priority number attached to a route, where a lower number is preferred. Another is path length. By setting different priorities, you can make one link the primary and another the backup, or give equal priorities and let traffic share the paths, known as equal-cost multi-path or ECMP.

Failure detection

A link may look up and yet carry nothing, for example when a middle device fails. To notice quickly, routers use BFD, Bidirectional Forwarding Detection, a lightweight check sent many times per second. If replies stop, the session is marked down within a short time and routes shift to other paths. Without it, BGP can take much longer to notice a problem.

Limits

Cloud Router and VPC networks have limits on how many routes can be learned and announced, how many sessions can exist, and so on. Large networks should summarize routes into bigger ranges, rather than announcing thousands of tiny ones. Check the current quotas before design.

Let us follow one piece of data from your office to a cloud server, and then follow the life of the link itself from order to retirement.

One packet, step by step

Suppose an application in your data center wants to send a message to a virtual machine in Google Cloud at the private address 10.20.1.15. Here is what happens.

flowchart LR
  APP["App on a server in your data center"] --> RT1["Your edge router checks its route table"]
  RT1 -->|"route learned from Cloud Router"| CAB["Cable to colocation port"]
  CAB --> VA["VLAN attachment tags and accepts the frame"]
  VA --> VPCR["VPC network routing"]
  VPCR --> FW{"Firewall rules allow it?"}
  FW -->|"yes"| VMX["Virtual machine at 10.20.1.15"]
  FW -->|"no"| DROP["Packet dropped"]
  VMX -->|"reply follows return route"| RT1

1. The application sends the message. The server passes it to your edge router.
2. Your router looks up 10.20.1.15 in its route table. It finds a route learned from the Cloud Router through BGP and sends the packet toward the link.
3. The packet crosses the cable to the colocation port and enters the VLAN attachment.
4. The VPC network routes the packet to the right subnet.
5. VPC firewall rules are checked. If a rule allows the traffic, the packet reaches the machine. If not, it is dropped silently.
6. The machine replies. The reply needs a return route, which is the on-premises range learned by Cloud Router. If this route is missing, replies are lost and the connection hangs.

Why return routes matter

Many beginners test one direction only. Packets go out, but replies cannot come back because the other side does not know the return path. Always verify routes in both directions, and also check that each firewall allows both the requests and the replies.

The life of a link

This chapter explains the order of work in plain words. It is a roadmap, not a command list. Always check the current Google documentation for exact screens and options.

Before you start

• Choose the connection type and the number of links.
• Create or choose the Google Cloud project and the VPC network. Many teams use a Shared VPC and a dedicated networking project for Interconnect.
• Decide on address ranges for both sides. Make sure they do not overlap.
• Decide on ASN values and the BGP design, including primary and backup roles.
• Check roles and permissions, so the right people can create connections and attachments.
• Check which services you need to reach, such as Google APIs, so you can plan the route advertisements.

Dedicated Interconnect steps

1. Order the connection. Pick the facility, the metro, the circuit speed, the number of circuits and the availability domain.
2. Receive the letter of authorization. Give it to the colocation provider so they can connect your cable to Google’s port.
3. Complete the physical connection. Your equipment must match the optical type. Google tests the connection, and when the light levels are good it reports the circuit as ready.
4. Create VLAN attachments. Choose the region, the VPC network, the capacity and the Cloud Router.
5. Configure your router. Use the VLAN ID and the link-local addresses shown on the attachment, and set up the BGP session.
6. Activate the attachment. Some attachments need to be turned on or approved before use.
7. Check routes and firewalls. Confirm that routes appear on both sides and that firewall rules allow the intended traffic.
8. Test. Test normal traffic, high load and failover.

Partner Interconnect steps

1. Choose a provider. Look at the supported provider list for your locations.
2. Create the attachment. Select the region, the Cloud Router and the redundancy design. Receive the pairing key.
3. Give the key to the provider. They complete their side and offer the service.
4. Activate the attachment. After the provider is ready, review and activate it.
5. Set up BGP. For a Layer 2 partner, you configure BGP with Cloud Router. For a Layer 3 partner, the provider handles BGP and you follow their guidance.
6. Test. Verify routes, firewalls, speed and failover.

Cross-Cloud Interconnect steps

1. Choose the remote cloud provider, the location and the capacity.
2. Request the connection. Google arranges the link with the other provider.
3. Create attachments on the Google side, and do the matching setup in the other cloud.
4. Set up routing and firewalls in both clouds, and test.

Infrastructure as code

Once you understand the pieces, describe them in code with a tool such as Terraform. Then every connection, attachment and router is recorded, can be reviewed like a software change and can be rebuilt in the same way every time. This is especially important for the dozens of settings that Interconnect involves.

A link is only useful if it is up when you need it. This chapter explains how to build links that survive failures.

What can fail

• A fiber cable can be cut or unplugged.
• A router or optical module on either side can break.
• A whole building can lose power or be taken offline.
• Google or a provider can perform planned maintenance on a part of the path.
• A wrong configuration can break a BGP session.

The redundancy ladder

Google describes recommended designs by the level of availability they target. The exact numbers and rules are in the current documentation, but the general idea is stable: the more separate paths you build, the better your uptime target, and the higher your cost.

flowchart TD
  subgraph SITE["Your Network"]
    R1["Router A"]
    R2["Router B"]
  end
  subgraph M1["Metro 1"]
    E1["Edge domain 1 link"]
    E2["Edge domain 2 link"]
  end
  subgraph M2["Metro 2"]
    E3["Edge domain 1 link"]
    E4["Edge domain 2 link"]
  end
  R1 --> E1
  R1 --> E3
  R2 --> E2
  R2 --> E4
  E1 --> CRA["Cloud Router in region A"]
  E2 --> CRA
  E3 --> CRB["Cloud Router in region B"]
  E4 --> CRB
  CRA --> VPCX["VPC Network"]
  CRB --> VPCX

Reading the high-availability picture

Every layer has two or more of everything. There are two routers on your side, so one router failing does not cut the link. There are two metros, so one city’s problem does not cut the connection. Within each metro there are two availability domains, so planned maintenance on one does not remove the other. Two Cloud Routers in two regions give the cloud side the same protection.

Active-active and active-passive

In an active-active design, all links carry traffic at the same time, using equal priorities and ECMP. This uses capacity well, but you must remember that after one link fails, the rest must be able to carry everything. In an active-passive design, one link carries traffic and the other waits, with lower priority. It wastes some capacity, but behavior is simple and easy to predict.

Capacity during failure

A common mistake is to size the links for normal traffic only. If two 10-gigabit links each run at 80 percent, a failure of one leaves a single link facing 16 gigabits of demand. Plan so that the remaining links can carry the full peak load, or decide in advance which traffic can be delayed or dropped when capacity is short.

VPN as a backup

Some designs use a Cloud VPN over the internet as a lower-capacity backup to the Interconnect. It gives a different kind of path, which survives problems that affect the physical Interconnect, but it is slower and so has a lower priority. Test it, so you know what the application does when it takes over.

A private link is a good start for security, but it is not the whole story. This chapter shows the layers you should consider.

What the private path gives you

Traffic on Cloud Interconnect does not travel over the public internet, and it uses private addresses. This removes whole classes of risk, such as attackers on the open internet reaching your internal services. But once traffic is on the link, anyone with access to the physical path in the middle could, in theory, read it, and any machine on either side can send traffic across it unless rules stop them.

Encryption options

Each option has a different trade-off. MACsec adds little delay and keeps full speed on supported links, but protects only that hop. A VPN over Interconnect protects the whole path between the endpoints, but it adds overhead and limits the speed of each tunnel. TLS protects the data in every situation, but each application must use it. Many security teams combine more than one layer.

Control who can reach what

• VPC firewall rules decide which on-premises ranges can reach which cloud resources, and the reverse. Use the smallest ranges and the fewest ports that work.
• On-premises firewalls do the same for traffic coming in from the cloud.
• Route filters limit which ranges are announced. Do not announce more than necessary.
• Network tags and service accounts help write rules around identities rather than address lists.

Control who can change the link

IAM, the identity and access system, decides who can create, change or delete connections, attachments and routers. Give these powers to a small group, separate them from the powers over workloads, and record changes in audit logs. An accidental change to a Cloud Router can cut off an entire company.

Private access to Google services

On-premises machines without public addresses can still reach Google APIs through the Interconnect, using special private ranges and settings for private Google access from on-premises hosts. This keeps even API traffic off the internet. The DNS and route settings need care, so follow the current documentation.

Shared VPC and project design

In large companies, attachments are often created in a central networking project and shared with many other projects through Shared VPC. This gives the network team control over the link, and lets application teams use it without changing it. Clear ownership prevents accidents.

A link can be up and still feel slow. This chapter explains the factors that decide real-world speed and how to tune them.

Bandwidth is not the same as speed

A big pipe lets many streams flow at once, but a single connection may still be slow. Two things limit one connection: the delay of the round trip and the way the sending protocol, usually TCP, grows its speed. On a long path, a single stream may use only a part of the link. To fill a large link, use several parallel streams or tools designed for bulk transfer.

Latency

Latency depends mostly on distance. A link into a nearby Google region will have lower delay than one into a far region. Choose the region for your cloud workload with the on-premises site in mind, and avoid sending traffic through a faraway region when the nearest one will do. Chatty applications, which make many small round trips, suffer most from delay.

MTU

The MTU, maximum transmission unit, is the largest packet size allowed on a link. If the MTU does not match across the path, large packets are broken up or dropped, causing mysterious slowness or hanging connections. Interconnect attachments support certain MTU values, and some options allow larger packets than the default. Larger packets can improve efficiency for bulk data, but every device on the path must agree. Set and test the MTU on purpose.

Spreading load across links

With ECMP, traffic is divided between equal paths, usually by hashing details of each connection. This works well with many different flows, but one very large flow stays on one path. If a single backup job must move a huge stream, it may fill one link while others sit idle. Consider running several streams.

Cloud-side location choices

• Place workloads in the region that is closest to your Interconnect location.
• Use global routing only when needed, because cross-region traffic adds delay and cost.
• Keep chatty services together on the same side of the link when possible.

On-premises side choices

• Use routers and switches that can really forward at the link speed.
• Keep the optical modules and cables clean and within specification.
• Avoid sending traffic through slow firewalls or inspection boxes if they cannot keep pace.

Measuring

Measure before and after any change. Useful tests include round-trip time, throughput with parallel streams, packet loss and the behavior of real applications. Measure at different times, as a test at midnight may hide problems seen at noon.

A link you cannot see is a link you cannot trust. This chapter lists what to watch and how to find the cause when something goes wrong.

What to watch

Tools in Google Cloud

• Cloud Monitoring collects metrics from connections, attachments and Cloud Routers, draws dashboards and sends alerts.
• Cloud Logging stores logs, including audit logs that show who changed what, and Cloud Router logs about route and session events.
• VPC Flow Logs sample the conversations between machines, which helps to see who is using the link and how much.
• Network Intelligence Center offers tools such as connectivity tests that check whether a path should work, and topology views that show how parts are connected.
• Maintenance notifications tell you about planned work that can affect a link, so you can prepare.

Alerts that earn their keep

1. A circuit goes down or loses redundancy.
2. A BGP session drops or flaps, meaning it goes up and down repeatedly.
3. Light levels move outside the safe range.
4. Utilization stays above a chosen level, such as 70 percent, for a while.
5. The number of learned routes changes sharply.
6. A planned maintenance notice arrives.

A simple troubleshooting path

Private links cost money, and the bill has several parts. Prices change by location and over time, so use the official pages and the calculator for real numbers. This chapter explains the shape of the cost.

What you pay for

Why the fixed part matters

Because the circuit charges run all day and night, the fixed cost can be large compared with the data charges. A link that is mostly empty is wasteful, while a link that is always full is a bargain. The best value comes from using capacity well, but without leaving yourself without room for failures and growth.

Comparing with other paths

Internet egress and VPN traffic leaving Google Cloud usually cost more per gigabyte than traffic leaving over Interconnect. For a business sending large amounts of data out every month, the savings in data charges can pay for part of the circuit cost. Do the arithmetic with your own volumes, and include redundant links in the calculation.

Cost control habits

• Choose attachment capacities from real traffic measurements, and review them every quarter.
• Avoid sending traffic through distant regions needlessly, because cross-region transfer adds charges.
• Remove unused attachments and connections, since they bill whether used or not.
• Compress and deduplicate bulk transfers where possible.
• Use labels and billing reports to see which teams use the link.
• Set budgets and alerts so a surprise is seen in days, not months.

A pattern is a proven solution to a common problem. An anti-pattern looks fine at first but causes pain later.

Patterns that work

Anti-patterns to avoid

This chapter collects the habits of teams that run private connectivity well, and the mistakes of those who learned the hard way.

Best practices: planning

• Measure traffic before you choose. Use real numbers for peak and average load, in both directions.
• Check the current list of supported locations, providers and sizes before designing.
• Create one company-wide address plan, with room to grow.
• Decide the target availability first, and let it drive the number of links.
• Start physical orders early, because they take the longest.

Best practices: building

• Use a hub networking project with Shared VPC for central control.
• Describe everything as code, and keep it in version control.
• Use clear names that show metro, zone, region and purpose.
• Use BGP authentication where it is supported, and filter routes in both directions.
• Enable BFD for faster failure detection when your design supports it.

Best practices: operating

• Monitor circuits, light levels, sessions, routes and utilization, with alerts.
• Read maintenance notifications, and arrange traffic before planned work.
• Review capacity and cost every quarter.
• Run failure drills, and write down the results.
• Keep contact details and diagrams where on-call staff can find them.

Common mistakes

Let us look at the kinds of work that Cloud Interconnect carries in practice. Companies seldom publish exact designs, so these examples describe typical patterns.

What these stories share

Every example uses the same habits. They measure first, choose the smallest option that meets the need, build for failure, secure both the link and the traffic, and watch the link all the time. The industries differ, but the thinking is the same.

A link by itself is only a road. To make it useful for a whole company, you need patterns for who connects to what, how names are found and how Google services are reached privately.

Hub and spoke

In a hub and spoke design, one central network, the hub, holds the Interconnect attachments, Cloud Routers and shared services such as security inspection and name servers. Many other networks, the spokes, connect to the hub and use its link to reach on-premises systems. This avoids buying a separate link for every project or team. The hub team runs the road, and the spoke teams drive on it.

Ways to connect the spokes

• Shared VPC lets many projects use subnets of one central network, so they all use the same attachments naturally.
• VPC Network Peering joins two VPC networks so they can talk privately. Routes learned from Interconnect in one network can be exported to a peer, if you enable the options, but peering is not transitive by default, so plan the paths with care.
• Network Connectivity Center is a Google service that acts as a central manager for connecting many networks and sites, including Interconnect attachments, in a hub model. It can let sites send traffic to each other through Google’s network.
• Cloud VPN can link smaller sites to the hub when they do not need an Interconnect of their own.

Transit through Google

Some companies have several offices or data centers, each with its own link into Google. A well-known idea is to use Google’s backbone to carry traffic between those sites, so site A reaches site B through the cloud. This can replace a costly private network between offices. It needs the right services and routing settings, and the current rules and charges must be checked, since not every combination is allowed or free.

Private Google Access from on-premises

Your on-premises machines often need Google services such as storage or analytics. Normally these services have public addresses. With private Google access for on-premises hosts, requests travel over the Interconnect to special private ranges that lead to Google APIs, instead of crossing the open internet. Your machines do not need public addresses, and traffic stays on the private path.

To use it, you announce the special API ranges from Cloud Router to your network, so your routers know to send API traffic over the link. You also set up your DNS so that the names of Google services resolve to those private ranges. If DNS gives public addresses, traffic will try to leave through the internet instead.

DNS across the link

DNS, the domain name system, turns names into IP addresses. In a hybrid setup, two sets of names exist: on-premises names such as a company file server, and cloud names such as a database in a project. Machines on each side should resolve names on the other side without help from the public internet. Google Cloud DNS offers private zones and forwarding rules. Forwarding sends questions for on-premises names to your DNS servers, and your on-premises DNS can forward cloud names to Google. A resolver link that works in both directions keeps names consistent.

Services reached through private connections

Besides Google APIs, many managed services can be reached by private addresses inside your VPC network. When on-premises systems need such services, the same routes that serve your virtual machines can serve them, as long as the routes are exported and the firewall rules allow it. Check each service’s own documentation, because some services use special peering arrangements that affect whether on-premises routes reach them.

Keeping the design simple

1. Choose one place, the hub, for connections and routers.
2. Give each team its own subnets and projects, and share the hub through Shared VPC or a similar model.
3. Use the same naming and tagging rules everywhere.
4. Write down which ranges are announced where, so nobody has to guess.
5. Test name lookups, routes and firewall rules after every change.

Most teams already have some connection: a VPN, a leased line or just the internet. This chapter explains how to move to Cloud Interconnect without a night of panic.

Starting from a VPN or the internet

If your systems talk through a VPN or the public internet today, the move is mostly about adding a new, better path and then shifting traffic to it. The old path can stay in place as the backup, which makes the move safer, since you can step back at any time.

Starting from a private line to another place

Some companies have a private network from the older days, with leased lines between offices. When moving workloads to the cloud, you can keep that network and attach it to Google through Partner Interconnect, using the same provider. This reuses what exists, and the office networks continue to work as before.

A safe migration path

Using route priorities for a gentle cutover

Because Cloud Router and your router use BGP, you can control which path wins by changing priorities. Initially, the old path has the better priority. When you are ready, you make the new path better, and traffic moves within seconds. If something goes wrong, you reverse the change. This is much safer than changing addresses or applications.

Things that surprise teams

• Different paths, different delay. The new path may be shorter or longer. Applications that were tuned for the old delay may behave differently.
• Asymmetric routing. Requests may use the new path while replies use the old, which stateful firewalls often reject. Make sure both directions follow the same path.
• MTU differences. A VPN reduces the packet size, while Interconnect may allow larger ones. Mixed settings can cause failures.
• Firewall changes. Traffic may now arrive from different source ranges, and rules need to follow.
• Monitoring gaps. Dashboards built for the old link do not cover the new one. Prepare them first.
• Contract and lead time. Physical links take weeks, and providers may have notice periods for the old service.

Building the link is only the beginning. Running it for years needs planning, routines and clear rules about who may do what.

Capacity planning

Capacity planning means making sure that the link is big enough today and tomorrow, without paying for much more than you need. Start with data. Collect throughput measurements for each attachment, including the peak, the average and the pattern over the week and the month. Then add growth: new applications, more users, bigger datasets.

1. Find the peak. Use short measuring periods, because averages hide spikes.
2. Add headroom. Keep utilization well under the limit in normal operation, so bursts and failures do not cause drops.
3. Include the failure case. Ask what happens to load when one link or one metro is gone.
4. Forecast growth. Use history and business plans, such as new launches and migrations.
5. Plan lead time. Physical changes take time, so start the upgrade before the link is full.

Growing a connection

Capacity can often be increased in a few ways. You can raise the capacity of a VLAN attachment, up to the circuit’s limit. You can add more circuits to a Dedicated connection. You can add more attachments and spread traffic with ECMP. For Partner Interconnect, you can ask the provider for a larger service. Each method has rules about downtime and cost, so check them and schedule changes in quiet periods.

Maintenance

Google and providers perform maintenance from time to time, and they announce it ahead. With a redundant design in separate availability domains, maintenance on one side should not cut service. Before each event, check that the other links are healthy and have room for the load. After the event, check that sessions are back and routes are normal. Keep a calendar of upcoming events.

Change management

• Make every change through code review and a ticket.
• Test in a separate environment when possible.
• Apply changes to one link at a time, never to both of a redundant pair together.
• Have a way back for every change.
• Record who changed what and when, using audit logs.

Roles and responsibilities

Runbooks and reviews

A runbook is a short guide for handling a known problem, for example “a BGP session is down.” Write runbooks for the common failures, keep them short and test them. Hold a regular review, such as every quarter, to look at capacity, cost, incidents, upcoming maintenance and planned projects. A calm routine prevents most emergencies.

Support and escalation

Know in advance how to open a support case with Google and with your provider, what information they ask for, and who in your company may do it. During an outage, the time to find a phone number is too late. Keep the details of IDs, locations and contacts together in one document.

Reading about failures before they happen is cheaper than learning from them at night. Here are common situations, what causes them and how to think about each.

Scenario 1: The BGP session will not come up

What you see: The circuit is up and the attachment is active, but the session stays down and no routes appear.

Likely causes: The VLAN ID on your router does not match the attachment. The BGP addresses are entered wrongly or on the wrong interface. The ASN is wrong. An authentication key differs on the two sides. A filter or firewall on your router blocks BGP packets.

How to think: Compare every value on your router with the values shown on the attachment and the Cloud Router, one by one. BGP is strict, and a single wrong digit keeps the conversation from starting.

Scenario 2: Routes are learned, but nothing works

What you see: The BGP session is established and both sides show the expected routes, yet connections time out.

Likely causes: Firewall rules in the VPC network or on-premises block the traffic. The return route is missing or points elsewhere. The request and the reply take different paths, and a stateful firewall rejects the reply. The test is aimed at a machine that itself has a local firewall.

How to think: Test in small steps. Check whether a request arrives, whether a reply leaves, and where it goes. Connectivity tests and flow logs help to show where packets disappear.

Scenario 3: Small requests work, big transfers hang

What you see: Simple pings and short web calls succeed, but file copies or database syncs stall.

Likely causes: The MTU is different on parts of the path, and large packets are dropped while small ones pass. Some firewalls also block the messages that tell senders to use smaller packets, which makes the problem harder to see.

How to think: Test with packets of increasing size, and find the largest that passes. Then align the MTU settings along the path, or lower the packet size on the endpoints.

Scenario 4: The link is fast in tests, slow for the application

What you see: A bulk test reaches a high speed, but the real application moves data slowly.

Likely causes: The application uses a single stream, and the delay limits its speed. It makes many small requests one after another, so delay adds up. A device in the middle, such as a firewall or inspection box, cannot keep up. The disk or database on one side is the real bottleneck.

How to think: Do not blame the link first. Measure each part: the network alone, the application alone and the storage alone. Use parallel streams for bulk jobs and fewer round trips for chatty ones.

Scenario 5: One link fails and everything slows down

What you see: A planned or unplanned loss of one link causes slow responses and many errors, even though a second link exists.

Likely causes: The remaining link is too small for the full load. All traffic shifted to one side of an unbalanced design. BGP priorities were set so that the backup path is the slow VPN. Failure detection is slow, so traffic keeps going into a broken path for a long time.

How to think: Compare the real peak load to the capacity that survives a failure. Check priorities and consider BFD for faster detection. Then repeat a controlled test.

Scenario 6: Traffic goes out over the internet by surprise

What you see: Bills show internet transfer, or logs show public addresses where private ones were expected.

Likely causes: DNS returns public addresses for Google services or for internal names. A more specific route over the internet or VPN wins over the Interconnect route. The ranges for private Google access were never announced to the on-premises network.

How to think: Follow a single request from start to end. Check which address the name resolves to, which route the router picks, and which interface the packet leaves through.

Scenario 7: A route disappears after a change

What you see: After a configuration change, part of the network can no longer be reached.

Likely causes: A custom advertisement replaced the default one and left out some ranges. A route filter is too strict. The number of routes reached a limit and extra routes were dropped. Someone removed a subnet that was still in use.

How to think: Compare the announced and learned routes before and after the change. This is why changes should come through code review, so the difference is visible in the change itself.

A general method

1. State the problem exactly. Which source, which destination, which port, since when, and what changed.
2. Find the layer. Is it physical, link, routing, firewall, name resolution or application?
3. Check from both ends. A problem often looks different from each side.
4. Change one thing at a time. Otherwise you cannot know which change helped.
5. Write down the result. Add it to a runbook so the next person solves it faster.

Short, direct answers to questions that beginners ask most often.

You have now seen Cloud Interconnect from the outside and the inside. Here is the whole story in one place.

Cloud Interconnect gives your network a private road into Google Cloud. You choose Dedicated Interconnect for a direct physical link, Partner Interconnect to go through a provider, or Cross-Cloud Interconnect to join another cloud. A VLAN attachment ties each link to a Cloud Router and a VPC network, and BGP keeps both sides informed about where each address range lives. You then build redundancy, add encryption and firewalls, watch the link closely and keep a firm hand on cost.