ExpressRoute in Azure

ExpressRoute in Azure

A complete, plain-English guide to the private highway between your own network and Microsoft's cloud: how it is built, how traffic finds its way, how to keep it alive when things break, and how to run it well in production.

Imagine your company has a head office and a huge warehouse in another city. Every day, trucks must carry valuable goods between them. You could send the trucks along the public roads, sharing lanes with every other driver. Most days that works. But on a busy Friday evening the traffic crawls, one accident closes a road, and now and then a thief eyes a truck at a red light. So you decide to build your own private road between the office and the warehouse, with no other traffic and a guard post at each end. The goods arrive on time, every time, and nobody outside can see what is on the road.

That private road is the idea behind Azure ExpressRoute. Instead of sending your company data to Azure across the shared public internet, ExpressRoute gives you a private, dedicated connection between your own network and Microsoft’s cloud. In this guide you will learn what it is, why it exists, every part inside it, how traffic travels along it, and the mistakes that trip up even experienced engineers. Each new word is explained the first time it appears, so you can begin with no background at all.

1What ExpressRoute Is

Start with the big picture: what you get, who provides it, and what makes it different from an ordinary internet connection.

ExpressRoute is a service that lets you create private connections between your on-premises network and Microsoft cloud services. The word on-premises simply means the computers and networks that sit in your own buildings, such as an office, a factory, or a rented data center, as opposed to computers that live in the cloud. A private connection means your data does not travel across the public internet. It travels across a line that is set aside for you, supplied by a connectivity provider, which is a telecommunications company such as Equinix, Megaport, AT&T, or Verizon that runs the physical cables and hands the connection over to Microsoft.

The connection typically offers more reliability, faster and more consistent speeds, and lower delays than ordinary internet links. Three words are worth defining right away. Bandwidth is how much data can flow per second, like the width of a pipe. Latency is how long a single message takes to travel from one end to the other, like the time a car takes to drive the road. Jitter is how much that travel time varies from one message to the next. The public internet can swing widely on all three. A private line is far steadier.

Analogy

The public internet is like a crowded city road system where your delivery van shares lanes with everyone. ExpressRoute is like a private railway line built between two stations. It does not depend on traffic lights, rush hour, or strangers, so the schedule stays the same day after day.

Private
Traffic does not cross the public internet
99.95%
Availability target stated for a circuit
BGP
The routing language used to exchange network paths

A short history

In the early days of cloud computing, companies reached the cloud over the internet, often through encrypted tunnels. That was fine for small workloads. But large businesses wanted to move databases, backups, and entire applications to the cloud, and they needed predictable speed and service guarantees, just like the leased lines they already used between their own offices. Microsoft launched ExpressRoute in 2013 to meet that need, partnering with telecommunications companies so that customers could reach Azure through connections they already trusted. Since then the service has grown with higher speeds, a direct-to-Microsoft option for very large customers, link encryption, a feature that joins your own sites together through Microsoft’s network, and many more meeting points around the world.

i
Tip

Prices, speeds, limits, and the list of supported locations change over time. Use the ideas in this guide as a map and confirm exact numbers in Microsoft’s current documentation before you finalize a design.

2The Problem It Solves

Every tool exists because something hurt. Here is the pain that ExpressRoute removes.

Suppose a bank wants to run its customer account system partly in Azure and partly in its own data center. The two halves must talk to each other all day long, sending small requests thousands of times per second and large files once per night. If that traffic uses the public internet, several things can go wrong.

First, unpredictable speed. Internet routes change, and a message may take a longer path at some times than at others. Applications that make many small round trips, such as a program that asks a database a hundred questions to build a single page, feel every extra millisecond. Second, shared risk. Congestion, outages at other companies, and attacks aimed at someone else can still slow down your traffic. Third, exposure. Even with encryption, traffic that crosses the open internet passes through many networks owned by strangers, and some regulators and security teams simply do not accept that for sensitive data. Fourth, cost at scale. Moving hundreds of terabytes over the internet is slow and, with encryption overhead, harder to tune.

The usual alternative, a VPN, or virtual private network, builds an encrypted tunnel across the internet. A VPN is simple, cheap, and good for many cases. But the tunnel still rides on the internet, so it inherits the same swings in speed and the same shared risk. When your needs outgrow it, you need something dedicated.

Pain 1

Unpredictable delay

Internet paths change, so speed and latency vary from hour to hour.

Pain 2

Shared outages

Congestion and failures elsewhere on the internet can still hit your traffic.

Pain 3

Compliance worries

Some rules and security policies prefer or require traffic that avoids the public internet.

Pain 4

Huge data moves

Large migrations and nightly backups need steady, high throughput.

Everyday example: the nightly backup

A hospital group copies several terabytes of medical imaging to Azure every night. Over the internet the copy sometimes finishes at dawn and sometimes does not finish at all. After moving to a dedicated ExpressRoute line, the job finishes at roughly the same time each night, and the hospital no longer worries about the data crossing the public internet.

3Core Concepts: Circuit, Provider, Location, and Key

ExpressRoute has its own small vocabulary. Learn these six words and every diagram will make sense.

The central object is the ExpressRoute circuit. A circuit is the logical connection between your network and Microsoft. It is called logical because it is an Azure resource you create in the portal and configure, while the physical wires underneath are handled by the provider. When you create a circuit you choose a provider, a peering location, a bandwidth, and a SKU (the edition, which we will cover in a later chapter).

A peering location, sometimes called a meet-me location, is a real building, usually a large data center, where Microsoft has network equipment and where providers and customers can plug in. Think of it as a train station where your private railway meets Microsoft’s network. It is not the same as an Azure region. An Azure region is where your virtual machines and databases run. The peering location is only where the connection enters Microsoft’s network. From there, Microsoft’s own private backbone carries traffic to the right region.

When you create a circuit, Azure gives you a service key, a long code that works like a ticket number. You hand the key to your provider, and the provider uses it to connect your line to your circuit. The circuit reports a provider status that moves from not provisioned, to provisioning, to provisioned. Only when it says provisioned is the line ready for configuration.

Inside Microsoft’s building sit special routers called Microsoft Enterprise Edge routers, or MSEEs. Every circuit connects to two of them, a primary and a secondary, so that if one fails the other keeps working. Your side has its own routers, called customer edge routers or CE routers.

TermPlain meaningWhy it matters
CircuitThe logical connection resource in AzureEverything else attaches to it
ProviderThe telecom company supplying the wireDetermines the models, locations, and speeds on offer
Peering locationThe building where the line meets MicrosoftAffects latency and which regions you can reach
Service keyThe ticket code for the circuitLets the provider link the right line
MSEEMicrosoft’s edge routers, two per circuitProvide built-in redundancy on Microsoft’s side
CE routerYour own edge routerSpeaks routing language to the MSEEs
Analogy

Think of booking a private shuttle between your office and an airport terminal. The circuit is your booking. The provider is the shuttle company. The peering location is the terminal building. The service key is your booking reference. The MSEEs are the two airport staff waiting at two different doors, so if one door is locked, you use the other.

!
Warning

Creating a circuit in the portal starts the billing clock for the circuit even if your provider has not finished the line. Coordinate with your provider so that you do not pay for a circuit that is not yet carrying traffic.

4Architecture and Components

Now let us walk the road from your building to your virtual machine in Azure, stopping at every checkpoint.

A complete ExpressRoute setup has pieces on three sides. On your side are your on-premises network and your customer edge routers. In the middle is the provider’s network and the peering location where the cables meet Microsoft. On Microsoft’s side are the MSEE routers, Microsoft’s private backbone, and finally your virtual network, shortened to VNet, which is your own private network inside Azure.

One more piece is needed to get into the virtual network. An ExpressRoute virtual network gateway is a managed Azure component that sits inside a special slice of your VNet named the gateway subnet. A subnet is a smaller piece of a network. The gateway is the doorman at your VNet’s entrance: it accepts traffic coming from the circuit and delivers it to your virtual machines, and it also sends their replies back out. The link between the circuit and the gateway is called a connection.

flowchart LR
    ONP["On-premises network"] --> CE["Customer edge routers"]
    CE -->|"Provider line"| PROV["Provider network"]
    PROV --> LOC["Peering location"]
    LOC --> MSEE1["Microsoft edge router - primary"]
    LOC --> MSEE2["Microsoft edge router - secondary"]
    MSEE1 --> BB["Microsoft backbone"]
    MSEE2 --> BB
    BB --> GW["ExpressRoute gateway"]
    GW --> VNET["Virtual network"]
    VNET --> VM["Virtual machines and services"]
    
Figure 1. The full road from your building to a virtual machine in Azure.

Read the diagram from left to right. A packet leaves your office network, reaches your edge router, and travels over the provider’s line to the peering location. There it enters one of two Microsoft routers. Microsoft’s own backbone, a private global network, carries the packet to the Azure region where your virtual network lives. The gateway there accepts it and hands it to the destination virtual machine. The reply takes the same road in reverse.

Your side

CE routers

Your routers that connect your network to the provider line and speak the routing language to Microsoft.

Middle

Provider and peering location

The cables and the building where your line meets Microsoft’s network.

Microsoft side

MSEE pair

Two edge routers per circuit working together for redundancy.

Azure

ExpressRoute gateway

The managed doorman inside your virtual network.

Link

Connection

The Azure object that joins a circuit to a gateway.

Destination

Virtual network

Your private network holding machines, databases, and services.

5Connectivity Models

There is more than one way to physically reach Microsoft. Which one you can use depends on where you are and who your provider is.

Microsoft describes several connectivity models. Each is a different way for the line to get from your site to the peering location.

ModelHow it worksBest for
Cloud exchange co-locationYou place equipment in the same building as the peering location and connect to Microsoft through a virtual cross-connect at the exchangeCompanies already hosted in a major co-location data center
Point-to-point EthernetA provider supplies a dedicated link directly between your site and the peering locationSites that need a simple, dedicated line
Any-to-any networkYour provider’s wide area network, often a private MPLS network that already joins your offices, adds Azure as another siteCompanies with many branch offices on one provider network
ExpressRoute DirectYou connect straight into Microsoft’s routers at the peering location using your own high speed portsVery large customers needing massive bandwidth or special control

The first three are provider-based, which means a connectivity provider sits in the middle and sells you the circuit. The fourth, ExpressRoute Direct, removes the middle step. You rent whole ports on Microsoft’s routers, available at high speeds such as ten or one hundred gigabits per second, and then you carve those ports into one or more circuits yourself. It also unlocks features such as encryption at the link layer, which we will meet in the security chapter.

Analogy

A provider-based circuit is like buying a seat on a charter bus that runs between your office and the airport. ExpressRoute Direct is like buying your own bus, and then deciding yourself how many seats, which routes, and what rules to follow inside. You get more control, but you also take on more responsibility.

i
Tip

Start by asking which providers serve your site and which peering locations they reach. Your choices are often limited by geography, and a good provider conversation saves weeks of planning.

6Bandwidth, SKUs, and Data Plans

Circuits come in sizes and editions. Choosing well keeps both performance and cost under control.

For provider-based circuits, bandwidth is picked from a menu that begins at fifty megabits per second and rises through several steps to ten gigabits per second. Larger bandwidths cost more. A useful fact is that you can increase a circuit’s bandwidth later without tearing it down, but decreasing it generally means building a new circuit. For ExpressRoute Direct, the ceiling is the size of your ports, and you divide it among circuits.

The SKU is the edition of the circuit, and there are three. The Local SKU lets you reach only the Azure regions in or near the same metropolitan area as the peering location. The Standard SKU lets you reach all Azure regions inside the same geopolitical area, such as all of Europe or all of North America. The Premium SKU is a global edition: it lets one circuit reach Azure regions around the world, supports more routes and more virtual network links, and is the usual choice for global companies.

SKUReachData planBest for
LocalRegions in or near the peering location’s metro areaUnlimited data onlyCost-conscious setups whose workloads sit close to the peering location
StandardAll regions in the same geopolitical areaMetered or unlimitedMost regional business workloads
PremiumGlobal reach to all regions, with higher limitsMetered or unlimitedGlobal enterprises and large route tables

Metered and unlimited data

For Standard and Premium circuits you choose how data leaving Azure is billed. With a metered plan you pay a lower monthly circuit fee plus a charge for each gigabyte that flows out of Azure toward your network. Data flowing into Azure is not charged. With an unlimited plan the circuit fee is higher but all outbound data is included. If you move a lot of data, unlimited usually becomes cheaper beyond a break-even point. If traffic is light, metered saves money.

Real number example

A company sends about two terabytes per day out of Azure to its own analytics system. At that volume the per-gigabyte charges of a metered plan add up quickly, so the finance team compares the monthly total against the higher fixed price of unlimited. If unlimited is lower, they switch. Switching from metered to unlimited is allowed, while going the other way is generally not.

!
Warning

The bandwidth you buy is shared by every virtual network and service using that circuit. Size it for the combined peak, not for one application, and watch the utilization metric so you can raise it before it is saturated.

7Peering Types: Private and Microsoft

A circuit is only a pipe. Peerings decide what kind of traffic is allowed to flow through it.

The word peering means two networks agreeing to exchange routes and traffic. An ExpressRoute circuit can carry more than one peering, each serving a different purpose. Each peering is a separate routing relationship with its own set of addresses.

Azure private peering

Azure private peering connects your on-premises network to your own virtual networks in Azure, using their private IP addresses. Private addresses are the ones used inside company networks, such as those beginning with 10, and they are not reachable from the internet. With private peering your virtual machines look like part of your own network, so you can reach them without public addresses. This is by far the most common peering and the foundation for hybrid designs.

Microsoft peering

Microsoft peering connects your network to Microsoft’s public-facing services, which are reached by public IP addresses. This covers Azure services that live on public addresses, such as storage endpoints, and certain Microsoft online services. You need your own public addresses for network address translation on your side, and you use a route filter, a list that picks exactly which Microsoft services you want to receive routes for. Some Microsoft services follow special rules and approvals for use over ExpressRoute, so check Microsoft’s current guidance before planning around them.

An older third option called Azure public peering has been retired. Its job is now done by private endpoints, which place a private address for a service inside your own virtual network, or by Microsoft peering with route filters.

flowchart TB
    ONP["On-premises network"] --> CIR["ExpressRoute circuit"]
    CIR --> PRIV["Private peering"]
    CIR --> MS["Microsoft peering"]
    PRIV --> VN["Your virtual networks - private IPs"]
    MS --> PUB["Microsoft public services - public IPs"]
    
Figure 2. One circuit can carry two peerings that lead to two different kinds of destinations.
PeeringReachesAddress typeMain use
Private peeringYour Azure virtual networksPrivate IPsHybrid applications, databases, migrations
Microsoft peeringMicrosoft public servicesPublic IPsPublic Azure services and selected Microsoft services
Analogy

Think of the circuit as a private tunnel under a city. Private peering is the staff entrance that leads straight into your own offices in a tower. Microsoft peering is the public lobby entrance that leads to the shops and services shared by everyone in the building.

8Routing and BGP

A private road is useless if no one knows which towns are at the end of it. Routing is how the two sides learn that.

Routers find their way using a shared language called BGP, the Border Gateway Protocol. It is the same language the whole internet uses. In BGP, each side announces, “I can reach these addresses,” and the other side remembers which doorway leads where. A router that learns the same address from two places picks the better one using rules such as the shortest path. ExpressRoute requires BGP between your edge routers and Microsoft’s. Static routes are not used on the circuit itself.

Every network that speaks BGP has an identifier called an autonomous system number, or ASN. Microsoft uses the ASN 12076 for ExpressRoute. You use your own ASN, which may be a private one or a public one. When you set up a peering you also provide a VLAN ID, a tag that separates this peering’s traffic from others on the same physical wire, and a pair of small address blocks called /30 subnets, one for the primary link and one for the secondary link, which give the two ends of each link their own addresses. A BGP session runs over each link, so there are two sessions per peering, one with each Microsoft router.

1

Links come up

The provider line is lit and both ends can see each other on the agreed VLAN.

2

BGP sessions start

Your edge router and each Microsoft router introduce themselves using their ASNs.

3

You advertise your prefixes

You tell Azure which on-premises address ranges live behind you.

4

Azure advertises its prefixes

Microsoft tells you which virtual network address ranges it can reach for you.

5

Traffic flows

Both sides now have a map and can send packets the right way.

Choosing between two paths

When you have two circuits or an ExpressRoute plus a VPN, the network may learn the same destination from both. You steer it with BGP tools. AS path prepending makes a path look longer by repeating your ASN, so it becomes less attractive. Longer prefixes win: a route for a small address range beats a route for a large one that includes it. On Azure’s side, a connection weight lets you prefer one circuit over another for traffic leaving a virtual network.

!
Warning

There is a limit on how many address prefixes a peering will accept. If your edge router advertises more than the limit, the BGP session can be dropped, which cuts the connection. Summarize your routes before advertising them, and check the current limits for your SKU.

Analogy

BGP is like two neighboring towns swapping road maps. Each town says which villages it can reach. If two roads lead to the same village, the drivers pick the shorter one, unless a town puts up a sign saying the long road is the better one.

9Gateways, Connections, and FastPath

The circuit gets you to Microsoft. The gateway gets you into your own network.

As you saw earlier, an ExpressRoute virtual network gateway sits in a dedicated gateway subnet inside your virtual network. You do not manage servers for it. Azure runs it as a pair of managed instances, and you pick a gateway SKU that decides how much traffic it can handle. Smaller SKUs handle around one gigabit per second, middle ones around two, and the largest ones around ten gigabits per second. Newer zone-redundant versions spread the instances across availability zones, which are physically separate buildings within a region, so the loss of one building does not take the gateway down. A scalable option also exists in which the gateway adds or removes capacity units to match demand.

To use the gateway you create a connection that links it to a circuit. If the circuit belongs to a different Azure subscription, the circuit’s owner creates an authorization, which works like a signed permission slip, and the gateway’s owner uses it to make the link. A single circuit can connect to several virtual networks, and one virtual network can connect to more than one circuit, up to limits that depend on the SKU.

FastPath

Normally every packet from your network to a virtual machine passes through the gateway, which inspects and forwards it. That is one extra step. FastPath is a feature that lets the data skip the gateway and flow directly from the Microsoft routers to the virtual machine. Because the gateway is no longer in the data path, you get lower latency and higher packets per second, especially valuable for very busy workloads such as trading systems or large databases. The gateway is still needed to exchange routes, but the actual data does not go through it. FastPath needs the faster gateway SKUs and has some limits on which features work with it, so check the current list of supported scenarios before relying on it.

flowchart LR
    MSEE["Microsoft edge router"] -->|"Normal path"| GW["ExpressRoute gateway"]
    GW --> VM1["Virtual machine"]
    MSEE -->|"FastPath - bypasses gateway"| VM1
    GW -.->|"Still exchanges routes"| MSEE
    
Figure 3. With FastPath the data skips the gateway while the gateway keeps managing routes.
Analogy

The gateway is a security desk at the entrance of an office building where everyone signs in. FastPath is a special express door for regular, trusted staff who go straight to their floor. The security desk still keeps the guest list up to date, but it no longer slows down the daily rush.

i
Tip

Pick a gateway SKU large enough for your peak traffic, since the gateway is often the first thing to saturate. Moving to a bigger SKU later is possible but may need planned downtime, so size with growth in mind.

!
Warning

The gateway subnet is special and must not hold other resources. Give it enough addresses from the start, because resizing it later is disruptive.

10Internal Working and Data Flow

What actually happens, step by step, when an on-premises program talks to an Azure virtual machine?

Suppose an application server in your office sends a request to a database server in Azure at the private address 10.1.0.4. First, your server checks its routing table. It does not own that address, so it sends the packet to its default gateway, a nearby router inside your network. Your internal routers have learned, through your own routing, that traffic for the Azure address range should go to the customer edge router. That router, in turn, has learned through BGP from Microsoft that the Azure range is reachable through the two Microsoft edge routers.

The edge router sends the packet over the provider line to the peering location. Along the way, the provider treats it as ordinary Ethernet frames carrying your VLAN tag. At the peering location, the packet enters one of the Microsoft routers. Both routers can accept traffic, since the pair works in an active-active way, meaning both are working at the same time rather than one waiting idle as a spare. Microsoft’s router sends the packet across the private backbone to the Azure region that holds the destination virtual network. There the gateway, or the direct FastPath route, delivers it to the database server. The reply comes back along the same logical route.

sequenceDiagram
    participant App as Office server
    participant CE as Customer edge router
    participant MS as Microsoft edge router
    participant GW as ExpressRoute gateway
    participant DB as Azure database VM
    App->>CE: Packet for 10.1.0.4
    CE->>MS: Forward over provider line using BGP route
    MS->>GW: Carry across Microsoft backbone
    GW->>DB: Deliver inside virtual network
    DB-->>App: Reply returns by the reverse route
    
Figure 4. One request and its reply traveling the private road.
Step 1

Look up the route

The office server and routers consult tables built by BGP.

Step 2

Cross the provider line

Frames travel on a private VLAN to the peering location.

Step 3

Ride Microsoft’s backbone

Microsoft carries traffic privately to the right region.

Step 4

Enter the virtual network

The gateway or FastPath delivers the packet to the machine.

Active-active redundancy inside every circuit

Remember that a circuit always has two links to two different Microsoft routers. Your edge routers should establish a BGP session on each and use both at the same time. If one link or one Microsoft router fails, traffic shifts to the other in moments. A feature called BFD, bidirectional forwarding detection, helps both sides notice a broken link in a fraction of a second, instead of waiting for slower timers.

Analogy

Imagine a courier company with two drivers who both run the same route. Normally they share the work. If one breaks down, the dispatcher notices at once and gives all deliveries to the other. Customers may see a short delay but never an empty street.

!
Warning

If you connect only one of the two links, you lose the built-in redundancy and the circuit’s availability guarantee no longer helps you. Always configure both, and test a failover before going live.

11ExpressRoute Global Reach

A clever extra: use Microsoft’s network to link your own offices to each other.

Suppose you have an office in London with its own ExpressRoute circuit and another in Singapore with a different circuit. Both already connect to Microsoft. Normally traffic between London and Singapore would use your own expensive wide area links. With Global Reach, you tell Azure to connect the two circuits to each other. Your London and Singapore networks can then exchange traffic across Microsoft’s private global backbone, with no extra provider contract between those two sites.

It is useful for joining data centers, for disaster recovery between two sites, and for reaching an acquired company’s network. Global Reach is available only in selected countries and regions, and only for private peering. Because traffic crosses Microsoft’s backbone, you also pay a Global Reach add-on fee and normal data charges that depend on the zones the traffic crosses.

flowchart LR
    LON["London office"] --> C1["ExpressRoute circuit 1"]
    SIN["Singapore office"] --> C2["ExpressRoute circuit 2"]
    C1 |"Global Reach"| C2
    C1 --> AZ["Azure virtual networks"]
    C2 --> AZ
    
Figure 5. Global Reach lets two on-premises sites talk through Microsoft’s backbone.
Analogy

It is like two companies that each rent a private lane on the same airline’s network. Once the airline links their lanes, the companies can send packages directly to one another through the airline, without building a new road between themselves.

i
Tip

Global Reach is not a replacement for a full wide area network design. It suits moderate site-to-site needs. Weigh its cost and your traffic patterns against your existing provider links before relying on it.

12Redundancy and Resiliency

A single circuit can fail in ways that its two internal links cannot cover. Plan for the bigger failures too.

The 99.95 percent availability figure applies to the circuit’s connection to Microsoft, but your real uptime depends on your whole design. Several things can fail outside that promise: the provider’s cable can be cut by a construction crew, your own edge router can break, or the whole peering location can lose power. Microsoft’s guidance describes three levels of resiliency.

LevelDesignSurvivesDoes not survive
StandardOne circuit at one peering locationFailure of one link or one Microsoft routerProvider failure, site failure, peering location outage
HighTwo circuits in two different peering locations of the same metro areaThe loss of one peering locationA disaster affecting the whole metro area
MaximumTwo circuits in two different metros, with redundant gear and paths on your sideThe loss of a whole metro area and many provider failuresVery rare multi-region disasters
flowchart TB
    DC1["Data center A"] --> CA["Circuit A - provider 1"]
    DC2["Data center B"] --> CB["Circuit B - provider 2"]
    CA --> LA["Peering location - metro 1"]
    CB --> LB["Peering location - metro 2"]
    LA --> AZ["Azure region"]
    LB --> AZ
    
Figure 6. Maximum resiliency uses two circuits, two providers, and two metro areas.

Failure scenarios and recovery

A

One link or Microsoft router fails

The other link of the pair carries the load. Applications see a short blip at most.

B

Your edge router fails

If you have a second router connected to the second link, traffic moves. With only one router you lose the circuit.

C

The provider’s cable is cut

The circuit goes down. A second circuit through another provider or path takes over.

D

A peering location is lost

A circuit at a different location keeps traffic flowing. Without one, you depend on a backup VPN.

E

The gateway has a problem

Zone-redundant gateways survive a zone loss. Others may need a repair or replacement.

F

BGP route limit exceeded

The session drops. Summarize routes and reconnect.

i
Tip

Run a planned failover test. Microsoft provides tools in the portal to rehearse failing a peering or connection so that you can see how your network behaves before a real outage forces the issue.

13Coexistence with VPN

The internet is not your enemy. It makes a fine backup road.

A site-to-site VPN is an encrypted tunnel from your network to an Azure VPN gateway across the internet. It can run side by side with ExpressRoute in the same virtual network, which Microsoft calls coexistence. The usual pattern is to use ExpressRoute as the main path and the VPN as a standby. When the circuit fails, traffic automatically falls back to the VPN. The fallback is slower and less steady, but your applications stay reachable.

For this to work, the network needs to prefer ExpressRoute when both are up. Azure prefers routes learned from ExpressRoute over routes learned from a VPN for the same address range, as long as the prefixes are equal. If your VPN advertises a more specific, longer prefix, it would win instead and quietly steal traffic, so keep the prefixes the same or less specific on the VPN side.

FeatureExpressRouteSite-to-site VPN
PathPrivate provider lineAcross the public internet
EncryptionNot built in by default; add if neededEncrypted with IPsec by default
Speed and consistencyHigh and steadyGood, but varies with the internet
Setup timeDays to weeks with a providerMinutes to hours
CostHigherLower
Best rolePrimary path for important workloadsQuick start, small sites, or backup path
Analogy

ExpressRoute is your private expressway and the VPN is the older public highway with a locked, armored truck. When the expressway is closed for repairs, trucks use the highway. It takes longer, but the goods still arrive safely.

!
Warning

When a firewall sits on your side, make sure return traffic follows the same path as the request. If a request goes out over ExpressRoute but the reply returns over the VPN, many firewalls drop it as suspicious. This is called asymmetric routing and is a classic cause of mysterious failures during failover.

14Security and Encryption

Private does not automatically mean encrypted. Here is how to think about protection.

A common misunderstanding is that ExpressRoute encrypts your data. It does not, by default. The line is private, meaning not shared with the public internet, but the data on it is carried as normal network traffic that your provider could in theory observe. For many companies this level of privacy is enough. For others, regulation or policy demands encryption too. You have several choices, and you can combine them.

Application layer

TLS and similar

Encrypt inside your applications, as you would on any network. This is the most common and flexible choice.

Link layer

MACsec

With ExpressRoute Direct you can encrypt the link between your routers and Microsoft’s routers at the port level.

Network layer

IPsec over ExpressRoute

Run a VPN tunnel across the private line to add end-to-end encryption between your site and Azure, at the cost of some speed.

Authentication

BGP password

Optionally require a shared secret on the BGP sessions so only trusted routers can join.

Beyond encryption, think about who can reach what. Traffic arriving from ExpressRoute lands inside your virtual network, so it is subject to the same controls as any other internal traffic. Use network security groups, which are rule lists that allow or deny traffic at subnets and machine network cards, and place an Azure Firewall or a partner firewall appliance in a central hub network to inspect traffic between your office and Azure. Use private endpoints so that Azure services such as storage and databases get a private address inside your network, reachable across private peering without ever touching a public address.

At the management level, role based access control decides who can create, change, or delete circuits, gateways, and connections. Deleting a connection or advertising the wrong route can cut your entire hybrid network, so protect these with tight roles and resource locks.

Analogy

A private road keeps strangers off your route, but cargo in open trucks can still be seen by the road’s operators. Encryption is the sealed container. MACsec seals the truck on the road, IPsec seals the container end to end, and TLS seals each package individually.

i
Tip

Treat ExpressRoute as a trusted extension of your network, not a trusted network itself. Keep the usual layers of segmentation and inspection in place, because a problem on your office network can now reach Azure over the same road.

15Network Topologies: Hub and Spoke and Virtual WAN

Where you place the gateway shapes how the whole cloud network behaves.

Most companies organize their Azure networks in a hub and spoke pattern. A central hub virtual network holds shared services: the ExpressRoute gateway, a firewall, and management tools. Many spoke virtual networks hold the actual workloads, one for each application or team. VNet peering, a feature that joins two virtual networks so they behave as one for traffic purposes, links each spoke to the hub. The result is that one ExpressRoute gateway and one circuit serve every spoke, and all traffic can be inspected at the hub.

flowchart TB
    OFF["Office and data center"] --> ERC["ExpressRoute circuit"]
    ERC --> HUBGW["ExpressRoute gateway in hub network"]
    HUBGW --> FW["Firewall in hub"]
    FW --> S1["Spoke - application A"]
    FW --> S2["Spoke - application B"]
    FW --> S3["Spoke - data platform"]
    
Figure 7. One gateway and one firewall in a hub serve many spoke networks.

For companies with many sites and regions, Azure Virtual WAN offers a managed version of this idea. Microsoft runs the hubs, including ExpressRoute gateways, VPN gateways, and routing, and you attach circuits, branches, and virtual networks to them. This takes away much of the manual route management. It suits large and growing networks, while a hand-built hub and spoke gives more control for smaller or highly customized ones.

OptionStrengthTrade-off
Hand-built hub and spokeFull control and clear understanding of every routeYou manage routes, peerings, and firewalls yourself
Virtual WANManaged hubs and automated routing at scaleLess low-level control and its own pricing model
!
Warning

Address ranges must not overlap between your office and Azure, or between connected virtual networks. Overlap makes routing ambiguous and is very costly to fix after the fact. Plan address space for the whole company before you build.

16Monitoring, Logging, and Metrics

You cannot manage what you cannot see. Azure shows you the health of every part of the road.

The main tool is Azure Monitor, the central place for numbers, charts, and alerts in Azure. A metric is a measurement taken over time. ExpressRoute publishes metrics for the circuit, for each peering, and for the gateway. A few matter most. ARP availability shows whether the basic link-level handshake with each Microsoft router is working. BGP availability shows whether the routing sessions are up. Bits in per second and bits out per second show how full the pipe is. Dropped bits reveal when traffic is being lost, often a sign of saturation. On the gateway, CPU utilization, the number of routes learned and advertised, and the packets per second show whether it is coping.

A topology view in Azure’s network insights draws a live map of circuits, connections, gateways, and virtual networks, and colors parts that are unhealthy. Connection Monitor in Network Watcher actively sends test traffic between your on-premises machines and Azure machines, measuring real latency and loss along the path. ExpressRoute Traffic Collector can sample the flows on an ExpressRoute Direct circuit and send them to a Log Analytics workspace, a searchable store, so you can see which applications use the bandwidth.

What to watchWhy it mattersTypical action
BGP availabilityA dropped session means routes disappear and traffic stopsAlert instantly and check route limits and provider status
ARP availabilityShows the physical and link status per Microsoft routerContact the provider if one link is down
Bits in and out per secondReveals utilization against purchased bandwidthRaise bandwidth before it reaches saturation
Dropped bitsIndicates loss from congestion or limitsInvestigate bursts and consider a bigger circuit
Gateway CPU and packets per secondThe gateway can be the first bottleneckMove to a larger SKU or use FastPath
Routes learned and advertisedA sudden jump may signal a mistake or a limitReview route summaries and filters

Everyday example: the weekly slowdown

Every Monday morning a company’s cloud application becomes sluggish. The utilization metric shows the circuit reaching its limit between nine and ten o’clock, exactly when a large report job starts. The fix is not a rewrite, but a bigger circuit and moving that job to run after hours.

17How a Connection Is Set Up, Step by Step

Understanding the order of events helps you plan the calendar, because some steps depend on people outside your company.

Setting up ExpressRoute is part cloud work and part physical-world work. The cloud steps take minutes, but the provider’s cable and equipment steps can take days or weeks, so start early. A typical sequence looks like this.

1

Plan the design

Choose the provider, peering location, bandwidth, SKU, data plan, and resiliency level. Plan address ranges so nothing overlaps.

2

Create the circuit in Azure

Azure generates the service key. The circuit’s provider status shows not provisioned.

3

Give the key to the provider

The provider builds the line, connects it to Microsoft, and updates the status to provisioned.

4

Configure the peering

Set the VLAN, the /30 address blocks, and the ASN, and bring up the BGP sessions on both links.

5

Create the gateway

Build the gateway subnet and an ExpressRoute gateway of a suitable size in each virtual network that needs it.

6

Link the circuit to the gateway

Create the connection, using an authorization if the circuit lives in another subscription.

7

Test and harden

Check routes in both directions, test failover, set up alerts, and then move real workloads.

i
Tip

Treat the provider’s delivery date as the most uncertain item on your project plan. Build a temporary VPN first, so that teams can start working in Azure while the circuit is being delivered.

Everyday example: the delayed circuit

A retailer plans to migrate its inventory system on the first of the month. The provider’s line arrives two weeks late. Because the team had already built a VPN and tested everything over it, they simply switched the traffic to ExpressRoute once the circuit was live, and the migration date held.

18Cost and Capacity Management

ExpressRoute is a premium service. Knowing how the bill is built is the best way to keep it friendly.

The bill has several parts. The first is the circuit fee, a monthly charge that depends on bandwidth and SKU. The second is the data charge: with a metered plan you pay per gigabyte leaving Azure, while an unlimited plan includes it. The third is the gateway, billed by the hour according to its size. Fourth come the provider’s own charges for the line itself, which are separate from the Azure invoice and can be a large share of the total. Optional extras include Global Reach, ExpressRoute Direct ports, and premium add-ons. Data coming into Azure is generally free.

Cost leverHow it saves moneyTrade-off
Right-size bandwidthAvoids paying for unused capacityNeeds monitoring so you can grow before saturation
Choose Local where it fitsLower price with unlimited data for nearby regionsLimited to nearby regions
Pick metered or unlimited by volumeMatches the plan to real data movementSwitching back from unlimited to metered is generally not allowed
Share one circuit across many networksSpreads the fixed fee over many workloadsAll workloads compete for the same bandwidth
Use the correct gateway sizeAvoids paying for an oversized gatewayAn undersized one becomes a bottleneck
Keep chatty traffic in one placeReduces data crossing the circuitMay need architectural changes
Analogy

The circuit fee is like the rent on a private railway line, whether or not trains are running. The metered charge is like a toll for each load carried from Azure to you. If you carry loads every hour of every day, a flat yearly pass, the unlimited plan, is cheaper than paying the toll each time.

!
Warning

Two circuits for resiliency means roughly double the cost. Decide which workloads truly need that protection, and put only those on the more resilient design.

19Advantages, Disadvantages, and Trade-offs

No tool is right for everything. A fair look at both sides helps you choose with open eyes.

Advantages

  • Private connection that avoids the public internet.
  • More predictable latency and throughput than internet-based links.
  • High bandwidth options, including very large ports with ExpressRoute Direct.
  • Built-in redundancy with two links to two Microsoft routers.
  • Works with your existing provider network and routing practices.
  • Can reach many regions with the Premium SKU, and join sites with Global Reach.
  • Often helps meet compliance and security expectations.

Disadvantages

  • Higher cost than a VPN, and provider charges on top.
  • Longer setup time because of physical delivery.
  • Not encrypted by default, so extra work is needed for encryption.
  • Needs BGP skills and careful address and route planning.
  • Depends on a provider and on the availability of peering locations near you.
  • A single circuit still has failure risks outside Microsoft’s guarantee.

The big trade-offs

Cost versus predictability. You pay more to get steadier performance, so the question is whether your workloads genuinely need it. Simplicity versus resilience. One circuit is simple but fragile, while two circuits in two metros is strong but more expensive and complex. Privacy versus encryption. A private line is not an encrypted line, so strong security needs extra layers that may cost some speed. Control versus convenience. A hand-built hub and spoke lets you control every route, while Virtual WAN takes over most of the work but hides some details.

20Design Patterns That Work

A pattern is a proven recipe. These recipes appear again and again in healthy ExpressRoute designs.

Pattern

Hub and spoke with one gateway

Put the gateway and firewall in a central hub and share them with every spoke.

Pattern

Two circuits, two metros

Use two providers and two peering locations for the most important workloads.

Pattern

VPN as safety net

Keep a site-to-site VPN ready as a lower-cost backup path.

Pattern

Private endpoints for services

Reach storage and databases by private addresses across private peering.

Pattern

Encrypt what needs it

Use TLS in applications, MACsec on Direct ports, or IPsec over the line for sensitive traffic.

Pattern

Route summarization

Advertise summarized address ranges so you stay below route limits.

Choosing among connection options

NeedGood fitWhy
Quick, low-cost link for a small officeSite-to-site VPNFast to set up and encrypted by default
Remote users connecting from laptopsPoint-to-site VPNDesigned for individual devices
Steady, high throughput for production systemsExpressRoutePrivate and predictable
Very large bandwidth and link encryptionExpressRoute DirectHigh-speed ports and MACsec
Many branches and regions with managed routingVirtual WAN with ExpressRoute and VPNManaged hubs and automatic routing
Resilient connection for critical systemsTwo ExpressRoute circuits plus a VPN backupLayers of redundancy
i
Tip

Write down each workload’s needs for speed, delay, data volume, and recovery targets first. Then the choice of circuit size, resiliency level, and backup design nearly makes itself.

21Anti-Patterns to Avoid

An anti-pattern is a habit that looks sensible but leads to pain. Learn these now so you do not learn them in an outage.

ANTI-PATTERN 01Avoid
What people do

Buy one circuit and call the design highly available because the circuit has two links.

Why it hurts

The two links protect against a Microsoft router failure, not against a provider cable cut, a failure of your own router, or the loss of a peering location.

Better approach

Choose the resiliency level that matches the value of the workload, and add a second circuit or at least a VPN backup.

ANTI-PATTERN 02Avoid
What people do

Assume ExpressRoute traffic is encrypted.

Why it hurts

Sensitive data may travel unprotected across the provider’s network, which could break policy or regulation.

Better approach

Use TLS in applications and add MACsec or IPsec where required.

ANTI-PATTERN 03Avoid
What people do

Use overlapping address ranges between the office and Azure.

Why it hurts

Routers cannot tell which network a packet belongs to, causing failures that are hard to diagnose and expensive to fix later.

Better approach

Plan a company-wide address scheme before building anything.

ANTI-PATTERN 04Avoid
What people do

Advertise thousands of unsummarized routes to Azure.

Why it hurts

Exceeding the prefix limit drops the BGP session and cuts the connection.

Better approach

Summarize routes and monitor the number of prefixes.

ANTI-PATTERN 05Avoid
What people do

Never test failover, trusting that the backup path will simply work.

Why it hurts

Asymmetric routing, missing firewall rules, or a longer prefix on the VPN can turn a failover into an outage.

Better approach

Rehearse failover in a quiet period and fix what you find.

ANTI-PATTERN 06Avoid
What people do

Send all internet-bound traffic from Azure back through the office over ExpressRoute without need.

Why it hurts

It adds delay, consumes circuit bandwidth, and increases data charges for traffic that could leave Azure directly.

Better approach

Send only the traffic that must be inspected on-premises across the circuit, and let the rest use Azure’s own internet path through a firewall.

22Best Practices and Common Mistakes

A short checklist that separates calm operations from late-night emergencies.

1

Design for failure first

Pick the resiliency level based on business impact, and avoid single points of failure on your side.

2

Use both links of every circuit

Configure BGP on the primary and secondary and enable fast failure detection.

3

Plan address space and routes

Avoid overlap, summarize prefixes, and document which side advertises what.

4

Size circuits and gateways with headroom

Measure real peaks and leave room to grow, and watch utilization metrics.

5

Encrypt where required

Combine TLS, MACsec, or IPsec with the private line according to your policies.

6

Monitor and alert early

Alert on BGP and ARP availability, utilization, drops, and gateway load.

7

Test and rehearse

Run planned failover tests, and review the results with your provider and your security team.

8

Automate and lock down

Describe circuits, gateways, and connections as code, restrict who can change them, and apply resource locks.

Common mistakes

  • Paying for a circuit that is not provisioned. Billing may start before the provider finishes the line.
  • Forgetting the gateway limits. The circuit is large, but the gateway is small, and it limits speed.
  • Misconfigured VLAN or address blocks. Mismatches between your router and the Azure peering keep BGP from coming up.
  • Letting a VPN prefix win. A more specific route over the VPN can silently draw traffic away from ExpressRoute.
  • Ignoring DNS. Names for Azure services and on-premises systems must resolve across the connection, or applications fail even though the network works.
  • Assuming ExpressRoute speeds up internet access. It does not. It only improves the path to Microsoft’s network.
!
Warning

A route change on your edge router can affect your entire Azure presence in seconds. Use change control, review route advertisements before applying them, and keep a rollback plan ready.

23Real-World Usage Patterns

Where does ExpressRoute actually shine? These are the jobs it is most often chosen for.

Hybrid applications

A company keeps its core database in its own data center but runs web and application servers in Azure. These tiers chat constantly, so a steady, low-delay private link keeps pages fast and avoids the swings of the internet.

Large data migrations and backups

Moving hundreds of terabytes to Azure, or sending nightly backups to cloud storage, needs sustained throughput. A dedicated circuit makes the transfer time predictable, and unlimited data plans keep the cost steady.

Regulated industries

Banks, healthcare groups, and public bodies often have rules about how data may travel. A private connection, combined with encryption and strong access controls, helps them show that sensitive traffic does not cross the open internet.

Large enterprise databases and SAP

Business systems such as SAP landscapes and big Oracle or SQL Server databases span on-premises and cloud sides during long migrations. ExpressRoute provides the stable connection these systems expect, including during replication between sites.

Disaster recovery

A company replicates its on-premises systems to Azure for failover. The private circuit carries the steady stream of changes, and Global Reach or a second circuit supports recovery between sites.

Virtual desktops and media workloads

Companies delivering virtual desktops or moving large video and design files between studios and the cloud rely on high bandwidth and low jitter to keep screens smooth and file transfers quick.

Choose ExpressRoute when the connection itself is part of the application’s performance and trust story, and treat it as one layer in a larger resilient design.

24Frequently Asked Questions

Q1Is ExpressRoute the same as a VPN?

No. A VPN builds an encrypted tunnel across the public internet. ExpressRoute uses a private connection through a provider and does not cross the public internet, though it is not encrypted by default.

Q2Does ExpressRoute encrypt my data?

Not by default. You can add TLS in your applications, MACsec on ExpressRoute Direct ports, or IPsec tunnels over the circuit.

Q3What is the difference between a circuit and a peering?

A circuit is the logical connection to Microsoft. A peering is a routing relationship on that circuit, such as private peering for virtual networks or Microsoft peering for public Microsoft services.

Q4Why does every circuit have two links?

So that if one link or one Microsoft router fails, the other keeps traffic flowing. You should connect and use both.

Q5Which routing protocol does it use?

BGP. Static routes are not used on the circuit itself, and you run a BGP session on each of the two links.

Q6What is a peering location, and is it the same as an Azure region?

No. The peering location is the building where your line meets Microsoft’s network. The region is where your Azure resources run. Microsoft’s backbone carries traffic from one to the other.

Q7Can one circuit reach several regions?

Yes. Local reaches nearby regions, Standard reaches the whole geopolitical area, and Premium reaches regions around the world.

Q8Do I pay for data going into Azure?

Data coming into Azure is generally free. Data leaving Azure is charged on a metered plan and included in an unlimited plan.

Q9What is FastPath?

A feature that lets traffic skip the gateway and flow directly to virtual machines, lowering latency and raising performance for demanding workloads.

Q10What is Global Reach?

A feature that connects two of your ExpressRoute circuits together so that your on-premises sites can exchange traffic through Microsoft’s backbone.

Q11Can I use a VPN as a backup?

Yes. ExpressRoute and a site-to-site VPN can coexist, with ExpressRoute preferred and the VPN taking over if the circuit fails. Test it for asymmetric routing.

Q12Will it make my internet browsing faster?

No. It improves the private path between your network and Microsoft’s cloud, not general internet access.

25Summary and Key Takeaways

Key Takeaways

  • What it is: ExpressRoute is a private, provider-delivered connection between your network and Microsoft’s cloud that avoids the public internet and offers steadier speed and delay.
  • How it is built: A circuit connects through a provider to a peering location, then to two Microsoft edge routers, then over Microsoft’s backbone to a gateway in your virtual network.
  • How it routes: BGP sessions on both links exchange your office prefixes and Azure prefixes, over private peering for virtual networks and Microsoft peering for public Microsoft services.
  • How to choose: Pick the connectivity model, bandwidth, SKU, and data plan based on location, regions needed, and monthly data volume. Use ExpressRoute Direct for very large needs.
  • How to survive failure: A circuit’s two links are only the start. Add a second circuit in another location, a VPN backup, and test failover, because many failures sit outside Microsoft’s guarantee.
  • How to stay safe: Private is not encrypted. Add TLS, MACsec, or IPsec where needed, and keep network security groups, firewalls, and private endpoints in place.
  • How to run it well: Plan address space, summarize routes, size gateways, use FastPath for hot paths, monitor BGP and utilization, and control changes carefully.