Every Connection Is Two Tunnels, by Design, Not by Choice

The single most important advanced fact about AWS Site-to-Site VPN, and the one most frequently glossed over in basic tutorials: every VPN connection provisions exactly two IPsec tunnels terminating at two different, physically separate AWS endpoints — not as an optional high-availability upgrade you can decline, but as the default, only way a connection is created. This is a deliberate design decision reflecting the reality that AWS’s own infrastructure performs maintenance and can experience localized failures, and a single-tunnel design would tie your hybrid connectivity’s availability to a single endpoint’s uptime. The advanced implication: a customer-side design that only terminates one of the two tunnels (common when a customer router has a single WAN interface) discards half of AWS’s built-in redundancy before it ever gets used, even though the connection is billed and provisioned as if both were active.

Virtual Private Gateway Versus Transit Gateway as the VPN Attachment Point

A Site-to-Site VPN connection terminates at either a Virtual Private Gateway (VGW), attached to a single VPC, or a Transit Gateway, which can aggregate many VPCs, many VPN connections, and Direct Connect attachments behind one routing domain. The advanced distinction goes beyond “VGW is older” — a Transit Gateway attachment supports Equal-Cost Multi-Path (ECMP) routing across multiple VPN connections, which a VGW attachment does not, making Transit Gateway the only path to aggregate throughput beyond a single connection’s per-tunnel ceiling. Any architecture anticipating growth beyond one VPC or needing throughput above a single tunnel’s limits should default to Transit Gateway from the outset, since migrating from a VGW-terminated design later is a non-trivial re-architecture, not a simple setting change.

Static Routing Versus BGP: Not Just “Which Is Easier to Set Up”

Static routing requires manually specifying the customer-side CIDR ranges the AWS side should route toward, but critically, it gives AWS no dynamic signal about tunnel health beyond IPsec-layer liveness — a static-routed tunnel that’s technically still up but whose customer-side network path is broken can continue receiving traffic that silently blackholes. BGP, by contrast, actively advertises reachable prefixes and withdraws them the moment a route becomes genuinely unavailable on the customer side, giving AWS a real-time, application-independent signal to shift traffic to the healthy tunnel. This is the advanced reason BGP is the default recommendation for any production deployment: it isn’t about configuration convenience, it’s about whether failure detection actually reflects true end-to-end reachability or merely tunnel-layer liveness.

Slack’s Hybrid Connectivity During Early Cloud Migration

Companies migrating incrementally from on-premises data centers commonly use Site-to-Site VPN as the first hybrid connectivity layer — fast to provision (minutes, not the weeks a physical Direct Connect circuit requires), and sufficient for the traffic volumes typical of an early-stage migration where most services still live on-premises and only a subset of traffic crosses the hybrid boundary. The advanced lesson embedded in this common pattern: VPN and Direct Connect aren’t mutually exclusive alternatives chosen once — many production hybrid networks run both simultaneously, with VPN serving either as the initial bridge before Direct Connect is provisioned, or as the resilient failover path once Direct Connect becomes the primary link.

Route Propagation Into the VPC’s Own Route Tables

A subtlety that trips up otherwise-solid designs: routes learned by a Virtual Private Gateway or Transit Gateway from a VPN connection’s BGP session don’t automatically appear in every VPC subnet’s route table. Route propagation must be explicitly enabled on the relevant route tables (or, for Transit Gateway, the attachment must be associated with the correct Transit Gateway route table), and forgetting this step is one of the most common “the tunnel is up, BGP shows routes, but traffic still can’t reach the on-premises network” support cases — the VPN’s routing layer working correctly is necessary but not sufficient if the VPC’s own routing configuration never learned about it.

Transitive Routing Behavior Differs Between VGW and Transit Gateway

A Virtual Private Gateway attached to a single VPC has no concept of routing traffic onward to a different VPC — any multi-VPC connectivity requires a separate mechanism like VPC peering layered on top. A Transit Gateway, by contrast, natively supports transitive routing between all its attachments (subject to route table associations), meaning a single VPN connection into a Transit Gateway can reach every attached VPC without any additional peering configuration. This is one of the clearest, most concrete reasons advanced multi-VPC hybrid architectures default to Transit Gateway rather than VGW from the start.

flowchart TB
    ONPREM["On-Premises Router
Customer Gateway Device"] -->|"Tunnel 1 (IPsec)"| AZ1["AWS VPN Endpoint
Availability Zone A"]
    ONPREM -->|"Tunnel 2 (IPsec)"| AZ2["AWS VPN Endpoint
Availability Zone B"]
    AZ1 --> TGW["Transit Gateway
or Virtual Private Gateway"]
    AZ2 --> TGW
    TGW --> VPC1["VPC A"]
    TGW --> VPC2["VPC B"]
    ONPREM -.->|"BGP session per tunnel"| AZ1
    ONPREM -.->|"BGP session per tunnel"| AZ2
        

IKE Negotiation: Two Phases Before Any Data Moves

Each tunnel establishes through the standard IPsec IKE (Internet Key Exchange) process in two phases, and understanding both matters for real troubleshooting. IKE Phase 1 negotiates a secure, authenticated channel between the two endpoints themselves — exchanging and validating the pre-shared key (or certificate, if configured), agreeing on encryption and hashing algorithms, and establishing what’s called the ISAKMP security association. IKE Phase 2 then negotiates the actual IPsec security associations that will protect data traffic, using the secure channel Phase 1 established. A tunnel stuck in a perpetual “negotiating” or flapping state is, in the overwhelming majority of real cases, a Phase 1 mismatch — an incompatible encryption algorithm, hashing method, Diffie-Hellman group, or a pre-shared key typo — and advanced troubleshooting starts by checking Phase 1 parameters match exactly on both sides before looking anywhere else.

sequenceDiagram
    participant CGW as Customer Gateway
    participant VPN as AWS VPN Endpoint

    CGW->>VPN: IKE Phase 1 — negotiate ISAKMP SA
    VPN-->>CGW: Phase 1 SA established (authenticated channel)
    CGW->>VPN: IKE Phase 2 — negotiate IPsec SA
    VPN-->>CGW: Phase 2 SA established (data channel ready)
    CGW->>VPN: BGP session establishes over tunnel
    VPN-->>CGW: Route advertisements exchanged
    Note over CGW,VPN: Data traffic now flows, encrypted, over the tunnel
        

Dead Peer Detection as the Tunnel-Layer Health Signal

Dead Peer Detection (DPD) is the mechanism by which each side periodically verifies the other endpoint is still responsive at the IPsec layer, independent of whether any actual data traffic is flowing — critical because a tunnel with no active data traffic for a period could otherwise sit in a stale, half-open state indefinitely without either side realizing the peer is gone. DPD’s configured timeout and retry behavior directly affects how quickly a genuinely dead tunnel is detected and torn down so failover can occur — an advanced tuning consideration distinct from BGP’s own timers, and one that matters independently because DPD operates below the routing layer entirely.

MTU and Fragmentation: A Frequently Underestimated Detail

IPsec encapsulation adds overhead to every packet, meaning the effective MTU available to traffic traversing the tunnel is smaller than a typical 1500-byte Ethernet MTU — AWS’s Site-to-Site VPN tunnels typically support an effective MTU around 1436 bytes. Traffic that doesn’t account for this and sends packets at the full 1500-byte MTU either gets fragmented (adding processing overhead and potential performance degradation) or silently dropped if the “don’t fragment” bit is set and Path MTU Discovery isn’t functioning correctly across the tunnel — a subtle, advanced-level root cause behind reports of certain large-payload requests failing over VPN while smaller requests work fine.

The Inside-Tunnel CIDR and Why It’s Easy to Overlook

Each tunnel requires its own small, dedicated /30 CIDR block for the IPsec-layer addressing between the two tunnel endpoints themselves — distinct from both the on-premises network’s addressing and the VPC’s own CIDR range entirely. AWS provides a default range for this if unspecified, but organizations with strict internal IP address management policies, or those running many VPN connections that might otherwise collide on the default range, need to explicitly assign non-overlapping inside-tunnel CIDRs per connection. Overlooking this as “just an internal implementation detail” can create genuine addressing conflicts at scale, particularly in environments already using significant portions of the private address space for other purposes.

Idle Timeout and Tunnel Re-Establishment After Inactivity

A tunnel with no BGP keepalive activity and no application traffic for an extended period can, depending on configuration and DPD settings, be torn down and require full re-establishment — including the complete IKE Phase 1 and Phase 2 negotiation sequence — rather than simply resuming. For traffic patterns with genuine idle periods (batch jobs that run once nightly, for instance), this re-establishment latency is worth accounting for explicitly in any end-to-end latency budget for the first request after an idle period, since it can add a meaningfully longer delay than steady-state tunnel traffic would suggest.

Encapsulation and the Path a Packet Actually Takes

A packet destined for a resource across the VPN boundary is encrypted and encapsulated in IPsec (typically ESP in tunnel mode) at the originating side’s tunnel endpoint, transmitted across the underlying network path (the public internet by default, or AWS’s own backbone if Accelerated VPN is configured) to the receiving endpoint, decrypted and de-encapsulated, and then routed normally within the destination network according to whatever routing table entries exist there. The advanced point worth internalizing: the VPN tunnel itself is purely a secure transport mechanism between two endpoints — actual routing decisions about where a packet goes once it’s inside the VPC, or once it’s inside the on-premises network, are governed entirely by each side’s own routing tables and have nothing to do with the tunnel’s own configuration beyond which prefixes are advertised into or out of it.

BGP Route Propagation and Path Selection

When BGP is used, each tunnel runs its own independent BGP session, meaning the same on-premises prefix can be advertised over both tunnels simultaneously, and standard BGP path-selection rules (AS path length, local preference, and other attributes you can configure) determine which tunnel a given prefix’s traffic actually prefers when both are healthy. Advanced designs deliberately configure asymmetric BGP attributes — a lower AS path prepend on the primary tunnel, a longer prepend on the secondary — specifically to control which tunnel carries traffic under normal conditions, rather than leaving path selection to whatever default the routers negotiate, which can result in unpredictable or asymmetric routing across your two tunnels.

What Actually Happens During a Tunnel Failover

When the active tunnel fails, the sequence of events differs meaningfully between BGP and static routing. With BGP, the failing tunnel’s peer stops sending keepalives, the BGP session times out (governed by the hold-timer, typically tens of seconds unless tuned more aggressively), the routes learned over that session are withdrawn, and traffic shifts to the surviving tunnel’s routes — a process that, with well-tuned BGP timers, can complete in single-digit seconds. With static routing, there is no equivalent automatic mechanism at the routing layer; failover depends entirely on whether the customer-side device is configured to detect the tunnel’s IPsec-layer failure and manually withdraw or deprioritize the affected static route, a process that’s both slower and dependent on correct customer-side configuration that AWS has no visibility into or influence over.

Asymmetric Routing Risks Introduced by Multi-Path Hybrid Designs

When a network has multiple potential paths between on-premises and AWS — two VPN tunnels, or a VPN alongside a Direct Connect circuit — traffic in one direction can end up taking a genuinely different physical and logical path than the return traffic, unless routing policy is deliberately configured to keep flows symmetric. This matters beyond mere untidiness: stateful devices in the path (firewalls, NAT gateways, intrusion detection systems) that only see one direction of a given connection’s traffic often can’t properly track its state, leading to connections being dropped or misclassified. Advanced hybrid network designs treat symmetric routing as an explicit requirement to verify, not an assumption that naturally holds just because both paths are technically reachable.

Traffic Encryption Overhead’s Effect on Effective Throughput

Beyond the hard per-tunnel throughput ceiling covered in the performance chapter, the IPsec encryption and decryption process itself consumes CPU cycles on both the AWS-side endpoint and the customer-gateway device, meaning the practically achievable throughput for a given workload can fall meaningfully short of the theoretical ceiling if the customer-side device’s own encryption processing capacity — often dependent on whether it has hardware-accelerated IPsec support — is the actual bottleneck rather than AWS’s side. Advanced capacity planning verifies the customer-gateway device’s own throughput characteristics under real IPsec load, not just its stated maximum interface bandwidth.

At the advanced level, the honest framing isn’t “VPN connects your data center to AWS” — it’s “VPN trades the higher throughput ceiling, more predictable latency, and higher cost of a dedicated Direct Connect circuit for fast provisioning, no physical infrastructure dependency, and encryption built into the transport by default.”

The Trade-off Interviewers Actually Care About

The most tested trade-off is provisioning speed and flexibility versus throughput and latency predictability. VPN can be stood up and torn down in minutes, making it ideal for temporary needs, rapid prototyping of hybrid connectivity, or disaster-recovery scenarios where a new connection needs to exist quickly. Direct Connect requires physical circuit provisioning measured in weeks, but delivers a dedicated, non-internet-dependent path with far more predictable latency and no per-tunnel throughput ceiling. Advanced architecture decisions treat this as a genuine spectrum rather than a binary choice — many organizations run VPN for newly onboarded sites or temporary needs while a Direct Connect circuit is provisioned in parallel, then transition primary traffic to Direct Connect once it’s ready, keeping VPN as the ongoing failover path rather than decommissioning it.

Operational Ownership Trade-offs Between the Two Options

A less-discussed but real trade-off: Direct Connect typically involves an external telecom or colocation provider as a party in the relationship, adding a vendor management dimension to troubleshooting and change requests that a purely AWS-and-customer-router VPN connection doesn’t have. This means a VPN-only design keeps the entire troubleshooting surface within two parties’ direct control, which can meaningfully speed up incident resolution compared to a Direct Connect issue that might require coordinating with a third-party circuit provider — a consideration worth weighing separately from the pure throughput and latency comparison most discussions focus on.

The Per-Tunnel Throughput Ceiling Is a Hard Physical Constraint

Each individual IPsec tunnel is capped at approximately 1.25 Gbps of throughput — a limit rooted in the computational cost of IPsec encryption/decryption at the endpoint, not an arbitrary AWS-imposed quota that can be requested away. A workload that needs sustained throughput beyond this ceiling cannot get there by asking AWS for a higher limit on a single tunnel; the only native paths are aggregating multiple VPN connections via ECMP through a Transit Gateway, or moving to Direct Connect, which has a fundamentally different throughput profile because it isn’t encrypting every packet through IPsec by default.

flowchart LR
    ONPREM["On-Premises Network"] -->|"VPN Connection 1
~1.25 Gbps ceiling"| TGW["Transit Gateway
ECMP Distribution"]
    ONPREM -->|"VPN Connection 2
~1.25 Gbps ceiling"| TGW
    ONPREM -->|"VPN Connection 3
~1.25 Gbps ceiling"| TGW
    TGW --> VPC["Attached VPCs"]
        

ECMP Requires Genuinely Equal-Cost Paths, Not Just Multiple Tunnels

Simply having multiple VPN connections attached to the same Transit Gateway doesn’t automatically grant proportional throughput scaling — ECMP requires the routes learned from each connection to actually be equal-cost from the routing protocol’s perspective (matching AS path length and other BGP attributes AWS’s ECMP implementation considers). A common advanced mistake is configuring multiple connections with mismatched BGP attributes for other reasons (like the primary/secondary preference pattern discussed in the routing chapter) and then being confused why traffic isn’t actually load-balancing evenly across them — the two goals, controlled failover preference and ECMP load balancing, are in direct tension and a design has to pick which one it’s actually optimizing for on a given set of connections.

Accelerated VPN’s Latency Improvement Is Path-Dependent, Not Universal

Accelerated VPN routes the AWS-bound leg of tunnel traffic through AWS’s global backbone via Global Accelerator’s anycast addressing, which tends to improve latency and reduce jitter specifically for customer sites geographically distant from the target AWS Region, where the public internet path would otherwise involve more hops and more variable routing. For a customer site already close to the target Region with a good direct internet path, the improvement is often marginal — advanced adoption decisions benchmark actual latency with and without acceleration for the specific customer-site-to-Region pair in question, rather than assuming a uniform benefit across every deployment.

Jitter and Packet Loss Sensitivity for Real-Time Workloads

Beyond raw throughput and average latency, workloads sensitive to jitter and packet loss — voice traffic, real-time control systems — face an additional consideration specific to VPN’s default reliance on the public internet: variable internet routing can introduce inconsistent latency and occasional packet loss that a dedicated Direct Connect circuit’s predictable path largely avoids. Advanced deployments for genuinely real-time-sensitive hybrid traffic either use Accelerated VPN specifically to reduce this variability, or conclude that VPN alone isn’t the right transport for that traffic class and pair it with Direct Connect instead, reserving VPN for traffic more tolerant of variable network conditions.

Capacity Planning Ahead of Known Volume Increases

Because the per-tunnel throughput ceiling is fixed and ECMP scaling requires provisioning additional VPN connections ahead of time (not something that can be requested as an instant, on-demand burst the way EC2 Auto Scaling handles compute), advanced capacity planning for a growing hybrid workload tracks utilization against the ceiling proactively and provisions additional ECMP-balanced connections well before sustained utilization approaches the limit, rather than discovering the ceiling has been reached only when application-layer performance degradation is already occurring.

Dual Tunnels Protect Against AWS-Side Failure, Not Customer-Side Single Points of Failure

The two AWS-provisioned tunnels terminate at two different Availability Zones specifically to protect against a localized AWS-side failure or maintenance event. What they do not protect against, by themselves, is a failure on the customer’s own side — if both tunnels terminate at the same single customer-gateway device, that device remains a single point of failure regardless of how well AWS’s side is architected. True end-to-end high availability requires a second customer gateway device (often at a second physical location or at least a redundant device at the same site) with its own VPN connection, giving genuinely independent failure domains on both ends of the connection, not just the AWS side.

Multiple VPN Connections for True Redundant-Site Designs

Advanced high-availability topologies commonly provision two entirely separate VPN connections — each with its own pair of tunnels — terminating at two physically distinct customer locations or devices, attached to the same Transit Gateway, with BGP attributes configured so one is clearly preferred under normal conditions and the other serves as a genuine, independent failover path. This is meaningfully more resilient than relying on a single connection’s built-in tunnel pair alone, because it removes the shared customer-side device as a common point of failure across both tunnels.

BGP Timer Tuning as a Direct Lever on Recovery Time

The default BGP hold-timer and keepalive interval represent a trade-off between failover speed and false-positive sensitivity — more aggressive timers detect a genuine failure faster but also increase the risk of a transient, brief network hiccup triggering an unnecessary failover event (and the “flapping” that comes with routes rapidly appearing and disappearing). Advanced designs tune these timers deliberately based on the actual reliability characteristics of the underlying network path, rather than leaving default values that may be either too slow for the business’s actual recovery-time requirements or too aggressive for a genuinely noisy internet path.

Pre-Shared Keys Versus Certificate-Based Authentication

Pre-shared keys are the simpler, more common authentication method for the IKE handshake, but they carry the operational burden of secure distribution and periodic rotation — a leaked pre-shared key compromises the tunnel’s authentication entirely until rotated on both sides simultaneously, which itself requires a coordinated maintenance window. Certificate-based authentication, where supported by the customer gateway device, avoids the shared-secret distribution problem entirely and integrates with existing PKI infrastructure for rotation and revocation, at the cost of more complex initial setup. Advanced security postures for long-lived, high-value VPN connections increasingly favor certificate-based authentication specifically to avoid the pre-shared key’s rotation and distribution risk.

Encryption Algorithm and Diffie-Hellman Group Selection

Tunnel options let you specify which encryption algorithms, integrity algorithms, and Diffie-Hellman groups are permitted for both IKE phases — and the advanced security discipline is restricting these to modern, strong choices explicitly (AES-256, SHA-2 family, higher-numbered DH groups) rather than accepting the broadest possible default set for compatibility, since a tunnel configured to accept weaker legacy algorithms as a fallback is only as strong as the weakest option it’s willing to negotiate down to if a peer requests it.

The Tunnel Encrypts Transport, Not Application-Layer Trust

It’s worth stating plainly because it’s a genuine, advanced-level misconception: a Site-to-Site VPN tunnel establishes an encrypted, authenticated network path between two networks — it says nothing about whether traffic arriving over that tunnel should be implicitly trusted at the application layer. A compromised device anywhere within the on-premises network that the VPN connects still has network-layer reachability to whatever the VPN’s routing advertises, and security groups, NACLs, and application-layer authentication within the VPC remain exactly as necessary as they would be for any other network path — the VPN’s encryption protects data in transit between the two networks, not the trustworthiness of everything already inside either network.

Limiting Advertised Routes to the Minimum Necessary Scope

It’s operationally convenient to advertise an entire on-premises supernet over BGP rather than carefully scoping advertisements to only the specific subnets that genuinely need hybrid connectivity, but this convenience has a real security cost: it grants the VPC-side network reachability to the entire advertised range, including systems that may have no legitimate reason to be reachable from AWS at all. Advanced network security reviews specifically check whether BGP advertisements are scoped tightly to the actual required subnets on both sides, treating an overly broad advertisement as a genuine finding worth remediating, not just a minor tidiness issue.

Auditing Tunnel Option Changes as a Security-Relevant Event

Because tunnel options — encryption algorithms, DH groups, pre-shared keys — directly determine the cryptographic strength of a production hybrid connection, changes to them are exactly the kind of event that should generate an audit trail and, ideally, a change-approval record, via CloudTrail on the AWS side and the customer-gateway device’s own audit logging. A silent downgrade of a tunnel’s permitted encryption algorithms — whether accidental or malicious — is the kind of change that could otherwise go unnoticed for a long time if nobody is specifically watching for it.

Tunnel State Alone Is an Insufficient Health Signal

CloudWatch’s TunnelState metric reports whether a tunnel’s IPsec layer is up, but as covered in the failover chapter, a tunnel can be “up” at that layer while the actual routes needed for real traffic are unavailable or the BGP session itself is unhealthy. Advanced monitoring dashboards track TunnelState alongside BGP session state and route count metrics together, because a healthy tunnel with zero advertised routes, or a tunnel whose BGP session has flapped repeatedly in a short window, represents a real operational problem that tunnel state alone would report as entirely fine.

Bytes-In and Bytes-Out Asymmetry as a Diagnostic Signal

A significant, sustained asymmetry between a tunnel’s inbound and outbound byte counts — far more traffic flowing one direction than the other, beyond what the actual application traffic pattern would explain — is often an early indicator of asymmetric routing (traffic going out over one tunnel but return traffic coming back over the other, or through a completely different path like Direct Connect if both exist), which can itself cause connection-tracking and stateful firewall issues on either side. Advanced network teams treat this metric as a routing-correctness check, not just a capacity or utilization number.

Logging BGP Session Events for Post-Incident Reconstruction

BGP session state changes — established, idle, active, the specific timestamps of route withdrawals — are essential forensic data during any post-incident review of a failover event, and advanced practice ensures these are captured either via the customer-gateway device’s own logging (typically the richer source) or CloudWatch Logs where the AWS side exposes them, rather than relying purely on tunnel-level up/down events that don’t capture the routing-layer detail needed to reconstruct exactly when and why traffic actually shifted between tunnels.

Transit Gateway as the Hub for Multi-VPC, Multi-Connection Topologies

For any organization with more than a single VPC needing hybrid connectivity, attaching VPN connections to a Transit Gateway rather than individual VGWs per VPC centralizes routing management, enables the ECMP throughput scaling covered earlier, and lets Direct Connect and VPN attachments share the same routing domain — meaning a single Transit Gateway route table can express policies like “prefer Direct Connect, fail over to VPN” across the entire attached VPC fleet at once, rather than needing that logic replicated per VPC.

Multi-Region VPN Designs and Their Genuine Complexity

A Transit Gateway is regional, so a genuinely multi-Region hybrid network requires either separate VPN connections into a Transit Gateway per Region (with Transit Gateway peering connecting the Regions), or routing all hybrid traffic through a single Region’s Transit Gateway and relying on inter-Region VPC peering or Transit Gateway peering from there. Advanced designs weigh this complexity against actual multi-Region traffic requirements — a workload with only occasional cross-Region hybrid traffic may not justify the operational overhead of VPN connections duplicated per Region, while one with substantial, latency-sensitive traffic to multiple Regions likely does.

Infrastructure as Code for VPN Configuration Drift Prevention

Because a VPN connection’s correctness depends on exact agreement between the AWS-side Customer Gateway resource and the actual on-premises router configuration — IP addresses, ASN, pre-shared keys, tunnel options — manual console changes on either side that aren’t mirrored on the other are a common, entirely avoidable source of outages. Advanced practice manages both the AWS-side resources (via Terraform or CloudFormation) and, where the customer-gateway vendor supports it, the corresponding on-premises configuration through the same change-review process, specifically to prevent the two sides drifting out of sync silently.

Multi-Account Hybrid Connectivity via Shared Transit Gateway

In an AWS Organizations multi-account setup, a Transit Gateway in a dedicated networking account can be shared via Resource Access Manager with other accounts in the organization, letting a single set of VPN connections and their associated hybrid connectivity be reused across many application-owning accounts without each one needing its own separate VPN connection back to the same on-premises network. This centralization is an advanced organizational pattern that significantly reduces both the operational overhead of managing many redundant connections and the on-premises router’s own connection count, at the cost of requiring careful Transit Gateway route table segmentation so accounts only see the routes they’re actually meant to reach.

Testing Failover as Part of the Deployment Pipeline, Not Just Initial Setup

Beyond the one-time validation of a newly provisioned connection, advanced operational maturity treats periodic, scheduled failover testing as an ongoing practice — deliberately disabling the primary tunnel or connection in a controlled window and confirming traffic shifts to the secondary within the expected time, with the expected impact. Configuration changes elsewhere in the network (a firmware update on the customer-gateway device, a change to BGP timers, a new route added upstream) can silently break a failover path that worked correctly when it was last tested, and only a recurring test schedule catches that kind of regression before an actual incident does.

Documenting the Actual, Current BGP Preference Design

As a hybrid network accumulates multiple VPN connections, possibly alongside Direct Connect, over time, the intended traffic-preference logic (which path should carry traffic under normal conditions, which is the failover, which combination is meant to provide ECMP scaling) becomes genuinely difficult to reconstruct from router configuration alone months or years later, especially across personnel changes. Advanced teams maintain an explicit, current network diagram and preference-rationale document alongside the actual configuration, specifically so a new engineer troubleshooting a routing anomaly can quickly understand what the intended behavior was supposed to be, rather than having to reverse-engineer it from BGP attributes scattered across multiple device configurations.

Across these examples, the pattern repeats: the organizations getting the most value from Site-to-Site VPN treat it either as a deliberate, permanent connectivity choice suited to their actual throughput and latency needs, or as an explicitly time-bound bridge — never as an assumed-adequate default applied without checking whether the workload’s real requirements exceed what it can provide.