Overview
IPsec VPN to AWS and Azure works for low-throughput workloads but maxes out around 1.25Gbps per tunnel and adds 20–50ms latency over the internet. AWS Direct Connect and Azure ExpressRoute are dedicated private circuits from your data center to the cloud provider's edge — predictable latency, up to 100Gbps, and no public internet exposure for sensitive workloads. This guide covers both paths and how to design them together for redundancy.
Connectivity Options Comparison
Architecture: Direct Connect + VPN Redundancy
Part 1 — AWS Site-to-Site VPN
Cisco IOS-XE to AWS VGW
AWS provides two tunnel endpoints per VPN connection. Always configure both for redundancy.
! Tunnel 1 (AWS provides the outside IP — replace with your VGW endpoint)crypto isakmp policy 10 encryption aes 256 hash sha256 authentication pre-share group 14 lifetime 28800crypto isakmp key AWS-PSK-TUNNEL-1 address 203.0.113.100crypto isakmp key AWS-PSK-TUNNEL-2 address 203.0.113.101crypto ipsec transform-set AWS-TSET esp-aes 256 esp-sha256-hmac mode tunnelcrypto map AWS-VPN-MAP 10 ipsec-isakmp set peer 203.0.113.100 set transform-set AWS-TSET set pfs group14 match address ACL-AWS-VPNcrypto map AWS-VPN-MAP 20 ipsec-isakmp set peer 203.0.113.101 set transform-set AWS-TSET set pfs group14 match address ACL-AWS-VPNinterface GigabitEthernet0/0 crypto map AWS-VPN-MAP! BGP over the tunnels (AWS assigns link-local BGP IPs — check AWS console)router bgp 65001 neighbor 169.254.10.1 remote-as 7224 ! 7224 = AWS BGP ASN for VPN connections neighbor 169.254.10.1 timers 10 30 neighbor 169.254.10.1 local-preference 100 ! Lower LP = VPN is backup to Direct ConnectPart 2 — AWS Direct Connect BGP Design
Direct Connect uses a Private Virtual Interface (VIF) with BGP. You peer with AWS using your own AS and AWS's ASN (7224 for VPN; you get a private ASN for DX VIF — check console).
! BGP on the Direct Connect routerrouter bgp 65001 neighbor 192.168.100.1 remote-as 64512 ! 192.168.100.1 = AWS-side VIF IP, 64512 = AWS private ASN for your DX connection neighbor 192.168.100.1 description AWS-DirectConnect-Primary neighbor 192.168.100.1 local-preference 200 ! Higher LP = prefer DX over VPN ! Advertise on-premises routes to AWS neighbor 192.168.100.1 route-map RM-TO-AWS out! Only advertise your own prefixes to AWS — never advertise a default or transitroute-map RM-TO-AWS permit 10 match ip address prefix-list MY-ON-PREM-PREFIXESip prefix-list MY-ON-PREM-PREFIXES permit 10.0.0.0/8 le 24ip prefix-list MY-ON-PREM-PREFIXES permit 172.16.0.0/12 le 24Part 3 — Azure ExpressRoute
Azure ExpressRoute uses two BGP sessions (primary and secondary path) over the circuit. Route filtering in Azure uses BGP communities.
On-Premises Router BGP for ExpressRoute
! ExpressRoute — peer with Microsoft Enterprise Edge (MSEE)! Azure provides /30 peering IPs — check Azure portalrouter bgp 65001 ! Primary path neighbor 192.168.200.1 remote-as 12076 ! 12076 = Microsoft's BGP ASN for ExpressRoute neighbor 192.168.200.1 description Azure-ER-Primary neighbor 192.168.200.1 local-preference 200 neighbor 192.168.200.1 route-map RM-FROM-AZURE in neighbor 192.168.200.1 route-map RM-TO-AZURE out ! Secondary path (mandatory — Azure requires both sessions active) neighbor 192.168.200.5 remote-as 12076 neighbor 192.168.200.5 description Azure-ER-Secondary neighbor 192.168.200.5 local-preference 190 ! Slightly lower — prefer primary MSEE! Filter routes received from Azure — accept only your subscribed regionsroute-map RM-FROM-AZURE permit 10 match community AZURE-EASTASIA-COMMUNITY ! Use Azure BGP communities to filter by region: 12076:51010 = East Asia! Advertise only your prefixes to Azureroute-map RM-TO-AZURE permit 10 match ip address prefix-list MY-ON-PREM-PREFIXESPart 4 — Route Propagation and VPC/VNet Design
AWS — Propagate DX and VPN Routes to VPC Route Tables
# AWS CLI — enable route propagation from VGW to VPC route tablesaws ec2 enable-vgw-route-propagation \ --route-table-id rtb-0a1b2c3d \ --gateway-id vgw-0a1b2c3d# Verify routes propagated to VPC route tableaws ec2 describe-route-tables --route-table-id rtb-0a1b2c3d \ --query 'RouteTables[0].Routes[?Origin==`EnableVgwRoutePropagation`]'Azure — Add Gateway Subnet and Configure ExpressRoute Gateway
# Create gateway subnet (must be named exactly "GatewaySubnet")az network vnet subnet create \ --vnet-name MyVNet --resource-group MyRG \ --name GatewaySubnet --address-prefix 10.200.255.0/27# Create ExpressRoute gatewayaz network vnet-gateway create \ --name ER-Gateway --resource-group MyRG \ --vnet MyVNet --gateway-type ExpressRoute \ --sku Standard --location eastasia# Connect the gateway to your ExpressRoute circuitaz network vpn-connection create \ --name ER-Connection --resource-group MyRG \ --vnet-gateway1 ER-Gateway \ --express-route-circuit2 /subscriptions/.../expressRouteCircuits/MyCircuitReal-World Scenario
The situation: A company runs a financial reporting application in an on-premises data center. They move the app to AWS. During peak month-end reporting, the IPsec VPN saturates (1.25Gbps limit) and transfers take 3× longer than on-prem to on-prem. The app is also latency-sensitive — round-trip over VPN is 45ms vs the required 10ms SLA.
Root cause:
- VPN throughput: IPsec on Cisco IOS maxes at ~1.25Gbps, and data transfer hits that ceiling
- VPN latency: public internet path adds 20ms+ jitter unpredictably
Fix — deploy Direct Connect:
# Step 1: Order DX circuit via AWS Console → Direct Connect → Connections# Bandwidth: 10Gbps | Location: nearest AWS DX location# Step 2: Create Private VIF after circuit is provisionedaws directconnect create-private-virtual-interface \ --connection-id dxcon-abc123 \ --new-private-virtual-interface \ virtualInterfaceName=Corp-VIF,vlan=100,asn=65001,\ amazonAddress=192.168.100.1/30,customerAddress=192.168.100.2/30,\ virtualGatewayId=vgw-0a1b2c3d# Step 3: Configure BGP on the CE router (see Part 2 above)# Step 4: Keep VPN as standby with lower local-preference# Result: 8Gbps sustained throughput, 3ms latency, month-end reports complete on timeTroubleshooting
BGP not establishing over Direct Connect
Symptom: DX circuit is Up but BGP session stays in Active or Idle.
Cause: BGP timer mismatch, wrong ASN on either side, or incorrect VIF peering IPs entered.
Fix:
! Verify BGP config matches AWS console VIF details exactlyR1# show bgp neighbors 192.168.100.1! Check: "BGP state = Active" — means TCP established but BGP handshake failing! Check: remote AS must match what AWS shows in the console! Enable BGP debugR1# debug ip bgp 192.168.100.1 events! "open failed: bad AS" = ASN mismatch! "Notification: hold time expired" = timer mismatch (set timers 10 30)AWS VPC not receiving on-premises routes
Symptom: BGP session up, but EC2 instances can't reach on-premises. AWS route table shows no propagated routes.
Cause: Route propagation disabled in VPC route table, or on-premises router isn't advertising the correct prefix.
Fix:
# Check what AWS is receiving from your BGP peeraws directconnect describe-virtual-interfaces \ --query 'virtualInterfaces[?virtualInterfaceName==`Corp-VIF`].bgpPeers'# Verify prefix is being advertised from on-premisesR1# show bgp neighbors 192.168.100.1 advertised-routes! Must show your on-prem prefix hereExpressRoute failover not working — traffic doesn't shift to secondary path
Symptom: Primary MSEE BGP session drops, but traffic stays down instead of failing over to secondary.
Cause: Secondary path BGP session was never established, or local-preference on secondary is lower than a competing VPN route.
Fix:
! Verify both MSEE sessions are establishedR1# show bgp neighbors 192.168.200.5! Secondary must also be in Established state at all times! Check that secondary LP (190) is still higher than VPN backup LP (100)R1# show bgp ipv4 unicast | include 10.200.! Should show two paths — primary via .1, secondary via .5Hybrid Cloud Connectivity Checklist
- Both VPN tunnels (AWS provides two endpoints) configured and in Up state — single-tunnel VPN has no SLA
- Direct Connect HA: two DX connections from different providers, or one DX + VPN fallback with tested failover
- BGP local-preference values documented and consistent: DX=200, ER=200, VPN=100
- Only advertise your own prefixes to cloud providers — no default route, no transit routes
- VPC/VNet route propagation enabled and verified — BGP up does not mean routes propagated
- ExpressRoute: both primary and secondary BGP sessions established before go-live
- MTU verified end-to-end — AWS VPN max MTU is 1446, DX supports 9001 (jumbo frames supported)
- BGP hold timer set to 10s/30s or lower on cloud-facing sessions — default 90s too slow for failover
- Cost allocation tags on DX/ER connections for chargeback tracking
- Quarterly failover test: shut primary path, verify secondary activates within 30 seconds