Overview
When an IPSec site-to-site VPN tunnel fails to establish or drops unexpectedly, the root cause usually falls into one of four categories: a Phase 1 IKE mismatch, a Phase 2 transform-set or proxy ACL mismatch, a missing NAT exemption, or a routing/policy issue. This guide walks through a systematic, step-by-step approach to isolating and resolving these failures on both Cisco ASA and Palo Alto NGFW โ with real CLI commands, debug output interpretation, and hardening tips.
| Parameter | Phase 1 (ISAKMP) | Phase 2 (IPSec) |
|---|---|---|
| Encryption | AES-256 | AES-256 |
| Hash / Auth | SHA-256 | SHA-256 |
| DH / PFS | Group 14 | Group 14 (PFS) |
| Lifetime | 86400s | 3600s |
| Proxy ACL | 10.10.10.0/24 โ 10.20.20.0/24 โ must match exactly on both peers | |
| Common fails | Mismatched transform-set ยท Proxy ACL mismatch ยท NAT exemption missing | |
| Quick check | show crypto ipsec sa ยท show vpn ipsec-sa ยท grep mp-log ikemgr.log | |
Step 1 โ Check Tunnel Status
# Quick overview of all IKE and IPSec SAs
ASA# show crypto isakmp sa
ASA# show crypto ipsec sa
# If tunnel shows MM_WAIT_MSG3 or MM_WAIT_MSG5 โ Phase 1 is failing
# If tunnel shows QM_IDLE but no traffic โ Phase 2 or ACL issue
Step 2 โ Enable Real-Time Debug
โ ๏ธ Run debugs on a maintenance window or low-traffic period. Always disable after capturing output.
# Set debug to peer IP only (avoid flooding the console)
ASA# debug crypto condition peer 203.0.113.10
# Phase 1 debug
ASA# debug crypto isakmp 7
# Phase 2 debug
ASA# debug crypto ipsec 7
# Send output to syslog (recommended over terminal)
ASA# logging enable
ASA# logging console debugging
ASA# logging host inside 10.10.10.50
# Clear tunnel and re-initiate to capture fresh negotiation
ASA# clear crypto isakmp sa
ASA# clear crypto ipsec sa
Step 3 โ Interpret Phase 1 Failures
MM_WAIT_MSG3 โ Peer not responding. Check:
# Confirm peer IP is reachable
ASA# ping 203.0.113.10
# Check if ISAKMP is enabled on the outside interface
ASA# show run crypto isakmp
! Should show: crypto isakmp enable outside
# Enable it if missing
ASA(config)# crypto isakmp enable outside
MM_WAIT_MSG5 โ Proposal mismatch or wrong PSK. Check:
# View current Phase 1 policy
ASA# show run crypto isakmp policy
# Typical mismatch areas:
# - encryption: aes vs aes-256
# - hash: sha vs sha256
# - DH group: group2 vs group14
# - lifetime: 86400 vs 28800
# Example correct Phase 1 config:
ASA(config)# crypto isakmp policy 10
ASA(config-isakmp-policy)# authentication pre-share
ASA(config-isakmp-policy)# encryption aes-256
ASA(config-isakmp-policy)# hash sha256
ASA(config-isakmp-policy)# group 14
ASA(config-isakmp-policy)# lifetime 28800
Wrong pre-shared key:
# View tunnel group (PSK is hashed but you can re-enter it)
ASA# show run tunnel-group 203.0.113.10
# Re-enter PSK to rule out typos
ASA(config)# tunnel-group 203.0.113.10 ipsec-attributes
ASA(config-tunnel-ipsec)# ikev1 pre-shared-key MySecretKey123!
Step 4 โ Interpret Phase 2 Failures
If Phase 1 is UP (shows MM_ACTIVE or IKEv2 SA ESTABLISHED) but no traffic flows:
# Check Phase 2 SA detail
ASA# show crypto ipsec sa detail
# Key things to check:
# - "encaps/decaps" counters โ if zero, interesting traffic not matching
# - "failed SA creation" โ proposal mismatch
# - "#pkts invalid identity" โ proxy ID / crypto ACL mismatch
Crypto ACL mismatch โ the most common Phase 2 failure:
# View crypto ACL
ASA# show run access-list CRYPTO-ACL-TO-PEER
# Must be mirror image on both ends:
# Local end: permit ip 10.10.10.0/24 โ 10.20.20.0/24
# Remote end: permit ip 10.20.20.0/24 โ 10.10.10.0/24
# Fix example:
ASA(config)# access-list CRYPTO-ACL-TO-PEER extended permit ip 10.10.10.0 255.255.255.0 10.20.20.0 255.255.255.0
Phase 2 proposal mismatch:
# View transform set
ASA# show run crypto ipsec transform-set
# Example fix:
ASA(config)# crypto ipsec ikev1 transform-set TS-AES256-SHA1 esp-aes-256 esp-sha-hmac
ASA(config)# crypto ipsec ikev1 transform-set TS-AES256-SHA256 esp-aes-256 esp-sha256-hmac
Step 5 โ Check NAT Exemption
Missing NAT exemption is a silent killer โ traffic hits the NAT rule before the crypto ACL.
# Check NAT rules
ASA# show nat detail
# NAT exemption must be FIRST (lowest line number) or use nat-before-crypto
# Example NAT exemption:
ASA(config)# nat (inside,outside) 1 source static INSIDE-NET INSIDE-NET destination static REMOTE-NET REMOTE-NET no-proxy-arp route-lookup
# Verify with packet-tracer (simulates a real packet)
ASA# packet-tracer input inside tcp 10.10.10.10 1234 10.20.20.10 80 detail
! Look for "IPSEC" phase โ should show "ACTION: ENCRYPT"
! If it shows NAT before IPSEC โ NAT exemption is wrong
Step 6 โ Packet Capture on ASA
# Capture on the outside interface for peer traffic
ASA# capture CAP-VPN interface outside match ip host 203.0.113.2 host 203.0.113.10
# View capture
ASA# show capture CAP-VPN detail
# Save to PCAP for Wireshark
ASA# copy /pcap capture:CAP-VPN tftp://10.10.10.50/vpn-debug.pcap
# Clean up
ASA# no capture CAP-VPN
Part 2 โ Palo Alto NGFW Troubleshooting
Step 1 โ Check Tunnel Status in GUI and CLI
# CLI: Check IKE gateway status
admin@PA> show vpn ike-sa
admin@PA> show vpn ike-sa gateway {gateway-name}
# CLI: Check IPSec tunnel status
admin@PA> show vpn ipsec-sa
admin@PA> show vpn ipsec-sa tunnel {tunnel-name}
# Look for:
# State: active = UP
# State: init = Phase 1 negotiating
# State: (blank) = not initiated or failed
Step 2 โ Check IKE Phase 1
# View IKE gateway config
admin@PA> show vpn ike-sa gateway {gw-name} detail
# Common issues โ check these in Panorama or CLI:
# Network > IKE Gateways > [your gateway]
# - IKE Version matches peer (IKEv1 / IKEv2)
# - Peer IP address correct
# - Pre-shared key matches exactly (case-sensitive)
# - IKE Crypto Profile matches peer proposals
# View IKE crypto profiles
admin@PA> show vpn ike-crypto-profile
Enable IKE debug on Palo Alto:
# Set debug level (warning: verbose, use only for troubleshooting)
admin@PA> debug ike global on debug
# Initiate tunnel
admin@PA> test vpn ike-sa gateway {gateway-name}
# View debug logs
admin@PA> less mp-log ikemgr.log
# Filter by IP, hostname, or keyword pattern
admin@PA> grep mp-log ikemgr.log pattern 203.0.113.10
admin@PA> grep mp-log ikemgr.log pattern {gateway-name}
admin@PA> grep mp-log ikemgr.log pattern "IKE_INIT\|IKE_AUTH\|DELETE"
# Tail the log live while re-initiating
admin@PA> tail follow yes mp-log ikemgr.log
# Clear tunnel and re-test
admin@PA> clear vpn ike-sa gateway {gateway-name}
admin@PA> test vpn ike-sa gateway {gateway-name}
# Turn off debug when done
admin@PA> debug ike global off
Step 3 โ Check IPSec Phase 2
# View IPSec SA details
admin@PA> show vpn ipsec-sa tunnel {tunnel-name}
# Check IPSec crypto profile
admin@PA> show vpn ipsec-crypto-profile
# Common Phase 2 mismatches on PA:
# - Encryption: aes-128-cbc vs aes-256-cbc
# - Authentication: sha1 vs sha256
# - DH group (PFS): group2 vs group14 vs no-pfs
# - Lifetime seconds: 3600 vs 28800
Step 4 โ Check Proxy IDs
Palo Alto requires explicit Proxy IDs for policy-based VPNs (connecting to Cisco ASA, Juniper, Fortinet etc.).
# View proxy IDs on existing tunnel
admin@PA> show vpn ipsec-sa tunnel {tunnel-name} detail
# Proxy IDs must exactly match the peer's crypto ACL:
# PA Local: 10.10.10.0/24
# PA Remote: 10.20.20.0/24
# Must match ASA crypto ACL mirror image exactly
# In Panorama/GUI: Network > IPSec Tunnels > [tunnel] > Proxy IDs tab
# Add: Local 10.10.10.0/24, Remote 10.20.20.0/24, Protocol: any
Step 5 โ Check Security Policies and NAT
# Verify traffic is hitting the tunnel zone policy
admin@PA> show session all filter source 10.10.10.10 destination 10.20.20.10
# Check NAT โ NAT exemption must exist for VPN traffic
# Panorama: Policies > NAT
# Add a no-NAT rule ABOVE the internet NAT rule:
# Source zone: trust | Dest zone: untrust | Source: 10.10.10.0/24 | Dest: 10.20.20.0/24
# Action: No SNAT, No DNAT
# Test policy lookup
admin@PA> test security-policy-match source 10.10.10.10 destination 10.20.20.10 protocol 6 destination-port 80 from trust to vpn-zone
Step 6 โ Packet Capture on Palo Alto
# Stage 1: Before parse (raw ingress)
admin@PA> debug dataplane packet-diag set filter match source 10.10.10.10 destination 10.20.20.10
# Enable capture
admin@PA> debug dataplane packet-diag set capture stage ingress file vpn-capture.pcap
admin@PA> debug dataplane packet-diag set capture on
# Generate traffic then stop
admin@PA> debug dataplane packet-diag set capture off
# Export via SCP (from ops mode)
admin@PA> scp export packet-capture from mgmt/vpn-capture.pcap to [email protected]:/captures/
# Or view in Monitor > Packet Capture in the GUI
Quick Reference โ Common Error Messages
Final Checklist Before Escalating to Vendor TAC
# Run through this before opening a TAC case:
# 1. Both ends reachable
ping source
# 2. UDP 500 / 4500 open (check upstream firewall/ACL)
ASA# show access-list | include 500
ASA# show access-list | include 4500
# 3. Phase 1 proposals match exactly (encryption, hash, DH, lifetime)
show crypto isakmp policy # ASA
show vpn ike-crypto-profile # PA
# 4. PSK identical on both ends (re-enter to be sure)
# 5. Phase 2 proposals match (encryption, hash, PFS, lifetime)
show crypto ipsec transform-set # ASA
show vpn ipsec-crypto-profile # PA
# 6. Crypto ACL is mirror image on both sides
# 7. NAT exemption exists and is above PAT/NAT rules
# 8. Routing exists for interesting traffic via tunnel interface
show route 10.20.20.0 # ASA
show routing route virtual-router default # PA
If all of the above check out and the tunnel still will not come up, capture traffic on both ends simultaneously and open a TAC case with the full debug output, packet captures, and running configurations from both devices.