VXLAN Best Practices: Overlay Design, EVPN Control Plane, and Troubleshooting

March 10, 2026·10 min read

Overview

VXLAN (Virtual Extensible LAN) solves the fundamental scaling limits of traditional Layer 2 networks by encapsulating Ethernet frames inside UDP packets, allowing L2 segments to stretch across any IP underlay. Combined with an EVPN (Ethernet VPN) control plane, VXLAN enables modern data center fabrics to provide multi-tenant L2/L3 connectivity with the scale, automation, and fast convergence that enterprise and cloud workloads demand. Understanding the interplay between the VXLAN data plane and the EVPN control plane is essential for anyone building or operating a modern spine-leaf fabric.

Part 1 — VXLAN Fundamentals

Key Terminology

VTEP (VXLAN Tunnel Endpoint) — the device (leaf switch) that originates and terminates VXLAN tunnels. Each VTEP has a unique loopback IP used as the tunnel source.
VNI (VXLAN Network Identifier) — a 24-bit segment ID (up to 16 million segments) that identifies a Layer 2 or Layer 3 segment. Equivalent to a VLAN ID but globally unique across the fabric.
NVE (Network Virtualization Edge) — the logical interface on a switch that handles VXLAN encapsulation/decapsulation. On Cisco NX-OS this is the nve1 interface.
BUM traffic — Broadcast, Unknown unicast, and Multicast. The VXLAN control plane must handle BUM flooding without creating loops.
EVPN (Ethernet VPN) — the BGP address family (AFI 25 / SAFI 70) used as the VXLAN control plane. Distributes MAC and IP reachability information between VTEPs, replacing flood-and-learn.

Why EVPN Over Flood-and-Learn

Flood-and-learn (multicast or ingress replication) works but does not scale — every VTEP must flood unknown unicast to all other VTEPs. EVPN distributes MAC/IP bindings via BGP before traffic needs to flow, eliminating most BUM flooding and enabling:

MAC mobility (VM live migration detected and handled via sequence numbers)
Distributed anycast gateway (same gateway IP/MAC on every leaf)
Multi-tenancy with per-VRF routing (L3 VNI per tenant)
ARP suppression (leaf answers ARP locally instead of flooding)

Part 2 — VNI and VRF Design

L2 VNI vs L3 VNI

Type	Purpose	Maps To	Best Practice
L2 VNI	Extends a VLAN across the fabric	One VLAN ↔ one L2 VNI	Use a consistent offset: VLAN 100 → VNI 10100
L3 VNI	Routes between L2 VNIs within a tenant VRF	One VRF ↔ one L3 VNI	Assign from a dedicated range: VRF TENANT-A → VNI 50001

Recommended VNI Numbering Scheme

bash

# L2 VNIs: VLAN-ID + offset (e.g., 10000)# VLAN 100 → VNI 10100# VLAN 200 → VNI 10200# VLAN 300 → VNI 10300# L3 VNIs: sequential from 50000+# VRF TENANT-A → VNI 50001# VRF TENANT-B → VNI 50002# VRF MGMT    → VNI 50099# Route Distinguisher and Route Target: use ASN:VNI pattern# RD  65001:10100 (per VTEP — must be unique)# RT  65001:10100 (same on all VTEPs for same VNI — for import/export)

Part 3 — Cisco NX-OS VXLAN EVPN Configuration

Step 1 — Enable Features

cisco

Leaf-1(config)# feature nv overlayLeaf-1(config)# feature vn-segment-vlan-basedLeaf-1(config)# feature bgpLeaf-1(config)# nv overlay evpn

Step 2 — Map VLANs to VNIs

cisco

Leaf-1(config)# vlan 100Leaf-1(config-vlan)#  vn-segment 10100Leaf-1(config)# vlan 200Leaf-1(config-vlan)#  vn-segment 10200! L3 VNI VLAN (internal — do not assign access ports)Leaf-1(config)# vlan 3001Leaf-1(config-vlan)#  vn-segment 50001

Step 3 — VRF and Anycast Gateway

cisco

Leaf-1(config)# vrf context TENANT-ALeaf-1(config-vrf)#  vni 50001Leaf-1(config-vrf)#  rd autoLeaf-1(config-vrf)#  address-family ipv4 unicastLeaf-1(config-vrf-af-ipv4)#   route-target both auto evpn! Anycast gateway — same IP and MAC on every leafLeaf-1(config)# fabric forwarding anycast-gateway-mac 0001.0001.0001Leaf-1(config)# interface Vlan100Leaf-1(config-if)#  vrf member TENANT-ALeaf-1(config-if)#  ip address 10.100.0.1/24Leaf-1(config-if)#  fabric forwarding mode anycast-gatewayLeaf-1(config-if)#  no shutdown! L3 VNI SVI (no IP needed — just a carrier for the L3 VNI)Leaf-1(config)# interface Vlan3001Leaf-1(config-if)#  vrf member TENANT-ALeaf-1(config-if)#  ip forwardLeaf-1(config-if)#  no shutdown

Step 4 — NVE Interface

cisco

Leaf-1(config)# interface nve1Leaf-1(config-if-nve)#  no shutdownLeaf-1(config-if-nve)#  source-interface loopback1Leaf-1(config-if-nve)#  host-reachability protocol bgp! L2 VNILeaf-1(config-if-nve)#  member vni 10100Leaf-1(config-if-nve-vni)#   mcast-group 239.1.1.100Leaf-1(config-if-nve-vni)#   suppress-arpLeaf-1(config-if-nve)#  member vni 10200Leaf-1(config-if-nve-vni)#   mcast-group 239.1.1.100Leaf-1(config-if-nve-vni)#   suppress-arp! L3 VNILeaf-1(config-if-nve)#  member vni 50001 associate-vrf

Step 5 — EVPN BGP

cisco

Leaf-1(config)# router bgp 65001Leaf-1(config-router)#  router-id 10.1.1.1Leaf-1(config-router)#  neighbor 10.0.0.1 remote-as 65001Leaf-1(config-router)#  neighbor 10.0.0.1 update-source loopback0Leaf-1(config-router)#  neighbor 10.0.0.1 send-community extendedLeaf-1(config-router)#  neighbor 10.0.0.2 remote-as 65001Leaf-1(config-router)#  neighbor 10.0.0.2 update-source loopback0Leaf-1(config-router)#  neighbor 10.0.0.2 send-community extendedLeaf-1(config-router)#  address-family l2vpn evpnLeaf-1(config-router-af)#   neighbor 10.0.0.1 activateLeaf-1(config-router-af)#   neighbor 10.0.0.1 route-reflector-clientLeaf-1(config-router-af)#   neighbor 10.0.0.2 activateLeaf-1(config-router-af)#   neighbor 10.0.0.2 route-reflector-client! EVPN MAC/IP advertisement per VLANLeaf-1(config)# evpnLeaf-1(config-evpn)#  vni 10100 l2Leaf-1(config-evpn-evi)#   rd autoLeaf-1(config-evpn-evi)#   route-target import autoLeaf-1(config-evpn-evi)#   route-target export auto

Part 4 — Symmetric vs Asymmetric IRB

Integrated Routing and Bridging (IRB) is how VXLAN performs inter-VLAN routing. There are two models:

Asymmetric IRB — Route on ingress leaf, bridge on egress leaf. Simple but requires all VLANs present on all leaves (VLAN stretching). Not recommended for large fabrics.

Symmetric IRB — Route on both ingress and egress leaf using L3 VNI as a transit VNI between VTEPs. Each leaf only needs the VLANs of its locally connected hosts. This is the recommended model for production.

bash

# Symmetric IRB requires:#   1. L3 VNI per tenant VRF (configured above)#   2. SVI for each L2 VNI as anycast gateway#   3. EVPN Type-5 routes for inter-subnet routing#   4. VRF with route-target import/export## Traffic flow (VM-A on Leaf-1 to VM-B on Leaf-2, different subnets):# 1. VM-A sends to default gateway (anycast GW on Leaf-1)# 2. Leaf-1 routes to Leaf-2 using L3 VNI 50001 in VXLAN header# 3. Leaf-2 decapsulates, bridges to VM-B VLAN# Return traffic follows the same path in reverse — symmetric

Part 5 — Troubleshooting VXLAN EVPN

Step 1 — Verify EVPN BGP Sessions

cisco

Leaf-1# show bgp l2vpn evpn summary! All spines should show Up/Established! Check: PfxRcd > 0 means MAC/IP routes are being receivedLeaf-1# show bgp l2vpn evpn! Shows all EVPN routes: Type-2 (MAC/IP), Type-3 (IMET/BUM), Type-5 (prefix)Leaf-1# show bgp l2vpn evpn 10.100.0.10/32! Look up a specific host IP — should show originating VTEP as next-hop

Step 2 — Verify NVE and Tunnel State

cisco

Leaf-1# show nve peers! Should show all remote VTEPs as Up! LearnType: CP = control plane (EVPN) — correct! LearnType: DP = data plane (flood-and-learn) — investigate why EVPN not workingLeaf-1# show nve vni! Verify L2 VNIs and L3 VNIs are in Up stateLeaf-1# show nve interface nve1 detail

Step 3 — MAC and ARP Table Verification

cisco

! Check local and remote MAC addressesLeaf-1# show mac address-table vlan 100! Local MACs: interface = Ethernet port! Remote MACs: interface = nve1, VTEP IP shows as next-hop! Check ARP suppression table (should have remote host ARPs)Leaf-1# show ip arp suppression-cache detail! Check VRF routing tableLeaf-1# show ip route vrf TENANT-ALeaf-1# show ip route vrf TENANT-A 10.200.0.0/24! Remote subnets should show via nve1 with VTEP IP as next-hop

Step 4 — End-to-End Connectivity

cisco

! Ping between VTEPs (underlay check)Leaf-1# ping 10.1.1.2 source loopback1! Ping from tenant VRF (overlay check)Leaf-1# ping 10.200.0.10 vrf TENANT-A source 10.100.0.1! Traceroute to verify pathLeaf-1# traceroute 10.200.0.10 vrf TENANT-A source 10.100.0.1

Quick Reference — EVPN Route Types

Route Type	Name	Carries	Used For
Type-1	Ethernet Auto-Discovery	ESI, Ethernet tag	Multi-homing fast convergence
Type-2	MAC/IP Advertisement	MAC + optional IP binding	Host MAC/ARP distribution — most common
Type-3	Inclusive Multicast (IMET)	VTEP IP + VNI	BUM traffic — announces VTEP participation in a VNI
Type-4	Ethernet Segment	ESI	Designated Forwarder election for multi-homing
Type-5	IP Prefix Route	IP prefix + gateway MAC	Inter-subnet routing in symmetric IRB

VXLAN EVPN Hardening Checklist

Underlay IGP (OSPF/IS-IS) is stable before configuring VXLAN overlay — fix underlay issues first
Each VTEP loopback (NVE source) is advertised into the underlay IGP
VNI numbering follows a documented scheme — L2 VNIs and L3 VNIs in separate ranges
Symmetric IRB is used — not asymmetric — VLANs are not unnecessarily stretched across all leaves
ARP suppression is enabled on all L2 VNIs — reduces BUM flooding significantly
route-target both auto evpn is used consistently — manual RT mismatches are a common failure mode
Spines are configured as EVPN Route Reflectors — leaves peer only with spines (not full-mesh)
send-community extended is set on all EVPN BGP neighbors — without it, route targets are stripped
Anycast gateway MAC is identical on all VTEPs — any mismatch causes ARP issues
L3 VNI VLAN SVIs have ip forward only — no IP address assigned