Table Of Content

Virtual Extensible LAN (VXLAN) A Practical guide to VXLAN solution Toni Pasanen, CCIE 28158 ii iii Copyright © Toni Pasanen, All rights reserved. Revision History August 2019: First Edition October 2019: Second Edition The second edition includes additional chapter 19: “Tenant Routed Multicast (TRM)” October 2019: Editing numering of examples and figures (ch. 15-17). November 2019: Editing numering of examples and figures. (ch. 1-14, 18-19) iv About the Author: Toni Pasanen. CCIE No. 28158 (Routing and Switching), Presales Engineer at Fujitsu Finland. Toni started his IT-carrier in 1998 at Tieto, where he worked as a Service Desk Specialist moving via LAN team to Data Center team as a 3rd. Level Network Specialist. Toni joined Teleware (Cisco Learning partner) in 2004, where he spent two years teaching network technologies focusing routing/switching and MPLS technologies. Toni joined Tieto again in 2006, where he spent next six years as a Network Architect before joining Fujitsu. In his current role, Toni work closely with customers helping them in selecting the right network solutions not only from the technology perspective but also from the business perspective. Dedications To my lovely wife Tiina, thanks for pushing me forward when I was about the throw this procject out of the window (more than twice). v vi Table of Contents Chapter 1: Underlay Network – OSPF Operation 1 Introduction 1 OSPF 2 Link-State Database (LSDB) optimization 3 Shortest-Path First (SPF)/Dijkstra Algorithm 8 SPF Run – Phase I: Building a Shortest-Path Tree 9 First iteration round 10 Second iteration round 11 Third iteration round 12 Fourth iteration round 13 Fifth iteration round 14 Sixth iteration round 15 Seventh iteration round 16 SPF Run – Phase II: Adding Leafs to Shortest-Path Tree 17 References: 19 Chapter 2: Underlay Network – Comparison of OSPF and IS-IS 20 Scenario-1: Interface loopback 50 down on Leaf-101 (IS-IS) 22 Scenario-2: Interface loopback 50 down on Leaf-101 (OSPF) 24 Scenario-3: OSPF Incremental SPF – L55 Down on Leaf-101 (Stub) 26 Scenario-4: OSPF Incremental SPF – Interface g0/3 Down on Spine-12 (transit link does not participate in SPT) 27 Scenario-5: IS-IS SPF – Interface g0/3 Down on Spine-12 (Full SPF computation) 28 Scenario-6: IS-IS Incremental SPF – Interface g0/3 Down on Spine-12 29 (transit link does not participate in SPT) 29 Conclusion 30 Chapter 3: Underlay Network: iBGP in Underlay Network 32 Next-Hop-Self consideration 36 Case-1: Next-hop-self is changed by RR Spine-11. 36 Case-2: RR Spine-11 does not change Next-hop-self. 37 Chapter 4: Underlay Network: Two-AS eBGP 42 Underlay Network Control Plane eBGP 42 Overlay Network Control Plane: eBGP 46 References: 58 Chapter 5: eBGP as an Underlay Network Routing Protocol: Multi-AS eBGP 59 vii Underlay Network Control Plane: IPv4 eBGP peering 59 Overlay Network Control Plane: L2VPN EVPN eBGP peering 61 References: 69 Chapter 6: Layer 2 Multi-Destination Traffic - Anycast-RP with PIM. 70 Step 1: Configuring Anycast-RP cluster 71 Step 2: Assign unique Cluster Member IP and define members 71 Step 3: Assign unique Cluster Member IP and define members 72 Configuring NVE interface 74 Anycast-PIM Control Plane Operation 75 Phase 1: PIM Join 75 Phase 2: PIM Registration 76 Phase 3: PIM Registration-Stop 78 Phase 4: Anycast-RP peer notification 78 Data Plane operation 82 ARP Request 82 ARP Reply 83 References: 85 Chapter 7: Layer 2 Multi-destination traffic - PIM BiDir. 86 Configuration 86 Control Plane Operation 87 References 90 Chapter 8: BGP EVPN VXLAN Configuration and building blocks. 91 BGP EVPN VXLAN Building Blocks for Intra-VNI switching 91 Underlay Network: OSPF configuration 92 Overlay Network: BGP L2VPN EVPN configuration 93 Overlay Network: NVE Peering 93 Overlay Network: Host Mobility Manager 94 Overlay Network: Anycast Gateway (AGW) 94 Overlay Network: VLAN based service 95 Overlay Network: TCAM modification 95 Intra-VNI service (L2VNI) in VXLAN Fabric 96 Tenant based Inter-VNI Routing (L3VNI) in VXLAN Fabric 100 References 103 Chapter 9: BGP EVPN VXLAN Control and Data Plane Operation. 104 MAC address learning process 105 Phase 1: MAC Address-Table update 105 Phase 2: L2RIB Update 106 viii Phase 3: BGP MAC Route Export on Local VTEP 107 Phase 4: BGP AFI L2EVPN MAC Route Import on Remote VTEP 110 Phase 5: MAC VRF on Remote VTEP 112 Phase 6: MAC Address Table on Remote VTEP 113 L2VNI: Intra-VNI Data Plane 113 ARP Request 114 ARP Reply 116 ICMP Request 118 ICMP Reply 120 Summary 121 MAC-IP address learning process 122 Phase 1: ARP Table on Local VTEP 123 Phase 2-3: MAC-IP on Local VTEP 123 Phase 4: BGP Route Export on Local VTEP 125 Phase 5: BGP Route Import on Remote VTEP 126 Phase 6: IP VRF on Remote VTEP 129 ARP-Suppression 130 Host route Advertisement: Inter-VNI routing (L3VNI) 132 Phase 1. Host Route in Local Routing Information Base (RIB) 133 Phase 2. Host Route BGP Process on Local VTEP 133 Phase 3. Host Route BGP Process on Remote VTEP 134 Phase 4. Installing Host Route into RIB of Remote VTEP 135 Data Plane operation 137 Phase 1. Switching in VNI30000 on VTEP-102 137 Phase 2. Routing from VNI30000 to VNI 10077 on VTEP-102 138 Phase 3. Routing from VNI10077 to VNI 10000 on VTEP-101 138 Summary 139 Prefix Advertisement 139 Phase 1: vmBeef start pinging to vmBebe 140 Phase 2: Local VTEP Leaf-101: ARP process 141 Phase 3: Remote VTEP Leaf-102: ARP process - Request 142 Phase 4: vmBebe: ARP process - Reply 143 Phase 5: remote VTEP switch Leaf-102: BGP Update 144 Phase 6: Local VTEP switch Leaf-102: BGP Update 144 Data Plane testing 148 Phase 1: vmBeef start pinging to vmBebe 149 Phase 2: Local VTEP Leaf-101: Routing 149 Phase 3-4: Remote VTEP Leaf-102: ARP request 150 ix Phase 5: vmBebe: ARP Reply 151 Phase 6: Remote VTEP Leaf-102: ICMP Request forwarding 152 Phase 7: vmBebe: ICMP reply 152 Phase 8-9: Remote VTEP Leaf-102: Routing decision and ICMP reply 152 Phase 10-11: Local VTEP Leaf-101: Routing decision and ICMP reply 153 Summary 156 References 157 Chapter 10: VXLAN fabric External Connections 158 eBGP Configuration between Border Leaf-102 and Ext-Ro01 158 Starting point 160 Chapter 11: Multihoming with vPC 190 Virtual Port Channel Configuration 190 Some other consideration for vPC: 197 VTEP redundancy with vPC 198 Advertising Primary IP address 204 References: 210 Chapter 12: Multihoming - vPC and Graceful Insertion and Removal (GIR) operation 211 Loopback addressing 211 Graceful Insertion and Removal (GIR) 212 Verifications. 213 Example-2 summary: BGP EVPN peering and NVE1 using the same Loopback interface. 218 Conclusion 219 References: 220 Chapter 13: Using vPC Peer Link as an Underlay Backup Path 221 Configuration 222 Verification 223 References: 227 Chapter 14: VXLAN Fabric Firewall Implementation 228 Protected segment 229 Non-Protected segment 230 Connectivity Testing 238 References: 240 Chapter 15: EVPN ESI Multihoming 241 Introduction 241 x Ethernet Segment Identifier (ESI) and Port-Channel 242 Designated Forwarder (DF) 243 Designated Forwarder 246 References: 248 Chapter 16: EVPN ESI Multihoming - Fast Convergence and Load Balancing 249 Ethernet A-D per ES route - Fast Convergence in the all-Active mode 249 Fast Convergence 254 Load Balancing (Aliasing) 257 Summary 258 References: 259 Chapter 17: EVPN ESI Multihoming - Data Flows and link failures 260 Introduction 260 Intra-VNI (L2VNI): Unicast Traffic 262 Scenario 1: Link E1/2 down on Leaf-102 262 Scenario 2: Core link down on Leaf-102. 265 Intra-VNI (L2VNI): Broadcast, Unknown Unicast and Multicast (BUM) traffic 266 Scenario 1: Traffic flow from Designated Forwarder 266 Scenario 2: Traffic flow from non-Designated Forwarder 267 CHAPTER 18: VXLAN EVPN Multi-Site 269 Shared EVPN domain limitations 269 EVPN Multi-Site Architecture Introduction 270 Intra-Site EVPN Domain (Fabric) 271 Intra-Site NVE peering and VXLAN tunnels 272 Summary 278 Shared Common EVPN Domain Connections 278 Border Gateway setup 279 Multi-Destination traffic forwarding 287 Designated Forwarder 287 Ingress-Replication 293 Fabric Link Failure 299 Normal State 300 Fabric-Link Failure 302 Fabric-Link Recovery 304 DCI-Link Failure 307 Normal State 308 DCI Link Failure 309 DCI Link Recovery 310