Building Data Centers with VXLAN BGP EVPN

Lukas Krattiger, CCIE No. 21921 (Routing/Switching and Data Center), is principal engineer, Technical Marketing, with more than 15 years of experience in data center, Internet, and application networks. Within Cisco, he specializes in data center switching, overlay architectures, and solutions across platforms. Lukas is a double-CCIE (R&S and Data Center) with several other industry certifications and has participated in various technology leadership and advisory groups. Prior to joining Cisco, Lukas was a senior network engineer with System Integrators and Service Providers, where he was responsible for data center and Internet networks. Since joining Cisco, he has covered various technologies within the data center as well as enterprise networks portfolio, and he has built foundational solutions for customers and partners. Shyam Kapadia is a principal engineer in the Data Center Group at Cisco Systems. With more than a decade of experience in the networking industry, Shyam holds more than 30 patents and has coauthored the book Using TRILL, FabricPath, and VXLAN: Designing MSDC with Overlays. In his 10 years at Cisco, Shyam has worked on a number of products, including the Catalyst and Nexus families of switches, with special emphasis on end-to-end data center solutions, including automation and orchestration. He holds a Ph.D. and master’s degree from the University of Southern California in the field of computer science. Over the past 15 years, Shyam has been the Program Chair for the Southern California Linux Exposition (SCALE). David Jansen, CCIE No. 5952 (Routing/Switching), is a distinguished systems engineer (DSE) for Cisco, specializing in data center, campus, branch/WAN, and cloud architectures. He has 20 years of experience in the industry and has earned certifications from Novell, VMware, Microsoft, TOGAF, and Cisco. His focus is working with global enterprise customers to address their challenges with comprehensive end-to-end data center, enterprise, WAN/Internet, and cloud architectures. David has been with Cisco for more than 19 years; for the last 4 years or so as a DSE, he has gained unique experiences in building next generation data center solutions.

• Introduction xv Chapter 1 Introduction to Programmable Fabric 1 Today’s Data Center Challenges and Requirements 2 The Data Center Fabric Journey 3 Cisco Open Programmable Fabric 10 Fabric-Related Terminology 13 Data Center Network Fabric Properties 14 Server or Endpoint Connectivity Options 15 Summary 17 References 17 Chapter 2 VXLAN BGP EVPN Basics 21 Overlays 23 Introduction to VXLAN 27 VXLAN Flood and Learn (F&L) 30 Introduction to BGP EVPN with VXLAN 32 MP-BGP Features and Common Practices 34 IETF Standards and RFCs 37 Host and Subnet Route Distribution 40 Host Deletion and Move Events 46 Summary 48 References 49 Chapter 3 VXLAN/EVPN Forwarding Characteristics 53 Multidestination Traffic 54 Leveraging Multicast Replication in the Underlying Network 55 Using Ingress Replication 58 VXLAN BGP EVPN Enhancements 60 ARP Suppression 60 Distributed IP Anycast Gateway 65 Integrated Route and Bridge (IRB) 69 Endpoint Mobility 73 Virtual PortChannel (vPC) in VXLAN BGP EVPN 76 DHCP 81 Summary 85 References 85 Chapter 4 The Underlay 87 Underlay Considerations 88 MTU Considerations 91 IP Addressing 93 IP Unicast Routing 99 OSPF as an Underlay 100 IS-IS as an Underlay 102 BGP as an Underlay 103 IP Unicast Routing Summary 106 Multidestination Traffic 107 Unicast Mode 107 Multicast Mode 109 PIM Any Source Multicast (ASM) 112 BiDirectional PIM (PIM BiDir) 114 Summary 119 References 119 Chapter 5 Multitenancy 121 Bridge Domains 123 VLANs in VXLAN 124 Layer 2 Multitenancy: Mode of Operation 129 VLAN-Oriented Mode 130 BD-Oriented Mode 131 VRF in VXLAN BGP EVPN 132 Layer 3 Multitenancy: Mode of Operation 134 Summary 137 References 138 Chapter 6 Unicast Forwarding 139 Intra-Subnet Unicast Forwarding (Bridging) 139 Non-IP Forwarding (Bridging) 147 Inter-Subnet Unicast Forwarding (Routing) 149 Routed Traffic to Silent Endpoints 158 Forwarding with Dual-Homed Endpoint 164 IPv6 167 Summary 169 Chapter 7 Multicast Forwarding 171 Layer 2 Multicast Forwarding 171 IGMP in VXLAN BGP EVPN Networks 174 Layer 2 Multicast Forwarding in vPC 178 Layer 3 Multicast Forwarding 182 Summary 184 References 184 Chapter 8 External Connectivity 185 External Connectivity Placement 185 External Layer 3 Connectivity 189 U-Shaped and Full-Mesh Models 190 VRF Lite/Inter-AS Option A 192 LISP 195 MPLS Layer 3 VPN (L3VPN) 200 External Layer 2 Connectivity 203 Classic Ethernet and vPC 204 Extranet and Shared Services 206 Local/Distributed VRF Route Leaking 207 Downstream VNI Assignment 210 Summary 212 Reference 212 Chapter 9 Multi-Pod, Multifabric, and Data Center Interconnect (DCI) 213 Contrasting OTV and VXLAN 213 Multi-Pod 219 Interconnection at the Spine Layer 227 Interconnection at the Leaf Layer 227 Multifabric 228 Inter-pod/Interfabric 231 Interfabric Option 1: Multi-Pod 232 Interfabric Option 2: Multifabric 233 Interfabric Option 3 (Multisite for Layer 3) 235 Interfabric Option 4 (Multisite for Layer 2) 236 Summary 238 References 238 Chapter 10 Layer 4—7 Services Integration 241 Firewalls in a VXLAN BGP EVPN Network 242 Routing Mode 242 Bridging Mode 244 Firewall Redundancy with Static Routing 245 Static Route Tracking at a Service Leaf 248 Static Routing at a Remote Leaf 248 Physical Connectivity 249 Inter-Tenant/Tenant-Edge Firewall 250 Services-Edge Design 254 Intra-Tenant Firewalls 254 Mixing Intra-Tenant and Inter-Tenant Firewalls 260 Application Delivery Controller (ADC) and Load Balancer in a VXLAN BGP EVPN Network 262 One-Armed Source-NAT 262 Direct VIP Subnet Approach 263 Indirect VIP Subnet Approach 264 Return Traffic 265 Service Chaining: Firewall and Load Balancer 267 Summary 271 References 271 Chapter 11 Introduction to Fabric Management 273 Day-0 Operations: Automatic Fabric Bring-Up 275 In-Band Versus Out-of-Band POAP 276 Other Day-0 Considerations 278 Day-0.5 Operations: Incremental Changes 279 Day-1 Operations: Overlay Services Management 280 Virtual Topology System (VTS) 282 Nexus Fabric Manager (NFM) 282 Data Center Network Manager (DCNM) 283 Compute Integration 283 Day-2 Operations: Monitoring and Visibility 285 VXLAN OAM (NGOAM) 294 Summary 299 References 299 Appendix A VXLAN BGP EVPN Implementation Options 303 9781587144677, TOC, 3/16/17