VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for...

48
© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation ID VXLAN 1

Transcript of VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for...

Page 1: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation ID

VXLAN

1

Page 2: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Virtual Overlay Encapsulations and Forwarding •  Ethernet Frames are encapsulated into an IP frame format •  New control logic for learning and mapping VM identity (MAC address) to Host

identity (IP address) •  Two main Hypervisor based Overlays

•  VXLAN Virtual Extensible Local Area Newtork •  NVGRE, Network Virtualization Generic Router Encapsulation

•  GENEVE Generic Network Virtualization Encapsulation ( draft ) •  Network Based Overlays

•  OTV, Overlay Transport Virtualization •  VPLS, EVPN •  FabricPath •  VXLAN and NVGRE

Page 3: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

What is a Virtual Overlay Technology ? •  Servers perform data encapsulation and forwarding

•  SW based virtual switches instantiate customer topologies

VM1

VM2

VM3

Virtual Switch

Hypervisor

VM4

VM5

VM6

Virtual Switch

Hypervisor

IP Network

Ethernet Frames

IP/UDP Packets

IP Addr 2.2.2.2

IP Addr 1.1.1.1

Page 4: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Technologies Intra-DC and Inter-DC Requirement Intra-DC Inter-DC Layer 2 connectivity FabricPath, VXLAN OTV, VPLS

IP Mobility LISP, FP, AnyCast Gateway LISP, OTV

Secure Segmentation VXLAN / Segment-ID LISP, MPLS-IP-VPNs

DC-east DC-west

POD POD POD POD

IP Network

Fabric Path (Intra-DC L2)

OTV, VPLS (Inter-DC L2-x-L3)

App

OS

App

OS

App

OS

App

OS

EF, LISP, VXLAN (Intra-DC x-L3)

LISP IP mobility (Inter-DC)

Fabric Path (Intra-DC L2)

App

OS

App

OS

EF, LISP (Intra-DC mobility)

Page 5: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation ID

VXLAN Properties

5

Page 6: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Virtual Extensible Local Area Network (VXLAN) •  Ethernet in IP overlay network

•  Entire L2 frame encapsulated in UDP •  50 bytes of overhead ( 54 bytes if VLAN

tag is used )

•  Include 24 bit VXLAN Identifier •  16 M logical networks •  Mapped into local bridge domains

•  VXLAN can cross Layer 3

•  Tunnel between VEMs •  VMs do NOT see VXLAN ID

•  IP multicast used for L2 broadcast/multicast, unknown unicast

•  Technology submitted to IETF for standardization •  With Cisco, Arista, VMware, Citrix, Red Hat and

Others

Outer MAC DA

Outer MAC SA

Outer 802.1Q

Outer IP DA

Outer IP SA

Outer UDP

VXLAN ID (24 bits)

Inner MAC DA

Inner MAC SA

Optional Inner 802.1Q

Original Ethernet Payload

CRC

VXLAN Encapsulation Original Ethernet Frame

For Your Reference

Page 7: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

What is a vApp? •  A Cloud Provider using vCloud Director offers catalogs of vApps to their Users •  When cloned, new vApps retain the same MAC and IP addresses •  Duplicate MACs within different vApps requires L2 isolation •  Duplicate IP addresses requires L2/L3 isolation (NAT of externally facing IP

addresses) •  Usage of vApps causes an explosion in the need for isolated L2 segments

7

vApp

DB VM’s

App VM’s

Web VM’s

VLAN 55 VXLAN 5500 VXLAN 5501 VXLAN 5502

Page 8: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Destination is in another segment. Packet is routed to the new

segment

VXLANORANGE VXLANBLUE

Ingress VXLAN packet on Orange segment

VXLAN Router

VXLAN L2 and L3 Gateways Connecting VXLAN to the broader network

L2 Gateway: VXLAN to VLAN Bridging VXLANORANGE

Ingress VXLAN packet on Orange segment

Egress interface chosen (bridge may .1Q tag the

packet)

VXLAN L2 Gateway

SVI

Egress interface chosen (bridge may .1Q tag the packet)

L3 Gateway: VXLAN to X Routing •  VXLAN •  VLAN VLAN100 VLAN200

Page 9: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Data Plane Learning Dedicated Multicast Distribution Tree per VNI

VTEP VTEP VTEP

PIM Join for Multicast Group 239.1.1.1

PIM Join for Multicast Group 239.1.1.1

PIM Join for Multicast Group 239.2.2.2

PIM Join for Multicast Group 239.2.2.2

Web VM

Web VM

DB VM

DB VM

Multicast-enabled Transport

See Slide 50 for Multicast Configuration on the spine

Page 10: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Data Plane Learning Learning on Broadcast Source - ARP Request Example

VM 1 VM 3 VM 2

VTEP 1 1.1.1.1

VTEP 3 3.3.3.3

VTEP 2 2.2.2.2

IP A è G ARP Req

MAC IP Addr VM 1 VTEP 1

MAC IP Addr VM 1 VTEP 1

ARP Req

IP A è G ARP Req

ARP Req ARP Req

Multicast-enabled Transport

Page 11: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Data Plane Learning Learning on Unicast Source - ARP Response Example

VM 1 VM 3 VM 2

VTEP 1 1.1.1.1

VTEP 3 3.3.3.3

VTEP 2 2.2.2.2

ARP Resp

MAC IP Addr VM 2 VTEP 2

Multicast-enabled Transport

VTEP 2 è VTEP 1 ARP Resp ARP Resp

MAC IP Addr VM 1 VTEP 1

Page 12: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

VXLAN Gateway Functionality For Your Reference

PLATFORM  VXLAN  Bridging  and/or  VXLAN  

Rou7ng)  Star7ng  Release   PLATFORM  

VXLAN  Bridging  and/or  VXLAN  

Rou7ng)  Star7ng  Release  

DATA  CENTER   ENTERPRISE  Networking      

Nexus  1000v   Yes:  Both  4.2(1)SV1(5.1) (MCast

ASR  1K   Bridging  only   IOS XE 3.13S (Bridging)

5.2(1)SV3 (BGP CP)

Nexus  3100   Bridging  Only   6.0(2)U3(2)

Nexus  5600   Yes  Both   7.1(0)N1(1a)  

Nexus  7x00  with  F3     Yes  Both   7.2  ASR  9K   Yes  Both  

IOS XR 5.2.0 (Bridging and

Routing) Nexus  9300  (Standalone)   Yes  Both   7.0(3)  

Page 13: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Multicast Enabled Underlay

•  PIM-ASM or PIM-BiDir (Different hardware has different capabilities) •  Spine and Aggregation Switches make good Rendezvous-Point (RP); much lile RR •  PIM-ASM (sparse-mode)

•  Source-trees, build a couple of unidirectional trees from RP; (s,g) •  Every VTEP is Source and Destination •  PIM-Anycast RP vs MSDP for example

•  PIM-BiDir •  No Sources tree use a bi-directional shared tree •  No (S,G), we have (*,G) •  Phanton RP (Leverages Unicast for convergence)

•  Each VNI does not need the same a different Multicast Group

Underlay

13

Nexus 1000v Nexus 3000 Nexus 5600 Nexus 7000/F3 Nexus 9000 ASR 1000 CSR 1000 ASR 9000

Multicast Mode IGMP L2/L3 PIM ASM PIM BiDir PIM ASM / PIM BiDir PIM ASM PIM BiDir PIM ASM / PIM BiDir

Page 14: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation ID

The Underlay

14

Page 15: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Deployment Considerations

•  MTU and Overlays

•  Unicast Routing Protocol and IP Addressing

•  Multicast for BUM Traffic Replication

Underlay

15

Page 16: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Building your IP Network – Interface Principles

•  Know your IP addressing and IP scale requirements •  Best to use single Aggregate for all

Underlay Links and Loopbacks •  IPv4 only •  For each Point-2-Point (P2P)

connection, minimum /31 required •  Loopback requires /32

•  Routed Ports/Interfaces •  Layer-3 Interfaces between Spine and

Leaf (no switchport)

•  VTEP uses Loopback as Source-Interface

Underlay

16

L2 L1

L3

Page 17: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Building your IP Network – Routing Protocols; OSPF

•  OSPF – watch your Network type •  Network Type Point-2-Point (P2P)

•  Preferred (only LSA type-1) •  No DR/BDR election •  Suits well for routed interfaces/ports

(optimal from a LSA Database perspective) •  Full SPF calculation on Link Change

•  Network Type Broadcast •  Suboptimal from a LSA Database

perspective (LSA type-1 & 2) •  DR/BDR election •  Additional election and Database Overhead

Underlay

17

L2 L1

L3

Page 18: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Building your IP Network – Routing Protocols; IS-IS

•  IS-IS – what was this CLNS? -  Independent of IP (CLNS) -  Well suited for routed interfaces/ports -  No SPF calculation on Link change;

only if Topology changes -  Fast Re-convergence -  Not everyone is familiar with it

Underlay

18

L2 L1

L3

Page 19: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

What is the key point to Remember?

Overlay Convergence = Underlay Convergence!

19

Page 20: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation ID

The Control Plane Evolution

20

Page 21: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

EVPN Multi-vendor Interoperability Demonstrated MPLS SDN World Congress Hosted by EANTC EVPN Interoperability Testing Participants: Cisco, Juniper, Alcatel-Lucent, Ixia Results:

•  All participating vendor platforms can interoperate for VXLAN Layer-2 bridging

•  Nexus 9300 functioned as both EVPN iBGP route reflector spine and VTEP leaf

•  Nexus 9000 and Nexus 7000 demonstrated EVPN symmetric routing

White paper by EANTC: http://www.eantc.de/fileadmin/eantc/downloads/events/2011-2015/MPLSSDN2015/EANTC-MPLSSDN2015-WhitePaper_online.pdf

Page 22: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

EVPN – Ethernet VPN VXLAN Evolution

22

Control- Plane

EVPN MP-BGP draft-ietf-l2vpn-evpn

Data- Plane

Multi-Protocol Label Switching (MPLS)

draft-ietf-l2vpn-evpn

Provider Backbone Bridges (PBB)

draft-ietf-l2vpn-pbb-evpn

Network Virtualization Overlay (NVO)

draft-sd-l2vpn-evpn-overlay

Ø  EVPN over NVO Tunnels (VXLAN, NVGRE, MPLSoE) for Data Center Fabric encapsulations

Ø  Provides Layer-2 and Layer-3 Overlays over simple IP Networks

For Your Reference

Page 23: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

What is VXLAN/EVPN? •  Standards based Overlay (VXLAN) with Standards based Control-Plane (BGP)

•  Layer-2 MAC and Layer-3 IP information distribution by Control-Plane (BGP)

•  Forwarding decision based on Control-Plane (Flood Prevention Optimized ARP forwarding)

•  Integrated Routing/Bridging (IRB) for Optimized Forwarding in the Overlay

Page 24: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

VXLAN Control Plane Options

24

PLATFORM  VXLAN  Bridging  and/or  VXLAN  

Rou7ng)  Star7ng  Release   Mul7cast   EVPN  Support   VTS  support   Layer  2  

Gateway  Layer  3  Gateway    

Mul7cast  Groups  

Ingress  Replica7on  

DATA  CENTER  

Nexus  1000v   Yes:  Both  4.2(1)SV1(5.1)  (MCast  

    Planned  for  Fall                      5.2(1)SV3  (BGP  CP)  

Nexus  3100   Bridging  Only   6.0(2)U3(2)   PIM  Sparse  mode                          

Nexus  5600   Yes  Both   7.1(0)N1(1a)  PIM  BiDir  

Engineer  Release  August  

Trying  for  support  in  2.0     650   650   200      

Nexus  7x00  with  F3     Yes  Both   7.2  

PIM  Sparse  and  BiDir  Mode  

7.2  Layer  3  only  Helsinki  full  Layer  2     VTS  2.0                  

Nexus  9300  (Standalone)   Yes  Both   7.0(3)   PIM  Sparse  

mode   Bronte   VTS  1.5     1024   900       Camden  

Page 25: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Host and Subnet Route Distribution

•  Host Route Distribution decoupled from the Underlay protocol

•  Use MultiProtocol-BGP (MP-BGP) on the Leaf nodes to distribute internal Host/Subnet Routes and external reachability information

•  Route-Reflectors deployed for scaling purposes

VXLAN/EVPN

25

RR RR

L2 L1

L3

BGP Route-Reflector RR

iBGP Adjacency

Page 26: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Protocol Learning & Distribution (1) VXLAN/EVPN

26

Host A MAC_A / IP_A Host B

MAC_B / IP_B

Virtual Switch

Host C MAC_C / IP_C

Host Y MAC_Y / IP_Y

RR RR

L2 L1

L3

1

1

1 VTEPs advertise Host Routes (IP+MAC) for the Host within the Control-Plane

1

Page 27: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Protocol Learning & Distribution (2) VXLAN/EVPN

27

Host A MAC_A / IP_A Host B

MAC_B / IP_B

Virtual Switch

Host C MAC_C / IP_C

Host Y MAC_Y / IP_Y

RR RR

L2 L1

L3

2 2

2

2 BGP propagates routes for The Host to all other VTEPs

MAC, IP VNI NH

MAC_A, IP_A 30000 IP_L1

MAC_B, IP_B 30000 IP_L2

MAC, IP VNI NH

MAC_A, IP_A 30000 IP_L1

MAC_C, IP_C 30000 IP_L3

MAC_Y, IP_Y 30001 IP_L3

3 VTEPs obtain host routes for remote hosts and install in RIB/FIB

3 3

3

MAC, IP VNI NH

MAC_B, IP_B 30000 IP_L2

MAC_C, IP_C 30000 IP_L3

MAC_Y, IP_Y 30001 IP_L3

NH = VTEP IP Address

Page 28: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Host Moves

1.  Host Moves to L3

2.  L3 detects Host A and advertises it with Seq #1

3.  L1 sees more recent route and withdraws its advertisement

VXLAN/EVPN

28

BGP Route-Reflector RR

iBGP Adjacency

MAC, IP VNI (L2)

VNI (L3)

NH Encap Seq

0050.56ac.0773, 192.168.101.101 30001 50000 0.0.0.0 8:VXLAN 1

Host A MAC_A / IP_A

RR RR

L3 L1

L3

Host A MAC_A / IP_A

L1# sh bgp l2vpn evpn 192.168.101.101BGP routing table information for VRF default, address family L2VPN EVPNRoute Distinguisher: 10.254.254.102:32868 (L2VNI 30001)BGP routing table entry for [2]:[0]:[0]:[48]:[0050.56ac.0773]:[32]:[192.168.101.101]/272, version 30Paths: (1 available, best #1)Flags: (0x00030a) on xmit-list, is not in l2rib/evpn

Advertised path-id 1 Path type: local, path is valid, is best path, no labeled nexthop AS-Path: NONE, path locally originated 10.254.254.102 (metric 0) from 0.0.0.0 (10.254.254.102) Origin IGP, MED not set, localpref 100, weight 32768 Received label 30001 50000 Extcommunity: RT:65501:30001 RT:65501:50000 MAC Mobility Sequence:00:1

Path-id 1 advertised to peers: 10.254.254.101

L2 RT Like TCP S# L3 RT

Page 29: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Additional Functions of VXLAN/EVPN ARP

Termination

Distributed Anycast Gateway

Suppresses flooding for Unknown Unicast ARP

Authenticate VTEPs through BGP peer authentication

Seamless and Optimal mobility

Active/Active Multipathing

Active/Active and Resilient Multipathing with vPC

Ingress Replication

Unicast Alternative to Multicast underlay

Security & Authentication

Page 30: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

ARP Suppression VXLAN/EVPN

30

Host A MAC_A / IP_A Host B

MAC_B / IP_B

Virtual Switch

Host C MAC_C / IP_C

Host Y MAC_Y / IP_Y

RR RR

L2 L1

L3

1 ARP Request sent for IP_B sent from Host A

MAC, IP VNI NH

MAC_A, IP_A 30000 IP_L1

MAC_B, IP_B 30000 IP_L2

MAC, IP VNI NH

MAC_A, IP_A 30000 IP_L1

MAC_C, IP_C 30000 IP_L3

MAC_Y, IP_Y 30001 IP_L3

2 L1 knows about IP_B and can respond. No need for ARP forwarding across the Network

MAC, IP VNI NH

MAC_B, IP_B 30000 IP_L2

MAC_C, IP_C 30000 IP_L3

MAC_Y, IP_Y 30001 IP_L3 ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF

1 2

ARP Response for IP_B Src MAC: MAC_B Dst MAC: MAC_A

Page 31: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

ARP Handling on Lookup “Silent host” (1) VXLAN/EVPN

31

Host A MAC_A / IP_A Host B

MAC_B / IP_B

Virtual Switch

Host C MAC_C / IP_C

Host Y MAC_Y / IP_Y

RR RR

1 ARP Request sent for IP_B sent from Host A

MAC, IP VNI NH

MAC_A, IP_A 30000 IP_L1

2 Miss of IP_B. Forward ARP Request to all Ports except source-port (ARP snooping)

MAC, IP VNI NH

MAC_C, IP_C 30000 IP_L3

MAC_Y, IP_Y 30001 IP_L3

ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF

1

Missing “B”

2

2

L2 L1

L3

MAC, IP VNI NH

MAC_A, IP_A 30000 IP_L1

MAC_C, IP_C 30000 IP_L3

MAC_Y, IP_Y 30001 IP_L3

ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF

ARP Request for IP_B Src MAC: MAC_A Dst MAC: FF:FF:FF:FF:FF:FF

Page 32: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

ARP Handling on Lookup “Silent host” (2) VXLAN/EVPN

32

Host A MAC_A / IP_A Host B

MAC_B / IP_B

Virtual Switch

Host C MAC_C / IP_C

Host Y MAC_Y / IP_Y

RR RR

3 ARP Response is sent to L2

MAC, IP VNI NH

MAC_A, IP_A 30000 L1

4 L2 will populate this information in the control-plane (learn) and forward it subsequently

MAC, IP VNI NH

MAC_C, IP_C 30000 L3

MAC_Y, IP_Y 30001 L3

ARP Response from IP_B Src MAC: MAC_B Dst MAC: MAC_A

3 MAC, IP VNI NH

MAC_A, IP_A 30000 IP_L1

MAC_B, IP_B 30000 IP_L2

ARP Response for IP_B Src MAC: MAC_B Dst MAC: MAC_A

4

4

MAC, IP VNI NH

MAC_A, IP_A 30000 IP_L1

MAC_C, IP_C 30000 IP_L3

MAC_Y, IP_Y 30001 IP_L3

L2 L1

L3

MAC, IP VNI NH

MAC_C, IP_C 30000 IP_L3

MAC_Y, IP_Y 30001 IP_L3

MAC_B, IP_B 30000 IP_L2

Page 33: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Gateway Functions in VXLAN

Centralized Gateway •  Extra Bridging hop before and after Routing •  Centralized Gateway (Aggregation) for Routing •  Large amounts of state => convergence issues •  Scale problem for large Layer-2 domains •  Works with VXLAN Flood & Learn or EVPN

VXLAN Routing

Distributed Gateway •  Route or Bridge at Leaf •  Distributed Gateway (Anycast) for Routing •  Disaggregate state by scale out •  Optimal Scalability •  Requires VXLAN/EVPN!

33

L1

L3

L2

L1

L3

VX

VY

L2

Layer-3 Boundary

Layer-3 Boundary

Page 34: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Consistent Configuration with Distributed Gateway

•  Logical Configuration only instantiated at respective Leaf (scoped)

•  ARP & MAC state only for local hosted VLAN/VNI and SVI

•  Flooding only to respective Leaf (where VLAN/VNI is instantiated)

•  Host demands provisioning; two models available •  top-down Orchestration, push to Leaf •  bottom-up Orchestration, pull by Leaf

VXLAN/EVPN

34

L1

L3

L2

Page 35: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Asymmetric IRB

•  Asymmetric •  Similar to Inter-VLAN routing •  Source and Destination VNI has to exist

on Switch where routing happens •  Post Routing traffic shares destination

VNI with Bridged traffic •  Not very suitable for distributed Routing

•  From Host A via VLAN/VNI “blue” routed at L1 to VNI “red” reaching destination VLAN “red”

•  From Host Y via VLAN/VNI “red” routed at L3 to VNI “blue reaching destination VLAN “blue”

VXLAN Routing

35

Host Y VNI 30001

Host A VNI 30000

L2

L1

L3

Page 36: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Symmetric IRB

•  Symmetric •  Similar to creating a Transit Segment •  Regardless of where Source or

Destination VNI exists •  Post Routing traffic uses different VNI

than Bridged traffic •  Additional VNI for Routing traffic (per

VRF) •  From Host A via VLAN “blue” routed at L1 to

VNI “purple” reaching destination VLAN “red”

•  From Host Y via VLAN “red” routed at L3 to VNI “purple” reaching destination VLAN “blue”

•  Used in Cisco VXLAN/EVPN

VXLAN Routing

36

Host Y VNI 30001

Host A VNI 30000

L3

L2

L1

Page 37: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821 37

L1# sh bgp l2vpn evpn 192.168.101.101 (IPA) BGP routing table information for VRF default, address family L2VPN EVPN Route Distinguisher: 10.254.254.102:32868 (L2VNI 30001) BGP routing table entry for [2]:[0]:[0]:[48]:[0050.56ac.0773]:[32]:[192.168.101.101]/272, version 30 Paths: (1 available, best #1) Flags: (0x00030a) on xmit-list, is not in l2rib/evpn Advertised path-id 1 Path type: local, path is valid, is best path, no labeled nexthop AS-Path: NONE, path locally originated 10.254.254.102 (metric 0) from 0.0.0.0 (10.254.254.102) Origin IGP, MED not set, localpref 100, weight 32768 Received label 30001 50000 Extcommunity: RT:65501:30001 RT:65501:50000 MAC Mobility Sequence:00:1 Path-id 1 advertised to peers: 10.254.254.101

L2 RT Like TCP S# L3 RT

Page 38: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

MP-BGP EVPN Address-Family: What’s in it?

Address-family (AF) is L2VPN EVPN, comprised of:

•  route-type 2 = MAC/48,IP/32

•  route-type 5 = IP Prefix

Format of route-type 2: MAC / IP / L2VNI / L3VNI / NH (Host Information) Format of route-type 5: IP Prefix / L3VNI / NH (Subnet Information, classic routing)

38

Note: No multicast routing in the overlay today; IETF has not concluded on implementation details / proposals.

Page 39: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821 39

L1# sh bgp l2vpn evpn 192.168.101.101 BGP routing table information for VRF default, address family L2VPN EVPN Route Distinguisher: 10.254.254.102:32868 (L2VNI 30001) BGP routing table entry for [2]:[0]:[0]:[48]:[0050.56ac.0773]:[32]:[192.168.101.101]/272, version 30 Paths: (1 available, best #1) Flags: (0x00030a) on xmit-list, is not in l2rib/evpn Advertised path-id 1 Path type: local, path is valid, is best path, no labeled nexthop AS-Path: NONE, path locally originated 10.254.254.102 (metric 0) from 0.0.0.0 (10.254.254.102) Origin IGP, MED not set, localpref 100, weight 32768 Received label 30001 50000 Extcommunity: RT:65501:30001 RT:65501:50000 MAC Mobility Sequence:00:1 Path-id 1 advertised to peers: 10.254.254.101

L2 RT Like TCP S# L3 RT

MP-BGP EVPN Type 2 Routes

Page 40: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

MP-BGP EVPN Type 5 Routes

40

L1# sh bgp l2vpn evpn 192.168.101.0 BGP routing table information for VRF default, address family L2VPN EVPN Route Distinguisher: 10.254.254.102:3 (L3VNI 50000) BGP routing table entry for [5]:[0]:[0]:[24]:[192.168.101.0]:[0.0.0.0]/224, version 11 Paths: (1 available, best #1) Flags: (0x00000a) on xmit-list, is not in l2rib/evpn Path type: internal, path is valid, not best reason: Weight, no labeled nexthop Imported from 10.254.254.103:3:[5]:[0]:[0]:[24]:[192.168.101.0]:[0.0.0.0]/120 AS-Path: NONE, path sourced internal to AS 10.254.254.103 (metric 3) from 10.254.254.101 (10.254.254.101) Origin IGP, MED not set, localpref 100, weight 0 Received label 50000 Extcommunity: RT:65501:50000 ENCAP:8 Router MAC:5087.89a2.157f Originator: 10.254.254.103 Cluster list: 10.254.254.101 Advertised path-id 1 Path type: local, path is valid, is best path, no labeled nexthop AS-Path: NONE, path locally originated 10.254.254.102 (metric 0) from 0.0.0.0 (10.254.254.102) Origin IGP, MED not set, localpref 100, weight 32768 Received label 50000 Extcommunity: RT:65501:50000 Path-id 1 advertised to peers: 10.254.254.101

ENCAP:8 = VXLAN

Page 41: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public Presentation ID

VXLAN Multi-tenancy

41

Page 42: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Leaf Tenant Configuration: L2 VLAN(s) Configuration

leaf

Eth

erne

t VLA

N 1

02

Eth

erne

t VLA

N 1

01

interface Ethernet1/1 switchport mode trunk switchport trunk allowed vlan 101-104 spanning-tree port type edge trunk spanning-tree bpduguard enable spanning-tree rootgaurd

vlan 101 vn-segment 30001 vlan 102 vn-segment 30002 vlan 103 vn-segment 30003 vlan 104 vn-segment 30004

*vn-segment (namespace) is for Layer 2 isolation; similar to Private-VLAN(s)

VN 30001 VN 30002

VN 50000

interface nve1 (VTEP) source-interface loopback0 host-reachability protocol bgp member vni 30001 mcast-group 239.239.239.100 member vni 30002 mcast-group 239.239.239.100 member vni 30003 mcast-group 239.239.239.100 member vni 30004 mcast-group 239.239.239.101

VN 30003

Eth

erne

t VLA

N 1

03

Page 43: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Leaf Tenant Configuration: L2 Configuration Allows mac-address to be advertised via EVPN

leaf

Eth

erne

t VLA

N 1

02

Eth

erne

t VLA

N 1

01

VN 30001 VN 30002

VN 50000

VN 30003

Eth

erne

t VLA

N 1

03

evpn vni 30001 l2 rd auto route-target import auto route-target export auto vni 30002 l2 rd auto route-target import auto route-target export auto

vni 30003 l2 rd auto route-target import auto route-target export auto vni 30004 l2 rd auto route-target import auto route-target export auto

Page 44: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Leaf Tenant Configuration: Layer 3 Definitions

leaf vrf: CiscoLive

Eth

erne

t VLA

N 1

01

Eth

erne

t VLA

N 1

00

VX

LAN

VN

I 500

00

interface vlan 100 no shutdown vrf member CiscoLive ip address 192.168.100.1/24 tag 5952 fabric forwarding mode anycast-gateway interface vlan 101 no shutdown vrf member CiscoLive ip address 192.168.101.1/24 tag 5952 fabric forwarding mode anycast-gateway interface vlan 2500 no shutdown mtu 9216 vrf member CiscoLive ip forward

VN 30000 VN 30001

VN 50000

vrf context CiscoLive vni 50000 rd auto (router-id + segment-id) address-family ipv4 unicast route-target both auto (BGPAS + segment-id) route-target both auto evpn address-family ipv6 unicast route-target both auto route-target both auto evpn

vlan 2500 vn-segment 50000

Page 45: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Leaf Tenant Configuration: Anycast GW MAC

leaf

vrf: CiscoLive

Eth

erne

t VLA

N 1

01

Eth

erne

t VLA

N 1

00

VX

LAN

VN

I 500

00

VN 30000 VN 30001

VN 50000

Anycast Gateway MAC (AGM) for SVI 100 and 101 fabric forwarding anycast-gateway-mac 2020.DEAD.BEEF interface vlan 100 no shutdown vrf member CiscoLive ip address 192.168.100.1/24 tag 5952 fabric forwarding mode anycast-gateway interface vlan 101 no shutdown vrf member CiscoLive ip address 192.168.101.1/24 tag 5952 fabric forwarding mode anycast-gateway

Page 46: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Leaf Tenant Configuration: BGP Configuration

leaf

vrf: CiscoLive

Eth

erne

t VLA

N 1

01

Eth

erne

t VLA

N 1

00

VX

LAN

VN

I 500

00

VN 30000 VN 30001

interface nve1 (VTEP) source-interface loopback0 host-reachability protocol bgp member vni 50000 associate-vrf (vrf routing information to be transported) router bgp 65501 vrf CiscoLive address-family ipv4 unicast advertise l2vpn evpn redistribute direct route-map FABRIC-RMAP-REDIST-SUBNET maximum-paths ibgp 2

route-map FABRIC-RMAP-REDIST-SUBNET permit 10 match tag 5952

Direct: Subnet Advertisement /24 SVI as well as more specific /32 host routes.

Page 47: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Leaf Tenant Configuration w/ VPC

leaf

vrf: CiscoLive

VX

LAN

VN

I 500

00

leaf

vrf: CiscoLive

VX

LAN

VN

I 500

00

App-x VNI 30001

vPC peer-link;PO10

# VPC Domain Configuration vpc domain 10 peer-switch peer-keepalive destination s1 source s2 peer-gateway ip arp synchronize # VPC Peer-Link interface port-channel10 switchport mode trunk vpc peer-link # VPC Domain Routing Adjacency interface Vlan3999 no shutdown ip address 10.254.254.1/30 ip router ospf 1 area 0.0.0.0 ip ospf network point-to-point ip pim sparse-mode

Routed Interface for routing adjacency across vPC peer-link

Page 48: VXLAN - Amazon Simple Storage Service · Participants: Cisco, Juniper, Alcatel-Lucent, Ixia ... for VXLAN Layer-2 bridging • Nexus 9300 functioned as both EVPN iBGP route reflector

© 2015 Cisco and/or its affiliates. All rights reserved. Cisco Public TECDCT-2821

Leaf Tenant Configuration w/ VPC

leaf

vrf: CiscoLive

VX

LAN

VN

I 500

00

leaf

vrf: CiscoLive

VX

LAN

VN

I 500

00

# VLAN to VNI mapping vlan 101 vn-segment 30001 # VTEP Loopback0 interface loopback0 ip address 10.10.10.10/32 ip address 10.10.10.100/32 secondary # VTEP configuration using Loopback0 as source. interface nve1 source-interface loopback0 host-reachability protocol bgp member vni 30001 mcast-group 239.239.239.100 suppress-arp member vni 50000 associate-vrf

App-x VNI 30001

Secondary is for vPC Hosts and Orphan-ports. Same on both peers