VMworld 2015: VMware NSX Deep Dive

VMware NSX - Deep DiveJacob Rapp, VMware, Inc

NET5560

#NET5560

• This presentation may contain product features that are currently under development.

• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

• Technical feasibility and market demand will affect final delivery.

• Pricing and packaging for any new technologies or features discussed or presented have not been determined.

Disclaimer

CONFIDENTIAL 2

What You’ve Done with NSX

CONFIDENTIAL 3

NSX Customers700+

Production Deployments(adding 25-50 per quarter)

100+

Organizations invested US$1M+ in NSX

65+

What You’re Doing Next

EXPANDED SECURITY

New security partners, integrations, and projects and applications of NSX.

DEEPER INTEGRATION

New infrastructure and operations partners, integrations, and frameworks for IT organizations

√APPLICATION CONTINUITY

New functionality to scale deployments across vCenter instances, with the ability to:

• Pool resources from multiple data centers• Recover from disasters faster• Deploy a hybrid cloud architecture

• NSX 6.2 contains over 20 new features• Tested against over 1000 new scenarios

Session Objectives• Provide you with an in-depth understanding of the NSX architecture and components

• Understand how networking functions and services are implemented within the NSX platform

• Analyze key workflows for configuring virtual network & security services

• Provide pointers to reference design sessions and guides

CONFIDENTIAL 4

CONFIDENTIAL 5

ProvidesA Faithful Reproduction of Network & Security Services in Software

Management APIs, UI

Switching Routing

Firewalling

LoadBalancing

VPN

Connectivity to Physical Networks

Policies, Groups, Tags

Data Security Activity Monitoring

Physical Workloads

Security PoliciesSecurity Groups

Logical Switching, Routing, Firewall, Load Balancing

Web

App

Database

Web“Standard Web”

Firewall – allow inbound HTTP/S, allow outbound ANY IPS – prevent DOS attacks, enforce acceptable use

Database“Standard Database”

Firewall – allow inbound SQL Vulnerability Management –

Weekly Scan

App“Standard App”

Firewall – allow inbound TCP 8443, allow outbound SQL

VM VM

VM VM VM

VM

“Default” Firewall – Access shared services (DNS, AD) Anti-Virus – Scan Daily

Default

Creating Sophisticated Application Topologies

CONFIDENTIAL 6

Agenda

1 NSX Architecture and Components

2 Switching

3 Routing

4 Distributed Firewall & Micro-Segmentation

5 Services

6 Summary & Next Steps

CONFIDENTIAL 7

NSX Architecture and ComponentsCloud Consumption • Self Service Portal

• vCloud Automation Center, OpenStack, Custom

Data Plane

NSX Edge

ESXi Hypervisor Kernel Modules

Distributed Services

• High – Performance Data Plane• Scale-out Distributed Forwarding Model

Management PlaneNSX Manager

• Single configuration portal• REST API entry-point

Control Plane

NSX Controller• Manages Logical networks• Control-Plane Protocol• Separation of Control and Data Plane

FirewallDistributed Logical Router

LogicalSwitch

Logi

cal N

etw

ork

Phys

ical

Net

wor

k

…

…

HW VTEP

CONFIDENTIAL 8

NSX Data Plane ComponentsData Plane

Edge Clusters and HW VTEP (Physical-to-Virtual)

DFWVXLAN DLRSecurity

NSX Edge Service Gateways• VM form factor• Highly Available• Dynamic Routing:

• OSPF, IS-IS, BGP• L3-L7 Services:

• NAT, DHCP, Load Balancer, VPN, Firewall

• vSphere Distributed Switch • VMkernel Modules

• Logical Switching (VXLAN)• Distributed Logical Router• Distributed Firewall

ESXi

Hypervisor Kernel Modules (VIBs)

DistributedFirewall

Distributed Logical Router

LogicalSwitch

vSphere Components

DFWVXLAN DLRSecurity DFWVXLAN DLRSecurity

… …Compute Clusters

HW VTEP• ToR Switch

• Bandwidth and physical ports scale-out

• VLANs for Physical workloads local to a rack

CONFIDENTIAL 9

NSX Control Plane Components

• Properties– Virtual Form Factor (4 vCPU, 4GB RAM)– Data plane programming– Control plane Isolation

• Benefits– Scale Out – High Availability– VXLAN - no Multicast– ARP Suppression

NSX Controllers

vSphere Cluster vSphere HA DRS with Anti-affinity

VM ESXi VM VM

Host Agent

Data-Path Kernel Modules

10CONFIDENTIAL 10

Management Plane Components

• Runs as a Virtual Machine

• Provisioning and Management ofNetwork and Network services• VXLAN Preparation• Logical Network Consumption• Network Services Configuration

NSX Manager

NSX Manager1:1Management Plane

vRA/Openstack/Custom

vCenter

NSX REST APIsvSphere APIs

3rd Party Management Console

NSX Manager vSphere Plugin

Single Pane of Glass

Enabling Automation with NSX and vRANET5362

CONFIDENTIAL 11

OpenStack with NSX Deep DiveNET5836

12

NSX Component Interaction - Deployment and Configuration Deploy NSX Manager

12

53

Register with vCenter

Deploy NSX Controllers

4

Prepare HostsConfigure and deploy NSX

Edge Gateway(s) and network services

NSX Manager

vSphere Cluster 1 vSphere Cluster 2 vSphere Cluster N

NSX Controller

vCenter

NSX Edge Services GW

CONFIDENTIAL

CONFIDENTIAL 13

Management Plane ComponentsMulti-vCenter

Local VC Inventory Local VC Inventory Local VC Inventory

vCenter & NSX Manager A

Universal Object Configuration(NSX UI & API) Universal Configuration Synchronization

Universal Controller

Cluster

Primary Secondary

vCenter & NSX Manager B vCenter & NSX Manager H

Secondary

Universal LogicalSwitches

Universal Distributed Logical Router

UniversalDFW

Multi-VC Solutions with NSXNET5989

Deploying and Configuring VMware NSXDeploy VMware NSX

NSXEdge

NSXMgmt

Virtual Infrastructure

Deploy NSX Manager

Deploy NSX Controller Cluster

Component Deployment

Host Preparation

Logical Network Preparation

Preparation O

ne T

ime

ProgrammaticVirtual

Network Deployment

Logical Networks

+ + +Consumption

Rec

urrin

g

Deploy Logical Switches per tier

Create Bridged Network

Logical Network/Security Services

Deploy Distributed Logical Router or connect to existing

CONFIDENTIAL 14

Agenda


2 Switching

3 Routing


5 Services


CONFIDENTIAL 15

NSX Logical Switching

• Per Application/Multi-tenant segmentation • VM Mobility requires L2 everywhere• Large L2 Physical Network Sprawl – STP

Issues • HW Memory (MAC, FIB) Table Limits

• Scalable Multi-tenancy across data center• Enabling L2 over L3 Infrastructure • Overlay Based with VXLAN, etc.• Logical Switches span across Physical Hosts

and Network Switches

Challenges Benefits

VMw

are

NSX

Logical Switch 1 Logical Switch 2 Logical Switch 3

CONFIDENTIAL 16

Logical View: VMs in a Single Logical Switch

Web LS172.16.10.0/24

172.16.10.11 172.16.10.12 172.16.10.13

VM1 VM3VM2

172.16.20.12

VM5

172.16.20.11

VM4App LS172.16.20.0/24

CONFIDENTIAL 17

Physical View: VMs in a Single Logical Switch

VM1

vSphere Distributed Switch

VM2

Logical Switch 5001

VM3

Transport Subnet A 192.168.150.0/24

Physical Network

192.168.150.51 192.168.150.52 192.168.250.51

172.16.10.11 172.16.10.12 172.16.10.13

CONFIDENTIAL 18

19

IP Fabric

Host A Host B


Traffic Flow on a VXLAN Backed VDS

• In this setup, VM1 and VM2 are on different hosts but belong to the same logical switch

• When these VMs communicate, a VXLAN overlay is established between the two hosts

dvUplink-PG

Logical SW A

VM1

dvUplink-PG

dvPG-VTEP

VTEP

dvPG-VTEP

VTEP

VXLAN Overlay

Logical SW A

VM2

CONFIDENTIAL

Host BHost A


Traffic Flow on a VXLAN Backed VDS

• Assume VM1 sends some traffic to VM2:

dvUplink-PG

Logical SW A

VM1

dvUplink-PG

dvPG-VTEP

VTEP

dvPG-VTEP

VTEP

Logical SW A

VM2L2 frame L2 frame

IP FabricVXLAN Overlay

IP/UDP/VXLANL2 frame

VM1 sends L2 frame to local VTEP

1 VTEP adds VXLAN, UDP & IP headers2 Physical Transport

Network forwards as a regular IP packet

3 Destination Hypervisor VTEP de-encapsulates frame

4 L2 frame delivered to VM2

5

CONFIDENTIAL 20

NSX for vSphere VXLAN Replication Modes• NSX for vSphere provides three modes

of traffic replication (two which are Controller based, and onewhich is Data Plane based

• Unicast Mode– All replication occurs using unicast

• Hybrid Mode– Local replication offloaded to physical

network, while remote replication occurs via unicast

• Multicast Mode– Requires IGMP for a Layer 2 topology and

Multicast Routing for L3 topology

• All modes require an MTU of 1600 bytes

CONFIDENTIAL 21

Agenda


2 Switching

3 Routing


5 Services


CONFIDENTIAL 22

23

NSX Logical Routing Introduction

DLR Kernel Module

NSX Edge

ESXi

Hypervisor Kernel Modules (VIBs)

Distributed Logical Router

Distributed Logical RoutingOptimized for E-W Traffic Patterns

Centralized RoutingOptimized for N-S Routing

vSphere Host

LIF1 LIF2

Logical Routing Deep DiveNET5826

CONFIDENTIAL

NSX Routing: Distributed, Feature-Rich

• Physical Infrastructure Scale Challenges – Routing Scale

• VM Mobility is a challenge• Multi-Tenant Routing Complexity• Traffic hair-pins

Challenges

• Distributed Routing in Hypervisor• Dynamic, API based Configuration• Full featured – OSPF, BGP, IS-IS• Logical Router per Tenant• Routing Peering with Physical Switch

Benefits

SCALABLE ROUTING – Simplifying Multi-tenancy

L2

L2

Tenant A

Tenant B

L2

L2

L2 Tenant C

L2

L2

L2

CMP

CONFIDENTIAL 24

Logical View: VMs in a Single Logical Switch

VM1 VM3VM2

VM5VM4

Web LS172.16.10.0/24

172.16.10.11 172.16.10.12 172.16.10.13

172.16.20.12172.16.20.11App LS172.16.20.0/24

CONFIDENTIAL 25

Logical View: VMs with Distributed Routing

172.16.10.1

192.168.10.0/29192.168.10.1

Distributed Logical Router Service

VM1 VM3VM2

VM5VM4

Web LS172.16.10.0/24

172.16.10.11 172.16.10.12 172.16.10.13

172.16.20.12172.16.20.11App LS172.16.20.0/24

172.16.20.1

CONFIDENTIAL 26

Physical View: VMs in a Single Logical Switch

VM1


VM2

Logical Switch 5001

VM3

Physical Network

Transport Subnet A 192.168.150.0/24

192.168.150.51 192.168.150.52 192.168.250.51

172.16.10.11 172.16.10.12 172.16.10.13

CONFIDENTIAL 27

Physical View: Logical Routing

VM5VM1


VM2

Logical Switch 5001

VM3

Physical Network

VM4

Logical Switch 5002 Controller

Management Cluster

L3 Control Plane Programming

Data Plane

Transport Subnet A 192.168.150.0/24 Transport Subnet B 192.168.250.0/24

192.168.150.51 192.168.150.52 192.168.250.51

CONFIDENTIAL 28

29

NSX Logical Routing : Components Interaction

NSX Edge (Acting as next hop router)

172.16.10.0/24 172.16.20.0/24

DLR

192.168.10.1

192.168.10.2

External Network

192.168.10.3

DLR Control VM

DataPath

Control

Controller Cluster

Control

NSX Mgr

Dynamic routing protocol is configured on the logical router instance1

OSPF/BGP peering between the NSX Edge and logical router control VM3

Learnt routes from the NSX Edge are pushed to the Controller for distribution4

Controller sends the route updates to all ESXi hosts5

Routing kernel modules on the hosts handle the data path traffic6

1

34

5

6

Controller pushes new logical router Configuration including LIFs to ESXi hosts2

2

Peering

OSPF, BGP

Peering

OSPF, BGP

172.16.30.0/24

CONFIDENTIAL

Distributed East-West Routing Traffic FlowDifferent Hosts

30

vSphere Host

VM1

VDS

VXLAN Transport Network

VXLAN 5001

VM2

VXLAN 5002 1

4

vSphere HostLIF2 - ARP Table

DA: vMACSA: MAC1

DA: 20.20.20.20SA: 10.10.10.10

5002

MAC1 MAC25

172.16.10.10

2VM IP VM MAC

172.16.20.10 MAC2

PayloadL2 IP

DA: 172.16.20.10SA: 172.16.10.10

PayloadL2 IP

L2 IP UDP VXLAN PayloadL2 IP

172.16.20.10

LIF1LIF2 vMAC

LIF1LIF2 vMAC

Host 1 Host 2

3

10.10.10.10/24 20.20.20.20/24

3

DA: MAC2SA: vMAC

Example: Enterprise Routing Topology

VLAN 20

Core

Physical Routers

Web1 App1 DB1 Webn Appn DBnWeb2 App2 DB2

VXLAN 5020Uplink

Distributed Routing

Routing

Peering

Routing

Peering

E3 E8E1

Physical Routers

E2 …

Core

Routing Adjacencies

VXLAN

VLAN

Routing Adjacencies

CONFIDENTIAL 31

What Have We Seen Thus Far ..1. NSX architecture

2. An on-demand application deployment

3. Logical switching configuration

4. Understand logical networks

5. Logical routing and possible designs

CONFIDENTIAL 32

Agenda


2 Switching

3 Routing


5 Services


CONFIDENTIAL 33

NSX Distributed Firewalling

• Centralized Firewall Model• Static Configuration • IP Address based Rules• 40 Gbps per Appliance• Lack of visibility with encapsulated traffic

• Distributed at Hypervisor Level• Dynamic, API based Configuration• VM Name, VC Objects, Identity-based Rules • Line Rate ~20 Gbps per host• Full Visibility to encapsulated traffic

Challenges Benefits

PHYSICAL SECURITY MODEL DISTRIBUTED FIREWALLING

Firewall Mgmt

VMware NSX

API

CMP

NSX DFW Deep DiveSEC5589

CONFIDENTIAL 34

Distributed Firewall FeaturesVM5

VM1


Web-LS1

VM4App-LS1

Management Cluster192.168.150.51 192.168.150.52 192.168.250.51

VM2

• Firewall rules are enforced at VNIC Level• Policy independent of location (L2 or L3 adjacency)• State persistent across vMotion• Enforcement based on VM attributes like Tags, VM Names, Logical Switch, etc

Capabilities

CONFIDENTIAL 35

Distributed Firewall Rules VM5

VM1


Web-LS1

VM4App-LS1


VM2

Rules Based on VM NamesCONFIDENTIAL 36

Distributed Firewall Rules VM5

VM1


Web-LS1

VM4App-LS1


VM2

Rules Based on Logical Switches

CONFIDENTIAL 37

Example Building a Web DMZ

Web-Tier

App-Tier

External Network

Source Destination Service Policy

Any Web-Tier LS HTTPS Allow

Web-VM1 Web-VM2 Block

Any Web-Tier LS Block

Web-Tier LS App-Tier LS TCP 8443 Allow

Any App-Tier LS Block

STOP

Client to Web HTTPS Traffic

Web to App TCP/8443

CONFIDENTIAL 38

39External Network

VDS

Guest VM

Partner Services VM

vCenter Partner Console

DFW

Filtering Module

Slot 2

Slot 4Traffic RedirectionModule

NSX Distributed Firewall Packet WalkDFW, Filtering Module and Traffic Redirection Module

CONFIDENTIAL

Agenda


2 Switching

3 Routing


5 Services


CONFIDENTIAL 40

Features SummaryNSX Edge

Gateway Services

Rule configuration with IP, Port ranges, Grouping Objects, VC ContainersFirewall

Configuration of IP Pools, gateways, DNS servers and search domains.DHCP

IPSec site to site VPN between two Edges or other vendor VPN terminators.Site-to-Site VPN

Stretch your layer 2 across datacenters.L2VPN

Allow remote users to access the private networks behind Edge GSW.SSL VPN

Configure Virtual Servers and backend pools using IP addresses or VC ObjectsLoad Balancing

Source and Destination NAT capabilities.Network Address Translation

Active-Standby HA capability which works well with vSphere HA.High Availability

Static as well as Dynamic Routing protocols support (OSPF, BGP, ISIS)Routing

Allow configuring DNS relay and remote syslog servers.DNS/Syslog

NSX Edge Integrated Network Services

….

FirewallLoad BalancerVPNRouting/NAT

DHCP/DNS relayDDI

VM VM VM VM VM

Overview

• Integrated L3 – L7 services• Virtual appliance model to provide

rapid deployment and scale-out

Benefits

• Real time service instantiation• Support for dynamic service differentiation

per tenant/application• Uses x86 compute capacity

CONFIDENTIAL 42

NSX Load Balancing

• Application Mobility• Multi-tenancy• Configuration complexity – manual

deployment model

• On-demand load balancer service• Simplified deployment model for

applications – one-arm or inline • Layer 7, SSL, …

Challenges Benefits

LOAD BALANCER – Per Tenant Application Availability Model

Tenant A

VM1 VM2 VM1 VM2

Tenant B

NSX Load Balancing Deep DiveNET5612

CONFIDENTIAL 43

NSX L2VPN

Use Cases

• Brownfield NSX deployments (VLAN -> VXLAN)• Data Center Migrations (P2V, V2V)• Disaster Recovery & Testing• Cloud Bursting & Onboarding

Best Fit for L2 extensions with

• Long Distance / High Latency• Multiple management domains• NSX present only on a single site• Max 1500 byte MTU on WAN

Highlights

• SSL secured L2 extension over any IP network• Independent of vCenter Server boundaries• Can co-exist with existing default gateway • No specialized hardware required• Supports up to 750Mb/s per Edge• AES-NI supported if available

L2 VPN

Internet / WAN

Enterprise

L2 VPN

Internet / WAN

Hybrid Cloud

PublicCloud

Connecting Remote Sites with NSXNET5352

Agenda


2 Switching

3 Routing


5 Services


CONFIDENTIAL 45

46

VMware NSX – Summary and Takeaways • Faithful reproduction of L2 – L7 network & security services

• Services design for scale-out

• Central API for provisioning & monitoring

• All NSX components designed with resiliency

• Extensive 3rd party ecosystem for NSX platform

CONFIDENTIAL

NSX Ecosystem

CONFIDENTIAL 47

Service Insertion“Leverage full automation and

service insertion for NSX”

NSX aware“Leverage NSX API and

metadata to bring a solution”

Co-existence“Let’s meet in the network”

Works with any switching fabric

Works with routing ecosystem using

traditional protocols

Existing Physical firewall provide security sitting in front of NSX Edge at layer 3

Existing Physical/virtual ADC services can connect to NSX at layer 2 or layer 3

Network Virtualization Next Steps with VMware NSX

CONFIDENTIAL 48

virtualizeyournetwork.com

The online resource for the people, teams and organizations that are adopting network virtualization

communities.vmware.com

Connect and engage with network virtualization experts and fellow VMware NSX users

vmware.com/go/NVtraining

Build knowledge and expertise for the next step in your career

labs.hol.vmware.com

Test drive the capabilities of VMware NSX

VMware NSX - Deep DiveJacob Rapp, VMware, Inc

NET5560

#NET5560

VMworld 2015: VMware NSX Deep Dive

Technology

Transcript of VMworld 2015: VMware NSX Deep Dive