Achieving High Performance with NFV Data Plane Workloads ...

#vmworld

T5G1105BU

Achieving High Performance with NFV Data Plane Workloads: The Secrets

Jambi Ganbar, VMware, Inc.Dharma Rajan, VMware, Inc.

#T5G1105BU

VMworld 2019 Content: Not for publication or distribution

©2019 VMware, Inc.

Disclaimer

This presentation may contain product features or functionality that are currently under development.

This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.

Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.

Technical feasibility and market demand will affect final delivery.

Pricing and packaging for any new features/functionality/technology discussed or presented, have not been determined.

2

The information in this presentation is for informational purposes only and may not be incorporated into any contract. There is no commitment or obligation to deliver any items presented herein. VMworld 2019 Content: Not for publication or distribution

©2019 VMware, Inc.

Agenda

3

Data Plane Performance and NSX Virtual Distributed Switch (N-VDS)

10G, 40G Performance Testing using vLAN and Overlay

VMware & Nokia Partnership


4©2019 VMware, Inc.

Data Plane Performance and NSX Virtual Distributed Switch (N-VDS)


©2019 VMware, Inc. 5

TELCO CLOUDS: CRITICAL FOR 5G SUCCESS

VMware: The Essential, Ubiquitous Digital Foundation across all 4 Clouds

VMware’s Vision - Any Cloud Strategy

ANY DEVICE

ANY APPLICATION Cloud Native SaaSTraditional

ANY CLOUD HYBRID PUBLIC

TELCO

EDGE

75

TELCOVMworld 2019 Content: Not for publication or distribution


Use Cases Needing High Infrastructure Performance

NFV

vEPC

vIMS

vCPE

SD-WAN, uCPE

vRAN

MEC

Augmented Reality

Virtual Reality

Gaming

Video caching

Camera as a service

5G

eMBB, URLLC, mMTC

Connected cars

Real-time Telematics

AI / ML based systems

IoT

Factories of the Future

Narrow Band IoT

NFV - Network Function Virtualization

EPC – Evolved Packet Core

IMS – IP Multimedia Subsystem

CPE – Customer Premises Equipment

SD-WAN – Software Defined Wide Area Network

RAN – Radio Access Network

eMBB - enhanced Mobile Broadband

URLLC - Ultra Reliable Low Latency Communications

mMTC - massive Machine Type Communications



Data Plane Performance Requirement

NFV

64 - 1500 bytes packets

Packet loss < 0.001%

Latency < 50+msec

No Loss of virtualization function

MEC

High data volume

Real time decision

Low backhaul traffic

Near zero to 5 mslatency

Business continuity

5G

Data Rate 1 to 20Gb/s

Latency Data Plane 1-4 ms

Control Plane 20 ms

Density 1M device/sq km

Reliability – Six nines

IoT

%age of Devices up

Active users

Massive traffic increase

Significant IPV6 deployments



NFV Platform and N-VDS Secret

Data Plane VM

VMXNET3

Data Plane VM

VMXNET3

ESXi Host

N-VDS Enhanced Data Path mode

Dedicated CPU Cores for Networking

Dedicated Poll Mode Drivers

Service Provider Network

NSX-T Bare Metal Edge



virtual Evolved Packet Core Network

Data Plane Functions Connecting N-VDS

Management Plane Function

Data Plane Function

vSphere Distributed Switch

N-VDS SwitchEnhanced

Control Plane Function

N-VDS SwitchStandard

EMSMME

SGW-C

SGW-D

PGW-C

PGW-D

S5 (GTP-C)

S5 (GTP-U)

S11 (G

TP

-C)

S1 (GTP-U) UE

eNB

eNB



Effect of Packet Gap and Overlay Encapsulation HeaderHeader contributing to Performance

VLAN (802.1Q) RFC 3069 -- Max 4096 VLANS – Good for Non-Cloud DC

40GbE NIC

Original L2 Frame

VXLAN Header (8 bytes)

Outer UDP Header (8 bytes)

Outer Ethernet Header (14 bytes)

VXLAN Encapsulated Frame

Outer FCSOuter IP Header

(20 bytes)

Overlay Networks: VXLAN RFC 7348 -- Max 16 Million segments - Good for Cloud DC

40GbE NIC

Original L2 Frame

Geneve Header (8 bytes+ 8 byte option)

Outer UDP Header (8 bytes)

Outer Ethernet Header (14 bytes)

Geneve Encapsulated Frame

Outer FCSOuter IP Header

(20 bytes)

*https://datatracker.ietf.org/doc/draft-ietf-nvo3-geneve/

Overlay Networks: GENEVE IETF Draft* -- Variable length Options Good for Cross Cloud Architecture


https://datatracker.ietf.org/doc/draft-ietf-nvo3-geneve/


Header contributing to Performance Gbps

Effect of Gap, Preamble and Header for 40G



Header contributing to Performance MPPS

Effect of Gap, Preamble and Header for 40G


©2019 VMware, Inc.

vCloud NFV Verified Data Plane Performance KPIs

Paper Published in IEEE July 2019Achieving High Performance with Virtualized Data Plane Workloads for 5G Networks- D.Rajan

2019 Sixth International Conference on Software Defined Systems (SDS)https://ieeexplore.ieee.org/document/8768646

Packet Size bytes 64 128 256 512 IMIX 1024

Line rate Throughput ✓ ✓ ✓ ✓ ✓ ✓

Packet loss < 0.001% ✓ ✓ ✓ ✓ ✓ ✓

10 Gbps Scalability ✓ ✓ ✓ ✓ ✓ ✓

Logical cores/N-VDS (E) to saturate 10G link

4 3 2 1 1 1

Bi-Directional forwarding

✓ ✓ ✓ ✓ ✓ ✓

40 Gbps Scalability ✓ ✓ ✓ ✓ ✓ ✓

Logical cores/N-VDS (E)to saturate 40G link

4+ 4+ 4+ 4 3 3

Multiple links supported ✓ ✓ ✓ ✓ ✓ ✓

Sub second Latency ✓ ✓ ✓ ✓ ✓ ✓

Sub second Jitter ✓ ✓ ✓ ✓ ✓ ✓VMworld 2019 Content: Not for publication or distribution

https://ieeexplore.ieee.org/document/8768646


NFV Performance Results and Configuration



A brief vCloud NFV performance tuning guide – Host Level

Secrets of High Data Plane Performance

• CPUs with a high count of cores per NUMA • Fast pNICs

Server Class

• BIOS tuned for performance - did you catch “Extreme Performance session” yesterday?

• Turbo Boost• Hyper Threads

Server BIOS

• Intel Fortville• Mellanox ConnectX-5

Network Interface Cards (examples)



NSX-T Configuration for Data PlaneHost-level configuration

Identify and install driver

Install PMD drivers and Firmware



NSX-T Configuration for Data PlaneNSX-T Manager-level configuration

Create a transport zone

Create a Logical Switch

Assign Cores to N-VDS (E)

Disable DFWVMworld 2019 Content: Not for publication or distribution


Use recent DPDK version in the VM

Use VMXNET3

Use the latest VM hardware version

Tune your buffers

Align VM to NUMA

Set Latency Sensitivity (and pre-requisites)

The Secrets to Tuning Your VMs for PerformanceSimplified Configuration with vCloud NFV 3.x



Data Plane Acceleration with Overlay TrafficTest environment logical topology

Bidirectional Traffic

Acceptable packet loss of 0.001%

40GbE Intel XL710 NICs

PROX VMPROX VM

ESXi Host

N-VDS (E)

PROX VMPROX VM

ESXi Host

N-VDS (E)

Layer 2 Fabric

Spirent TestCenter

NSX BM Edge

Uplink

Downlink



If I use overlay, what performance can I expect from my host?

Single Host Overlay Performance

4 logical cores

3.1

3 4.5

5 6.8

6

7.9

5

8.9

4 11.4

8

11.7

3

0

2

4

6

8

10

12

14

1 2 3 4 5 6 7

Packet Size (Bytes)

Th

rou

gh

pu

t (M

PP

S)

GeneveLine Rate

GeneveLine Rate

GeneveLine Rate



3.1

6 4.5

3

5.2

8 6.8

6 8.4

7

14.9

7

18.1

2

18.1

4

0

2

4

6

8

10

12

14

16

18

20

1 2 3 4 5 6 7 8

Th

rou

gh

pu

t (M

PP

S)

Packet Size (Bytes)

BM Edge Scalability Test Results With Overlay Traffic – 40GbESmall NSX BM Edge – 12 cores

Genève Line RateGenève

Line Rate

Genève Line Rate

Genève Line Rate

Genève Line Rate

Genève Line Rate



3.1

6

4.5

3

5.2

8

6.8

6

8.4

7

14.9

7

24

.27

25

.10

0

5

10

15

20

25

30

1 2 3 4 5 6 7 8

Th

rou

gh

pu

t (M

PP

S)

Packet Size (Bytes)

BM Edge Scalability Test Results With Overlay Traffic – 40GbELarge NSX BM Edge – 20 cores

Genève Line RateGenève

Line Rate

Genève Line Rate

Genève Line Rate

Genève Line Rate

Genève Line Rate

Genève Line Rate



Lets SummarizeVMware vCloud NFV – Telco platform for all workloads

Use case and Requirements

vCloud NFV Architecture

Test Results and

Configuration

Fireside Chat



VMware & Nokia PartnershipFireside Chat

Allwyn Sequiera SVP VMwareRon Heberman CTO Nokia Software


Achieving High Performance with NFV Data Plane Workloads ...

Documents

Transcript of Achieving High Performance with NFV Data Plane Workloads ...