Bird's Eye View onNeutron & SDN

Xin Wu @ Big Switch Networks

Logical View: Same as Wireless Router at Home

externalnetworktenant

Arouter

host1

host2

host3

192.168.1.1/24 192.168.2.1/24

tenantB

router

host4

host5

host6

192.168.1.1/24 192.168.2.1/24

1.1.1.2 1.1.1.3

Physical View

spine

leaf1 leaf2

server1

vswitch1

host1 host3 host4 host6

host2 host5

Tenant Router Is Missing

Key Question: Where to Implement Tenant Router?Option 1: one software router on a server

Option 2: distributed router on switches

Option 1: One Software Router on a Server

spine

leaf1 leaf2

server1

vswitch1

host1 host3 host4 host6

host2 host5

leaf3

server2

vswitch2

tenant A router tenant B router

openstack controller

SDN controller

Extremely Simplified Control Plane Flow

openstack controller

SDN controller vswitch agents

1. create VM & attach VM to vswitch

2. cr

eate

/upd

ate

serv

er-id

,

p

ort,

vlan,

mac

, IP,

net

work3. program flow entries

Option 1: One Software Router on a ServerPros: server only

Option 1: One Software Router on a ServerCons 1: cannot support non-vm workloads

Solution: offload tunnel to physical switch

spine

leaf1 leaf2

server1

vswitch1

host1 host3 host4 host6

host2 host5

leaf3

server2

vswitch2

tenant A router tenant B router

openstack controller

SDN controller

Option 1: One Software Router on a ServerCons 2: suboptimal routing

Solution: distributed virtual routing (DVR)

spine

leaf1 leaf2

server1

vswitch1

host1 host3 host4 host6

host2 host5

openstack controller

SDN controller

Key Question: Where to Implement Tenant Router?Option 1: one software router on a server

Option 1: One Software Router on a Server

Pros: server only (no longer the case for non-vm workloads)

Cons 1: cannot support non-vm workloads offload tunnel to physical switch

Cons 2: suboptimal routing distributed virtual routing

Option 2: Distributed Router on Switches

spine

leaf1 leaf2

server1

vswitch1

host1 host3 host4 host6

host2 host5

openstack controller

SDN controller

Pros 1: Support both vm and non-vm workloads

Pros 2: Always optimal forwarding/routing

Applications that Drives Neutron and SDN Evolvement

1. NFV DPDK, SR-IOV

2. Docker 4-tier networking

NFV Intel x86 Data Plane Development Kit (DPDK)NFV: networking function running in VM

OVS/linux bridge: expensive interrupt and data copy between kernel and NIC

NUMA: non-uniform memory access

kernel

vswitch

firewall VM 1

NIC

firewall VM 2

kernel

user space vswitch

NIC

firewall VM 1 firewall VM 2

NUMA1

CPU1

NIC

core

core

memory

NUMA2

CPU2

NIC

core

core

memory

firewall VM 1 firewall VM 2

DPDKs Impact on Control Plane

openstack controller

SDN controller vswitch agents

1. create NFV VM & attach it to NUMA1

2. cr

eate

/upd

ate

serv

er-id

,

p

ort,

vlan,

mac

, IP,

net

work3. program flow entries

NFV Single Root I/O Virtualization (SR-IOV)SR-IOV NIC

DMA between NIC and VM

No CPU is involved

kernel

vswitch

firewall VM 1

NIC

firewall VM 2firewall VM 1

NIC (physical)

firewall VM 2

(virtual)

SR-IOVs Impact on Control Plane

physicalswitch

SDN controller

2. cr eate/upda te server-id , port, vl an, mac, IP, network 3. program flow entries openstack controller agents1. cr eate NFV VM &assign vir tual NIC to it vswitch

Applications that Drives Neutron and SDN Evolvement

1. NFV DPDK, SR-IOV

2. Docker 4-tier networking

Docker on Physical Server: Solved Problem

spine

leaf1 leaf2

server1

vswitch1

docker1 docker3 docker4 docker6

host2 host5

Docker on VM

spine

leaf1 leaf2

server1

vswitch1

vm1

vm3

host2 host5

vswitch2

docker1 docker2

Solution 1: run Kubernetes on top of VMs separate IP space

Solution 2: SDN controller manages vswitches in VMs

openstack controller

SDN controller

Neutron & SDN

Where to implement tenant router?

option 1: one software router on a server

option 2: distributed router on switches

Killer application

NFV DPDK, SR-IOV

Docker 4-tier networking