Distributed Routing in Ironic Integrated OpenStack Cloud

Rajeev Grover

Maruti Kamat

Vivek Narasimhan

Jonathan BryceExecutive DirectorOpenstack Foundation

“Embracing Datacenter Diversity” Austin Summit 2016 Keynote Address

DVR in Ironic-integrated Clouds

Distributed Routing in Neutron Improves performance

Scales with size of compute farm

Limited failure domain (per compute node)

Bare Metal Continues to be relevant

Prominent use cases• Three tier applications with Database servers

• Compliance & Licensing

• Specialized hardware functions

• Application architectures

Challenge Distributed Routing along with BMs

Compute node

Network node

Compute node

VM1 VM2 RtrADefault

SNATRtrA’’RtrA’

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex

vm1 vm3

br-tun-cn1

vm5 vm6

br-ex

br-int-cn1

L3-Agent

Floating IP

Router

br-int-nn

br-tun-nn

L3-Agent

br-ex

SNATRouter

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex

vm2 vm4

br-tun-cn2

vm7 vm8

br-ex

br-int-cn2

L3-Agent

Floating IP

Router

Data Network

External Network

Terms and Notations

Network Service Node

East-West - Traffic across VMs in compute nodes North-South – Traffic from VMs on compute nodes to/from

external networkNamespaces

Compute node 1 Compute node 2

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex

vm1 vm3

br-tun-cn1

vm5 vm6

br-ex

br-int-cn1

L3-Agent

Floating IP

Router

br-int-nn

br-tun-nn

L3-Agent

br-ex

SNATRouter

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex

vm2 vm4

br-tun-cn2

vm7 vm8

br-ex

br-int-cn2

L3-Agent

Floating IP

Router

Data Network

External Network

DVR overview – East West

Network Service Node

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex

vm1 vm3

br-tun-cn1

vm5 vm6

br-ex

br-int-cn1

L3-Agent

Floating IP

Router

br-int-nn

br-tun-nn

L3-Agent

br-ex

SNATRouter

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex

vm2 vm4

br-tun-cn2

vm7 vm8

br-ex

br-int-cn2

L3-Agent

Floating IP

Router

Data Network

External Network

DVR overview – North South Floating IP Traffic

Network Service Node

DVR overview – North South SNAT

br-int-nn

br-tun-nn

L3-Agent

Data Network

br-ex

External Network

vm1 vm3

br-int-cn1

br-tun-cn1

IR

dvr-agent

vm5 vm6

br-ex

FIP

SNAT

vm1 vm3

br-int-cn1

br-tun-cn1

IR

dvr-agent

vm5 vm6

br-ex

FIP

vm1 vm3

br-int-cn1

br-tun-cn1

IR

dvr-agent

vm5 vm6

br-ex

FIP

vm1 vm3

br-tun-cn1

vm5 vm6

br-ex

br-int-cn1

L3-Agent

Floating IP

Router

Router

Ability to provision baremetal servers as opposed to virtual machines.

Provides lifecycle in Openstack for baremetal provisioning (Makes baremetal bring-up as easy as bringing-up virtual machines in cloud)

Integrated into Openstack as official project from Kilo Release.

Provides tenant network isolation with VLAN-based networks (from MitakaRelease)

Overview – Openstack Ironic

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex

vm1 vm3

br-tun-cn1

vm5 vm6

br-ex

br-int-cn1

L3-Agent

Floating IP

Router

Data Network

External Network

BM1

BM2

VLAN frames

Switching - VLAN-network isolation(Ironic with Neutron)vm1 on VLAN

segment in Network N

BM1 on VLAN segment in

same Network N

Bridges two segments (can be different underlays) , providing semantics of a single L2 broadcast domain

Segments that are bridged, can be Neutron orchestrated (or) they can be segments outside the cloud

Multi-segment network in Neutron – one way to compose bridged segments both being Neutron orchestrated

Typical deployments use Neutron L2 Gateway to bridge Neutron orchestrated VXLAN (or GRE) segments, with existing VLAN segments in an enterprise

L2Gateway as a service, is available from ‘Kilo’ Release of Openstack

Overview – Openstack Neutron L2 Gateway

Uses a single Multi-Segment network of Neutron that comprises- one VLAN Segment (used by Ironic for network pinning baremetals)- one VXLAN segment (used for Virtual Machine spin-offs)

Virtual Instances transmits/receives packets on VXLAN-Segment

Baremetal Instances transmits/receives packets on VLAN-Segment

L2 Gateway provides bridging between the VXLAN and VLAN-segments on a Multi-Segment network transparently

- i.e., L2Gateway retains Neutron Network data path semantics.

Switching - VXLAN-network isolation(Neutron with Ironic)

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex

vm1 vm3

br-tun-cn1

vm5 vm6

br-ex

br-int-cn1

L3-Agent

Floating IP

Router

Data Network

External Network

L2 Gateway

BM1

BM2

Switching - VXLAN-network isolation(Ironic with Neutron)

vm1 onVXLAN

segment in Multi-

Segment Network N

BM1 on VLAN segment in same Multi-

Segment Network N

VXLAN packets

VLAN frames

Distributed Routing East West

Extend DVR concept to Ironic-managed baremetal servers

Enable DVR on VLAN-based tenant networks for Ironic-managed baremetalservers

Enable DVR on VXLAN-based tenant networks for Ironic-managed baremetalservers (with L2Gateway )

Provide a highly-available Distributed Virtual Routing solution for ironic-integrated cloud deployments

Distributed Virtual Routing (DVR) on Ironic-integrated clouds( Initial Goals )

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex

vm1 vm3

br-tun-cn1

vm5 vm6

br-ex

br-int-cn1

L3-Agent

Floating IP

Router

Data Network

External Network

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex br-tun-nnbr-ex

br-int-nn

L3-Agent

Floating IP

DVR-L

L2 Gateway

BM1

BM2

Network Service Node

Distributed Routing East-West VM3 initiating traffic to BM1vm3 on

VXLAN Network N1

BM1 onVLAN

Network N2

VXLAN packets VLAN frames

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex

vm1 vm3

br-tun-cn1

vm5 vm6

br-ex

br-int-cn1

L3-Agent

Floating IP

Router

Data Network

External Network

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex br-tun-nnbr-ex

br-int-nn

L3-Agent

Floating IP

DVR-L

L2 Gateway

BM1

BM2

Network Service Node

Distributed Routing East-WestBM2 initiating traffic to VM1vm1 on

VXLAN segment in Network N1

BM2 onVLAN

segment in Network N2

VXLAN packets VLAN frames

Distributed Routing North South

Distributed Routing North South(SNAT)

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex

vm1 vm3

br-tun-cn1

vm5 vm6

br-ex

br-int-cn1

L3-Agent

Floating IP

Router

br-int-nn

br-tun-nn

L3-Agent

br-ex

SNATDVR-L

Data Network

External Network

L2 Gateway

BM1

BM2

Network Service Node

Distributed Routing North South (Floating IP)

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex

vm1 vm3

br-tun-cn1

vm5 vm6

br-ex

br-int-cn1

L3-Agent

Floating IP

Router

Data Network

External Network

br-int-cn1

br-tun-cn1

agentbr-int-cn1

br-tun-cn1

agent

ex

br-int-cn1

br-tun-cn1

agent

ex br-tun-nnbr-ex

br-int-nn

L3-Agent

Floating IP

DVR-L

L2 Gateway

BM1

BM2

Network Service Node

Present same Use model as VMs FIPs, SNAT Services

No Touch Model No agents, modules or helpers required in the BM

Architectural compatibility Work within the current framework of OpenStack

High availability Leverage SNAT HA for DVR-L HA

Scalability Intelligent scheduling of DVR-L routers on different nodes Optimize use of tenant available IP Address and MAC Address scopes

Alternate Explorations.. DVR done in hardware with L2Gateway Device that is OpenFLOW Capable.

(Avoids need for service entity in virtual cloud) Networking agent in BMs

Design Considerations

Thank you

21