BGP Dynamic Routing and Neutron

Ryan Tidwell - HPJaume Devesa - Midokura

Vikram Choudhary - Huawei

Overview• Routing Cloud Network Traffic• Neutron BGP Dynamic Routing Service Overview• Applications of BGP Dynamic Routing with Neutron• Future Work• Q&A

Solutions for Routing Cloud Traffic• Neutron networks are typically stub networks with

default route and host routes statically defined for outbound flows by Neutron

• Outbound gateway IP determined by reading ‘gateway_ip’ from the external network subnet entity

• Next hop for inbound flows must be communicated to infrastructure routers, but how?

• Options• Static routing• Dynamic routing

• Static routing requires manual configuration of next-hops for each tenant network prefix or floating IP host route / prefix in upstream infrastructure routers

• Floating IP’s either must be confined to single L2 network, or host routes must be configured manually

• Operator intervention required each time a router is created or deleted

• Prefixes don’t move between Neutron routers easily

Static Routing of Cloud Traffic

Dynamic Routing• Operator configures routing

protocol amongst infrastructure routers

• Make Neutron insert routes into the routing protocol on subnet, router, and floating IP CRUD

• Isolate Neutron L3 agent from these changes, so use Neutron as a BGP route server and peer it with infrastructure routers

• Neutron will advertise routes to peers, but does not learn from peers

Why BGP?

• Separation between data plane and control plane• Work with different AS• Minimal topology to manage

Applications of Neutron BGP Dynamic Routing• Routed Model for Floating IP’s

• Unbind the floating range from the L2 network (see routed network segments)• As floating IP’s become unbound from the L2 network, we can advertise a host route for a

floating IP as it moves across different L2 network segments.• Directly Routable IPv4/IPv6 Tenant Networks

• Use BGP to advertise tenant prefixes for direct routing without floating IP’s or statically routing tenant prefixes

• DVR• Enable north-south DVR by advertising host routes with the compute node as the next-hop• Presents some scaling challenges (large number of host routes), is route aggregation possible

and would it help?

Future Applications of Neutron BGP Dynamic Routing• Routed Network Segments

• https://review.openstack.org/#/c/225384/• This spec is to support operators who want to be able to attach intances to the network using

an L3 domain as the identifier instead of an L2 network i.e. the traditional Neutron “network”• Offers a way of using Neutron to model L3 networks decoupled from the L2 segments they

span• Assigning a floating IP wouldn’t require a Neutron router. Use BGP to advertise the floating IP

• L3/BGP VPN (Potential Future Application)• Advertise route distinguisher for an address scope to PE routers• Not within scope for Mitaka, but is a potential enhancement

• Advertise Floating Range through a Neutron defined Gateway Router

Routed Model for Floating Range spanned in multiple L2 domains

Directly Routable Tenant Networks with Address Scopes

IPv6 networks don’ t need to be nattedSome small providers don’t want to use Floating IPsAddress Scopes will allow to define L3 routed domains instead of forcing NAT on tenant routers (public access to tenant networks)Subnet Pools allow to create non-overlapping SubnetsAddress Scopes will group non-overlapping Subnet Pools.BGP will automatically advertise new created subnetsExternal Gateway is the BGP peer

Routable Tenant Networks with Address Scopes

Advertise the Floating Range

With previous examples, cloud admin has to configure the Gateway Router to advertise Floating Range to ISP or other Enterprise RoutersIf Gateway Router belonged to Neutron model, we could associate the External Network to BGP speaker and advertise it.Some Neutron SDN controllers (like MidoNet) can implement the Dynamic Routing extension and advertise the Floating Range(s)

Advertise Floating Range

DVR With BGP• Each instance IP is advertised

as a host route with the compute node IP on the external network as the next-hop

• External network does not need to consume a routable prefix. It can be treated as if it were a link-local prefix.

• Large numbers of host routes won’t necessarily scale nicely. Is there a creative way to host aggregate routes?

Architecture Overview

Sample Deployment(simple)

• Entire cloud is treated as a single autonomous system

• Operator network runs in a separate autonomous system

• eBGP peering with operator network

Sample Deployment(advanced)

• External network for each rack

• Each rack is treated as an autonomous system

• Infrastructure routers redistribute Neutron routes learned by BGP into an IGP

• A single address scope can be shared across racks

Potential MPLS/BGP VPN Application

Why MPLS/BGP VPN- Omni presence of MPLS technology.

- Almost all the backbone routers understands MPLS.

- QoS guarantee

- Easier to manage

- Scales reasonably

L3 VPN Support• While not the target use case, L3VPN is important to consider.

• At the moment this effort is focused primarily on building the mechanism by which Neutron can “speak” BGP and advertise routers to neighbors

• In future cycles we will be looking to add L3VPN support and see how similar work in this arena can be discussed, combined and move forward.

Future Work• L3 VPN

• BGP-MPLS for tenant-only address scopes

• OSPF and IS-IS are very different protocols from BGP, and we don’t think we can leverage any work done on BGP.

• Route policing support

Resources• https://review.openstack.org/#/c/196452/•

https://review.openstack.org/#/q/status:open+project:openstack/neutron+branch:master+topic:bp/bgp-dynamic-routing,n,z

• http://ryu-zhdoc.readthedocs.org/en/latest/library_bgp_speaker.html• http://docs.openstack.org/developer/networking-bgpvpn/• https://review.openstack.org/#/c/225384

How To Get Involved• We’re here at the summit, feel free to corner us!• We’d love to hear from operators, your feedback on direction and utility is greatly valued• Reviews

• https://review.openstack.org/#/q/status:open+project:openstack/neutron+branch:master+topic:bp/bgp-dynamic-routing,n,z

• Code reviews are greatly valued• Let us know if you’re interested in contributing code!

Q&A