OpenStack Ops Meetup

of 24 /24
Romana Project Network and Security Automation romana.io June 2016 OpenStack Operators Meetup June 7, 2016

Transcript of OpenStack Ops Meetup

Page 1: OpenStack Ops Meetup

Romana Project

Network and Security Automation

romana.ioJune 2016

OpenStack

Operators Meetup

June 7, 2016

Page 2: OpenStack Ops Meetup

romana.io

New Networks, New Problems, New Solutions

• Legacy Apps/Enterprise Private

Cloud

• LAN Emulation to support vMotion

• Automated data center

infrastructure provisioning

• Cloud Native Apps

• Seamless public/private cloud

deployment and orchestration

• Docker and Container networking

• Endpoint explosion and compressed

lifecycle

• Whitebox and GIFEE Networks

• Enterprise SDN

• VMware/NSX

• Cisco ACI

• Others…

• Cloud Native Networks

• Network automation for

rapid provisioning

• Security automation

• Multi-cloud

romana.io

June 2016 Slide 1

Page 3: OpenStack Ops Meetup

Cloud Native vs. Enterprise Networks• Amazon AWS Style v. Enterprise Apps

• Service orientation (Cattle) v. Endpoint orientation (Pets)

• Network requirements

• Reachable IP addresses v. Auto discovered MAC (ARP on VLANs)

• Service orientation further decouples apps from infrastructure

• No VM migration

• No IP Failover

• Good News: Cloud Native apps don’t need layer 2 networks

• Avoiding Layer 2 networks eliminates a lot of SDN complexity

• Bad News: Layer 2 networks provided a convenient way to isolate apps

• Even a small number of VLANs were difficult to automate

Bottom Line: Need a new way to isolate networks

romana.ioJune 2016 Slide 2

Page 4: OpenStack Ops Meetup

Romana Network and Security Automation• Layer 3 based isolation and tenancy model

• Topology-aware addressing

• Embed tenant and segment IDs in IP addresses

• Requires nothing more than standard L3 routing

• Hierarchical design simplifies scalable deployment

• No virtual network required

• Native performance and visibility

• Eliminates overlays

• Routes map to services 1:1

• Simplifies composition, security and control

• Tightly integrated into Cloud Management/Orchestration IPAM

romana.ioJune 2016 Slide 3

Page 5: OpenStack Ops Meetup

SDN Complexity melts away• No VLANs, VXLANs, VTEP/VNID, OpenFlow, OVS/OVN/OVSDB

• Route aggregation simplifies network

• Static routing eliminates need for route distribution (BGP, XMPP, KVS)

• Reduces the number of firewall rules (i.e. network v. endpoint)

• Simplifies Operations

• Existing tools, techniques and diagnostics all just work

• Existing security, policy and control systems all work

• Firewalls, IDS, LB, etc., etc., etc.

June 2016 romana.io Slide 4

Page 6: OpenStack Ops Meetup

North/South Traffic• Neutron Network node

routes traffic between

segments

• Network node

performs all

L3 functions

• East/West traffic

encapsulated, but is direct

to destination host

romana.io

VXLAN Decap

VXLAN Decap

VXLAN Encap

VXLAN Encap

2 Top of Rack

Round Trips

East/West

Traffic

Per Instance

Security

June 2016 Slide 5

Page 7: OpenStack Ops Meetup

North/South Traffic• Latency dramatically

reduced

• No Network node

• No encap

• Identical path for

East/West traffic

romana.io

Eliminated

Bypassed

Bypassed

Romana

Router

Romana

Router

1 Top of Rack

Round Trip

Per Network

Security

June 2016 Slide 6

Page 8: OpenStack Ops Meetup

Network Latency

• North/South Latency reduced 50%-85%

• 10% improvement for East/West traffic between hosts (no encap)

• No performance penalty for local on-host East/West traffic

romana.io

North/South

(Routed)

East/West

(Switched)

Time (ms) Local Remote Local Remote

Native OpenStack 1.51* 1.51 0.24 0.85

Romana Networks 0.24 0.77 0.24** 0.77**

Relative Performance Local Remote Local Remote

Native OpenStack 100% 100% 100% 100%

Romana Networks 16% 51% 100% 90%

* All N/S OpenStack traffic

goes off host

** All Romana traffic is

routed

June 2016 Slide 7

Page 9: OpenStack Ops Meetup

How does it work?• Assign CIDR length for host (node), tenant and segment

• Example: host 16, tenant 20, segment 24

• On every host, each tenant gets a real physical CIDR

• Tenant can further sub-net for their own private segments

• Assign IP addresses that maintain reachability

• Apply layer 3 firewall rules for network isolation

• Configure next hop gateway for service composition

June 2016 romana.io Slide 8

Page 10: OpenStack Ops Meetup

Example

June 2016 romana.io Slide 9

Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Field

Capacity 0 0 0 0 1 0 1 0

Example: Bits Length Purpose

10/8 Network 8 10/8 Network

Hosts 8 Up to 255 Hosts

Tenants 4 Up to 255 Tenants

Segments 4 Up to 16 Segments per Tenant

Endpoints 8 Up to 16 Endpoints per Segment

Host 1 ID CIDR or IP Host 2 ID CIDR or IP Host 3 ID CIDR or IP

Physical Addr 192.168.0.10 Physical Addr 192.168.0.11 Physical Addr 192.168.0.12

Host 1 10.1/16 Host 2 10.2/16 Host 3 10.3/16

Tenant 0 10.1.0/20 Tenant 0 10.2.0/20 Tenant 0 10.3.0/20

Segment 1 10.1.1/24 Segment 1 10.2.1/24 Segment 2 10.3.2/24

VM 1 22 VM 1 22 VM 1 22

VM 2 33 VM 2 33 VM 2 33

Tenant 1 10.1.16/20 Tenant 1 10.2.16/20 Tenant 1 10.3.16/20

Segment 1 10.1.17/24 Segment 2 10.2.18/24 Segment 1 10.3.17/24

VM 3 44 VM 3 44 VM 3 44

VM 4 55 VM 4 55 VM 4 55

Endpoint ID

Up to 255 Hosts Up to 255 Tenants 255 Endpoints for each Tenant

20 17-20

10/8 Net Mask Host ID Bits (8) Tenant/Segment ID Bits (8)

Location

8 1-8

16 9-16

24 21-24

32 25-32

10.1.1.22

10.1.17.55 10.2.18.55 10.3.17.55

10.3.2.22

10.1.1.33 10.2.1.33 10.3.2.33

10.1.17.44 10.2.18.44 10.3.17.44

10.2.1.22

Page 11: OpenStack Ops Meetup

Physical Deployment

June 2016 romana.io Slide 10

192.168.0.10 192.168.0.11 192.168.0.12

Host 1

VM 1: 10.1.1.22

G/W: 10.1.0.1/16

VM 2: 10.1.1.33

VM 3: 10.1.17.44

VM 4: 10.1.17.55

10.2/16 -> 192.168.0.11

10.3/16 -> 192.168.0.12

Host 2

VM 1: 10.2.1.22

G/W: 10.2.0.1/16

VM 2: 10.2.1.33

VM 3: 10.2.18.44

VM 4: 10.2.18.55

10.1/16 -> 192.168.0.10

10.3/16 -> 192.168.0.12

Host 3

VM 1: 10.3.2.22

G/W: 10.3.0.1/16

VM 2: 10.3.2.33

VM 3: 10.3.17.44

VM 4: 10.3.17.55

10.1/16 -> 192.168.0.10

10.2/16 -> 192.168.0.11

Page 12: OpenStack Ops Meetup

ECMP

BGP/OSPF Area

Leaf 1

Every host gets /16 network, announces to Leaf

Leaf aggregates 64 /16 networks, announces /10 to Spine

Spine contains only four /10 networks

0.0.0.0 via Spine 1

10.0.0.0/16 via Port 1

10.1.0.0/16 via Port 2

10.2.0.0/16 via Port 3

10.3.0.0/16 via Port 4

10.63.0.0/16 via Port 64

Spine 1

0.0.0.0 via Internet

10.0.0.0/10 via Leaf 1

10.64.0.0/10 via Leaf 2

10.128.0.0/10 via Leaf 3

10.192.0.0/10 via Leaf 4

Spine 2

Leaf 2 Leaf 3 Leaf 4

0.0.0.0 via Internet

10.0.0.0/10 via Leaf 1

10.64.0.0/10 via Leaf 2

10.128.0.0/10 via Leaf 3

10.192.0.0/10 via Leaf 4

10.2/16 RIP to Leaf for distribution0.0.0.0 via Leaf 1, Port 8

Port 8

Host 221

10.194.3.710.0.0.0 via Leaf 4, Port 3

Port 3

Host 8

10.2.16.34

0.0.0.0 via Spine 1

10.192.0.0/16 via Port 1

10.193.0.0/16 via Port 2

10.194.0.0/16 via Port 3

10.195.0.0/16 via Port 4

10.255.0.0/16 via Port 64

romana.ioJune 2016 Slide 11

Endpoints on Host 8 must get address within 10.2.0.0/16

Endpoints on Host 221 must get address within 10.194.0.0/16

Announce route to ToR

Page 13: OpenStack Ops Meetup

Leaf 1

Spine 1 Spine 2

Leaf 2 Leaf 3 Leaf 4

10.2/16 RIP to Leaf for distribution172.16.1.25 host route0.0.0.0 via Leaf 1, Port 8

Host

10.194.3.710.0.0.0 via Leaf 4, Port 3Host 8

10.2.16.34

Edge/

NAT

Host routes to external service endpoints

June 2016 romana.io Slide 12

SLB

VMSLB get FIP as VIP

FIP 172.16.1.25

Page 14: OpenStack Ops Meetup

Security

Policy

Neutron Node

OpenStack Deployment

May 2016 romana.io

IPAM

Routes

Tenant

DB

Topology

Policy

Slide 13

Neutron

ML2IPAM

Compute Node n

VM

iptables

VM

Nova

Agent

Page 15: OpenStack Ops Meetup

Network/Security PolicyNetPolicy.json

{

"Name": "policy2",

"PolicyID": "CF2D2BE2-4553-4C28-BD02-140CF83617A2", # unique identifier across tenants, auto generated for POST.

"AppliedTo": [ # can attach multiple tenants to which the policy can be applied to.

{

"Tenant":"tenant2",

"Segment": "Segment1",

“HostCIDR": “10.23.0.0/0", # Apply policy to entire host

},

],

"Tags": [], # meta data attached to policies for various external environments like openstack/kubernetes

"Direction" : "Ingress", # can be Egress or Ingress.

"Peers": [

{

"CidrBlock": "0.0.0.0/0", # IP from L3 header

},

],

"Rules": [{

"Protocol": "ICMP",

"IcmpTypeCode": [0,8],

"IsStateful": true,

},],

"Description": "hello there, security policies are fun!",

}

June 2016 romana.io Slide 14

Page 16: OpenStack Ops Meetup

Scalable Deployments• Need more IP addresses

• Large OpenStack environments

• Container endpoint explosion

• Separate Romana deployment for each OpenStack cluster

• Clusters interact via service endpoints

• Explicitly manage overlapping IPs

• Use datacenter FIPs

• Support Overlapping in Romana IPAM

• Advantage of consistent policy across environment

• IPv6

June 2016 romana.io Slide 15

Page 17: OpenStack Ops Meetup

Cluster 2Cluster 1

Romana 1: 10/8

Shared Block: 10.0.1/24

Local FIPs: 10.0.1.128/25

Remote FIPs: 10.0.1.0/25

Edge

Large Scale Deployments

June 2016 romana.io Slide 16

Romana 2: 10/8

Shared Block: 10.0.1/24

Local FIPs: 10.0.1.0/25

Remote FIPs: 10.0.1.128/25Alternatively use FIPs from

DC addresses

Shared 172.16.1/24

FIPs

Page 18: OpenStack Ops Meetup

Security

Policy

k8s Master

Kubernetes Deployment

May 2016 romana.io

IPAM

Routes

Tenant

DB

Topology

Policy

Slide 17

Minion

Pod

iptables

Pod

Agent

Controllers

Scheduler

API

etcd

Pod/Service

Definition

CNI

Listener

Page 19: OpenStack Ops Meetup

Nested Container Networking

June 2016 romana.io Slide 18

Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Field

Capacity 0 0 0 0 1 0 1 0

Example: Bits Length Purpose

10.0 Network 8 Full Network (10/8)

Hosts 8 Up to 255 Hosts

Tenants 4 Up to 16 Tenants

Segments 4 Up to 16 Segments per Tenant

Endpoints 8 Up to 255 Endpoints per Segment

Bit location 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Field Host ID Bits (4)

Capacity 1 0 1 0 1 1 0 0 0 0 0 1 Up to 16 Hosts

Example: Bits Length Purpose

172.16 Network 12 Full Network (172.16/12)

Hosts 4 Up to 16 Hosts

Tenants 4 Up to 16 Tenants

Segments 4 Up to 16 Segments per Tenant

Endpoints 8 Up to 255 Endpoints per Segment

Endpoint ID

Up to 255 Hosts Up to 255 Tenant/Segments 255 Endpoints

Tenant and Segment ID Bits (8) Endpoint ID

Up to 255 Tenant/Segments 255 Endpoints

Location

12 1-12

16

20 17-20

10/8 Net Mask Host ID Bits (8) Tenant and Segment ID Bits (8)

Location

8 1-8

16 9-16

24 21-24

32 25-32

13-16

20 17-20

24 21-24

32 25-32

172.16/12 Net Mask

Page 20: OpenStack Ops Meetup

Nested Containers

June 2016 romana.io

192.168.0.10 192.168.0.11 192.168.0.12

Slide 19

Host 1

VM 1: 10.1.1.22

G/W: 10.1.0.1/16

10.2/16 -> 192.168.0.11

10.3/16 -> 192.168.0.12

172.17/16-> 192.168.0.11

172.18/16 -> 192.168.0.12

Pod 172.16.1.8

Pod 172.16.2.9

GW 172.16.0.1/16

172.17/16 -> 10.2.0.1

172.18/16 -> 10.3.0.1

Host 2

VM 1: 10.2.1.22

G/W: 10.2.0.1/16

Pod 172.17.6.8

Pod 172.17.2.11

GW 172.17.0.1/16

172.18/16 -> 10.3.0.1

172.16.16 -> 10.1.0.1

Host 3

VM 1: 10.3.1.22

G/W: 10.3.0.1/16

Pod 172.18.3.8

Pod 172.18.4.9

GW 172.18.0.1/16

172.16/16 -> 10.1.0.1

172.17/16 -> 10.2.0.1

10.1/16 -> 192.168.0.10

10.3/16 -> 192.168.0.12

172.16/16 -> 192.168.0.10

172.18/16 -> 192.168.0.12

10.1/16 -> 192.168.0.10

10.2/16 -> 192.168.0.11

172.16/16 -> 192.168.0.10

172.17/16-> 192.168.0.11

Page 21: OpenStack Ops Meetup

Ubernetes

June 2016 romana.io

192.168.0.10 192.168.0.11 192.168.0.12

Slide 20

Host 1

VM 1: 10.1.1.22

G/W: 10.1.0.1/16

10.2/16 -> 192.168.0.11

10.3/16 -> 192.168.0.12

172.17/16-> 192.168.0.11

172.18/16 -> 192.168.0.12

Pod 172.16.1.8

Pod 172.16.2.9

GW 172.16.0.1/16

172.17/16 -> 10.2.0.1

172.18/16 -> 10.3.0.1

Host 2

VM 1: 10.2.1.22

G/W: 10.2.0.1/16

Pod 172.17.6.8

Pod 172.17.2.11

GW 172.17.0.1/16

172.18/16 -> 10.3.0.1

172.16.16 -> 10.1.0.1

Host 3

VM 1: 10.3.1.22

G/W: 10.3.0.1/16

Pod 172.18.3.8

Pod 172.18.4.9

GW 172.18.0.1/16

172.16/16 -> 10.1.0.1

172.17/16 -> 10.2.0.1

10.1/16 -> 192.168.0.10

10.3/16 -> 192.168.0.12

172.16/16 -> 192.168.0.10

172.18/16 -> 192.168.0.12

10.1/16 -> 192.168.0.10

10.2/16 -> 192.168.0.11

172.16/16 -> 192.168.0.10

172.17/16-> 192.168.0.11

WAN

Page 22: OpenStack Ops Meetup

Networks Define Services• Tenant ID + Segment ID become a Network ID

• Natural fit for micro- and shared platform

services

• Route control to/from micro services enable

transparent service insertion/chaining and policy

enforcement

• Local/remote/hybrid cloud deployments

romana.io

IP

Int

IP

Int

IP

Int

IP

Int

L/B

Microservice

Endpoint

F/W

Shared Services

June 2016 Slide 21

Page 23: OpenStack Ops Meetup

Romana Project• Cloud Native network and security automation

• All details available at romana.io

• Open source

• Apache 2.0

• Written in Go

• www.github.com/romana

• OpenStack and Kubernetes integration

• Release v0.9 available now

romana.ioJune 2016 Slide 22

Page 24: OpenStack Ops Meetup

Demo• OpenStack on four physical machines

• Launch VMs on private 10/8 network

• Kubernetes running on VMs

• Kubernetes Network 172.16/12

• Container Network Interface (CNI) configuration of pods

• Romana IPAM allocates IPs for VMs and pods

• Chosen specially to maintain static routes and CIDRs to each host

and VM

• All IPs reachable by construction

June 2016 romana.io Slide 23