Inside the Architecture of Neutron

Mark McClain

mmcclain (at) yahoo-inc.com

http://yahoo-inc.com

Why Create Neutron?

Rich Topologies

Technology Agnostic

Extensible

Advance Services Support

Load Balancing, VPN, Firewall

The Basics

What does the user see?

Compute API

Network API

Storage APIGUI, CLI, API Libs

KVM

ML2 Plugin

Ceph

Abstractions

Net1

10.0.0.0/24

Nova

Neutron

L2 virtual network

virtual port

virtual server

virtual interface (VIF)

virtual subnet

VM1

10.0.0.2

VM2

10.0.0.2

Architecture

Design Goals

Unified API

Small Core

Pluggable Open Architecture

Extensible

OpenStack The Operator View

Basic Deployment

neutron-server

L2 AgentL2 AgentL2 AgentL2 AgentL2 AgentL2 Agent

L3 AgentL3 Agent

L3 AgentL3 Agent

Database

L3 Agent

DHCP Agent

L2 Agent

Message Queue

Adv Services

neutron-server

REST API SERVICE RPC SERVICE

PLUGIN

REST API SERVICE

REST API

HTTP(S) Python WSGI Application

Customary TCP port is 9696

Exposes logical resources

networks, subnets, ports, etc

Request/Response Serialization

neutron-server

REST API SERVICE RPC SERVICE

PLUGIN

RPC SERVICE

RPC Service

AMQP via Oslo messaging modules

Enables bidirectional agent communication

Optional

neutron-server

REST API SERVICE RPC SERVICE

PLUGIN

PLUGIN

Written in Python

Only one active

Must implement V2 API calls

Optional database access

Optional extension support

The Plugin

ML2Plugin

core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin

The Plugin

NeutronPluginBaseV2

NeutronDbPluginV2

ML2Plugin

core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin

Plugin Extensions

Add logical resources to the REST API

Discovered by server at startup

REST: /v2.0/extensions

Common Extensions

Binding, DHCP, L3, Provider, Quota, Security Group

Other Extensions

Allowed Addresses, Extra Routes, Metering

Monolithic Plugin

Full implementation of core resources

Two types:

Proxy

Direct control PLUGIN

ML2: Modular Layer 2 Plugin

Full V2 Plugin Implementation

Delegates calls to proper L2 drivers

Two kinds of drivers

Type Driver

Mechanism Driver Mech Mgr

PLUGIN

Type Mgr

L2 Agent

L2 Agent

Runs on hypervisor

Communicates with server via RPC

Watch and notify when devices added/removed

Wires new devices

Proper network segment

Security Group Rules

Dive Into the OVS Agent

OVS

What does it actually do?

How do we get isolation?

VLAN, Overlays: GRE, VXLAN

Processing loop

Linux Network Namespace

Isolated copy of network stack

private loopback

scope limited to namespace

can reuse addresses

Explicit configuration needed to connect

Processes can spawn within namespace

lo

eth1

eth0

lo

eth1

eth0

lo

eth1

eth0

Host A B

br-int

L3 Agents

Network Node

L3 Agent

Run on Network Node

Uses Namespaces

Metadata Agent (if enabled)

Network Node

Core

Hypervisor Hypervisor Hypervisor

L3 Agent How its implemented

Manages Collection of Network Namespaces

Isolated IP Stacks

Forwarding Enabled

net.ipv4.ip_forward=1

Static Routing

Metadata Proxy

lo

eth1

eth0

lo

qg-2

qr-1

lo

qg-b

qr-e

Host A B

br-ex

Configuration Agents

Configuration Agents: DHCP

RPC based notifications

dnsmasq

Isolation Support via Network Namespaces

Multiple copies for HA

Configuration Agents: Metadata Proxy

Proxies Metadata requests to Nova

Routed Networks

process embedded in router

Non-routed Networks

static route redirects traffic running in DHCP namespace

Configuration Agents: Metadata Proxy

curl http://168.254.169.254/openstack/latest/meta_data.json

Tenant

VM

Nova Metadata Service

Meta NS Proxy

Metadata Agent

Unix Domain Socket

Management Network

X-Router-Id: 2bc7c882-d612-438c-a334-0047f2b5c2d7 X-Forwarded-For: 10.0.0.1 X-Instance-ID: aaaaaaaa-aaaa-aaaa-aaaaaaaaaaaa

http://168.254.169.254/openstack/latest/meta_data.json

Booting a VM nova boot

Booting a VM nova boot

create port

notify DHCP of new port

Booting a VM nova boot

create port

notify DHCP of new port

create device

new in Icehouse wait

Booting a VM nova boot

create port

notify DHCP of new port

libvirt create device

new in Icehouse wait

wire port

Booting a VM nova boot

create port

notify DHCP of new port

libvirt create device

new in Icehouse wait

wire port

boot

Load Balancer as a Service

Service Plugin

Driver based

Agent w/Driver

Agent communicates over RPC

Open Source requires namespaces

Others interact with other systems

LB Agent

HAProxy

VPN as a Service

Service Plugin

Driver based

Agent w/Driver

Communicates over RPC

Openswan

L3 Agent

Router

Metadata Proxy

VPN Driver

Firewall as a Service

Edgewall

Service Plugin

Driver based

Agent w/Driver

Communicates over RPC

Experimental

L3 Agent

Router

Metadata Proxy

Firewall Driver

Differences

Different Design Decisions

Sync with backend system

L2 Agent Optional

Not all implement same extensions

Summary

Open vSwitch / Linux Bridge

Ryu OpenFlow

Controller

Unified API

Small Core

Pluggable Open Architecture

Multiple Vendor Support

Extensible

More Information

Cloud Administrator Guide

http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html

Network v2.0 API

http://developer.openstack.org/api-ref-networking-v2.html

http://docs.openstack.org/admin-guide-cloud/content/ch_networking.htmlhttp://developer.openstack.org/api-ref-networking-v2.html

Questions?