OpenStack Summit Austin 2016 · 4/28/2016 · P4: Programming Protocol-independent Packet...

OpenStack® Summit Austin 2016

Policy CanvasDraw your policies for OpenStack services

Joon-Myung Kang, Mario Ant Sánchez, Sujata Banerjee,

Hewlett Packard Labs{firstname.lastname}@hpe.com

Apr 28, 2016

2

Jeongkeun “JK” LeeBarefoot Networks

[email protected]

mailto:%[email protected]

Talk Outline

3

Policy management in OpenStack

Policy Canvas DemoPolicy Canvas for OpenStack

How to define policies

Nova

Neutron

Keystone

Heat

Glance

Swift

CinderIronic

Murano

Policy Management in OpenStack

4

Tacker

Nova

Neutron

Keystone

Heat

Glance

Swift

CinderIronic

Murano


5

Tacker

Cloud operator

NetworkAdministrator

TenantAdministrator

ApplicationDeveloper

User

Application

Multiple Users

Nova

Neutron

Keystone

Heat

Glance

Swift

CinderIronic

Murano


6

Tacker

Cloud operator

NetworkAdministrator

TenantAdministrator

ApplicationDeveloper

User

Application

ACL

QoS

Performance

Firewall

LoadBalancer

DPI

IDS

Multiple Users

Multiple Policies

Network Policy Management in OpenStack

– Security groups and rules– Allow/Deny protocols and ports– Associate to VMs or ports

– Automation (Heat, Murano, etc.)– Pre-define templates and application-specific policies

7

Low-level commandsFragmented interfacesStatic configurations

Network Policy Management in OpenStack



– Group-based Policy (GBP)– High-level Intent based management for OpenStack– Policies between Endpoint Groups (EPG)

8

Additional API SupportMultiple-writer problem




– Group-based Policy (GBP)– High-level Intent based management for OpenStack– Policies between Endpoint Groups (EPG)

– Congress– Global policy framework for OpenStack– SQL-like language (Datalog) defining policies

across services

9

Steep learning curveMultiple-writer problem

Policy Management in OpenStackChallenges

– Decouple high-level intent (the “what”) from the underlying implementation (the “how”)

– Easy to use, intuitive, high-level interfaces

– Automated deployment mechanisms

– Scalable

10

Policy CanvasDraw network policies for OpenStack

11

Policy Management for OpenStackIdea from Real World

12

Policy CanvasDraw your policies for OpenStack services

13

Policy Canvas

Policy Canvas: Graph Abstraction & Composition

14

Mktg&Cmp-B&Normal

Engg&Cmp-A&Normal

HTTP Web&Cloud

DNS

DB&Cloud

RemedyService

Engg&Cmp-A&Qn

Mktg&Cam-B&Qn

Ping,SSH

HTTP

monitor

SQL, monitorsync,

monitor

monitor

DNS DNS

*

*

BC

BC

BCLBFW

BCLBFW

DPIDPI

BC

BC

graph composition

QuarantinedRemedyService

Policy sources Graph Abstraction Unified, conflict-free policy graph Deploy

Proactive policy composition1) Resolve policy conflicts prior to runtime deployment2) Simplify policy enforcement & runtime trouble-shooting

Intuitive to human, readable to system

PGA: Policy Graph Abstraction

15

& Composition

P1 & ~P2

P1 & P2(conflict?)

~P1 & P2

vP1 & ~P2

P1 & P2(conflict-free)

~P1 & P2

PGA: Graph Model & Composition

Graph model Graph compositionLogical labels

WebMktg.

CloudCampus

LB

BChttps, ssh

httpsP1:

P2:

Exclusive to Mktg

Venn-diagrammerging

Web & Cloud

Mktg & Campus

~Web &Cloud

~Mktg & Campus

BC

BChttps, ssh

https LB

BC

https, ssh

Label relationsMktg ⊂ CampusWeb ⊂ Cloud

Node: End-Point Group (EPG)Edge: communication intent

Decouple policy from target specificsDefined from existing data sources

Compute the union of input policies Resolve conflicts in the intersectionNormalize into disjoint sets

App admin

Cloud operator

Current solutions: manual composition

Composition Algorithm

Group-based policy (GBP)

App admin P1:Allow Marketing employees exclusive https access to Web servers through Load Balancer (LB)Cloud operator P2:Allow https and ssh traffic from Campus to Cloud through Byte Counter (BC)

Placements: Marketing in CampusWeb in Cloud

Overlap and

conflict

Subject1: Web-Accesstcp, dstport=https : Permit, LB-BC chain

Subject2: Web-Block* : Deny

Subject3: Cloud-Accesstcp, dstport=https|ssh : Permit, BC chain

Clauses (prioritized) :1. Mktg Web: Web-Access2. * Web: Web-Block3. Campus Cloud: Cloud-Access

Mktg ⊂ CampusWeb ⊂ Cloud

WebMktg.

CloudCampus

LB

BChttps, ssh

httpsP1:

P2:

Exclusive to MktgApp admin

Cloud operator

Web & Cloud

Mktg & Campus

~Web &Cloud

~Mktg & Campus

BC

BChttps, ssh

https LB

BC

https, ssh

PGA continued• Formal graph model, data structure, algorithm• Label relations in Label Trees

• Scalable: 13 mins to compose 20K ACL + service policies• Intuitive reasoning & troubleshooting by graph walk

(a) Departmentsadmin

Engg.

MktgPing,SSH

Cloud

monitor

QuarantinedRemedyService

*

(b) Application admin

(d) Cloud operator(c) SDN app: Net Protector

Campus Cloud*

*

HTTPEmpl Web

SQLsync

DBLB

Normal DNSDNS

(a) Enterprise IT admin

DPI FW BC

BC

Label NamespaceLabel Trees

Label Mappings

Cmp-A

Status

Tenant

Empl App

Mktg

Web DBCampus Cloud Net

protector

Normal Qn

Location

Engg: Campus-A Mktg: Campus-BApp : CloudEmpl: Net protector

Cmp-BEngg

compose

Mktg&Cmp-B&Normal

Engg&Cmp-A&Normal

HTTP Web&Cloud

DNS

DB&Cloud

RemedyService

Engg&Cmp-A&Qn

Mktg&Cam-B&Qn

Ping,SSH

HTTP

monitor

SQL, monitorsync,

monitor

monitor

DNS DNS

*

*

BC

BC

BCLBFW

BCLBFW

DPIDPI

BC

BC

18

overlap

mutually exclusive

Intent APIs to Express Hidden Intents

Whitelisting WL = {HTTPS} MUST allows d

s d CAN communicate

s d Block

s d Conditional

src dstWL

Default edge: MUST

No edge: Conditional

BlacklistingBL = {SSH, ICMP}

src dstBL

{*} – WL : deny or don’t care?

Default edge: Block

No edge: CAN communicate

{*} – BL : allow or don’t care?

Intent ACL edges

Reduce %conflict: 50% 12.5%Enable systematic composition

PGA Adoption and Publicity

Adoption– ODL Network Intent Composition (NIC) project adopted graph compiler and intent ACL

https://wiki.opendaylight.org/view/Network_Intent_Composition:Graph

Paper, Demo– “PGA: Using graphs to express and automatically reconcile network policies”, ACM Sigcomm

2015, London. – “Network Policy Whiteboarding and Composition”, ACM Sigcomm 2015 Demo, London.

Talks– OpenStack Summit Vancouver #vBrownBag session, https://www.youtube.com/watch?v=jDCxUQ5D4Y4– OpenDaylight Summit 2015, Santa Clara, CA, https://www.youtube.com/watch?v=EHMSuWz3LHw

https://www.youtube.com/watch?v=jDCxUQ5D4Y4

https://www.youtube.com/watch?v=EHMSuWz3LHw

Policy Canvas System

21

Policy CanvasSystem overview

22

Policy Canvas (Horizon module)

Python-pgaclient(python bindings to PGA API)

PGA servicePGA extension- Input policy graph- Composed policy graph- Deployed policy graph- Label tree- Function box

REST APIs

Labeldrivers

Compilationdrivers

Enforcementdrivers

Policy CanvasPolicy graph creation/composition

23


PGA service

Label drivers

Draw policy graphs

Input policy graphs

Label tree(s)

Input policy graphs

Composed policy graph

…User/App1 User/App2 User/Appn


Label tree(s) Compilation drivers

PGA graph composer

OpenDaylight NIC graph compiler

Data sources(nova, keystone, ...)

Policy CanvasPolicy graph deployment

24

Congress


Deploy

Neutron

Admin


PGA service

enforcement driversCongress driver Neutron driver

Endpoint monitoring rules security groups, SFC, LB/FW …

Nodes and edges

Data sources(nova, keystone, ...)

actionclassifier

classifier + action

clients webLBHTTP

Policy CanvasAutomatic policy mapping

25

CongressNeutron

Admin


PGA service

enforcement driversCongress driver Neutron driver

Data sources(nova, keystone, ...)VM

notify membership change Associate security group

clients webLBHTTP

Policy Canvas1. Demo Scenario2. Demo

26

Demo Scenario

27

VM VM

VM VM

Availability Zone 1 (AZ1)

Availability Zone 2 (AZ2)

cnc4cnc3

cnc32 cnc27

HTTPSSHPING

Zone Manager

Host Manager

Global Policy

Policy Canvas Demo

28

Future Directions

29

Extending Policy Canvas

–Importing already-expressed policies directly from neutron or other services for composition and analysis

–Modeling and composing newlly-added neutron features QoS–Port-level connectivity

30

Policy Enforcement and Verification

Policy is eventually enforced at various network dataplanes

Need abstraction to capture capabilities and limitations Can this network implement my policy?

Need programmable dataplane to implement complex policiesTailor the network, rather than retrofit your policy into it

Need complete network visibility to verify policyDoes my policy get correctly enforced?

Industry-wide open-source effortsOpenFlow Data Plane Abstraction (OF-DPA)OpenCompute Switch Abstraction Interface (SAI)P4: Programming Protocol-independent Packet Processor

31


Neutron

Security Group, SFC, LB, FW …

Data planes (hw, sw, edge, core)

Network controllers(overlay, underlay)

P4: Programming Protocol-independent Packet Processor

High-level language to define and modify dataplane

Protocol/target independent

Open source, Apache 2.0 CLA

Adopted by OpenSwitch, OpenCompute SAI

Benefits in policy enforcementPick the best protocol for your policy, remove the othersCustom protocol carrying SFC metadataEmbed service functions (e.g., L4 LB) in the switchPolicy verification by In-Network Telemetry

/*********************************************//* Router MAC lookup *//*********************************************/action rmac_hit() {

modify_field(l3_metadata.rmac_hit, TRUE);}

action rmac_miss() {modify_field(l3_metadata.rmac_hit, FALSE);

}

table rmac {reads {

l3_metadata.rmac_group : exact;l2_metadata.lkp_mac_da : exact;

}actions {

rmac_hit;rmac_miss;

}size : ROUTER_MAC_TABLE_SIZE;

}

Copyright © 2016 P4 Language Consortium.

Fixed Data Plane

Driver

SBI Agent

Network controller

Southbound API (SBI)

Northbound APIProgram

Compile

Target Binary

Auto-Generated runtime API

Copyright © 2016 P4 Language Consortium.

Composed Policy

Policy down to Dataplane

Neutron

Programmable Data Plane (hw, sw)

Conclusions

– Policy Canvas– Simple and intuitive abstraction (Graph model)– Portable policies, decoupled from infrastructure specifics– Automatic and proactive composition of independently

specified policies

– Status– Horizon GUI– PGA service and API– Neutron/Congress drivers– Graph compiler in OpenDaylight NIC

– Collaboration with OpenStack community– More use cases and feedback– Code contribution to OpenStack

34


PGA service

Policies

Input policy graphs

…User/App1 User/App2 User/Appn

Infrastructure

composed policy graph

33

Thank you

35

OpenStack Summit Austin 2016 · 4/28/2016 · P4: Programming Protocol-independent Packet...

Documents

Transcript of OpenStack Summit Austin 2016 · 4/28/2016 · P4: Programming Protocol-independent Packet...