Distributed SDN controllers

Marco Canini

2

Network Policy Specification

10 Feb 2015

Controller Platform

Centralized Network Policy

Load Balancing

Access Control

Routing Monitoring Waypoint

3

Network Policy Specification

10 Feb 2015

Controller Platform

Centralized Network Policy

Load Balancing

Access Control

Routing Monitoring Waypoint

1.<Match, Action>2.<Match, Action>3.<Match, Action>4.<Match, Action>5.<Match, Action>6.…7.…

forward from X to Y

Y

X

4

Network Policy Specification

10 Feb 2015

Controller Platform

Centralized Network Policy

Load Balancing

Access Control

Routing Monitoring Waypoint

Y

X

Composition

Policy composition assembles data plane updates as a semantically sound set of rules

5

Policy Composition Review

10 Feb 2015

Composition

Reitblatt ‘12: Consistent network updates

Routing

Waypoint

Ferguson ‘12, ‘13: Policy treesfor multi-authorship

Srv Load Balancing Routing Monitoring>> U

Foster ‘11, Monsanto ‘13: Modular, parallel and sequential composition

6

Composing Policies

10 Feb 2015

Routing Monitoring

H

dst = H fwd(H) dport = 80 count

What to do with packet dst = H and dport = 80?Can Routing and Monitoring compose?

dst = H ∧ dport = 80 count; fwd(H)dst = H fwd(H)dport = 80 count

Composition

7

Ordering Policies

10 Feb 2015

Routing Waypoint

H

dst = H fwd(H) src = S fwd(IDS)

What to do with packet dst = H and src = S?Does Waypoint have precedence over Routing?

src = S fwd(IDS)src != S ∧ dst = H fwd(H)

Waypoint

Precedence

8

Conflicting Policies

10 Feb 2015

In the general case, policies might conflictExamples:• Overlapping domains and same precedence• Scarce flowtable resources

Must avoid conflicting policiesPick one and reject the other?

dst = H fwd(X) dst = H fwd(Y)Routing Load Balancing

Conflict X != Y

9

Centralized Network Control?

10 Feb 2015

Controller Platform

Centralized Network Policy

Load Balancing

Access Control

Routing Monitoring Waypoint

Fully centralized Inadequate availability, scalability and responsiveness

10

Distributed Network Control

10 Feb 2015

Controller Platform

Centralized Network Policy

Load Balancing

Access Control

Routing Monitoring Waypoint

Consistency and concurrency

Faults and asynchronous

communication

Composition?

11

Now, consider policy compositionin the distributed control plane...

10 Feb 2015

PolicyRouting Monitoring Waypoint

Switch Reader-Writer Model

Controller Platform

PolicyRouting Monitoring Waypoint

Controller Platform

State Distribution Model

Control LogicFactorization

12

Why should the programmer care?

• We believe she should not!• Enter Software Transactional Networking

– Let a dedicated component implement a general solution to all hard-to-solve, low-level concurrency and fault tolerance issues

10 Feb 2015

Transactional Interface

apply(π) ack / nack(reason)

13

Consistency: Linearizability of updates

10 Feb 2015

p1

p2

p3

apply(π1) ack

apply(π2) ack

apply(π3) nack

sw1

sw2

sw3

Original history

p1

p2

p3

apply(π1) ack

apply(π2) ack

sw1

sw2

sw3

Sequential equivalentWe don’t

control traffic!

Manipulate the network as though there is no concurrency

14

Can we realize a general distributed policy composition interface that is

agnostic to control logic, state distribution and reader-writer

model?

10 Feb 2015

15

Studied Two Underlying Problems

1. Consistent and Concurrent Policy Composition– Exploit optimistic concurrency [HotSDN’13]

2. Consistent and Robust Policy Composition– In face of controller failures [INFOCOM’15]

10 Feb 2015

16

Software Transactional Networking v1

10 Feb 2015

Routing Monitoring Waypoint

1. All-or-nothing semantics2. Optimistic concurrency for policy composition3. Non conflicting policies eventually installed4. Per-packet consistent updates

STN Interface

apply(π1) ack apply(π2) nack(reason)

HotSDN’13

17

Conceptualizing STN

10 Feb 2015

Routing Monitoring Waypoint

STN STN

Policy πunique tag τ 1. Internal ports

match on tag τ

2. Ingress portsapply tag τ

STN

apply(π) ack

18

STN v1 Details

• Explore design with no explicit communication between STN nodes– Avoid concurrency issues via an atomic read-

modify-write primitive at switch ports

• Problems with our simple algo:– # tags grows exponentially with # (conc.) policies

• Requires a cleaner to garbage collect unused rules

– No fault tolerance

10 Feb 2015

19

Software Transactional Networking v2:Consistent Policy Composition

10 Feb 2015

Routing Monitoring Waypoint

1. All-or-nothing semantics2. Tolerate up to f controller crash failures3. Non conflicting policies eventually installed and

at least one policy commits (among conflicting ones)4. Ensure updates affect traffic as a sequential

composition of their policies

CPC Interface

apply(π1) ack apply(π2) nack(reason)

Require robustness but not efficiency

Per-packet consistency

INFOCOM’15

20

Conceptualizing CPC

10 Feb 2015

Routing Monitoring Waypoint

CPC

Lock

Lock

Lock

Lock

Lock

Lock

Lock

Atomic Read-Modify-

Write

CPC

Reliable but asynchronous channel

Every controller receives and installs every policy

21

CPC implementation: model v1

• Controllers access switch ports with read and write

operations

• Controllers can communicate via asynchronous

message-passing

• Controllers may fail by crashing

• No synchrony assumptions

• Restrict policies to forwarding

– Compose if domains are disjoint or related by precedence

– Reject otherwise

10 Feb 2015

22

Asynchronous read-write CPC

10 Feb 2015

Theorem: 1-resilient read-write CPC is impossible

Proof sketch:• Two ingress ports 1 and 2 initially forward all

to the internal ports (π0)• π1 installed by p1 and π2 installed by p2, π2

refines π1 (higher precedence, same domain)• π1 and π2 propose different paths• p1 changes port 1 and is just about to change

2 (with a composition of π0 and π1), p2 takes no steps

• p2 wakes up and installs of π0 π1 π2, p1 takes no steps

• p1 changes port 2 with π0 π1: π2 is forgotten!

1 2?

p1 p2

π0 π1 π0 π1 π2

23

CPC implementation: model v2

• Controllers access ports with atomic read-modify-write ops RMW(f,g,v):– read the state v’– write f(v,v’) – return g(v,v’)

• Intuition: do not update if conflicts with currently installed policy

10 Feb 2015

RMW(f,g,v) g(v,v’)

Controller

24

Upper bound: FixTag algorithm

Operation:1. Unique tag per path2. Broadcast policy π to all other controllers3. Update ingress ports in predefined order4. … add rule to tag all packets matching dom(π) with the tag

corresponding to the path π(i) for ingress port i

Upsides: wait-free (tolerates all failure patterns)– Controllers only synchronize through the data plane

Downsides: overhead can be huge– Super-exponential in the size of the network

10 Feb 2015

25

Can we do better? No, if we get no feedback from the network

Tag t cannot be reused if a packet tagged with t is still “in flight”

Suppose, we can correctly evaluate the set of active tags Correct (but asynchronous) oracle

Single-controller scenario: one bit is enough! Upon policy update πi wait until (i mod 2)-traffic is over, and use tag i

mod 2

Two or more controllers: inherent price of concurrency? Between constant and super-exponential? Yes, if controllers coordinate use of tags

10 Feb 2015

26

ReuseTag: linear complexity

• Proportional to the level of resilience:– Up to f failures: f+2 tags needed (proved optimal)

• Controllers use consensus instances (eventual synchrony or « eventual leader ») – Replicated state machine that imposes a global order on the

policy updates and ensure coordinated use and reuse of tags• All requests are serialized

– Even non-conflicting ones– Can we do better?

10 Feb 2015

27

Summary

• Framework for concurrent and consistent policy composition in distributed SDN

• Transactional interface to manipulate the network as though there is no concurrency– Policies compose or conflict (and abort)

10 Feb 2015

2810 Feb 2015

Acknowledgements

Petr Kuznetsov Dan Levin Stefan Schmid

Supported by ARC grant 13/18-054 from Communauté française de Belgique, andEuropean Union’s Horizon 2020 ENDEAVOUR project (grant agreement 644960)

29

References

Software Transactional Networking: Concurrent and Consistent Policy CompositionHotSDN’13

STN: A Robust and Distributed SDN Control PlaneONS’14

A Distributed and Robust SDN Control Plane for Transactional Network UpdatesINFOCOM’15

10 Feb 2015

30

NetPL’15 – Call for Participation

• Workshop co-located with ECOOP’15– Organizers: Robert Soulé & Marco Canini

• Prague, July 5th or 6th 2015• “to bring together researchers and practitioners

from the fields of programming languages, formal methods, and networking”

• Program: speakers from academia and industry– Shriram Krishnamurthi, Hitesh Ballani, Colin Dixson,

Roger Wattenhofer, Fonta Dimitropoulos, Antonio Capone, Anil Madhavapeddy, Xia Yinben, ...

10 Feb 2015