Traffic Optimization in

Multi-Layered WANs using SDN Henrique Rodrigues1,2, Inder Monga2, Abhinava Sadasivarao3, Sharfuddin

Syed3, Chin Guok2, Eric Pouyoul2, Chris Liou3, Tajana Rosing1 1UCSD, 2ESNet/LBNL, 3Infinera

IEEE Symposium on High Performance Interconnects, August 2014

Wide Area Networks

• Critical resource for performance and reliability of the Internet

• Massive traffic from multiple applications over several long distance links

• Equipment from multiple vendors – Expensive to deploy, expensive to operate

• Problems: – Poor resource utilization (~30-50%)

– Low management flexibility

Hong%Kong%

Seoul%

Sea, le%

Los%Angeles%

New%York%

Miami%

Dublin%

Barcelona%

Tuesday, August 13, 13

Figure source: Microsoft SWAN SIGCOMM’2013

2 Hot Interconnect 2014 27/08/14

Recent work addressing these

problems in inter DC WAN:

• Google’s B4 (SIGCOMM’13)

• Microsoft SWAN (SIGCOMM’13)

Improved network utilization, flexibility, resilience with Centralized management + Software Defined Networking + OpenFlow

Inter DC Wide Area Networks

3 Hot Interconnect 2014 27/08/14

OpenFlow, BGP (B4, SWAN)

GMPLS, TL1

Proprietary Manual Operation

The hidden multi-layered infrastructure

TDM/Transport SONET, SDH

DWDM/OADM

IP OpenFlow Layer

Can we manage all layers using a unified abstraction?

4 Hot Interconnect 2014 27/08/14

Why is this important?

• Scenarios where dynamic management wins:

– Multi-layer traffic optimization

• Current WAN management assume static topology

• If demand grows, new paths are added manually

• In the limit, current solutions either throttle traffic

(SWAN, B4) or offer degraded service

– Bandwidth virtualization

• Static allocation of higher capacity optical pipes can

result in wasted capacity for variable demands

– Flows of different demand (mice vs. elephant)

• Interaction of flows might lead to lower utilization

5 Hot Interconnect 2014 27/08/14

Optical Transport Network

0

250

500

750

1000

0 10 20 30

Throughput(M

bps)

0

250

500

750

1000

0 10 20 30

Concurrent flows C = 4 Concurrent flows C = 8 Time (s) Time (s)

Packet Network

Site A Site B

Distinct TCP Flows vs. Utilization

10G Optical Circuit C concurrent small, short flows

Large flow

Congestion control triggered by intermittent small flows contributes

to poor utilization

6 Hot Interconnect 2014 27/08/14

Summary of Challenges for Unified

WAN Network Management

– Network representation:

• How to build a complete view of the network?

– Multiple management interfaces:

• OpenFlow, SNMP, GMPLS, TL1

– Equipment with different characteristics:

• Encapsulation, Forwarding, Queuing, Link Sharing

– Distinct Traffic visibility

• Packet Flows, TDM slots, Wavelengths

– Management granularity

• Single L3 flow vs. Wavelength with several flows

7 Hot Interconnect 2014 27/08/14

Multi-layer orchestration with

OSCARS-TE

REST/JSON

OpenFlow 1.0

Configuration Manager

Topology Exchange Multi-Layer

Path Engine Multi-Layer Provisioning

Multi-Layer Topology App

ESNet Circuits Reservation System (OSCARS)

SDN Controller

Floodlight

Traffic Optimization

Engine

OSCARSTE Multi-Layer SDN

Management Modules

Optical Transport Network

Packet Network

X

Y

Z

A, B, C – Packet Switches X, Y, Z – Optical Transport

A

B

C

Site A

8 Hot Interconnect 2014 27/08/14

Orchestrating a Multi-layer SDN:

Discovering and maintaining topology

• No inter-layer discovery protocols • Maintenance of topology likely manual in the near term

• Dynamic Topology construction for multi-layer • LLDP discovers L2 topology when L0/1 is in place

• Configuration manager communicates proprietary L0/L1 topology. Alternatively, L0/L1 topology can be scanned

• OSCARSTE constructs a multi-layer topology annotating link with capacities, granularity and flow capabilities

Configuration Manager

Topology Exchange Multi-Layer

Path Engine Multi-Layer Provisioning

Multi-Layer Topology App

ESNet Circuits Reservation System (OSCARS)

SDN Controller

Floodlight

Traffic Optimization

Engine

OSCARSTE Multi-Layer SDN

Management Modules

9 Hot Interconnect 2014 27/08/14

Orchestrating a Multi-layer SDN:

Opening New Paths

• Path Computation Engine multi-layer aware

• Multi-stage, multi-layer computation process – Prunable constraints (ex. Bandwidth), Additive constraints

(ex. Latency), non-additive constraints (ex. VLAN continuity), Cross-Layer adaptation constraints.

• In this work we flatten the topology into a single graph annotated with node/link capabilities

Configuration Manager

Topology Exchange Multi-Layer

Path Engine Multi-Layer Provisioning

Multi-Layer Topology App

ESNet Circuits Reservation System (OSCARS)

SDN Controller

Floodlight

Traffic Optimization

Engine

OSCARSTE Multi-Layer SDN

Management Modules

10 Hot Interconnect 2014 27/08/14

Orchestrating a Multi-layer SDN:

Provisioning at multiple layers

• Match capabilities of the layers with the provisioning action • Capabilities learnt from OF handshake

• Smart path setup with low impact on traffic • Ordered path updates between L2 and L1 devices

• Pluggable with public SDN controller APIs

Configuration Manager

Topology Exchange Multi-Layer

Path Engine Multi-Layer Provisioning

Multi-Layer Topology App

ESNet Circuits Reservation System (OSCARS)

SDN Controller

Floodlight

Traffic Optimization

Engine

OSCARSTE Multi-Layer SDN

Management Modules

0"

2.5"

5"

7.5"

10"

0" 5" 10" 15" 20"

Throughput"(Gbps)"

Time"(s)"

Htcp"

Cubic"

Reno"

Highspeed"

Regular topology update at 10s Planned topology update at 10s

0"

2.5"

5"

7.5"

10"

0" 5" 10" 15" 20"

Throughput"(Gbps)"

Time"(s)"

Htcp"

Cubic"

Reno"

Highspeed"

11 Hot Interconnect 2014 27/08/14

Orchestrating a Multi-layer SDN:

Dynamic provisioning based on demand

• This is our multi-layer optimization engine

• Offloading engine allocates new paths when demand grows and isolate traffic with different characteristics • Port-based monitoring with threshold-driven triggers

• Sub-flow insight using packet sampling

• Mapping flows to topology and new links requires multi-layer knowledge

Configuration Manager

Topology Exchange Multi-Layer

Path Engine Multi-Layer Provisioning

Multi-Layer Topology App

ESNet Circuits Reservation System (OSCARS)

SDN Controller

Floodlight

Traffic Optimization

Engine

OSCARSTE Multi-Layer SDN

Management Modules

12 Hot Interconnect 2014 27/08/14

Optical Transport Network

Packet Network

Site A Site B

Enabling predictable application performance

Small flows

Large flow

T = 0: Only small flows T = 30: Large data transfer started T = 55: Large data transfer offloaded to dynamically allocated circuit

13 Hot Interconnect 2014 27/08/14

Conclusion

• Multi-layer traffic optimization improves

network performance and utilization

• Planned topology minimize performance

degradation during offloading

• Intelligent multi-layered SDN control plane

enables practical bandwidth virtualization

and predictable application performance

14 Hot Interconnect 2014 27/08/14

Thank you!

Henrique Rodrigues, Tajana Rosing

{hsr,tajana}@eng.ucsd.edu

Inder Monga2, Chin Guok2, Eric Pouyoul2,

{inder,chin,lomax}@es.net

Chris Liou3, Abhinava Sadasivarao3, Sharfuddin Syed3,

{cliou,asadasivarao,ssyed}@infinera.com

15 Hot Interconnect 2014 27/08/14

This work was supported by Energy Sciences Network, which is funded by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research (ASCR). ESnet is operated by Lawrence Berkeley National Laboratory, which is operated by the University of California for the U.S. Department of Energy under contract DE-AC02-05CH11231.