A Narrow Waist for Multipath Routing

Murtaza MotiwalaBilal Anwer, Mukarram bin TariqDavid Andersen, Nick Feamster

Many Threats to Availability

• Natural disasters• Physical failures (node, link)• Router software bugs• Misconfiguration• Mis-coordination• Denial-of-service (DoS) attacks• Changes in traffic patterns

(e.g., flash crowd)• …

Idea: Backup/Multipath

• For intradomain routing– IP and MPLS fast re-route– Packet deflections [Yang 2006]– ECMP, NotVia, Loop-Free Alternates [Cisco]

• For interdomain routing– MIRO [Rexford 2006]

• Problem– Complexity and Scale: Protecting against arbitrary

failures requires storing lots of state, exchanging lots of messages

– Control: End systems can’t signal when they think a path has “failed”

Two Questions

• What is the appropriate mechanism for achieving multiple paths?– One example: Path Splicing

• What is the appropriate interface for allowing end systems access to multiple paths?– Path Bits: A “narrow waist” for Internet routing

Backup Paths: Promise and Problems

• Bad: If any link fails on both paths, s is disconnected from t

• Want: End systems remain connected unless the underlying graph has a cut

ts

Path Splicing: Main Idea

• Step 1 (Generate slices): Run multiple instances of the routing protocol, each with slightly perturbed versions of the configuration

• Step 2 (Splice end-to-end paths): Allow traffic to switch between instances at any node in the protocol

ts

Compute multiple forwarding trees per destination.Allow packets to switch slices midstream.

Generating Slices

• Goal: Each instance provides different paths• Mechanism: Each edge is given a weight that is

a slightly perturbed version of the original weight– Two schemes: Uniform and degree-based

ts

3

3

3

“Base” Graph

ts

3.5

4

5 1.5

1.5

1.25

Perturbed Graph

How to Perturb the Link Weights?

• Uniform: Perturbation is a function of the initial weight of the link

• Degree-based: Perturbation is a linear function of the degrees of the incident nodes– Intuition: Deflect traffic away from nodes where traffic

might tend to pass through by default

Forwarding Traffic

• One approach: shim header with forwarding bits

• Routers use lg(k) bits to index forwarding tables– Shift bits after inspection

• To access different (or multiple) paths, end systems simply change the forwarding bits– Incremental deployment is trivial– Persistent loops cannot occur

• Other variations are possible (2nd half of talk)

Reliability Approaches Optimal• Sprint (Rocketfuel) topology• 1,000 trials• p indicates probability edge was removed from base graph

Reliability approaches optimal

Average stretch is only 1.3

Sprint topology,degree-based perturbations

Simple Recovery Strategies Work Well

• Which paths can be recovered within 5 trials?– Sequential trials: 5 round-trip times– …but trials could also be made in parallel

Recovery approaches maximum possible

Adding a few more slices improves recovery beyond best possible reliability with fewer slices.

Two Questions

• What is the appropriate mechanism for achieving multiple paths?– One example: Path Splicing

• What is the appropriate interface for allowing end systems access to multiple paths?– Path Bits: A “narrow waist” for Internet routing

Allow for Innovation Above & Below

• Different applications & uses for multipath– Performance and load balancing– Availability

• Different mechanisms– Routing protocols: path splicing, ECMP, R-BGP,

NIRA, MIRO, …– Implementation platforms: proprietary solutions, Click,

NetFPGA, OpenFlow

Need: Common interface for multipath routing.

A Narrow Waist for Multipath Routing

Decouple End Hosts from Protocols

• Easy access to multiple paths

• Consistent path selection per-host– Same encoding should always yield the same path

• Interoperation among many different multipath routing protocols– Different switches and different networks may have

different implementations

Simple Interface and Implementation

• Ease of use at end systems

• Efficiently implementable in the network

• Scalable access to a large number of paths– End systems should not have to store state to

represent all paths– Routers should not have to store forwarding table

state for all paths

Path Bits: Three-Part Design

• Path bits interface– Level of control– Number of bits

• End system support– Monitoring framework– Packet interception

• Network control– Indexing into forwarding tables

Decision: Level of Control

• Path bits are opaque– Carry no explicit semantics– Instead, provide the following:

Changing the bits, will, with high probability, change the forwarding path

• Satisfies the two design goals– End-host interface is decoupled from implementation– Interface is simple: few changes required

Decision: Number of Bits

• A small number of bits can provide sufficient flexibility.– Example: Path splicing

• Possible to map many existing multipath implementations to the interface

End-System Implementation

• Socket capture library– Intercept connect() and

sendto()

• Kernel interface– Click module– Keeps track of active

flows & bits for those flows

• Monitoring daemon

Network Implementations

• Software implementations in Click– Splicing, Deflections, ECMP++

• Hardware implementations– Intel IXP: Splicing (~30 lines of C++ plug-in)– OpenFlow: kN entries per slice, ~20-25 lines at NOX

controller– NetFPGA: Small additional overhead beyond base

router implementation

Key Idea: Multiple forwarding table entires with bits to index. Different routing protocols simply change what is in the tables and how the bits are mapped to this interface.

Path Splicing in Click

Eight lines of C++.

Routing Deflections in ClickNine lines of C++.

NetFPGA Implementation

• Eight slices

• About 69% of available BRAM on NetFPGA(base router uses 53% of available BRAM)

NetFPGA Forwarding Performance

No noticeable performance penalty with four tables.

Applications with Path Bits: Recovery

Single-Path Routing(Baseline)

Path Splicing ECMP++

Conclusion• Many uses and applications for multipath routing

– Availability– Performance

• Many implementations of protocols– Path splicing: Simple, scalable, stable– Routing deflections

• Path bits: narrow waist allows applications and implementations to evolve independently– Decouples end host request from network

implementation/protocol– Affords very simple implementations– Many environments: Data center, interdomain, intradomain, etc.