REDUNDANCY ELIMINATION AS A NETWORK-WIDE SERVICE

Aditya AkellaUW-Madison

Shuchi Chawla Ashok Anand Chitra MuthukrishnanUW-Madison

Srinivasan Seshan Vyas SekarCMU

Ram RamjeeMSR-India

Scott ShenkerUC-Berkeley

Growing traffic vs. network performance2

Network traffic volumes growing rapidly Annual growth: overall (45%),

enterprise (50%), mobile (125%)*

Growing strain on installed capacity everywhere Core (Asian ISPs – 80-90%

core utilization), enterprise access, data center, cellular, wireless…

How to sustain robust network performance?

* Interview with Cisco CEO, Aug 2007, Network worldEnterprises

Mobile users Home users

Video

Data centers Web content

Other svcs(backup)

ISPcore

Strain on installed link capacities

Enterprises

Scale link capacities by suppressing duplicates

3

A key idea: suppress duplicates Popular objects, partial content

matches, backups, app headers Effective capacity improves ~ 2X

Many approaches Application-layer caches Protocol-independent schemes

Below app-layer WAN accelerators, de-duplication

Content distribution CDNs like Akamai, CORAL Bittorrent

Point solutions apply to specific link, protocol, or app

Mobile users Home users

Video

Data centers Web content

Other svcs(backup)

Wan Opt

Wan Opt

Dedup/archival

Dedup/archival

ISP HTTPcache

CDN

Universal need to scale capacities4

Wan Opt

Wan Opt

Dedup/archival

Dedup/archival ISP HTTP

cache

Network RedundancyElimination Service

Point solutions inadequate

RE: A primitive operation supported inherently in the network

o Applies to all links, flows (long/short), apps, unicast/multicast

o Transparent network service; optional end-point modifications

o How? Implications?

Architectural support to address universal need to

scale capacities? Implications?

Bittorrent

✗ Point solutions:Little or no benefit in the core

✗ Point solutions:Other links must re-implement specific RE mechanisms

✗ Point solutions: Only benefit system/app attached

How? Ideas from WAN optimization

5

5

Cache Cache

WAN link

Data center Enterprise Network must examine byte streams, remove duplicates, reinsert Building blocks from WAN optimizers: RE agnostic to application, ports or

flow semantics Upstream cache = content table + fingerprint index

Fingerprint index: content-based names for chunks of bytes in payload Fingerprints computed for content, looked up to identify redundant byte-

strings Downstream cache: content table

Internet2

Packet cache at every router

Network RE service: apply protocol-indep RE at the packet-level on network links IP-layer RE service

From WAN acceleration to router packet caches

6

Wisconsin

BerkeleyCMU

Router upstream removes redundant bytes

Router downstream reconstructs full packet

(Hop-by-hop works for slow links

Alternate approaches to scale to faster links…)

Implications overview: Performance and architectural benefits

7

Improved performance everywhere even if partially enabled Generalizes point deployments and app-specific approaches

Benefits all network end-points, applications Crucially, benefits ISPs

Improved switching capacity, responsiveness to sudden overload

Architectural benefits Enables new protocols and apps

Min-entropy routing, RE-aware traffic engineering (intra- and inter-domain) Anomaly detection, in-network filtering of unwanted traffic

Simplifies/improves apps: need not worry about using network efficiently Application control messages & headers can be verbose better diagnostics Controlling duplicate transmission in app-layer multicast is a non-issue

Internet2

Implications example: Performance benefits8

Network RE 12 pkts

(ignoring tiny packets)

Without RE 18 pkts

33% lower

Wisconsin

BerkeleyCMU

Generalizes pointdeployments

Benefits ISPs: improve effective switching capacity

62 packets

32 packets

32 packets

Wisconsin

Internet2

Implications example: New protocols

9

RE + routing 10 pkts

Simple RE 12 pkts

BerkeleyCMU

9

✓ Redundancy-based anomaly detectors

✓ Network-assisted spam filtering ✓ New content distribution

mechanisms

✓ Minimum-entropy routing✓ New, flexible traffic engineering

mechanisms✓ Inter-domain protocols

Network RE service: Quantitative results10

Analysis of 12 enterprises: traffic 15-60% redundant [SIGMETRICS 09] ~1GB of cache sufficient to identify redundancies DRAM or PCM (PRAM) on routers

Network RE benefits both ISPs and end-networks [SIGCOMM 08] Upto 15-50% better util, responsive TE, control inter-domain traffic impact Centralized algorithm for min-entropy routing (using “redundancy profiles”)

Reduces utilization by a further 10-25% in intra-domain case Inter-domain min-entropy routing: gains much more significant (50-80%)

Is network RE viable at high speeds? Not in its current form… Compression is slow: limits hop-by-hop speed at each hop to 2.5Gbps

Acceptable for access, wireless, cellular links, not for the core Also, wastes memory on multiple routers limits effectiveness

SmartRE: Concerted network-wide RE11

Toss out link-by-link view; treat RE as a network-wide problem per ISP [Current work]

Memory usage: each packet compressed/un-compressed once Throughput: allow reconstruction multiple hops away from compression

Stand-alone reconstruction much faster when freed from dependence on compression immediately upstream

Reconstructor can reconstruct a lot more, from multiple different compression agents Resource-awareness: carefully account for network and device resources,

and traffic Compression/reconstruction/caching locations decided based on memory capacity and

memory operations Also consider global TE objectives

Just 4% from ideal RE (no memory or processing constraints)

Summary and future directions12

RE service to scale link capacities everywhere Architectural niceties and performance benefits High speed router RE seems feasible Future directions

End-host participation Role of different memory technologies – DRAM, flash and PCM Theoretical issues – pricing and economics, routing policy, network

design Network coding as an alternative to compression