NET-REPLAY: A NEW NETWORK PRIMITIVE

Ashok Anand Aditya Akella

University of Wisconsin, Madison

2

Network is a black box

NetworkEnd hosts

Black box view

No standardized way for network to inform, where and why glitch occurred

This keeps network simple and efficient However, end hosts have to resort to

complicated logic to infer the nature of glitches

Packet lost or delayed

3

Current tools for locating glitches

Probing tools Tulip[SOSP 2003] sends multiple probes to

routers on the path, and use their response to infer the nature of glitches

Issues with such approach Out-of-band troubleshooting Probe packets can be treated differently Transient failures hard to detect

4

What if network could tell end-hosts about glitches?

NetworkEnd hosts

Where and why glitch occurred

End hosts can take better actions No need to reduce flow rate, if packet loss was not due to

congestion Route around glitches using alternate routes via multi-homing or

recent routing protocol enhancements like path-splicing etc Benefits emerging applications e.g. gaming, video

streaming etc to achieve more robust network performance

5

Network-assisted troubleshooting

Design requirement Keep the network as simple as possible While enabling end hosts to determine where and

why glitches occurred Our design: Net-Replay

Router remember the packets they have forwarded Routers annotate packet (e.g. with their identifier)

that they see for the first time When some glitch occurs, sender replays those

packets who had experienced the glitch Based on annotations, receiver determines the

nature of glitch experienced by the original packet

6

Using Net-Replay to characterize loss

Remember the green packetand annotate before forwarding

A B CA A AARe-play the lost

packet

B BReplayed packet was seen for the first time at B

Packet already present at router A

Sender

Receiver

Receiver infers that packet was dropped at A-B link

7

Outline Supporting Net-Replay functionality in

network

End hosts using Net-Replay

Discussion

8

Outline Supporting Net-Replay functionality in

network

End hosts using Net-Replay

Discussion

9

Basic support at router

PacketCompute

hash Hash

Compute hash Exclude mutable fields (e.g. TTL)

Finding if new packet was already seen by the router Look-up hash in Hashstore

Remember new packet as seen Store hash pointing to packet in

Hashstore Evict the oldest packet, if Hashstore

becomes full Simple hash table implementation in

DRAM for speeds like 2.4 Gbps SRAM for higher speed (40 Gbps) 16 MB SRAM currently available

Hash

Packet

Hashstore

10

High speed Hashstore implementation

Use bloom-filters What about false positives?

Can probabilistically report the location of glitches What about packet eviction?

Use 2 bloom filters: primary and secondary When primary is half filled, start using both When primary is fully filled, copy secondary to primary and

clear out secondary How much time worth packets are stored in 16 MB SRAM?

Up to 3s at 40 Gbps with average packet size of 600 bytes Greater than10 RTTs assuming RTT < 250 ms

Sufficient enough for end-applications to react

11

Outline Supporting Net-Replay functionality in

network

End hosts using Net-Replay

Deployment discussion, cheating Issues and new applications enabled by Net-Replay

12

How end hosts can use Net-Replay? Characterizing glitches

Packet loss Replay lost packet

Delay Router remembers which packets were delayed Replay delayed packet

Reordering If it happened due to route changes, sender

could know the first router where route changed Replay reordered packet

13

End host protocol stack

Higher layer should decide the policy of handling glitches TCP layer can tell higher layer the nature of glitches e.g. loss

After loss, TCP layer retransmits packet (in current TCP protocols) Uses retransmitted packet to find the nature of loss Receiver sends the information about loss back to sender along with ACK

TCP layer

Application or higher layer

Nature of glitches

Decide how to overcome glitches

MIRO/ Path splicingOr no action

14

Outline Supporting Net-Replay functionality in

network

End hosts using Net-Replay

Discussion

15

Deployment discussion Partial deployment

Net-replay can be deployed on few routers and can be used to find the nature of glitches in path segments Border routers of ISP and information per

domain Avoiding device modifications

Can be deployed in 2-port hardware switches as bumps in the wire

Net-Replay agnostic devices

Net-Replay aware bumps in the wire

16

Other applications Network tomography uses complicated logic

to infer link loss rates With Net-Replay, location of loss can be precisely

determined Simplifies network tomography

Packets can be moved from fast memory to disks in batches Can be used for debugging distributed

applications Useful for network operators to find the

performance at fine grained level

17

Conclusion Net-Replay helps applications perform in-

band characterization of glitches

Net-Replay requires simple support from network infrastructure

End hosts can get robust network performance using Net-Replay

18

Questions Thank you

19

Backup

20

Hashstore implementation

Simple hash table in DRAM (50 ns latency) good enough at 2.5 Gbps Lookup and store: 100 ns per packet 40B packets arrive every 128ns at 2.5 Gbps

However DRAM latency can’t match 40Gbps; requires faster memory like SRAM Current SRAM up to 16MB only; Need

space-efficient data structure

21

Cheating issues ISP inserts wrong annotations to ensure

that it is never considered accountable for glitches Chances are that ISP is caught

ISP modifies ACK packet, if it finds its router is causing glitches Use encrypted ACK

Possibly other issues and need to investigate