BUFFALO: Bloom Filter Forwarding Architecture for Large Organizations

Minlan YuPrinceton University

minlanyu@cs.princeton.eduJoint work with Alex Fabrikant, Jennifer Rexford

1

Large-scale SPAF Networks• Large layer-2 network on flat addresses

– Simple for configuration and management– Useful for enterprise and data center networks

• SPAF:Shortest Path on Addresses that are Flat– Flat addresses (e.g., MAC addresses)– Shortest path routing

• Link state, distance vector, spanning tree protocol

2

SH

S

S

S S

S S S S

S S S

S

H

H

H

H

HH

H

H

Challenges

• Scaling control plane– Topology and host information dissemination– Route computation – Recent practical solutions (TRILL, SEATTLE)

• Scaling data plane– Forwarding table growth (in # of hosts and switches)– Increasing link speed– Remains a challenge

3

Existing Data Plane Techniques• TCAM (Ternary content-addressable memory)

– Expensive, power hungry

• Hash table in SRAM– Crash or perform poorly when out of memory– Difficult and expensive to upgrade memory

4

BUFFALO: Bloom Filter Forwarding• Bloom filters in fast memory (SRAM)

– A compact data structure for a set of elements– Calculate s hash functions to store element x– Easy to check membership – Reduce memory at the expense of false positives

h1(x) h2(x) hs(x)01000 10100 00010

x

V0Vm-1

h3(x)

BUFFALO: Bloom Filter Forwarding• One Bloom filter (BF) per next hop

– Store all the addresses that are forwarded to that next hop

6

Nexthop 1

Nexthop 2

Nexthop T

……packet

query

Bloom Filters

hit

BUFFALO Challenges

7

BUFFALO Challenges

8

Optimize Memory Usage• Goal: Minimize overall false-positive rate

– The probability that one BF has a false positive• Input:

– Fast memory size M– Number of destinations per next hop– The maximum number of hash functions

• Output: the size of each Bloom filter– Larger Bloom filter for the next hop that has more

destinations

9

Optimize Memory Usage (cont.)

• Constraints– Memory constraint

• Sum of all BF sizes <= fast memory size M– Bound on number of hash functions

• To bound CPU calculation time• Bloom filters share the same hash functions

• Proved to be a convex optimization problem– An optimal solution exists– Solved by IPOPT (Interior Point OPTimizer)

10

Minimize False Positives

11

• A FIB with 200K entries, 10 next hop• 8 hash functions

Comparing with Hash Table

12

65%

• Save 65% memory with 0.1% false positive

BUFFALO Challenges

13

Handle False Positives• Design goals

– Should not modify the packet– Never go to slow memory– Ensure timely packet delivery

• BUFFALO solution – Exclude incoming interface

• Avoid loops in one false positive case– Random selection from matching next hops

• Guarantee reachability with multiple false positives

14

One False Positive• Most common case: one false positive

– When there are multiple matching next hops– Avoid sending to incoming interface

• We prove that there is at most a 2-hop loop with a stretch <= l(AB)+l(BA)

15

False positive

A

Shortest path

B

dst

Multiple False Positives• Handle multiple false positives

– Random selection from matching next hops– Random walk on shortest path tree plus a few

false positive links– To eventually find out a way to the destination

16

dst

Shortest path tree for dst

False positive link

Multiple False Positives (cont.)

• Provable stretch bound– Expected stretch with k false positive case is at most – Proved by random walk theories

• Stretch bound actually not bad– False positives are independent

• Different switches use different hash functions

– k false positives for one packet are rare– Probability of k false positives decreases

exponentially with k

17

(k 2 3k / 3)

Stretch in Campus Network

18

When fp=0.001%99.9% of the packets have

no stretch0.0003% packets have a stretch of shortest path

length

When fp=0.5%, 0.0002% packets have a stretch 6 times of shortest path

length

BUFFALO Challenges

19

Handle Routing Dynamics• Counting Bloom filters (CBF)

– CBF can handle adding/deleting elements– Take more memory than Bloom filters (BF)

• BUFFALO solution: use CBF to assist BF– CBF in slow memory to handle FIB updates– Occasionally re-optimize BF size based on CBF

20

1 0

Hard to expand to size 4

CBF BF 1 1

Easy to contract CBF to size 4

BUFFALO Switch Architecture

21

– Prototype implemented in kernel-level Click

Prototype Evaluation

• Environment– 3.0 GHz 64-bit Intel Xeon– 2 MB L2 data cache, used as fast memory size M

• Forwarding table– 10 next hops– 200K entries

22

Prototype Evaluation

• Peak forwarding rate– 365 Kpps, 1.9 μs per packet– 10% faster than hash-based solution EtherSwitch

• Performance with FIB updates– 10.7 μs to update a route– 0.47 s to reconstruct BFs based on CBFs

• On another core without disrupting packet lookup• Swap in new BFs is fast

23

Conclusion

• Improve data plane scalability in SPAF– Complementary to recent Ethernet advances

• Three nice properties of BUFFALO– Small memory requirement– Gracefully increase stretch with the growth of

forwarding table– React quickly to routing updates

24

Papers and Future Work• Paper in submission to CoNext

– www.cs.princeton.edu/~minlanyu/writeup/conext09.pdf

• Preliminary work in WREN Workshop– Use Bloom filters to reduce memory for enterprise

edge routers• Future work

– Reduce transient loops in OSPF using Random walk in shortest path tree

25

Thanks!

26

• Questions?