Bounds on the Independence Required for Cuckoo Hashingcseweb.ucsd.edu/~dakane/CuckooTalk.pdfBounds...

Bounds on the Independence Required for CuckooHashing

Daniel Kane

Department of MathematicsStanford University

[email protected]

April 18th, 2013

Joint work with Jeffrey Cohen.

D. Kane (Stanford) Cuckoo Independence April 2013 1 / 30

Dictionary Data Structures

Problem: Given a universe U = 1, 2, . . . ,N, and a subset S ⊆ U, with|S | = n, build a data structure that given x ∈ U can determine if x ∈ S .

Dynamic dictionary data structure also supports adding or removingelements from S .

Dictionary Data Structures

Problem: Given a universe U = 1, 2, . . . ,N, and a subset S ⊆ U, with|S | = n, build a data structure that given x ∈ U can determine if x ∈ S .

Dynamic dictionary data structure also supports adding or removingelements from S .

Hashing

Basic solution:

Create an array A of size m > n.

Find a hash function (i.e. a deterministically computable functionthat behaves like a random function) h : U → [m].

For x ∈ S store x in A[h(x)].

Query for x by checking if x = A[h(x)].

Hashing

Basic solution:

Hashing

Basic solution:

Hashing

Basic solution:

Collisions

Problem: What if h(x) = h(y)? Cannot store two values at A[h(x)]. Thisis called a collision between x and y under h.

Solution:

Randomness of h ensures not too many collisions

When you do have collisions do one of:I Bucket for extra values.I Secondary hashing.I Have multiple possible locations.

Collisions

Solution:

Collisions

Solution:

When you do have collisions do one of:I Bucket for extra values.

I Secondary hashing.I Have multiple possible locations.

Collisions

Solution:

When you do have collisions do one of:I Bucket for extra values.I Secondary hashing.

I Have multiple possible locations.

Collisions

Solution:

Cuckoo Hashing

Cuckoo Hashing is a dynamic dictionary data structure developed by Paghand Rodler in 2001.

Two hash functions h1, h2.

Two arrays A1,A2 of length m = 4n.

Store x in either A1[h1(x)] or A2[h2(x)].

Cuckoo Hashing

Queries and Removal

To determine if x ∈ S :

See if x is either A1[h1(x)] or A2[h2(x)].

To remove x from S :

If x = A1[h1(x)], set A1[h1(x)] to 0.

If x = A2[h2(x)], set A2[h2(x)] to 0.

Queries and Removal

To determine if x ∈ S :

See if x is either A1[h1(x)] or A2[h2(x)].

To remove x from S :

If x = A1[h1(x)], set A1[h1(x)] to 0.

If x = A2[h2(x)], set A2[h2(x)] to 0.

Insertions

To add x to S :

Let x = x1

If A1[h1(x1)] is 0, set it to x1.

Insertions

If A1[h1(x1)] 6= 0, let x2 = A1[h1(x1)], set A1[h1(x1)] to x1.

If A2[h2(x2)] is 0, set it to x2.

Otherwise, let x3 = A2[h2(x2)], set A2[h2(x2)] to x2.

Insertions

Generally trying to place xi .

If Ai [hi (xi )] = 0, set it to xi .

Otherwise, xi+1 = Ai [hi (xi )], set Ai [hi (xi )] to xi try to place xi+1.

Insertions

Sometimes problems will crop up if you hit a cycle.

If chain take more than log(n) replacements, pick new hash functionsand try again (known as rehashing).

Analysis

To have a chain of k − 1 replacements need:

x1, . . . , xk

h1(x1) = h1(x2), h2(x2) = h2(x3), . . .

Expected number of such chains is nk

mk−1 < n41−k .

k#chains of length k] = O(n).

Probably none of length more than log(n).

Also have trouble if there’s a loop:

x1, . . . , xk

h1(x1) = h1(x2), h2(x2) = h2(x3), . . . , hi (xk) = hi (x1)

mk < 4−k .

Expected number of loops is less than 1/3.

Analysis

x1, . . . , xk

h1(x1) = h1(x2), h2(x2) = h2(x3), . . .

mk−1 < n41−k .

x1, . . . , xk

mk < 4−k .

Analysis

x1, . . . , xk

h1(x1) = h1(x2), h2(x2) = h2(x3), . . .

mk−1 < n41−k .

x1, . . . , xk

mk < 4−k .

Analysis

x1, . . . , xk

h1(x1) = h1(x2), h2(x2) = h2(x3), . . .

mk−1 < n41−k .

x1, . . . , xk

mk < 4−k .

Analysis

x1, . . . , xk

h1(x1) = h1(x2), h2(x2) = h2(x3), . . .

mk−1 < n41−k .

x1, . . . , xk

mk < 4−k .

Analysis

Expected cost of n insertions is O(n).

Rehash expected O(1) times on n inputs. Cost O(n) per rehash.

Tighter analysis shows probability of rehash is O(1/n).

Theorem

Cuckoo Hashing uses O(n log(N)) space, performs queries and removals inworst case O(1) time, and insertions in expected, amortized O(1) time.

Analysis

Theorem

Analysis

Theorem

k-Independence

One problem with this construction is that you cannot use truly randomhash functions.

One common substitute is the following:

Definition

A k-Independent family of hash functions, is a probability distribution overfunctions h : U → [m] so that for every x1, . . . , xk ∈ U andy1, . . . , yk ∈ [m],

Prh(h(xi ) = yi , 1 ≤ i ≤ k) =1

Simple constructions to sample from k-independent families.

Analysis of Cuckoo Hashing only requires that h1, h2 be chosen fromlog(n)-independent families.

k-Independence

One problem with this construction is that you cannot use truly randomhash functions. One common substitute is the following:

Definition

Prh(h(xi ) = yi , 1 ≤ i ≤ k) =1

k-Independence

One problem with this construction is that you cannot use truly randomhash functions. One common substitute is the following:

Definition

Prh(h(xi ) = yi , 1 ≤ i ≤ k) =1

Required Independence

Unfortunately, using k-independent families for large k is impractical(common constructions require O(k) time per evaluation).

How much independence is actually required in order for Cuckoo Hashingto satisfy it’s standard guarantees?

k = log(n) is enough.

Is k = O(1) enough?

What about k = 2?

Unfortunately, using k-independent families for large k is impractical(common constructions require O(k) time per evaluation).How much independence is actually required in order for Cuckoo Hashingto satisfy it’s standard guarantees?

Is k = O(1) enough?

What about k = 2?

Is k = O(1) enough?

What about k = 2?

Is k = O(1) enough?

What about k = 2?

5-Independence is not Enough

Theorem (Cohen, K.)

For S ⊂ U with |S | = n, and m n26/25, there exists a 5-independentfamily of hash functions h : U → [m], so that for h1 and h2 pickedindependently from this family, Cuckoo Hashing is forced to rehash on Swith high probability.

Collision Graphs

Assume S = U.

After picking h1, h2 define the collision graph, G .

V (G ) = S .

(x , y) ∈ E (G ) if and only if h1(x) = h1(y) or h2(x) = h2(y).

Forced to rehash if G has pair of connected cycles

Collision Graphs

Assume S = U.After picking h1, h2 define the collision graph, G .

V (G ) = S .

Collision Graphs

Assume S = U.After picking h1, h2 define the collision graph, G .

V (G ) = S .

Value-Oblivious Families

We only care about the collisions caused by a function, not the values.

Definition

We say that a family of hash functions h : U → [m] is value-oblivious if forevery π ∈ Sm, the family selects h with the same probability that it selectsπ h.

Nice criterion for k-independence:

Proposition

Given a value-oblivious family of hash functions h : U → [m], the family isk independent if and only if for all x1, . . . , xk ∈ U and any equivalencerelation ∼ on xi with c equivalence classes

Prh(h(xi ) = h(xj), for all xi ∼ xj) =1

mk−c.

Definition

Proposition

mk−c.

Definition

Proposition

mk−c.

Definition

Proposition

mk−c.

2-Independent Construction

Let S = V = F`3

Pick h by:I Pick v ∈ VI For each line L = w + tv parallel to v have h collide the elements of

L with probability pI No other collisionsI For each class of colliding elements, assign a random distinct value of

Check 2-independence:

Probability that h(x) = h(y).

Probability 2/n that x − y is parallel to v .

Probability p that they collide if so.