X D Use of Amortized Analysis B T PUSH(T) PUSH(B) PUSH(X ... · 5.2 Fibonacci Heaps Frank...

5.2 Fibonacci HeapsFrank Stajano Thomas Sauerwald

Lent 2015

Use of Amortized Analysis

PUSH(T)

PUSH(B)

PUSH(X)

PUSH(D)

MULTIPOP(3)

58 30 10

43 4141 33 7070 32

54 66 82 51

Amortized Analysis

Fibonacci Heaps

Finding Shortest Paths

next week

⇡ two weeks

5.1: Amortized Analysis T.S. 2

Priority Queues Overview

Operation Linked list Binary heap Binomial heap

Fibon. heap

MAKE-HEAP O(1) O(1) O(1)

INSERT O(1) O(log n) O(log n)

MINIMUM O(n) O(1) O(log n)

EXTRACT-MIN O(n) O(log n) O(log n)

O(log n)

MERGE O(n) O(n) O(log n)

DECREASE-KEY O(1) O(log n) O(log n)

DELETE O(1) O(log n) O(log n)

O(log n)

5.2: Fibonacci Heaps T.S. 2

Priority Queues Overview

Operation Linked list Binary heap Binomial heap Fibon. heap

MAKE-HEAP O(1) O(1) O(1) O(1)INSERT O(1) O(log n) O(log n) O(1)

MINIMUM O(n) O(1) O(log n) O(1)EXTRACT-MIN O(n) O(log n) O(log n) O(log n)

MERGE O(n) O(n) O(log n) O(1)DECREASE-KEY O(1) O(log n) O(log n) O(1)

DELETE O(1) O(log n) O(log n) O(log n)

Binomial Heap vs. Fibonacci Heap: Costs

Operation Binomial heap Fibonacci heap

actual cost amortized cost

MAKE-HEAP O(1) O(1)INSERT O(log n) O(1)

MINIMUM O(log n) O(1)EXTRACT-MIN O(log n) O(log n)

MERGE O(log n) O(1)DECREASE-KEY O(log n) O(1)

DELETE O(log n) O(log n)

All these cost bounds holdif n is the size of the heap.

Binomial Heap: k/2 DECREASE-KEY+ k/2 INSERT

c1 = c2 = · · · = ck = O(log n)

⇒ ∑ki=1 ci = O(k log n)

c1 = c2 = · · · = ck = O(log n)

Fibonacci Heap: k/2DECREASE-KEY + k/2 INSERT

c̃1 = c̃2 = · · · = c̃k = O(1)⇒ ∑k

i=1 ci ≤∑k

i=1 c̃i = O(k)

c1 = c2 = · · · = ck = O(log n)

c̃1 = c̃2 = · · · = c̃k = O(1)

⇒ ∑ki=1 ci ≤

∑ki=1 c̃i = O(k)

c1 = c2 = · · · = ck = O(log n)

c̃1 = c̃2 = · · · = c̃k = O(1)⇒ ∑k

i=1 ci ≤∑k

i=1 c̃i = O(k)5.2: Fibonacci Heaps T.S. 3

Actual vs. Amortized Cost

O(1)2 · O(1)

14 · O(1)

1 2 14

Potential

∑ki=1 c̃i

∑ki=1 ci

Potential ≥ 0, but should bealso as small as possible

O(1)2 · O(1)

14 · O(1)

1 2 14

Potential

∑ki=1 c̃i

∑ki=1 ci

O(1)2 · O(1)

14 · O(1)

1 2 14

Potential

∑ki=1 c̃i

∑ki=1 ci

O(1)2 · O(1)

14 · O(1)

1 2 14

Potential

∑ki=1 c̃i

∑ki=1 ci

O(1)2 · O(1)

14 · O(1)

1 2 14

Potential

∑ki=1 c̃i

∑ki=1 ci

Outline

Structure

Operations

Glimpse at the Analysis

Reminder: Binomial Heaps

Binomial Trees

B(0) B(1) B(2) B(3) B(k)

B(k − 1)

Binomial Heap is a collection of binomial trees of different orders,each of which obeys the heap property

Operations:

MERGE: Merge two binomial heaps using Binary Addition ProcedureINSERT: Add B(0) and perform a MERGEEXTRACT-MIN: Find tree with minimum key, cut it and perform a MERGEDECREASE-KEY: The same as in a binary heap

Binomial Heaps

Binomial Trees

B(0) B(1) B(2) B(3) B(k)

B(k − 1)

Binomial Heap is a collection of binomial trees of different orders,each of which obeys the heap propertyOperations:

Binomial Heaps

Binomial Trees

B(0) B(1) B(2) B(3) B(k)

B(k − 1)

Binomial Heap is a collection of binomial trees of different orders,each of which obeys the heap propertyOperations:

Binomial Heaps

Merging two Binomial Heaps

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

4 9 11

12 17 13

4 9 11

12 17 13

0 0 1 1 1 = 70 1 0 1 1 = 111 1 1 1

1 0 0 1 0 = 18

Binomial Heap vs. Fibonacci Heap: Structure

Binomial Heap:consists of binomial trees, and every order appears at most onceimmediately tidy up after INSERT or MERGE

50 26 22

37 30 48

Fibonacci Heap:forest of MIN-HEAPslazily defer tidying up; do it on-the-fly when search for the MIN

37 22 5

50 38 26 19 7

Binomial Heap vs. Fibonacci Heap: Structure

Binomial Heap:consists of binomial trees, and every order appears at most onceimmediately tidy up after INSERT or MERGE

50 26 22

37 30 48

Fibonacci Heap:forest of MIN-HEAPslazily defer tidying up; do it on-the-fly when search for the MIN

37 22 5

50 38 26 19 7

Structure of Fibonacci Heaps

Forest of MIN-HEAPs

Nodes can be marked (roots are always unmarked)

Tree roots are stored in a circular, doubly-linked list

Min-Pointer pointing to the smallest element

Fibonacci Heap

58 30 10

54 66 82 51

How do we implement a Fibonacci Heap?

Forest of MIN-HEAPs

Fibonacci Heap

58 30 10

54 66 82 51

Forest of MIN-HEAPs

Fibonacci Heap

58 30 10

54 66 82 51

Forest of MIN-HEAPs

Fibonacci Heap

58 30 10

54 66 82 51

Forest of MIN-HEAPs

Fibonacci Heap

58 30 10

54 66 82 51

Forest of MIN-HEAPs

Fibonacci Heap

58 30 10

54 66 82 51

Forest of MIN-HEAPs

Fibonacci Heap

58 30 10

54 66 82 51

Forest of MIN-HEAPs

Fibonacci Heap

58 30 10

54 66 82 51

Forest of MIN-HEAPs

Fibonacci Heap

58 30 10

54 66 82 51

A single Node

payload

marked

degreeb f

Previous Sibling Next Sibling

Parent

One of the Children

Magnifying a Four-Node Portion

30 0 3

43 0 0 41 1 2 33 0 1

58 30 10

43 41 33 70 32

54 66 82 51

30 0 3

43 0 0 41 1 2 33 0 1

58 30 10

43 41 33 70 32

54 66 82 51

30 0 3

43 0 0 41 1 2 33 0 1

58 30 10

43 41 33 70 32

54 66 82 51

Outline

Structure

Operations

Fibonacci Heap: INSERT

Create a singleton tree

Add to root list

and update min-pointer (if necessary)

INSERT

17 24 23 7 3

30 26 46

18 52 41

Actual Costs: O(1)

Add to root list

INSERT

17 24 23 7 3

30 26 46

18 52 41

Actual Costs: O(1)

Add to root list

INSERT

17 24 23 7 3

30 26 46

18 52 41

Actual Costs: O(1)

Add to root list

INSERT

17 24 23 7 3

30 26 46

18 52 41

Actual Costs: O(1)

Add to root list and update min-pointer (if necessary)

INSERT

17 24 23 7 3

30 26 46

18 52 41

Actual Costs: O(1)

Add to root list and update min-pointer (if necessary)

INSERT

17 24 23 7 3

30 26 46

18 52 41

Actual Costs: O(1)

Fibonacci Heap: EXTRACT-MIN

Delete min

Meld childen into root list and unmark them

Consolidate so that no roots have the same degree

(# children) X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree=2 degree=01 2 0 0 1 0 1

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Every root becomes child of another root at most once!d(n) is the maximum degree of aroot in any Fibonacci heap of size n

Delete min

(# children) X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

(# children) X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

(# children) X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

(# children) X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

(# children) X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Meld childen into root list and unmark them X

(# children) X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

(# children) X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Consolidate so that no roots have the same degree (# children)

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree=2

degree=01 2 0 0 1 0 1

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree=2

degree=0

1 2 0 0 1 0 1

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree=2 degree=0

1 2 0 0 1 0 1

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

7 23 17

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Consolidate so that no roots have the same degree (# children) X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum X

EXTRACT-MIN

18 52 41

degree0 1 2 3

Actual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum X

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Delete min X

Update minimum X

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs:

O(trees(H) + d(n))

Every root becomes child of another root at most once!

d(n) is the maximum degree of aroot in any Fibonacci heap of size n

Delete min X

Update minimum X

EXTRACT-MIN

18 52 41

degree0 1 2 3

minActual Costs: O(trees(H) + d(n))

Every root becomes child of another root at most once!

d(n) is the maximum degree of aroot in any Fibonacci heap of size n

Fibonacci Heap: DECREASE-KEY (First Attempt)

Decrease the key of x (given by a pointer)

Check if heap-order is violated

If not

, then done.

Otherwise,

cut tree rooted at x and meld into root list (update min).

DECREASE-KEY of node x

24 17 23

26 46 30

Wide andshallow tree

Degree = 3,Nodes = 4

Peculiar Constraint: Make sure that each non-rootnode loses at most one child before becoming root

If not

, then done.

Otherwise,

24 17 23

26 46 30

If not

, then done.

Otherwise,

24 17 23

26 46 30

Decrease the key of x (given by a pointer)Check if heap-order is violated

If not

, then done.

Otherwise,

26 46 30

If not

, then done.

Otherwise,

26 46 30

If not

, then done.Otherwise,

26 46 30

If not, then done.

Otherwise,

26 46 30

If not, then done.Otherwise,

26 46 30

If not, then done.Otherwise, cut tree rooted at x and meld into root list (update min).

26 46 30

Fibonacci Heap: DECREASE-KEY

(Here we consider only cases where heap-order is violated)

⇒ Cut tree rooted at x , unmark x , meld into root list

Check if parent node is marked

If unmarked, mark it (unless it is a root)If marked,

unmark and meld it into root list and recurse (Cascading Cut)

24 17 23

26 46 30

5 26 247

minActual Cost:

O(# cuts)

1. DECREASE-KEY 46 15

24 17 23

26 46 30

5 26 247

minActual Cost:

O(# cuts)

and:Check if parent node is marked

24 17 23

26 46 30

5 26 247

minActual Cost:

O(# cuts)

24 17 23

5 26 247

minActual Cost:

O(# cuts)

24 17 23

5 26 247

minActual Cost:

O(# cuts)

24 17 23

5 26 247

minActual Cost:

O(# cuts)

⇒ Cut tree rooted at x , unmark x , meld into root list and:

24 17 23

5 26 247

minActual Cost:

O(# cuts)

⇒ Cut tree rooted at x , unmark x , meld into root list and:Check if parent node is marked

24 17 23

5 26 247

minActual Cost:

O(# cuts)

If unmarked, mark it (unless it is a root)

If marked,

24 17 23

5 26 247

minActual Cost:

O(# cuts)

If marked,

24 17 23

5 26 247

minActual Cost:

O(# cuts)

If marked,

24 17 23

5 26 247

minActual Cost:

O(# cuts)

1. DECREASE-KEY 46 15 X

If marked,

24 17 23

5 26 247

minActual Cost:

O(# cuts)

If marked,

24 17 23

5 26 247

minActual Cost:

O(# cuts)

If marked,

24 17 23

5 26 247

minActual Cost:

O(# cuts)

If marked,

24 17 23

5 26 247

minActual Cost:

O(# cuts)

If marked,

24 17 23

5 26 247

minActual Cost:

O(# cuts)

If marked,

24 17 23

26 247

minActual Cost:

O(# cuts)

24 17 23

26 247

Actual Cost:

O(# cuts)

If unmarked, mark it (unless it is a root)If marked, unmark and meld it into root list and recurse (Cascading Cut)

24 17 23

26 247

Actual Cost:

O(# cuts)

24 17 23

Actual Cost:

O(# cuts)

24 17 23

Actual Cost:

O(# cuts)

24 17 23

Actual Cost:

O(# cuts)

24 17 23

5 26 24

Actual Cost:

O(# cuts)

24 17 23

5 26 24

Actual Cost:

O(# cuts)

5 26 247

Actual Cost:

O(# cuts)

5 26 247

Actual Cost:

O(# cuts)

5 26 247

minActual Cost:

O(# cuts)

5 26 247

minActual Cost: O(# cuts)

Outline

Structure

Operations

Amortized Analysis via Potential Method

INSERT: actual O(1)

amortized O(1)

EXTRACT-MIN: actual O(trees(H) + d(n))

amortized O(d(n))

DECREASE-KEY: actual O(# cuts) ≤ O(marks(H))

amortized O(1)

Φ(H) = trees(H)+2·marks(H)

24 17 23

26 46 30

Lifecycle of a node

child C

onsolidate

Loses first child

INSERT: actual O(1)

amortized O(1)

amortized O(d(n))

amortized O(1)

24 17 23

26 46 30

Lifecycle of a node

child C

onsolidate

Loses first child

INSERT: actual O(1)

amortized O(1)

amortized O(d(n))

amortized O(1)

24 17 23

26 46 30

Lifecycle of a node

child C

onsolidate

Loses first child

INSERT: actual O(1)

amortized O(1)

amortized O(d(n))

amortized O(1)

24 17 23

26 46 30

Lifecycle of a node

child C

onsolidate

Loses first child

INSERT: actual O(1)

amortized O(1)

amortized O(d(n))

amortized O(1)

24 17 23

26 46 30

Lifecycle of a node

child C

onsolidate

Loses first child

INSERT: actual O(1) amortized O(1)EXTRACT-MIN: actual O(trees(H) + d(n)) amortized O(d(n))DECREASE-KEY: actual O(# cuts) ≤ O(marks(H)) amortized O(1)

24 17 23

26 46 30

Lifecycle of a node

ild Consolidate

Loses first child

X D Use of Amortized Analysis B T PUSH(T) PUSH(B) PUSH(X ... · 5.2 Fibonacci Heaps Frank...

Documents

Transcript of X D Use of Amortized Analysis B T PUSH(T) PUSH(B) PUSH(X ... · 5.2 Fibonacci Heaps Frank...