Post on 19-Dec-2015
Heaps
Chapter 6 in CLRS
Motivation
• Dijkstra’s algorithm for single source shortest path
• Prim’s algorithm for minimum spanning trees
Motivation
• Want to find the shortest route from New York to San Francisco
• Model the road-map with a graph
A Graph G=(V,E)
V is a set of vertices
E is a set of edges (pairs of vertices)
Model driving distances by weights on the edges
2 19
9
1
5
13
1710
148
21
V is a set of vertices
E is a set of edges (pairs of vertices)
Source and destination
V is a set of vertices
E is a set of edges (pairs of vertices)
2 19
9
1
5
13
1710
148
21
Dijkstra’s algorithm
• Assume all weights are non-negative
• Finds the shortest path from some fixed vertex s to every other vertex
s
s15
103
1
4
2s3
s2
s4
15 8
Example
Maintain an upper bound d(v) on the shortest path to v
0
∞
∞
∞
∞
s
s15
103
1
4
2s3
s2
s4
15 8
Example
s
s15
103
1
4
2s3
s2
s4
15 8
Maintain an upper bound d(v) on the shortest path to v
0
∞
∞
∞
∞
A node is either scanned (in S) or labeled (in Q)
Initially S = and Q = V
s
s15
103
1
4
2s3
s2
s4
15 8
Pick a vertex v in Q with minimum d(v) and add it to S
0
∞
∞
∞
∞
Initially S = and Q = V
s
s15
103
1
4
2s3
s2
s4
15 8
Pick a vertex v in Q with minimum d(v) and add it to S
0
∞
∞
∞
∞
Initially S = and Q = V
v
w15
For every edge (v,w) where w in Q relax(v,w)
s
s15
103
1
4
2s3
s2
s4
15 8
Relax(v,w)
If d(v) + w(v,w) < d(w) then d(w) ← d(v) + w(v,w) π(w) ← v
0
∞
∞
∞
∞
v
w
For every edge (v,w) where w in Q relax(v,w)
15
s
s15
103
1
4
2s3
s2
s4
15 8
0
∞
∞
∞
∞
S = {s}
Relax(s,s4)
s
s15
103
1
4
2s3
s2
s4
15 8
0
∞
∞
15
∞
Relax(s,s4)
S = {s}
s
s15
103
1
4
2s3
s2
s4
15 8
0
∞
∞
15
∞
Relax(s,s4)Relax(s,s3)
S = {s}
s
s15
103
1
4
2s3
s2
s4
15 8
0
∞
∞
15
10
Relax(s,s4)Relax(s,s3)
S = {s}
s
s15
103
1
4
2s3
s2
s4
15 8
0
∞
∞
15
10
Relax(s,s4)Relax(s,s3)Relax(s,s1)
S = {s}
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
∞
15
10
Relax(s,s4)Relax(s,s3)Relax(s,s1)
S = {s}
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
∞
15
10
S = {s}
Pick a vertex v in Q with minimum d(v) and add it to S
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
∞
15
10
S = {s,s1}
Pick a vertex v in Q with minimum d(v) and add it to S
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
∞
15
10
S = {s,s1}
Pick a vertex v in Q with minimum d(v) and add it to S
Relax(s1,s3)
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
∞
15
9
S = {s,s1}
Pick a vertex v in Q with minimum d(v) and add it to S
Relax(s1,s3)
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
∞
15
9
S = {s,s1}
Pick a vertex v in Q with minimum d(v) and add it to S
Relax(s1,s3)Relax(s1,s2)
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
6
15
9
S = {s,s1}
Pick a vertex v in Q with minimum d(v) and add it to S
Relax(s1,s3)Relax(s1,s2)
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
6
15
9
S = {s,s1}
Pick a vertex v in Q with minimum d(v) and add it to S
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
6
15
9
S = {s,s1,s2}
Pick a vertex v in Q with minimum d(v) and add it to S
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
6
15
9
S = {s,s1,s2}
Pick a vertex v in Q with minimum d(v) and add it to S
Relax(s2,s3)
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
6
15
8
S = {s,s1,s2}
Pick a vertex v in Q with minimum d(v) and add it to S
Relax(s2,s3)
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
6
15
8
S = {s,s1,s2}
Pick a vertex v in Q with minimum d(v) and add it to S
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
6
15
8
S = {s,s1,s2,s3}
Pick a vertex v in Q with minimum d(v) and add it to S
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
6
15
8
S = {s,s1,s2,s3,s4}
Pick a vertex v in Q with minimum d(v) and add it to S
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
6
15
8
S = {s,s1,s2,s3,s4}
When Q = then the d() values are the distances from s
The π function gives the shortest path tree
s
s15
103
1
4
2s3
s2
s4
15 8
0
5
6
15
8
S = {s,s1,s2,s3,s4}
When Q = then the d() values are the distances from s
The π function gives the shortest path tree
Implementation of Dijkstra’s algorithm
• We need to find efficiently the vertex with minimum d() in Q
• We need to update d() values of vertices in Q
Required ADT
• Maintain items with keys subject to
• Insert(x,Q)• min(Q)• Deletemin(Q)• Decrease-key(x,Q,Δ)
Required ADT
• Insert(x,Q)• min(Q)• Deletemin(Q)• Decrease-key(x,Q,Δ): Can simulate
by Delete(x,Q), insert(x-Δ,Q)
• Insert(x,Q)• min(Q)• Deletemin(Q)• Decrease-key(x,Q,Δ): Can simulate
by Delete(x,Q), insert(x-Δ,Q)
How many times we do these operations ?
n = |V|
n
n
m = |E|
Do we know an algorithm for this ADT ?
• Insert(x,Q)• min(Q)• Deletemin(Q)• Decrease-key(x,Q,Δ): Can simulate
by Delete(x,Q), insert(x-Δ,Q)Yes! Use balanced binary search trees (RB-trees)
Heap
• Heap is A complete binary tree The items at the children of v are
greater than the one at v
Heap-ordered tree
23
7
17
3365
1924 2629
7940
• Representing a heap with an array Get the elements from top to
bottom, from left to right
Array Representation
23
7
17
3365
1924 2629
7 23 17 24 29 26 19 65 33 40 79
7940
Q
Array Representation
23
7
17
3365
1924 2629
7 23 17 24 29 26 19 65 33 40 79
7940
• Left(i): 2i+1• Right(i): 2i+2• Parent(i): (i-1)/2
Q
Find the minimum
23
7
17
3365
1924 2629
7 23 17 24 29 26 19 65 33 40 79
7940
• Return Q[0]
Q
Delete the minimum
23
7
17
3365
1924 2629
7 23 17 24 29 26 19 65 33 40 79
7940
Q
Delete the minimum
23 17
3365
1924 2629
23 17 24 29 26 19 65 33 40 79
7940
Q
Delete the minimum
23
79
17
3365
1924 2629
79 23 17 24 29 26 19 65 33 40
40
Q
Delete the minimum
23
17
79
3365
1924 2629
17 23 79 24 29 26 19 65 33 40
40
Q
Delete the minimum
23
17
19
3365
7924 2629
17 23 19 24 29 26 79 65 33 40
40
Q
Delete the minimum
23
17
19
3365
7924 2629
17 23 19 24 29 26 79 65 33 40
40
Q
Q[0] ← Q[size(Q)-1]
Heapify-down(Q,0)
size(Q) ← size(Q)-1
Heapify-down(Q,i)• Heapify-down(Q, i)• l ← left(i)• r ← right(i)• smallest ← i• if l < size(Q) and Q[l] < Q[smallest]
then smallest ← l• if r < size(Q) and Q[r] < Q[smallest]
then smallest ← r• if smallest > i then Q[i] ↔ Q[smallest]• Heapify-down(Q,
smallest)
Inserting an item
23
17
19
3365
7924 2629
17 23 19 24 29 26 79 65 33 40
40
Q
Insert(15,Q)
Inserting an item
23
17
19
3365
7924 2629
17 23 19 24 29 26 79 65 33 40 15
40
Q
Insert(15,Q)Q[size(Q)] ← 15
15
Heapify-up(Q,size(Q))size(Q) ← size(Q) + 1
Inserting an item
23
17
19
3365
7924 2615
17 23 19 24 15 26 79 65 33 40 29
40
Q
29
Inserting an item
15
17
19
3365
7924 2623
17 15 19 24 23 26 79 65 33 40 29
40
Q
29
Inserting an item
17
15
19
3365
7924 2623
15 17 19 24 23 26 79 65 33 40 29
40
Q
29
Q
Heapify-up
• Heapify-up(Q, i)• while i > 0 and Q[i] <
Q[parent(i)]• do Q[i] ↔ Q[parent(i)]• i ← parent(i)
17
15
19
3365
7924 2623
40 29
15 17 19 24 23 26 79 65 33 40 29
Other operations
• Decrease-key(x,Q,Δ)• Delete(x,Q)
• Can implement them easily using heapify
Heapsort (Williams, Floyd, 1964)
• Put the elements in an array• Make the array into a heap• Do a deletemin and put the
deleted element at the last position of the array
Put the elements in the heap
65
79
26
3323
2924 1519
79 65 26 24 19 15 29 23 33 40 7
740
Q
65
79
26
3323
2924 1519
79 65 26 24 19 15 29 23 33 40 7
740
Q
Make the elements into a heap
Make the elements into a heap
65
79
26
3323
2924 1519
79 65 26 24 19 15 29 23 33 40 7
740
Q
Heapify-down(Q,4)
65
79
26
3323
2924 157
79 65 26 24 7 15 29 23 33 40 19
1940
Q
Heapify-down(Q,4)
65
79
26
3323
2924 157
79 65 26 24 7 15 29 23 33 40 19
1940
Q
Heapify-down(Q,3)
65
79
26
3324
2923 157
79 65 26 23 7 15 29 24 33 40 19
1940
Q
Heapify-down(Q,3)
65
79
26
3324
2923 157
79 65 26 23 7 15 29 24 33 40 19
1940
Q
Heapify-down(Q,2)
65
79
15
3324
2923 267
79 65 15 23 7 26 29 24 33 40 19
1940
Q
Heapify-down(Q,2)
65
79
15
3324
2923 267
79 65 15 23 7 26 29 24 33 40 19
1940
Q
Heapify-down(Q,1)
7
79
15
3324
2923 2665
79 7 15 23 65 26 29 24 33 40 19
1940
Q
Heapify-down(Q,1)
7
79
15
3324
2923 2619
79 7 15 23 19 26 29 24 33 40 65
6540
Q
Heapify-down(Q,1)
7
79
15
3324
2923 2619
79 7 15 23 19 26 29 24 33 40 65
6540
Q
Heapify-down(Q,0)
79
7
15
3324
2923 2619
7 79 15 23 19 26 29 24 33 40 65
6540
Q
Heapify-down(Q,0)
19
7
15
3324
2923 2679
7 19 15 23 79 26 29 24 33 40 65
6540
Q
Heapify-down(Q,0)
19
7
15
3324
2923 2640
7 19 15 23 40 26 29 24 33 79 65
6579
Q
Heapify-down(Q,0)
23
7
15
3379
2924 2619
6540
We have at most n/2 nodes heapified at height 1
How much time does it take to build the heap this way ?
We have at most n/4 nodes heapified at height 2We have at most n/8 nodes heapified at height 3Total time =
23
7
15
3379
2924 2619
6540
We have at most n/2 nodes heapified at height 1
How much time does it take to build the heap this way ?
We have at most n/4 nodes heapified at height 2We have at most n/8 nodes heapified at height 3Total time =
Summary
• We can build the heap in linear time
• We still have to deletemin the elements one by one in order to sort that will take O(nlog(n))
An example
• Given an array of length n, find the k smallest numbers.
77