1 CSE 326: Data Structures Priority Queues (Heaps) Lecture 9: Monday, Jan 27, 2003.
CSE 326 Data Structures Part 6: Priority Queues, AKA Heaps
description
Transcript of CSE 326 Data Structures Part 6: Priority Queues, AKA Heaps
1
CSE 326 Data Structures
Part 6:Priority Queues,
AKA Heaps
Henry Kautz
Autumn 2002
2
Not Quite Queues
• Consider applications– ordering CPU jobs– searching for the exit in a maze– emergency room admission processing
• Problems?– short jobs should go first– most promising nodes should be searched first– most urgent cases should go first
3
Priority Queue ADT
• Priority Queue operations– create
– destroy
– insert
– deleteMin
– is_empty
• Priority Queue property: for two elements in the queue, x and y, if x has a lower priority value than y, x will be deleted before y
F(7) E(5) D(100) A(4)
B(6)
insert deleteMinG(9) C(3)
4
Applications of the Priority Q
• Hold jobs for a printer in order of length
• Store packets on network routers in order of urgency
• Simulate events
• Anything greedy
5
Discrete Event Simulation
• An event is a pair (x,t) where x describes the event and t is time it should occur
• A discrete event simulator (DES) maintains a set S of events which it intends to simulate in time order
repeat {Find and remove (x0,t0) from S such that t0 is minimum;Do whatever x0 says to do, in the process new events (x2,t2)…(xk,tk) may be generated;Insert the new events into S;
}
6
Emergency Room Simulation • Patient arrive at time t with injury of criticality C
– If no patients waiting and a free doctor, assign them to doctor and create a future departure event; else put patient in the Criticality priority queue
• Patient departs at time t– If someone in Criticality queue, pull out most critical and
assign to doctor; create a future departure event
timequeue
arrive(t,c)criticality(triage)queueassign
patient todoctor
patientgenerator depart(t)
depart(t)
arrive(t,c)
7
Naïve Priority Queue Data Structures
• Unsorted list:– insert:
– deleteMin:
• Sorted list:– insert:
– deleteMin:
8
BST Tree Priority Queue Data Structure
4
121062
115
8
14137 9
•Regular BST:–insert:
–deleteMin:
•AVL Tree:–insert:
–deleteMin:Can we do better?
9
Binary Heap Priority Q Data Structure
201412911
81067
54
2
• Heap-order property– parent’s key is less than
children’s keys
– result: minimum is always at the top
• Structure property– complete tree with fringe
nodes packed to the left
– result: depth is always O(log n); next open location always known
How do we find the minimum?
10
201412911
81067
54
2
2 4 5 7 6 10 8 11 9 12 14 2012
1 2 3 4 5 6 7 8 9 10 11 12
1
2 3
4 5 6 7
8 9
10 11 12
Nifty Storage Trick• Calculations:
– child:
– parent:
– root:
– next free:
0
11
201412911
81067
54
2
2 4 5 7 6 10 8 11 9 12 14 2012
1 2 3 4 5 6 7 8 9 10 11 12
1
2 3
4 5 6 7
8 9
10 11 12
Nifty Storage Trick• Calculations:
– child: left = 2*node right=2*node+1– parent: floor(node/2)
– root: 1
– next free: length+1
0
12
DeleteMin
201412911
81067
54
202
201412911
81067
54
2
pqueue.deleteMin()
13
Percolate Down
1412911
81067
54
20
1412911
81067
520
4
1412911
810207
56
4
1420911
810127
56
4
14
DeleteMin CodeComparable deleteMin(){
x = A[1];
A[1]=A[size--];
percolateDown(1);
return x;
}
percolateDown(int hole) { tmp=A[hole]; while (2*hole <= size) { left = 2*hole; right = left + 1; if (right <= size && A[right] < A[left]) target = right; else target = left; if (A[target] < tmp) { A[hole] = A[target]; hole = target; } else break; } A[hole] = tmp;}runtime:
Trick to avoid repeatedly copying the value at A[1]
Move down
15
Insert
201412911
81067
54
2
201412911
81067
54
2
pqueue.insert(3)
3
16
Percolate Up
201412911
81067
54
2
3 201412911
8367
54
2
10
201412911
8567
34
2
10
17
Insert Code
void insert(Comparable x) {
// Efficiency hack: we won’t actually put x
// into the heap until we’ve located the position
// it goes in. This avoids having to copy it
// repeatedly during the percolate up.
int hole = ++size;
// Percolate up
for( ; hole>1 && x < A[hole/2] ; hole = hole/2)
A[hole] = A[hole/2];
A[hole] = x;
}
runtime:
18
Performance of Binary Heap
• In practice: binary heaps much simpler to code, lower constant factor overhead
Binary heap
worst case
Binary heap avg case
AVL tree worst case
BST tree avg case
Insert O(log n) O(1)
percolates 1.6 levels
O(log n) O(log n)
Delete Min
O(log n) O(log n) O(log n) O(log n)
19
Changing Priorities
• In many applications the priority of an object in a priority queue may change over time– if a job has been sitting in the printer queue for a long
time increase its priority
– unix “renice”
• Must have some (separate) way of find the position in the queue of the object to change (e.g. a hash table)
20
Other Priority Queue Operations
• decreaseKey – Given the position of an object in the queue, increase
its priority (lower its key). Fix heap property by:
• increaseKey– given the position of an an object in the queue, decrease
its priority (increase its key). Fix heap property by:
• remove– given the position of an an object in the queue, remove
it. Do increaseKey to infinity then …
21
BuildHeap
• Task: Given a set of n keys, build a heap all at once
• Approach 1: Repeatedly perform Insert(key)
• Complexity:
22
BuildHeapFloyd’s Method
5 11 3 10 6 9 4 8 1 7 212
pretend it’s a heap and fix the heap-order property!
27184
96103
115
12buildHeap(){
for (i=size/2; i>0; i--)
percolateDown(i);
}
23
Build(this)Heap
67184
92103
115
12
671084
9213
115
12
1171084
9613
25
12
1171084
9653
21
12
24
Finally…
11710812
9654
23
1
25
Complexity of Build Heap• Note: size of a perfect binary tree doubles (+1)
with each additional layer• At most n/4 percolate down 1 level
at most n/8 percolate down 2 levelsat most n/16 percolate down 3 levels…
log
11
log
1
1 2 3 ...4 8 16 2
(2)2 2 2
n
ii
n
ii
n n n ni
n i nn
O(n)
26
Heap Sort• Input: unordered array A[1..N]
1. Build a max heap (largest element is A[1])2. For i = 1 to N-1:
A[N-i+1] = Delete_Max()
7 50 22 15 4 40 20 10 35 25
50 40 20 25 35 15 10 22 4 7
40 35 20 25 7 15 10 22 4 50
35 25 20 22 7 15 10 4 40 50
27
Properties of Heap Sort
• Worst case time complexity O(n log n)– Build_heap O(n)– n Delete_Max’s for O(n log n)
• In-place sort – only constant storage beyond the array is needed
28
Thinking about Heaps
• Observations– finding a child/parent index is a multiply/divide by two
– operations jump widely through the heap
– each operation looks at only two new nodes
– inserts are at least as common as deleteMins
• Realities– division and multiplication by powers of two are fast
– looking at one new piece of data terrible in a cache line
– with huge data sets, disk accesses dominate
29
4
9654
23
1
8 1012
7
11
Solution: d-Heaps
• Each node has d children• Still representable by array• Good choices for d:
– optimize performance based on # of inserts/removes
– choose a power of two for efficiency
– fit one set of children in a cache line
– fit one set of children on a memory page/disk block
3 7 2 8 5 12 11 10 6 9112
30
Coming Up
• Mergeable heaps– Leftist heaps– Skew heaps– Binomial queues
• Read Weiss Ch. 6
• Midterm results
31
New Operation: Merge
Merge(H1,H2): Merge two heaps H1 and H2 of size O(N). – E.g. Combine queues from two different sources to run on one
CPU.
1. Can do O(N) Insert operations: O(N log N) time
2. Better: Copy H2 at the end of H1 (assuming array implementation) and use Floyd’s Method for BuildHeap.
Running Time: O(N)
Can we do even better? (i.e. Merge in O(log N) time?)
32
Binomial Queues
• Binomial queues support all three priority queue operations Merge, Insert and DeleteMin in O(log N) time
• Idea: Maintain a collection of heap-ordered trees
– Forest of binomial trees
• Recursive Definition of Binomial Tree (based on height k):
– Only one binomial tree for a given height
– Binomial tree of height 0 = single root node
– Binomial tree of height k = Bk = Attach Bk-1 to root of another Bk-1
33
Building a Binomial Tree• To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
• Binomial tree of height k has exactly 2k nodes (by induction)B0 B1 B2 B3
34
Building a Binomial Tree• To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
• Binomial tree of height k has exactly 2k nodes (by induction)B0 B1 B2 B3
35
Building a Binomial Tree• To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
• Binomial tree of height k has exactly 2k nodes (by induction)B0 B1 B2 B3
36
Building a Binomial Tree• To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
• Binomial tree of height k has exactly 2k nodes (by induction)B0 B1 B2 B3
37
Building a Binomial Tree• To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
• Binomial tree of height k has exactly 2k nodes (by induction)B0 B1 B2 B3
38
Building a Binomial Tree• To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
• Binomial tree of height k has exactly 2k nodes (by induction)B0 B1 B2 B3
39
Building a Binomial Tree• To construct a binomial tree Bk of height k:
1. Take the binomial tree Bk-1 of height k-1
2. Place another copy of Bk-1 one level below the first
3. Attach the root nodes
• Binomial tree of height k has exactly 2k nodes (by induction)B0 B1 B2 B3
40
Why Binomial?• Why are these trees called binomial?
– Hint: how many nodes at depth d?
B0 B1 B2 B3
41
Why Binomial?• Why are these trees called binomial?
– Hint: how many nodes at depth d?Number of nodes at different depths d for Bk =
[1], [1 1], [1 2 1], [1 3 3 1], …Binomial coefficients of (a + b)k = k!/((k-d)!d!)
B0 B1 B2 B3
42
Definition of Binomial Queues
3
Binomial Queue = “forest” of heap-ordered binomial trees
1
7
-1
2 1 3
8 11 5
6
5
9 6
7
21
B0 B2 B0 B1 B3
Binomial queue H15 elements = 101 base 2 B2 B0
Binomial queue H211 elements = 1011 base 2 B3 B1 B0
43
Binomial Queue PropertiesSuppose you are given a binomial queue of N nodes
1. There is a unique set of binomial trees for N nodes
2. What is the maximum number of trees that can be in an N-node queue?
– 1 node 1 tree B0; 2 nodes 1 tree B1; 3 nodes 2 trees B0 and B1; 7 nodes 3 trees B0, B1 and B2 …
– Trees B0, B1, …, Bk can store up to 20 + 21 + … + 2k = 2k+1
– 1 nodes = N.
– Maximum is when all trees are used. So, solve for (k+1).
– Number of trees is log(N+1) = O(log N)
44
Binomial Queues: Merge
• Main Idea: Merge two binomial queues by merging individual binomial trees– Since Bk+1 is just two Bk’s attached together, merging trees
is easy
• Steps for creating new queue by merging:1. Start with Bk for smallest k in either queue.
2. If only one Bk, add Bk to new queue and go to next k.
3. Merge two Bk’s to get new Bk+1 by making larger root the child of smaller root. Go to step 2 with k = k + 1.
45
Example: Binomial Queue Merge
31
7
-1
2 1 3
8 11 5
6
5
9 6
7
21
H1: H2:
46
Example: Binomial Queue Merge
31
7
-1
2 1 3
8 11 5
6
5
9 6
7
21
H1: H2:
47
Example: Binomial Queue Merge
3
1
7
-1
2 1 3
8 11 5
6
5
9 6
721
H1: H2:
48
Example: Binomial Queue Merge
3
1
7
-1
2 1 3
8 11 5
6
5
9 6
7
21
H1: H2:
49
Example: Binomial Queue Merge
3
1
7
-1
2 1 3
8 11 5
6
5
9 6
7
21
H1: H2:
50
Example: Binomial Queue Merge
3
1
7
-1
2 1 3
8 11 5
6
5
9 6
7
21
H1: H2:
51
Binomial Queues: Merge and Insert
• What is the run time for Merge of two O(N) queues?
• How would you insert a new item into the queue?
52
Binomial Queues: Merge and Insert
• What is the run time for Merge of two O(N) queues?– O(number of trees) = O(log N)
• How would you insert a new item into the queue?– Create a single node queue B0 with new item and
merge with existing queue– Again, O(log N) time
• Example: Insert 1, 2, 3, …,7 into an empty binomial queue
53
Insert 1,2,…,7
1
54
Insert 1,2,…,7
1
2
55
Insert 1,2,…,7
1
2
3
56
Insert 1,2,…,7
1
2
3
4
57
Insert 1,2,…,7
1
2 3
4
58
Insert 1,2,…,7
1
2 3
4
5
59
Insert 1,2,…,7
1
2 3
4
5
6
60
Insert 1,2,…,7
1
2 3
4
5
6
7
61
Binomial Queues: DeleteMin
• Steps:1. Find tree Bk with the smallest root
2. Remove Bk from the queue
3. Delete root of Bk (return this value); You now have a new queue made up of the forest B0, B1, …, Bk-1
4. Merge this queue with remainder of the original (from step 2)
• Run time analysis: Step 1 is O(log N), step 2 and 3 are O(1), and step 4 is O(log N). Total time = O(log N)
• Example: Insert 1, 2, …, 7 into empty queue and DeleteMin
62
Insert 1,2,…,7
1
2 3
4
5
6
7
63
DeleteMin
2 3
4
5
6
7
64
Merge
2 3
4
5
6
7
65
Merge
2 3
4
5
6
7
66
Merge
2
3
4
5
6
7
67
Merge
2
3
4
5
6
7
DONE!
68
Implementation of Binomial Queues
• Need to be able to scan through all trees, and given two binomial queues find trees that are same size– Use array of pointers to root nodes, sorted by size– Since is only of length log(N), don’t have to worry
about cost of copying this array– At each node, keep track of the size of the (sub) tree
rooted at that node• Want to merge by just setting pointers
– Need pointer-based implementation of heaps • DeleteMin requires fast access to all subtrees of root
– Use First-Child/Next-Sibling representation of trees
69
Efficient BuildHeap for Binomial Queues
• Insert one at a time - O(n log n)• Better algorithm:
– Start with each element as a singleton tree– Merge trees of size 1– Merge trees of size 2– Merge trees of size 4
• Complexity:
log log
1 1
log
1
(1 log1) (1 log 2) (1 log 4) ...2 4 8
(1 )( )
2 2
because 22
n n
i ii i
n
ii
n n n
n i iO n O n
i
70
Other Mergeable Priority Queues: Leftist and Skew Heaps
• Leftist Heaps: Binary heap-ordered trees with left subtrees always “longer” than right subtrees– Main idea: Recursively work on right path for Merge/Insert/DeleteMin– Right path is always short has O(log N) nodes– Merge, Insert, DeleteMin all have O(log N) running time (see text)
• Skew Heaps: Self-adjusting version of leftist heaps (a la splay trees)– Do not actually keep track of path lengths– Adjust tree by swapping children during each merge– O(log N) amortized time per operation for a sequence of M operations
• See Weiss for details…
71
Coming Up
• For Wednesday: Read Weiss on Leftist Heaps and Skew Heaps
• How do they work?
• How does their complexity compare to Binomial Queues?
• Which seems easiest to implement?
• Which is likely to have lowest overhead?
72
Leftist Heaps
• An alternative heap structure that also enables fast merges
• Based on binary trees rather than k-ary trees
73
Idea: Hang a New Tree
1213106
115
2
+
1014
49
1
=
141213106
49115
12
?
10
Now, just percolate down!
74
Idea: Hang a New Tree
1213106
115
2
+
1014
49
1
=
141213106
49115
?2
1
10
Now, just percolate down!
75
Idea: Hang a New Tree
1213106
115
2
+
1014
49
1
=
141213106
?9115
42
1
10
Now, just percolate down!
76
Idea: Hang a New Tree
1213106
115
2
+
1014
49
1
=
141213106
9115
42
1
10
Now, just percolate down!
Note: we just gave up the nice structural property on binary heaps!
77
Problem?
Need some other kind of balance condition…
2+ 4
1
=
2
?
12
1515
18
4
1
1215
1815
78
Leftist Heaps
• Idea: make it so that all the work you have to do in maintaining a heap is in one small part
• Leftist heap:– almost all nodes are on the left– all the merging work is on the right
79
the null path length (npl) of a node is the number of nodes between it and a null in the tree
Random Definition:Null Path Length
• npl(null) = -1
• npl(leaf) = 0
• npl(single-child node) = 0
000
001
11
2
another way of looking at it:npl is the height of complete subtree rooted at this node
0
80
Leftist Heap Properties
• Heap-order property– parent’s priority value is to childrens’ priority
values– result: minimum element is at the root
• Leftist property– null path length of left subtree is npl of right
subtree– result: tree is at least as “heavy” on the left as the
right Are leftist trees complete? Balanced?
81
Leftist tree examples
NOT leftist leftist
00
001
11
2
0
0
000
11
2
1
000
0
0
0
0
0
1
0
leftist
0
every subtree of a leftist tree is leftist, comrade!
82
Right Path in a Leftist Tree is Short• If the right path has length
at least r, the tree has at least 2r - 1 nodes
• Proof by inductionBasis: r = 1. Tree has at least one node: 21 - 1 = 1
Inductive step: assume true for r’ < r. The right subtree has a right path of at least r - 1 nodes, so it has at least 2r - 1 - 1 nodes. The left subtree must also have a right path of at least r - 1 (otherwise, there is a null path of r - 3, less than the right subtree). Again, the left has 2r - 1 - 1 nodes. All told then, there are at least:
2r - 1 - 1 + 2r - 1 - 1 + 1 = 2r - 1
• So, a leftist tree with at least n nodes has a right path of at most log n nodes
0
000
11
2
1
00
83
Merging Two Leftist Heaps
• merge(T1,T2) returns one leftist heap containing all elements of the two (distinct) leftist heaps T1 and T2
a
L1 R1
b
L2 R2
mergeT1
T2
a < b
a
L1
recursive merge
b
L2 R2
R1
84
Merge Continued
a
L1 R’
R’ = Merge(R1, T2)
a
R’ L1
npl(R’) > npl(L1)
constant work at each merge; recursively traverse RIGHT right path of each tree; total work = O(log n)
85
Operations on Leftist Heaps• merge with two trees of total size n: O(log n)• insert with heap size n: O(log n)
– pretend node is a size 1 leftist heap– insert by merging original heap with one node heap
• deleteMin with heap size n: O(log n)– remove and return root– merge left and right subtrees
merge
merge
86
Example
1210
5
87
3
14
1
0 0
1
0 0
0
merge
7
3
14
?
0
0
1210
5
8
1
0 0
0
merge
10
5?
0 merge
12
8
0
0
8
12
0
0
87
Sewing Up the Example
8
12
0
0
10
5?
0
7
3
14
?
0
0
8
12
0
0
10
51
0
7
3
14
?
0
08
12
0
0
10
5 1
0
7
3
14
2
0
0
Done?
88
Finally…
8
12
0
0
10
5 1
0
7
3
14
2
0
0
7
3
14
2
0
08
12
0
0
10
5 1
0
89
Skew Heaps
• Problems with leftist heaps– extra storage for npl
– two pass merge (with stack!)
– extra complexity/logic to maintain and check npl
• Solution: skew heaps– blind adjusting version of leftist heaps
– amortized time for merge, insert, and deleteMin is O(log n)
– worst case time for all three is O(n)
– merge always switches children when fixing right path
– iterative method has only one pass
What do skew heaps remind us of?
90
Merging Two Skew Heaps
a
L1 R1
b
L2 R2
mergeT1
T2
a < b
a
L1
merge
b
L2 R2
R1
Notice the old switcheroo!
91
Example
1210
5
87
3
14
merge
7
3
141210
5
8
merge7
3
1410
5
8
merge12
7
3
14108
5
12
92
Skew Heap Codevoid merge(heap1, heap2) {case {
heap1 == NULL: return heap2;heap2 == NULL: return heap1;heap1.findMin() < heap2.findMin():
temp = heap1.right;heap1.right = heap1.left;heap1.left = merge(heap2, temp);return heap1;
otherwise:return merge(heap2, heap1);
}}
93
Comparing Heaps
• Binary Heaps
• d-Heaps
• Binomial Queues
• Leftist Heaps
• Skew Heaps
94
Summary of Heap ADT Analysis• Consider a heap of N nodes• Space needed: O(N)
– Actually, O(MaxSize) where MaxSize is the size of the array– Pointer-based implementation: pointers for children and parent
• Total space = 3N + 1 (3 pointers per node + 1 for size)
• FindMin: O(1) time; DeleteMin and Insert: O(log N) time• BuildHeap from N inputs: What is the run time?
– N Insert operations = O(N log N)– O(N): Treat input array as a heap
and fix it using percolate down • Thanks, Floyd!
• Mergable Heaps: Binomial Queues, Leftist Heaps, Skew Heaps