Priority Queue and Heap

1

2

Priority Queue ADT A priority queue stores

a collection of entries Each entry is a pair

(key, value) Main methods of the

Priority Queue ADT insert(k, x)

inserts an entry with key k and value x

removeMin()removes and returns the entry with smallest key

Additional methods min()

returns, but does not remove, an entry with smallest key

size(), isEmpty()

Applications: Standby flyers Auctions Stock market

3

Total Order Relations Keys in a

priority queue can be arbitrary objects on which an order is defined

Two distinct entries in a priority queue can have the same key

Mathematical concept of total order relation Reflexive property:

x x Antisymmetric

property:x y y x x = y

Transitive property: x y y z x z

4

Priority Queue Sorting

We can use a priority queue to sort a set of comparable elements1. Insert the elements one by one with a

series of insert operations2. Remove the elements in sorted order

with a series of removeMin operations The running time of this sorting

method depends on the priority queue implementation

5

Sequence-based Priority Queue

Implementation with an unsorted list

Performance: insert takes O(1) time

since we can insert the item at the beginning or end of the sequence

removeMin and min take O(n) time since we have to traverse the entire sequence to find the smallest key

Implementation with a sorted list

Performance: insert takes O(n) time

since we have to find the place where to insert the item

removeMin and min take O(1) time, since the smallest key is at the beginning

4 5 2 3 1 1 2 3 4 5

Partially Ordered Tree Partially ordered tree is a complete

binary tree in which the following two properties hold:

6

1. The nodes of the tree are arranged in a pattern as close to that of a completely symmetrical tree as possible. Thus, the number of nodes along any path in the tree can never differ by more than one. Moreover, the bottom level must be filled in a strictly left-to-right order.

2. Each node contains a key that is always less than or equal to the key in its children. Thus, the smallest key in the tree is always at the root

Let’s add a node with the key 2193.

Is this a BST? Why/Why not?

Insertion Because of the requirement that

the lowest level of the tree be filled from left to right, the tree with the new node will be like…

But this violates the second property: a key in a node should be less than or equal to the keys in its children

To fix the problem, we can exchange the keys in those nodes, and get…

What will be the complexity?

7

Insertion complexity

A newly inserted key may need to be exchanged with its parent in a cascading sequence of changes until the root!

Because of the structure of the tree, the total number of such exchanges will never be more than O(log N).

So the complexity of insertion is O(log N). 8

Finding minimum

The smallest value in the tree is always at the root.

So the complexity of finding min is O(1).

But if we need to remove the minimum (the root), then we need to do some work to arrange the nodes in the tree! 9

Removing the root (minimum key)

The standard approach is to replace the key in the root with the key

in the rightmost node from the lowest level

delete that rightmost node, then swap keys down the tree until the

ordering property is restored. For example, let’s delete the root

node from our tree example10

Removing the root (minimum key)

Replace the key in the root node with the one in the rightmost node and delete that rightmost node.

Then we will have… But this violates the second

property: a key in a node should be less than or equal to the keys in its children

To fix the problem, we can exchange the keys and get…

What will be the complexity?

11

Like insertion, deleting the rootrequires O(log N) time.

Implementation So far we just talked about its

behavior For implementation, one could use

pointer-based structures as in BSTs. But a better alternative here is to use

an array-based structure called heap

12

HeapArray-based implementation of partially ordered tree (or complete binary tree)

The nodes in a partially ordered tree of size N can be stored in the first N elements of an array

Parent and child nodes always appear at an easily computed position.

13

parent(r) = (r−1) / 2 leftchild(r) = 2 * r + 1 rightchild(r) = 2 * r + 2

14

parent(r) = (r−1) / 2 if 0 < r < Nleftchild(r) = 2 * r + 1 if 2*r + 1 < Nrightchild(r) = 2 * r + 2 if 2*r + 2 < Nleftsibling(r) = r − 1 if r is even & 0 < r < Nrightsibling(r) = r + 1 if r is odd & 0 < r < N − 1

Heap

Max vs. Min Heap The values in the heap are partially

ordered and hence we can think of a heap as either A max heap which has the property that the value

of every node is greater than or equal to the values of its children or

A min heap which has the property that the value of every node is less than or equal to the values of its children (as we considered in previous examples)

Since max heaps and min heaps are symmetrical structures, we can assume, without loss of generality, that the heap which we are considering is a max heap

15

An example of a MAX heap and its array representation

16

Making a MAX heap out of a given set of elements

17

We will visualize the structure in terms of a tree structure as well as in the array. But this is not currently ordered and therefore is not a heap, yet!

Sifting down (also called heapifying)

In order to change previous array into a heap, we will start with the leaves and basically build upwards from each leaf making certain that we have a heap below each node. If we assume that the two children of a node are

heaps, then we can swap the node value with the largest son.

That new subtree may no longer be a heap since the new root might be smaller than one of its sons.

So we repeat the process until the value has moved into a position where it is larger than its two sons.

18

A simple example of sifting down

19

We have 7 at the node and the children are heaps with max values of 10 and 11. We would first swap 7 and 11. At this point, 7 is not larger than its children so we swap 7 with its largest child which is 9. At this point this entire subtree is a heap.

Now return to our full tree example

20

Start at level 3 with the leaves. The leaves are already heaps since they have no children and hence at level 3 and below we have all heaps.

In the following figures the dashed portion of a tree will either represent a portion of the tree not yet under consideration or a node and children which do not satisfy the condition that the node value is larger than the children values and hence have to be corrected before we move up a level.

Now consider the level two nodes

21

The 5 node is less than its 11 child so we swap the 5 and 11 nodes. The 10 node is larger than its children as is the 8 node and the 6 node.

Now consider the level one nodes

22

Swaps in previous slide yield the following partial heap and we look at level 1

The 7 node is swapped with the 11 node and after this swap the 7 subtree is a heap. We swap the 3 node with the 8 node. But …..the 3 node is not larger than its children.

23

the 3 node is not larger than its children. So we now swap the 3 node and the 4 node and 3 is now the head of a valid heap (since it is a leaf ).

Swaps in previous slide yields the following partial heap. But …

Now consider the level zero node

24

Swaps in previous slide yields the following partial heap and we look at level 0 … and swap the 1 node with the 11 node.

25

Swaps in previous slide yields the following partial heap … butThe 1 node is still smaller so we next swap it with its largest child, the 10 node.

26

Swaps in previous slide yields the following partial heap … butThe 1 node is not yet the head of a heap so we swap it with the 9 node

Finally we get the heap…

27

REMOVE THE MAX ELEMENT

The size of the array will be reduced by one, so what we will do is simply switch the maximum element with the last element in the array and decrement the array size by 1, so the maximum element, though still in the array is not consider to be part of the heap.

28

Now let the new root percolate down

29

Now let the new root percolate down by swapping with its maximum child until it is bigger than its children.

Heap after removing MAX

30

The heap.h file.

Define the macros that compute the children, parents and siblings from the node.

Let r be the index of the node and n be the number of elements in the heap.

31

#define PARENT(r,n) ( 0 < (r) && (r) < (n) ? ((r)-1)/2 : -1 )#define LEFT(r,n) ( 2*(r)+1 < (n) ? 2*(r)+1 : -1 )#define RIGHT(r,n) ( 2*(r)+2 < (n) ? 2*(r)+2 : -1 )void print_heap(int [], int);void build_heap(int [], int);void siftdown(int [], int, int);int valid_heap(int [], int);

Exercise: Write the void siftdown(int heap[], int r, int n) function

Now there is a very basic operation that we have seen, the siftdown or heapify operation which given a node whose children are heaps, sifts down the tree trading the node at each level with its biggest child until it is larger than its children.

The only tricky part here is remembering to recognize when there is no child and the LEFT/RIGHT macros return -1.

32

Now building a heap is simple.

You simply start at the first node which has children, and it is easy to see that this is ⌊n/2⌋ −1

So we start there and siftdown the node value and then move to the next one.

We end up with a heap and it can be shown that this is a linear operation, Θ(n) operations.

33

void build_heap(int heap[], int n) {int r;for ( r = n/2-1; r >= 0; r-- ) {

siftdown(heap,r,n);}}

print_heap() As always we need functions to help us

debug our functions for both small and large case, so print_heap() will be helpful in small cases.

34

void print_heap(int heap[], int n) {int i;for ( i=0; i<n; i++ ) {

fprintf(stderr,"\t%d\t%d\n",i,heap[i]);

}}

valid_heap() For larger examples we definitely want a

valid_heap() function. It should be noted that we are assuming a complete binary tree but this is implicit in how we are handling the data.

35

int valid_heap(int heap[], int n) {int r, left, right; for ( r = 0; r <= n/2-1; r++ ) {

left = LEFT(r,n);right = RIGHT(r,n);if ( left != -1 && heap[r] < heap[left] )

return(0);if ( right != -1 && heap[r] < heap[right] )

return(0);}return(1);

}

Driver Program

36

#include <stdio.h>#include <limits.h>#include "heap.h"int main(int argc, char *argv[]) {

int heap[100000];int i,num;i = 0;while ( scanf("%d",&heap[i]) == 1 ) {

i++;}num = i;build_heap(heap,num);if ( valid_heap(heap,num) )

printf("Valid heap\n");else

printf("Invalid heap\n");return(0);

}

Exercise: Write the int rmmax_heap(int heap[],int *n) function, which removes and returns the maximum element, maintaining max heap properties.

37

Add a new element While we can efficiently build a heap from

an array of data items, we will frequently encounter situations where we, as the algorithm progresses, need to add additional elements to the heap.

The answer is simple enough, simply add the new value to the end of the heap and sift it up into a correct position so we need two new functions.

38

Exercise: Write void siftup(int heap[], int r, int n) function, which takes a heap (heap,n) and a value index and if the parent value is less than the new value swaps them. This continues until there is no parent or the new value is less than the parent value.

Then the actual insert heap() function will simply insert the new value after the end of the heap, incrementing the size of the heap and then siftup the new value:

39

40

void insert_heap(int data, int heap[], int *n) {

heap[(*n)++] = data;siftup(heap,(*n)-1,*n);}