Binary heaps and Heapsort. Priority Queues n The smallest element is removed item = remove();item =...

Binary heaps and HeapsortBinary heaps and Heapsort

Priority QueuesPriority Queues

The The smallestsmallest element is removed element is removed• item = remove(); item = remove();

17 24 81 12 5 7 17 12 8

Priority Queue

Minimum element is 5


The The smallestsmallest element is removed element is removed• item = remove(); item = remove();

17 24 81 12 7 17 12 8

Priority Queue



Additions made whereverAdditions made wherever• add(15); add(15);

Priority Queue


17 24 81 12 17 12 817 24 81 12 17 12 8 15

List RepresentationsList Representations

Represent P.Q. as a sorted listRepresent P.Q. as a sorted list• insert item into the appropriate position = O(N)insert item into the appropriate position = O(N)• remove item at one end = O(1)remove item at one end = O(1)

Represent P.Q. as an unsorted listRepresent P.Q. as an unsorted list• insert at one end of list = O(1)insert at one end of list = O(1)• remove by finding minimum & deleting = O(N)remove by finding minimum & deleting = O(N)

Which to prefer?Which to prefer?

List RepresentationsList Representations

Sorted RepSorted Rep

• Insert 14Insert 14

• Remove-HighestRemove-Highest

• Remove-HighestRemove-Highest

Unsorted RepUnsorted Rep17 24 81 12 5 7 17 12 85 7 8 12 12 17 17 24 81

17 24 81 12 5 7 17 12 8 145 7 8 12 12 14 17 17 24 81

17 24 81 12 7 17 12 8 147 8 12 12 14 17 17 24 81

17 24 81 12 17 12 8 148 12 12 14 17 17 24 81

A Simpler RepresentationA Simpler Representation

Keep people semi-sorted by importanceKeep people semi-sorted by importance• most important person right at frontmost important person right at front• people near front are smallerpeople near front are smaller• people near back are biggerpeople near back are bigger• each person can have two people behindeach person can have two people behind

» one to left, one to rightone to left, one to right

People arranged in rows:People arranged in rows:• 1 in 11 in 1stst row, 2 in 2 row, 2 in 2ndnd row, 4 in 3 row, 4 in 3rdrd, 8 in 4, 8 in 4thth, …, …

A “Heap” of PeopleA “Heap” of People

Most important: 1Most important: 1stst row row• 22ndnd row less important than 1 row less important than 1stst

• 33rdrd row less important than 2 row less important than 2ndnd

• and so onand so on Each person can have 2Each person can have 2

more people behindmore people behind• the two people behind are less important the two people behind are less important

(bigger) than the person in front of them(bigger) than the person in front of them

Heap Order PropertyHeap Order Property

Highest priority item at rootHighest priority item at root• here that’s the 5here that’s the 5

Left & right children alsoLeft & right children alsoheap orderedheap ordered• highest priority in highest priority in that that sub-tree at its rootsub-tree at its root• here: 7 on left & 12 on righthere: 7 on left & 12 on right

Note: 5’s children are Note: 5’s children are notnot next two smallest next two smallest• they they could have beencould have been, but aren’t necessarily, but aren’t necessarily

5

127

881 24 17

Duplicate EntriesDuplicate Entries

Heaps allow duplicatesHeaps allow duplicates AA highest priority item highest priority item

appears at rootappears at root• others of same others of same

priority will be toppriority will be topin their sub-treein their sub-tree

• duplicates may also appear in separate sub-treesduplicates may also appear in separate sub-trees

1

51

18 8 14

1415 14

Starting the PQStarting the PQ

First person goes to front of “heap”First person goes to front of “heap” Second person arrivesSecond person arrives

• if smaller, goes to frontif smaller, goes to front» first person moves back to the first person moves back to the leftleft

• otherwise, new person goes to back and otherwise, new person goes to back and leftleft Third person arrivesThird person arrives

• if smaller than FRONT person, goes to frontif smaller than FRONT person, goes to front» front person goes back and to the front person goes back and to the rightright

• otherwise, new person goes to back and otherwise, new person goes to back and rightright

Growing the PQGrowing the PQ

As people arrive, they start in the back rowAs people arrive, they start in the back row• as far left in that row as they can goas far left in that row as they can go• if smaller than person immediately in front, the if smaller than person immediately in front, the

two exchange positionstwo exchange positions• new person continues pushing forward until new person continues pushing forward until

runs into someone smaller (more important)runs into someone smaller (more important)» or until reach front of PQor until reach front of PQ

New Person Joins the HeapNew Person Joins the Heap

Insert requires adding a new element to the Insert requires adding a new element to the array & making the array back into a heaparray & making the array back into a heap• small (high priority) itemssmall (high priority) items

must “percolate up”must “percolate up”• stop when parentstop when parent

is smaller…is smaller…• or reach rootor reach root

2

94

137 11 20

2318 15 14 12 16 8

8

20

9

88

Storing Heaps in ArraysStoring Heaps in Arrays

Heap can be stored compactlyHeap can be stored compactlyin an arrayin an array• do a “level order” traversal do a “level order” traversal

of the tree into the arrayof the tree into the array Left child of Left child of ii is in 2 is in 2ii Right child of Right child of ii is in 2 is in 2ii+1+1 Parent of Parent of ii is in is in ii/2/2 Leave location 0 empty!Leave location 0 empty!

5

127

881 24 17

5

127

881 24 17

5 127 881 24 17

1 2 3 4 5 6 7

Note: array index starts at 1

ExerciseExercise

Store the following heap into an arrayStore the following heap into an array

1

52

38 19 12

1415 20

ExerciseExercise

Draw the complete “heap” represented by Draw the complete “heap” represented by the following array:the following array:• [2, 4, 9, 7, 13, 11, 20, 18, 23, 15, 14, 12, 16, 19][2, 4, 9, 7, 13, 11, 20, 18, 23, 15, 14, 12, 16, 19]

Is it actually a heap?Is it actually a heap?• if not, can you make it into a heap?if not, can you make it into a heap?

Insert Insert itemitem into Heap into Heap

1.1. Let N be the next available position in the Let N be the next available position in the arrayarray

2.2. While N > 1 & A[N/2] > itemWhile N > 1 & A[N/2] > itema)a) set A[N] to A[N/2]set A[N] to A[N/2]b)b) set N to N/2set N to N/2

3.3. Set A[N] to the new itemSet A[N] to the new item

4.4. Add 1 to size of heapAdd 1 to size of heap

Insertion ExampleInsertion Example

Insert 6 into this heapInsert 6 into this heap

N = 8N = 8

5

127

881 24 17

5 127 881 24 17

1 2 3 4 5 6 7



N N 8 8 A[N/2] = A[4] = 81A[N/2] = A[4] = 81 81 > 6 81 > 6 percolate up percolate up

6

5

127

881 24 17

5 127 881 24 17

1 2 3 4 5 6 7 8

?



N = 4N = 4 A[N/2] = A[2] = 7A[N/2] = A[2] = 7 7 > 6 7 > 6 percolate up percolate up

81

5

127

86 24 17

5 127 881 24 17

1 2 3 4 5 6 7 8

81



N = 2N = 2 A[N/2] = A[1] = 5A[N/2] = A[1] = 5 5 5 6 6 percolating done percolating done Set A[N] = A[2] to 6Set A[N] = A[2] to 6

81

5

126

87 24 17

5 127 87 24 17

1 2 3 4 5 6 7 8

81

6

6

ExerciseExercise

Continuing the example:Continuing the example:• insert 14insert 14• insert 7insert 7• insert 4insert 4

81

5

126

87 24 17

5 126 87 24 17

1 2 3 4 5 6 7 8

81

Removing from a HeapRemoving from a Heap

Front person leaves heapFront person leaves heap How shall we fill the empty space?How shall we fill the empty space?

• NOT by pushing forward into the open spaceNOT by pushing forward into the open space• thatthat might might ruinruin our nice heap our nice heap

Instead we put the Instead we put the lastlast person in the front(!) person in the front(!) ThenThen people start pushing forward people start pushing forward

• starting with the smaller of the people in the starting with the smaller of the people in the second row of the heapsecond row of the heap

Deletion from HeapDeletion from Heap

Remove the rootRemove the root Promote last item to rootPromote last item to root Let it percolate downLet it percolate down

81

4

125

67 24 17

4 125 67 24 17

1 2 3 4 5 6 7 8 9 10 11

81

14 8 7

14 8 7

4



• choose smaller of twochoose smaller of twochildrenchildren

81

7

125

67 24 17

7 125 67 24 17

1 2 3 4 5 6 7 8 9 10

81

14 8

14 8 7




81

5

127

67 24 17

5 127 67 24 17

1 2 3 4 5 6 7 8 9 10

81

14 8

14 8 7




• (if it has two children)(if it has two children)81

5

126

77 24 17

5 126 77 24 17

1 2 3 4 5 6 7 8 9 10

81

14 8

14 8 7

Percolating DownPercolating Down

Check number of childrenCheck number of children• if none – then we are done percolatingif none – then we are done percolating• if one – it is the smallest childif one – it is the smallest child• if two – choose whichever is smallerif two – choose whichever is smaller

If smaller child is smaller than percolatorIf smaller child is smaller than percolator• move the child upmove the child up• continue percolating from the child’s positioncontinue percolating from the child’s position

Percolating DownPercolating Down

1.1. Let P be the last position in the heapLet P be the last position in the heap2.2. Let N be 1Let N be 13.3. If 2N<P (* has surviving children *)If 2N<P (* has surviving children *)

a)a) If 2N+1=P (* one child *), let S be 2NIf 2N+1=P (* one child *), let S be 2Nb)b) Else let S be (A[2N]<A[2N+1]) ? 2N : 2N+1Else let S be (A[2N]<A[2N+1]) ? 2N : 2N+1c)c) If (A[P]>A[S])If (A[P]>A[S])

i.i. let A[N] be A[S], N be S & go to step 3let A[N] be A[S], N be S & go to step 3

4.4. Let A[N] be A[P]Let A[N] be A[P]

NOTE: P is the position we are deleting from

Deleting from the HeapDeleting from the Heap

1.1. Set Set temptemp to A[1] to A[1]

2.2. Percolate DownPercolate Down

// from previous slide// from previous slide

3.3. Reduce the size of the heap by 1Reduce the size of the heap by 1

4.4. Return Return temptemp

ExerciseExercise

Delete from this heapDelete from this heap Delete again from this heapDelete again from this heap And againAnd again

81

5

126

177 24 24

5 126 177 24 24

1 2 3 4 5 6 7 8 9 10

81

14 18

14 18

Our Heap is Now a PQOur Heap is Now a PQ

remove method always gets smallest itemremove method always gets smallest item• and leaves next smallest item in its placeand leaves next smallest item in its place

add method inserts new item anywhereadd method inserts new item anywhere• but moves it up to front if it’s smallestbut moves it up to front if it’s smallest

We can add convenience methodsWe can add convenience methods• look at front, check if empty, make empty, …look at front, check if empty, make empty, …

17

17

““Almost” Heap SortAlmost” Heap Sort

Can sort an array using our heap/PQCan sort an array using our heap/PQ• add each element of array to heapadd each element of array to heap• remove items from heap back into the arrayremove items from heap back into the array

But we can do better!But we can do better!• we can do it all in the arraywe can do it all in the array• turn the array into a heapturn the array into a heap

5

127

81 24

5 1217 818 24 75 87 1712 24 81

8

12

7

1217

24

8

81

1217

2481

2481

Random Array to HeapRandom Array to Heap

Need to get smaller items up &Need to get smaller items up &bigger items downbigger items down• percolate up or downpercolate up or down

Need to be sure we getNeed to be sure we geteverything to its leveleverything to its level• only items withonly items with

children matterchildren matter

14

29

1319

820 9 22

29 1319 820 9 22

1 2 3 4 5 6 7 8 9 10

14

13 14

13 14

have children have no children


Start with the last item withStart with the last item withchildrenchildren

Percolate it downPercolate it down Move on to the previousMove on to the previous

nodenode• stop at the rootstop at the root

14

29

1319

820 9 22

29 1319 820 9 22

1 2 3 4 5 6 7 8 9 10

14

13 14

13 14



Item 5 is an 8Item 5 is an 8• its one child (item 10) is a 14its one child (item 10) is a 14• it’s OKit’s OK

Item 4 is a 20Item 4 is a 20• its smaller child its smaller child

(item 9) is a 13(item 9) is a 13• swap themswap them• now OKnow OK

14

29

1319

820 9 22

29 1319 820 9 22

1 2 3 4 5 6 7 8 9 10

14

13 14

13 14


13

20

13 20


Item 3 is a 13Item 3 is a 13• its smaller child is a 9its smaller child is a 9• swap & OKswap & OK

Item 2 is a 19Item 2 is a 19• smaller child is 8smaller child is 8

» swapswap

• itsits smaller child is 14: smaller child is 14:» swap againswap again

14

29

1319

813 9 22

29 1319 813 9 22

1 2 3 4 5 6 7 8 9 10

14

20 14

20 14


9

13

8

19

19

14

98 19 1914 13


Item 1 is a 29Item 1 is a 29• smaller child is 8smaller child is 8

» swapswap

• itsits smaller child is 13 smaller child is 13» swap againswap again

• its its smaller child is 14smaller child is 14» swap againswap again

• donedone

14

29

98

1413 13 22

29 98 1413 13 22

1 2 3 4 5 6 7 8 9 10

14

20 19

20 19


8

29

29

13

29

14

8 29 2913 2914

ExerciseExercise

Use the BuildHeap algorithm to build a Use the BuildHeap algorithm to build a heap from the following arrayheap from the following array• [9, 86, 54, 92, 91, 38, 28, 87, 93, 84, 88, 24, 64, [9, 86, 54, 92, 91, 38, 28, 87, 93, 84, 88, 24, 64,

96, 3, 18, 1, 72, 66, 8]96, 3, 18, 1, 72, 66, 8]

Code for Building a HeapCode for Building a Heap

Uses a modified version of percolate downUses a modified version of percolate down• say where to percolate down fromsay where to percolate down from

To BuildHeap(Array A, int Length)To BuildHeap(Array A, int Length)

i i Length ÷ 2; Length ÷ 2;

while (i while (i 1) 1)

PercolateDown(A, i, Length);PercolateDown(A, i, Length);

ii –– –;–;

Code for Percolate DownCode for Percolate Down

To PercolateDown(Array A, int N, int L)To PercolateDown(Array A, int N, int L)T T A[N]; A[N];while (2N while (2N L) L)

if (2N = L)if (2N = L) S S 2N; 2N;elseelse S S (A[2N]<A[2N+1]) ? 2N : 2N+1; (A[2N]<A[2N+1]) ? 2N : 2N+1;if (A[S]<T)if (A[S]<T) A[N] A[N] A[S], N A[S], N S; S;elseelse break;break;

A[N] A[N] T; T;

Building a Heap is O(N)Building a Heap is O(N)

There will be less than N swapsThere will be less than N swaps• establish an upper bound on the # of swapsestablish an upper bound on the # of swaps

Let k be the height of the treeLet k be the height of the tree Worst case # of swaps:Worst case # of swaps:

• # of swaps for left sub-tree# of swaps for left sub-tree• plus # of swaps for rightplus # of swaps for right• plus k swaps for the rootplus k swaps for the root• assume all leaves at same level (simplifies math)assume all leaves at same level (simplifies math)

Maximum Number of SwapsMaximum Number of Swaps

kk NN formulaformula max swapsmax swaps N – k – 1N – k – 1

00 11 00 00

11 33 2*0 + 12*0 + 1 11 11

22 77 2*1 + 22*1 + 2 44 44

33 1515 2*4 + 32*4 + 3 1111 1111

44 3131 2*11 + 42*11 + 4 2626 2626

Equality can be proven by induction on k

Heap ClassHeap Class

OperationsOperations• Insert, FindMin, DeleteMinInsert, FindMin, DeleteMin• IsEmpty, MakeEmptyIsEmpty, MakeEmpty

ConstructorsConstructors• default (creates empty heap)default (creates empty heap)• with vector (uses BuildHeap to make heap out with vector (uses BuildHeap to make heap out

of the vector)of the vector)

Heap SortHeap Sort

Turn array into a Turn array into a reversereverse heap heap• bigger numbers at the frontbigger numbers at the front• modified version of BuildHeapmodified version of BuildHeap

After you remove each item…After you remove each item… ……put it at the back of the arrayput it at the back of the array

• which just became available!which just became available!

Heap Sort ExampleHeap Sort Example

Original array:Original array: maxHeapify:maxHeapify: removeAll:removeAll:

17 7 22 9 5

9 722 17 5

9 722 17 5 22

517 22 17

9 7 17 9

57 9 7

5 7

5

Questions?Questions?

Binary heaps and Heapsort. Priority Queues n The smallest element is removed item = remove();item =...

Documents

Transcript of Binary heaps and Heapsort. Priority Queues n The smallest element is removed item = remove();item =...