2-11 Heapsort

Hengfeng Wei

hfwei@nju.edu.cn

May 12, 2020

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 1 / 24

Obama in a job interview at Google

“What is most efficient way to sort a million 32-bit integers?”

Obama: “The bubblesort would be the wrong way to go.”

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 2 / 24

Obama: “The bubblesort would be the wrong way to go.”

O Ω Θ

Best case Worst case Average case

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 3 / 24

Worst-case of Max-Heapify (TC 6.2-6)Show that the worst-case running time of Max-Heapify on ann-element heap is Ω(log n).

By Example.

1

5

2

6

34

Compare vs. Exchange

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 4 / 24

Worst-case of Max-Heapify (Section 6.2 of CLRS)Show that the worst-case running time of Max-Heapify on ann-element heap is O(log n).

W (n) ≤ H(n)

No Examples Here!Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 5 / 24

Worst-case of Max-HeapifyShow that the worst-case running time of Max-Heapify on ann-element heap is Θ(log n).

O Ω Θ

Worst-case “power” of A by example O = Ω

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 6 / 24

Worst-case of Heapsort (TC 6.4-4)Show that the worst-case running time of Heapsort is Ω(n log n).

By Example.

Non-proof.

Θ(n)︸ ︷︷ ︸Extract-Max

× Ω(log n)︸ ︷︷ ︸Max-Heapify

= Ω(n log n)

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 7 / 24

Worst-case of Heapsort (TC 6.4-4)Show that the worst-case running time of Heapsort is Ω(n log n).

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 8 / 24

Heap in decreasing order?

7

5

12

6

34

T (7) = 2 + 1 + 1 + 1 + 0 + 0 = 5

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 9 / 24

7

6

13

5

42

T (7) = 2 + 2 + 2 + 1 + 1 + 0 = 8

(Ex. 23, Section 5.2.3, TAOCP Vol 3)

8

7

63

5

42

1

n−1∑r=1

⌊log r⌋ = n⌊log n⌋ − 2⌊log n⌋+1 + 2 = Ω(n log n)

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 10 / 24

Worst-case of HeapsortShow that the worst-case running time of Heapsort is O(n log n).

n−1∑r=1

⌊log r⌋ = n⌊log n⌋ − 2⌊log n⌋+1 + 2 = O(n log n)

No Examples Here!

Θ(n)︸ ︷︷ ︸Extract-Max

× O(log n)︸ ︷︷ ︸Max-Heapify

= O(n log n)

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 11 / 24

Worst-case of HeapsortShow that the worst-case running time of Heapsort is Θ(n log n).

O Ω Θ

Worst-case “power” of A by example O = Ω

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 12 / 24

Algorithm A

Inputs I of size n

O Ω Θ

Best-case by example “weakness” of A O = Ω

Worst-case “power” of A by example O = Ω

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 13 / 24

Best-case of Heapsort (TC 6.4-5⋆)Show that when all elements are distinct, the best-case running time ofHeapsort is Ω(n log n).

Best-case of Heapsort (Ex. 32, Section 5.2.3, TAOCP Vol 3)Prove that the number of heapsort promotions, B, is always at least12N log N + O(N), if the keys being sorted are distinct.

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 14 / 24

Best-case of Heapsort (TC 6.4-5⋆)Show that when all elements are distinct, the best-case running time ofHeapsort is Ω(n log n).

Consider the largest m = ⌈n/2⌉ elements.The largest m elements form a subtree.

≥ ⌊m/2⌋ of m must be nonleaves of that subtree.

≥ ⌊m/2⌋ of m appear in the first ⌊n/2⌋ positions.

They must be promoted to the root before being Extract-Max.

⌊m/2⌋∑k=1

⌊log k⌋ = 12

m log m + O(m)

B(n) ≥ 14

n log n + O(n) + B(⌊n/2⌋) =⇒ B(n) ≥ 12

n log n + O(n)Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 15 / 24

Best-case of HeapsortShow that when all elements are distinct, the best-case running time ofHeapsort is O(n log n).

O Ω Θ

Best-case by example “weakness” of A O = Ω

Worst-case “power” of A by example O = Ω

12

n log n + O(n) ≤ ? ≤ n log n

O Ω Θ

Best-case ? ∼ 12n log n + O(n) O = Ω

Worst-case ∼ n log n ∼ n log n O = Ω

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 16 / 24

Best-case of HeapsortShow that when all elements are distinct, the best-case running time ofHeapsort is

B(n) ≤ 12

n log n + O(n log log n).

By Example.

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 17 / 24

“On the Best Case of Heapsort” (1994)

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 18 / 24

Best-case of HeapsortShow that when all elements are distinct, the best-case running time ofHeapsort is Θ(n log n).

O Ω Θ

Best-case by example “weakness” of A O = Ω

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 19 / 24

Algorithm A

Inputs I of size n

O Ω Θ

Best-case by example “weakness” of A O = Ω

Worst-case “power” of A by example O = Ω

Average-case ≤ ≥ O = Ω

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 20 / 24

Average-case of HeapsortAssume that all elements are distinct. Show that the average-caserunning time of Heapsort is Θ(n log n).

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 21 / 24

“By a surprisingly short counting argument.”“The Analysis of Heapsort” (Sedgewick ; 1992)

Robert Sedgewick D. E. Knuth

“It is elegant. see exercise 30.”Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 22 / 24

Heap Identity (Additional)

∀h ≥ 1 : ⌈log(⌊12

h⌋ + 1)⌉ + 1 = ⌈log(h + 1)⌉

⌈log(h + 1)⌉ = ⌊log h⌋ + 1, ∀h ≥ 1

⌊log⌊12

h⌋⌋ + 1 = ⌈log(⌊12

h⌋ + 1)⌉ = ⌈log(h + 1)⌉ − 1 = ⌊log h⌋

(Depth of the parent of h) + 1 = Depth of h

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 23 / 24

k-way Merging (TC 6.5-9)Give an O(n log k)-time algorithm to merge k sorted lists with nelements in total into one sorted list.

k = 2 =⇒ O(n)

Always maintain a min-heap of size k

whose root contains the next smallest element.

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 24 / 24

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 24 / 24

Office 302Mailbox: H016

hfwei@nju.edu.cn

Hengfeng Wei (hfwei@nju.edu.cn) 2-11 Heapsort May 12, 2020 24 / 24