Quicksort

Quicksort

Divide & Conquer

Recall that Mergesort uses the Divide & Conquer paradigm.To sort an array A:

Divide Split A into A[0..n/2− 1] and A[n/2..n − 1].

Conquer Recursively sort each subarray.

Combine Merge the two subarrays.

Quicksort

Quicksort

The Quicksort algorithm also uses Divide & Conquer:

Divide Partition A into A[0..q − 1], A[q], andA[q + 1..n − 1] such that

Each element in A[0..q − 1] is ≤ A[q].Each element in A[q + 1..n − 1] is > A[q].

The element A[q] is called pivot.

Conquer Recursively sort (in place) each subarray.

Combine Not needed.

57 182 3 6 4

Quicksort

Quicksort

The Quicksort algorithm also uses Divide & Conquer:

Divide Partition A into A[0..q − 1], A[q], andA[q + 1..n − 1] such that

Each element in A[0..q − 1] is ≤ A[q].Each element in A[q + 1..n − 1] is > A[q].

The element A[q] is called pivot.

Conquer Recursively sort (in place) each subarray.

Combine Not needed.

571 82 3 64

Quicksort

Quicksort

The Quicksort algorithm also uses Divide & Conquer:

Divide Partition A into A[0..q − 1], A[q], andA[q + 1..n − 1] such that

Each element in A[0..q − 1] is ≤ A[q].Each element in A[q + 1..n − 1] is > A[q].

The element A[q] is called pivot.

Conquer Recursively sort (in place) each subarray.

Combine Not needed.

571 82 3 64

Quicksort

Quicksort

The Quicksort algorithm also uses Divide & Conquer:

Divide Partition A into A[0..q − 1], A[q], andA[q + 1..n − 1] such that

Each element in A[0..q − 1] is ≤ A[q].Each element in A[q + 1..n − 1] is > A[q].

The element A[q] is called pivot.

Conquer Recursively sort (in place) each subarray.

Combine Not needed.

5 71 82 3 64

Quicksort

Quicksort

Quicksort(A, p, r) sorts the subarray A[p..r ].

Quicksort(A, 0, n − 1) sorts the entire array A.

Quicksort(A, p, r)

(1) if p < r(2) q ← Partition(A, p, r)(3) Quicksort(A, p, q − 1)(4) Quicksort(A, q + 1, r)

571 82 3 64

Quicksort

Partition

During the run of Partition(A, p, r), there are 4 zones ofA[p..r ].

A[r ] is the pivot (it will be moved to the correct locationat the end of the procedure).

A[j ..r − 1] contains unexamined elements.

A[i + 1..j − 1] contains elements that are > pivot.

A[p..i ] contains elements that are ≤ pivot.

57 182 3 6 4

571 82 3 6 4

Quicksort

Partition

Partition(A, p, r)

(1) i ← p − 1(2) for j = p to r − 1(3) if A[j ] ≤ A[r ](4) i ← i + 1(5) exchange A[i ] with A[j ](6) exchange A[i + 1] with A[r ](7) return i + 1

Time complexity: Θ(n).

57 182 3 6 4

Quicksort

Partition

Partition(A, p, r)

(1) i ← p − 1(2) for j = p to r − 1(3) if A[j ] ≤ A[r ](4) i ← i + 1(5) exchange A[i ] with A[j ](6) exchange A[i + 1] with A[r ](7) return i + 1

Time complexity: Θ(n).

57 182 3 6 4

Quicksort

Partition

Partition(A, p, r)

(1) i ← p − 1(2) for j = p to r − 1(3) if A[j ] ≤ A[r ](4) i ← i + 1(5) exchange A[i ] with A[j ](6) exchange A[i + 1] with A[r ](7) return i + 1

Time complexity: Θ(n).

57 182 3 6 4

Quicksort

Partition

Partition(A, p, r)

(1) i ← p − 1(2) for j = p to r − 1(3) if A[j ] ≤ A[r ](4) i ← i + 1(5) exchange A[i ] with A[j ](6) exchange A[i + 1] with A[r ](7) return i + 1

Time complexity: Θ(n).

57 182 3 6 4

Quicksort

Partition

Partition(A, p, r)

(1) i ← p − 1(2) for j = p to r − 1(3) if A[j ] ≤ A[r ](4) i ← i + 1(5) exchange A[i ] with A[j ](6) exchange A[i + 1] with A[r ](7) return i + 1

Time complexity: Θ(n).

571 82 3 6 4

Quicksort

Partition

Partition(A, p, r)

(1) i ← p − 1(2) for j = p to r − 1(3) if A[j ] ≤ A[r ](4) i ← i + 1(5) exchange A[i ] with A[j ](6) exchange A[i + 1] with A[r ](7) return i + 1

Time complexity: Θ(n).

571 82 3 6 4

Quicksort

Partition

Partition(A, p, r)

(1) i ← p − 1(2) for j = p to r − 1(3) if A[j ] ≤ A[r ](4) i ← i + 1(5) exchange A[i ] with A[j ](6) exchange A[i + 1] with A[r ](7) return i + 1

Time complexity: Θ(n).

571 82 3 6 4

Quicksort

Partition

Partition(A, p, r)

(1) i ← p − 1(2) for j = p to r − 1(3) if A[j ] ≤ A[r ](4) i ← i + 1(5) exchange A[i ] with A[j ](6) exchange A[i + 1] with A[r ](7) return i + 1

Time complexity: Θ(n).

571 82 3 6 4

Quicksort

Partition

Partition(A, p, r)

(1) i ← p − 1(2) for j = p to r − 1(3) if A[j ] ≤ A[r ](4) i ← i + 1(5) exchange A[i ] with A[j ](6) exchange A[i + 1] with A[r ](7) return i + 1

Time complexity: Θ(n).

571 82 3 64

Quicksort

Time complexity — worst case

In the worst case, in each step a subarray of size m ispartitioned into a subarray of size m − 1 and a subarrayof size 0.

The recurrence for the worst case is T (n) = T (n− 1) + nwhich gives T (n) = Θ(n2).

1 2 3 4p=0r=3

1 2 3p=0r=2

p=4r=3

1 2p=0r=1

p=3r=2

1p=0r=0

p=2r=1

Quicksort

Time complexity — best case

In the best case, in each step a subarray of size m ispartitioned into a subarray of size d(m − 1)/2e and asubarray of size b(m − 1)/2c.The recurrence for the worst case is T (n) = 2T (n/2) + nwhich gives T (n) = Θ(n log n).

p=0r=6

p=0r=2

p=4r=6

Quicksort

Average case — intuition

Suppose that in each step, a subarray of size m ispartitioned into a subarray of size 1

10m and a subarray of

size 910m. The time complexity in this case is Θ(n log n).

Quicksort

Average case — intuition

If the height of the recursion tree is h, the time is O(hn).

In a path in the recursion tree, some partitions may begood (at least 1-9 ratio), and some bad.

1000

199 800

4 795

294 500

Quicksort

Average case — intuition

For a random input (random permutation), theprobability of a good partition at some step is 0.8.

Therefore, along a path in the tree, there will be manygood partitions and the length of the path is O(log n)with high probability.

1000

199 800

4 795

294 500

Quicksort

Average case

Assuming random input (random permutation) theexpected running time of Quicksort is Θ(n log n).

If we do not assume the random input, we can apply arandom permutation on the input array.For every input array with distinct elements, the expectedtime complexity is Θ(n log n).

An equivalent algorithm is Randomized-Quicksort.

Quicksort

Randomized-Quicksort

RandomizedQuicksort(A, p, r)

(1) if p < r(2) q ← RandomizedPartition(A, p, r)(3) RandomizedQuicksort(A, p, q − 1)(4) RandomizedQuicksort(A, q + 1, r)

RandomizedPartition(A, p, r)

(1) i ← Random(p, r)(2) exchange A[i ] with A[r ](3) return Partition(A, p, r)

Quicksort

Analysis of Randomized-Quicksort

Suppose that the input is an array with n distinct elements.The expected time T (n) of Randomized-Quicksort satisfies

T (n) =1

n

n∑q=1

(T (q − 1) + T (n − q) + n).

The solution of this recurrence is T (n) = Θ(n log n).

Quicksort

Average run-time of sorting algorithms

Expected running time on a random permutation.

5 10 100 1000Insertion sort 1.00 3.18 232.1 22282.4Mergesort 2.52 5.88 96.8 1382.7Heapsort 2.15 6.02 119.8 1758.8Quicksort 1.62 3.61 56.7 810.9

Data taken from “Real-Time Performance of Sorting Algorithms” (Puschner 1999).

Quicksort

The selection problem

The selection problem

The selection problem

In the selection problem the input is an array A with ndistinct numbers and an integer i . The goal is to returnthe ith smallest number in A.

For i = 1 (finding the minimum) the problem is easy:

Minimum(A)

(1) min← A[0](2) for i = 1 to A.length− 1(3) if A[i ] < min(4) min← A[i ](5) return min

The selection problem

Randomized-Select

RandomizedSelect(A, p, r , i) returns the ith smallestelement in A[p..r ].The expected time of Randomized-Select is Θ(n).

RandomizedSelect(A, p, r , i)

(1) if p = r(2) return A[p](3) q ← RandomizedPartition(A, p, r)(4) k ← q − p + 1 (k = number elements in A[p..r ]

that are ≤ pivot)(5) if i = k(6) return A[q](7) else if i < k(8) return RandomizedSelect(A, p, q − 1, i)(9) else(10) return RandomizedSelect(A, q + 1, r , i − k)

681 92 4 75

The selection problem

Randomized-Select

RandomizedSelect(A, p, r , i) returns the ith smallestelement in A[p..r ].The expected time of Randomized-Select is Θ(n).

RandomizedSelect(A, p, r , i)

(1) if p = r(2) return A[p](3) q ← RandomizedPartition(A, p, r)(4) k ← q − p + 1 (k = number elements in A[p..r ]

that are ≤ pivot)(5) if i = k(6) return A[q](7) else if i < k(8) return RandomizedSelect(A, p, q − 1, i)(9) else(10) return RandomizedSelect(A, q + 1, r , i − k)

681 92 4 75

The selection problem

Randomized-Select

RandomizedSelect(A, p, r , i) returns the ith smallestelement in A[p..r ].The expected time of Randomized-Select is Θ(n).

RandomizedSelect(A, p, r , i)

(1) if p = r(2) return A[p](3) q ← RandomizedPartition(A, p, r)(4) k ← q − p + 1 (k = number elements in A[p..r ]

that are ≤ pivot)(5) if i = k(6) return A[q](7) else if i < k(8) return RandomizedSelect(A, p, q − 1, i)(9) else(10) return RandomizedSelect(A, q + 1, r , i − k)

681 92 4 75

The selection problem

Select

The Select algorithm is the same as Randomized-Select.The difference is the way the pivot is chosen.

Select(A, p, r , i)

(1) if p = r(2) return A[p](3) j ← ChoosePivot(A, p, r)(4) exchange A[j ] with A[r ](5) q ← Partition(A, p, r)(6) k ← q − p + 1(7) if i = k(8) return A[q](9) else if i < k(10) return Select(A, p, q − 1, i)(11) else(12) return Select(A, q + 1, r , i − k)

The selection problem

Choose-Pivot

ChoosePivot(A, p, r)

1 n← r − p + 1.

2 If n ≤ 10, return r .

3 Divide the n elements of A[p..r ] into dn/5e groups of 5elements each. If n is not divisible by 5, the last groupcontains n mod 5 elements.

4 Find the median of each group by sorting the group.Copy these medians to an array B .

5 x ← Select(B , 0,B .length− 1, dB .length/2e)(x is the median of B).

6 Find the index i such that A[i ] = x and return i .

2060 7082 83651022337284 11 1563 68 3091 922112 80878864 26

The selection problem

Choose-Pivot

ChoosePivot(A, p, r)

1 n← r − p + 1.

2 If n ≤ 10, return r .

3 Divide the n elements of A[p..r ] into dn/5e groups of 5elements each. If n is not divisible by 5, the last groupcontains n mod 5 elements.

4 Find the median of each group by sorting the group.Copy these medians to an array B .

5 x ← Select(B , 0,B .length− 1, dB .length/2e)(x is the median of B).

6 Find the index i such that A[i ] = x and return i .

2060 7082 83651022337284 11 1563 68 3091 922112 80878864 26

The selection problem

Choose-Pivot

ChoosePivot(A, p, r)

1 n← r − p + 1.

2 If n ≤ 10, return r .

3 Divide the n elements of A[p..r ] into dn/5e groups of 5elements each. If n is not divisible by 5, the last groupcontains n mod 5 elements.

4 Find the median of each group by sorting the group.Copy these medians to an array B .

5 x ← Select(B , 0,B .length− 1, dB .length/2e)(x is the median of B).

6 Find the index i such that A[i ] = x and return i .

2060 70828365142233 72841115 6368 3091922112 8087886426

The selection problem

Choose-Pivot

ChoosePivot(A, p, r)

1 n← r − p + 1.

2 If n ≤ 10, return r .

3 Divide the n elements of A[p..r ] into dn/5e groups of 5elements each. If n is not divisible by 5, the last groupcontains n mod 5 elements.

4 Find the median of each group by sorting the group.Copy these medians to an array B .

5 x ← Select(B , 0,B .length− 1, dB .length/2e)(x is the median of B).

6 Find the index i such that A[i ] = x and return i .

20 60 70828365142233 72841115 63 6830 91922112 8087886426

The selection problem

Time complexity of Select

At least half of the dn/5e medians of the groups are ≥ x .

For each such median, its group contains at least 3elements that are > x except for:

The group of x .The last group (the group that contains less than 5elements)

2060 70828365142233 72841115 6368 3091922112 8087886426

The selection problem

Time complexity of Select

At least half of the dn/5e medians of the groups are ≥ x .

For each such median, its group contains at least 3elements that are > x except for:

The group of x .The last group (the group that contains less than 5elements)

The number of elements > x is at least

3

(⌈dn/5e

2

⌉− 2

)≥ 3

10n − 6.

The number of elements < x is at least 310n − 6.

In the recursive call in line 10 or 12, the number ofelements is at most 7

10n + 6.

The selection problem

Time complexity of Select

The recurrence for the time complexity isT (n) = T (dn/5e) + T (7n/10 + 6) + Θ(n).

The 1st term is the recursive call on B insideChoosePivot. The 2nd term is the recursive call inline 10 or 12. The 3rd term is the call to Partition andthe non-recursive part of ChoosePivot.

Consider a simplified recurrence:T (n) = T (n/5) + T (7n/10) + n.

We show by induction that T (n) ≤ cn.T (n) = T (n/5) + T (7n/10) + n

≤ c · n5

+ c · 7n

10+ n =

9c

10n + n ≤ cn

The last inequality is true if n ≤ c10n, and this is true for

c ≥ 10.The selection problem