MA 252: Data Structures and Algorithms

Lecture 9http://www.iitg.ernet.in/psm/indexing_ma252/y12/index.htmlhttp://www.iitg.ernet.in/psm/indexing_ma252/y12/index.html

Partha Sarathi Mandal

Dept. of Mathematics, IIT Guwahati

• We are lucky if the partitioning is balance i.e.,

(n/2, n/2).

• We are unlucky if the partitioning is imbalance

i.e., (1, n-1) or (n-1, 1).

More about quicksort

i.e., (1, n-1) or (n-1, 1).

P. S. Mandal, IITG

More intuition

• Suppose we alternate lucky, unlucky, lucky, unlucky, lucky, ….

• L(n)= 2U(n/2) + Θ(n) lucky

• U(n)= L(n –1) + Θ(n) unlucky• U(n)= L(n –1) + Θ(n) unlucky

Solving:

L(n) = 2(L(n/2 –1) + Θ(n/2)) + Θ(n)

= 2L(n/2 –1) + Θ(n)

= Θ(nlg n)

How can we make sure we are usually lucky ?

P. S. Mandal, IITG

Randomized quicksort

• IDEA: Partition around a random element.

• Running time is independent of the input order.

• No assumptions need to be made about the • No assumptions need to be made about the input distribution.

• No specific input elicits the worst-case behavior.

• The worst case is determined only by the output of a random-number generator.

P. S. Mandal, IITG

Pseudocode for

Randomized quicksort

P. S. Mandal, IITG

Randomized quicksort analysis

• Let T(n) =the random variable for the running time of randomized quicksort on an input of size n, assuming random numbers are independent.

• For k= 0, 1, …, n–1, define the indicator random

variable:variable:

• E[Xk] = 0.Pr{Xk=0} + 1.Pr{Xk=1} = Pr{Xk = 1} = 1/n, since all splits are equally likely, assuming elements are distinct.

P. S. Mandal, IITG

Analysis (continued)

P. S. Mandal, IITG

Calculating expectation

P. S. Mandal, IITG

Calculating expectation

P. S. Mandal, IITG

Calculating expectation

P. S. Mandal, IITG

Calculating expectation

P. S. Mandal, IITG

Calculating expectation

P. S. Mandal, IITG

Exercise

(The k = 0, 1 terms can be absorbed in the Θ(n) )

Exercise: Exercise:

Prove: E[T(n)] ≤ anlg n for constant a > 0.

– Choose a large enough so that anlgn dominates

E[T(n)] for sufficiently small n ≥ 2.

Use fact:

P. S. Mandal, IITG

Substitution method

P. S. Mandal, IITG

Substitution method

P. S. Mandal, IITG

Substitution method

P. S. Mandal, IITG

Substitution method

P. S. Mandal, IITG

Quicksort in practice

• Quicksort is a great general-purpose sorting

algorithm.

• Quicksort is typically over twice as fast as

merge sort.merge sort.

• Quicksort can benefit substantially from code

tuning.

• Quicksort behaves well even with caching and

virtual memory.

P. S. Mandal, IITG

How fast can we sort?

• All the sorting algorithms we have seen so far are comparison sorts: only use comparisons to determine the relative order of elements.

• E.g., insertion sort, merge sort, quicksort, heapsort.heapsort.

• The best worst-case running time that we’ve seen for comparison sorting is O(nlg n).

Is O(nlg n) the best we can do ?

• Decision tree scan help us answer this question.

P. S. Mandal, IITG

Decision-tree example

• Each internal node is labeled i:j for i, j ∈ {1, 2, …, n}.

• The left subtree shows subsequent comparisons if ai ≤ aj.

• The right subtree shows subsequent comparisons if ai ≥ aj.

P. S. Mandal, IITG

Decision-tree example

• Each internal node is labeled i:j for i, j ∈ {1, 2, …, n}.

• The left subtree shows subsequent comparisons if ai ≤ aj.

• The right subtree shows subsequent comparisons if ai ≥ aj.

P. S. Mandal, IITG

Decision-tree example

• Each internal node is labeled i:j for i, j ∈ {1, 2, …, n}.

• The left subtree shows subsequent comparisons if ai ≤ aj.

• The right subtree shows subsequent comparisons if ai ≥ aj.

P. S. Mandal, IITG

Decision-tree example

• Each internal node is labeled i:j for i, j ∈ {1, 2, …, n}.

• The left subtree shows subsequent comparisons if ai ≤ aj.

• The right subtree shows subsequent comparisons if ai ≥ aj.

P. S. Mandal, IITG

Decision-tree example

• Each leaf contains a permutation ⟨π(1), π(2),…, π(n)⟩

to indicate that the ordering aπ(1) ≤ aπ(2) ≤ … ≤ aπ(n) has

been established.

P. S. Mandal, IITG

Decision-tree model

A decision tree can model the execution of any comparison sort:

• One tree for each input size n.

• View the algorithm as splitting whenever it compares two elements.compares two elements.

• The tree contains the comparisons along all possible instruction traces.

• The running time of the algorithm = the length of the path taken.

• Worst-case running time = height of tree.

P. S. Mandal, IITG

Lower bound for decision-tree sorting

Theorem: Any decision tree that can sort nelements must have height Ω(nlg n).

Proof: The tree must contain ≥ n! leaves, since there are n! possible permutations. A height-h binary tree has ≤ 2h leaves. Thus, n! ≤ 2h.

∴

tree has ≤ 2h leaves. Thus, n! ≤ 2h.

∴ h ≥ lg (n!) (lg is monotonically increasing)

≥ lg ((n/e)n) (Stirling’s formula)

= nlg n – nlg e

= Ω(nlg n)

P. S. Mandal, IITG

Lower bound for comparison sorting

Corollary: Heapsort and merge sort are

asymptotically optimal comparison sorting

algorithms.

P. S. Mandal, IITG