Lecture 13: Adversary Arguments continues … This lecture, we continue with the adversary

argument by giving more examples.

Problem: given a list of n elements, find the 2nd largest. [Reference: S. Baase, Computer Algorithms: Introduction to Design & Analysis, Second Edition, Addison-Wesley, 1991, pp. 128-133.]

Example 4. Largest and Second Largest

Find 2nd largest element

Naive algorithm: find the maximum element (n-1 comparisons) remove it from the list find the maximum

among those remaining (n-2 comparisons) total = 2n-3 comparisons

Tournament algorithm

Have players play in pairs, then winners play each other, etc., until a winner is determined. Then the "runner-up" (2nd largest) is among those defeated by winner. Why? Because any other player has at least TWO players superior to him/her.

10 6 3 5 12 4 7 11

\ / \ / \ / \ /

\ / \ / \ / \ /

10 5 12 11

\ / \ /

\ / \ /

10 12

\ /

\ /

12

Thus the total is n+logn -2

A matching lower bound for 2nd largest

We can prove that the algorithm just given is optimal, by an adversary argument similar to that we saw before. The result is due to S. S. Kislitsyn in 1964.

The idea. First, note that any algorithm to find the 2nd largest element must "know" the largest element. (For otherwise the 2nd largest might be the largest!) To determine the largest element requires n-1 comparisons. The adversary argument we will give shows that we can always force ceil(lg n) elements to be compared with the maximum. This will prove what we want, since the runner-up is the maximum of these ceil(lg n) elements, requiring ceil(lg n)-1 comparisons to decide.

Adversary strategy:

All keys are assigned weights w[i]

Weights are all initialized to 1

Adversary replies are based on weights

When x is compared to y

Weights Adversary Reply Weight Changes

w[x]>w[y] x>y w[x]:=w[x]+w[y]; w[y]:=0;

w[x]=w[y]>0 x>y w[x]:=w[x]+w[y]; w[y]:=0;

w[y]>w[x] y>x w[y]:=w[y]+w[x]; w[x]:=0;

w[x]=w[y]=0 Consistent None

Accumulation of weights

Solution of the problem requires all weights to be accumulated to one key

All other keys must have weight zero Since weight accumulates to the highest

weight, weight can at most double with each comparison, lg n comparisons are required to accumulate all weights to the highest weight.

And for these lg n comparisons, because the losers have non-0 weights, they have not lost before!

More formal argument

We now show that this adversary strategy forces ceil(lg n) comparisons of max with elements that have never previously lost a comparison.

Observe that w(k) ≤ 2 w(k-1) by the adversary strategy. It follows that w(k) ≤ 2k w(0). But w(0) =1. Hence w(k) ≤ 2k.

We end at w(k) = n. Hence n ≤ 2k. Hence k ≥ ceil(lg n). But k is the number of comparisons of max against elements

that have never previously lost (because they have non-zero weights), so we have shown there must be at least ceil(lg n) such comparisons. There must be at least n-1 comparisons to find the maximum, and among the ceil(lg n) elements that lost comparisons to max, we need to do at least ceil(lg n)-1 comparisons to find the 2nd largest.

Thus the lower bound is n+ lg n – 2, for the worst-case.

Example 5. Lower bound for Median

The best upper bound is 3n + o(n) Best lower bound is 2n + o(n) by Dor and Zwick (focs96). We prove a 1.5n + O(1) worst-case lower bound here. If x is the median, We must have the following situation:

* *

\ / elements > median

* *

| /

----------------x--------------------median--------

/ | \

* * * elements < median

|

*

Plan:

First, we call a comparison involving x crucial if it is the first comparison where (a) x < y and y ≤ median OR (b) x > y and y ≥ median.

So the algorithm must make n-1 crucial comparisons. We need to show the algorithm also makes (n-1)/2 non crucial comparisons. The idea is as follows: first, the adversary picks some value (not some element) to be the median. Then it assigns a label in {N, L, S} to each element, and also values, as appropriate. Initially each element is assigned an "N" (= "Never participated in any comparison"). "L" means Larger than the median and "S" means Smaller than the median.

Adversary strategy

The adversary follows the following strategy: Algorithm Adversary

compares responds

----------------------------------------------------------

(N,N) assign one element to be larger than

the median, one smaller; result is (L,S)

(S,N) or (N,S) assign the N-element to be larger than

the median; result is (S,L) or (L,S)

(L,N) or (N,L) assign the N-element to be smaller than

the median; result is (L,S) or (S,L)

(S,L) or (L,S) consistent with previously assigned

or (S,S) or (L,L) values

Proof continues …

This strategy continues until (n-1)/2 S's (or (n-1)/2 L's) have been assigned. If at some point (n-1)/2 S's are assigned, then the adversary assigns the remaining elements to be greater than the median, except for one, which IS the median. A similar thing is done if (n-1)/2 L's have been assigned.

The last element assigned is always the median. This strategy will always force the algorithm to perform (n-1)/2

non-crucial comparisons. For any time an N-element is compared, a non-crucial comparison is done (except at the very end, when a crucial comparison may be done with the median itself). The least number of comparisons with N-elements that can be done is (n-1)/2. The TOTAL number of comparisons is therefore n-1 + (n-1)/2 = (3/2)(n-1).

Summary

We have been extremely lucky with the problems we have discussed.

For most of other problems you will see in future, lower bounds, even the very weak ones, are extremely hard to get, if we can get them at all.

We learned two powerful methods: Adversary argument Incompressibility method.