CSC 1700 – Analysis of Algorithms: Lower Bounds€¦ · · 2016-12-13CSC 1700 – Analysis of...
Transcript of CSC 1700 – Analysis of Algorithms: Lower Bounds€¦ · · 2016-12-13CSC 1700 – Analysis of...
Villanova University Department of Computing Sciences
CSC 1700 – Analysis of Algorithms: Lower
Bounds
Professor Henry CarterFall 2016
Villanova University Department of Computing Sciences
Extra Credit
• Puzzle questions requiring the application of you algorithms knowledge
• 20 possible points, 4 questions
• Replaces the next two lowest quiz grades (after dropped quizzes)
• You may collaborate with one partner and submit your solutions jointly (cite your collaborator)
• Due Monday, Dec. 12
2
Villanova University Department of Computing Sciences
Recap
• Iterative improvement algorithms take a feasible solution and iteratively improve it until optimized
• Simplex method: linear equations
• Ford-Fulkerson: network flow
• Bipartite matching: matching cardinality or matching weight
3
Villanova University Department of Computing Sciences
How low can you go?
• Algorithmic power explored so far has been powerful
• Several implicit limitations have been encountered
• These limitations established through lower-bound proofs (big-Ω)
4
Villanova University Department of Computing Sciences
Proving the lower bound
• Trivial limits
• Problem reduction
• Adversarial argument
• Information-theoretic
5
Villanova University Department of Computing Sciences
Trivial Limits
• A simple lower bound can be established based on the size of the input and output
• Every input element must be read
• Every output element must be written
6
Villanova University Department of Computing Sciences
Examples
• Enumerating permutations of n elements
• Evaluating a polynomial
• N x N matrix multiplication
7
Villanova University Department of Computing Sciences
Tight Bounds
• The tightness of the bound indicates how close the lower bound is to the actual known lower bound
• If a lower bound is known along with an algorithm with the same efficiency, the bound is "tight"
• Examples:
‣ Polynomial evaluation
‣ N x N matrix multiplication
8
Villanova University Department of Computing Sciences
Problem Reduction
• If we have a lower bound for problem Q, it can be applied to other problems
• Reducing Q to P implies that the worst-case lower bound for Q also applies to P
• Since the worst-case Q requires Ω(n) operations no matter how you try to solve it, reducing the problem to P and solving P also faces the same lower bound
9
Villanova University Department of Computing Sciences
Common problems for reduction
10
Problem Lower Bound Tightness
sorting Ω(n log n) yes
Searching a sorted array
Ω(log n) yes
Element uniqueness Ω(n log n) yes
n-digit integer multiplication
Ω(n) unknown
n x n matrix multiplication
Ω(n2) unknown
Villanova University Department of Computing Sciences
(Matrix) Multiplication
• Integer multiplication:
• Matrix multiplication
12
x · y =(x+ y)2 � (x� y)2
4x
2 = x · x
X =
0 AAT 0
�Y =
0 BT
B 0
�
Villanova University Department of Computing Sciences
Adversarial Arguments
• An adversarial player generates inputs during execution to incur maximum comparisons
• The adversary must remain consistent with previous choices
• Determine how many comparisons the adversary can force
• Example: merging two sorted arrays
13
Villanova University Department of Computing Sciences
Information-Theoretic Arguments
• Determine the minimum number of questions required to gather the necessary "information"
• Example: guessing a number between 1 and n
• Enumerate possible execution paths with decision trees
• The maximum and average heights of the tree correspond to the worst-case and average number of questions (comparisons)
14
Villanova University Department of Computing Sciences
Sorting Lower Bound
• Lower bound on height:
• Stirling's formula: n! ≈ (2πn)1/2 (n/e)n
• Average height:
16
Villanova University Department of Computing Sciences
Searching Lower Bound
• Representation-dependent
• Ternary decision trees:
‣ Not tight
• Binary decision trees:
19
Villanova University Department of Computing Sciences
Practice: finding the median of three elements
• What is the information-theoretic lower bound using comparison-based algorithms?
• Draw a decision tree
• Is the bound illustrated by your decision tree tight? Can we get a better algorithm?
20
Villanova University Department of Computing Sciences
Recap
• Algorithmic power is broad but limited
• Lower bounds determine whether an algorithm can be improved by more than a constant factor
• Established through input size, problem reduction, adversarial generation, and information-theoretic decision trees
21