CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3)...
-
Upload
cecily-marshall -
Category
Documents
-
view
213 -
download
0
Transcript of CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3)...
![Page 1: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/1.jpg)
CS200 Algorithms and Data Structures Colorado State University
1
Part 5.Computational Complexity (3)
CS 200 Algorithms and Data Structures
![Page 2: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/2.jpg)
CS200 Algorithms and Data Structures Colorado State University
2
Outline
• Recurrence Relations• Divide and Conquer• Understanding the Complexity of Algorithms
![Page 3: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/3.jpg)
CS200 Algorithms and Data Structures Colorado State University
3
Recurrence Relations: An Overview
• What is a recurrence relation?– A recursively defined sequence
• Example– Arithmetic progression: a, a+d, a+2d, …, a+nd
• a0 = a
• an = an-1 + d
![Page 4: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/4.jpg)
CS200 Algorithms and Data Structures Colorado State University
4
Recurrence Relations:Formal Definition
• Sequence = Recurrence relation + Initial conditions (“base case”)
• Example: an = 2an-1 + 1, a1 = 1
![Page 5: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/5.jpg)
CS200 Algorithms and Data Structures Colorado State University
5
Compound Interest
• You deposit $10,000 in a savings account that yields 10% yearly interest. How much money will you have after 30 years? (b is balance, r is rate)
![Page 6: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/6.jpg)
CS200 Algorithms and Data Structures Colorado State University
6
Recursively defined functions and recurrence relations
• A recursive function Example:
f(0) = a (base case)
f(n) = f(n-1) + d for n > 0 (recursive step)
• The above recursively defined function generates the sequence a0 = a
an = an-1 + d
• A recurrence relation produces a sequence, an application of a recursive function produces a value from the sequence
![Page 7: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/7.jpg)
CS200 Algorithms and Data Structures Colorado State University
7
How to Approach Recursive Relations
Recursive Functions Sequence of Values
f(0) = 0 (base case) f(n) = f(n-1) + 2 for n >
0 (recursive part)
f(0) = 0 f(1) = f(0) +2 = 2 f(2) = f(1)+2 = 4 f(3) = f(2) +2 = 6
Substitution
Closed Form?(solution,
explicit formula)
![Page 8: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/8.jpg)
CS200 Algorithms and Data Structures Colorado State University
8
Find a recursive function(1/5)• Give an inductive definition of the factorial
function F(n)= n!
![Page 9: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/9.jpg)
CS200 Algorithms and Data Structures Colorado State University
9
*Is recursively defined functions well-defined?
• For every positive integer, the value of the function at this integer is determined in an unambiguous way.
• With given any of two parts of the definition, we obtain the same value no matter how we apply the two parts of the definition.
![Page 10: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/10.jpg)
CS200 Algorithms and Data Structures Colorado State University
10
Find a recursive function(2/5)• Give a recursive definition of an, where a is a
nonzero real number and n is a nonnegative integer.
• Rosen Chapter 5 example 3-2 pp. 346
![Page 11: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/11.jpg)
CS200 Algorithms and Data Structures Colorado State University
11
Find a recursive function(3/5) • Give a recursive definition of,
• Rosen Chapter 5 example 3-3 pp. 346
€
akk =0
n
∑
![Page 12: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/12.jpg)
CS200 Algorithms and Data Structures Colorado State University
12
Find a recursive function(4/5) • Give a recursive definition of l(w), the length of
the string w.
• Rosen Chapter 5 example 3-7 pp. 350
![Page 13: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/13.jpg)
CS200 Algorithms and Data Structures Colorado State University
13
Find a recursive function(5/5) • Find a recurrence relation and give initial
conditions for the number of bit strings of length n that do not have two consecutive 0s. How many such bit strings are there of length five?
![Page 14: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/14.jpg)
CS200 Algorithms and Data Structures Colorado State University
14
Solving recurrence relations (1)
Solve a0 = 2; an = 3an-1 , n > 0(1) What is the recursive function?
(2) What is the sequence of values?
Hint 1: Solve by substitution
![Page 15: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/15.jpg)
CS200 Algorithms and Data Structures Colorado State University
15
Solving recurrence relations (2)
Solve a0 = 2; an = 3an-1 + 2 for n > 0(1) What is the recursive function?
(2) What is the sequence of values?
Hint 1: Solve by substitution
Hint 2: Use formula for summing geometric series
1 + r + r2 + … + rn = (rn+1 - 1)/(r – 1)
if r ≠ 1
![Page 16: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/16.jpg)
CS200 Algorithms and Data Structures Colorado State University
16
Solving recurrence relations (3)
Solve a0 = 1; an = an-1 + n for n > 0
Hint 1: Solve by substitution
![Page 17: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/17.jpg)
CS200 Algorithms and Data Structures Colorado State University
17
Modeling with Recurrence• Suppose that the number of bacteria in a
colony triples every hour– Set up a recurrence relation for the number of
bacteria after n hours have elapsed.– 100 bacteria are used to begin a new colony.
![Page 18: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/18.jpg)
CS200 Algorithms and Data Structures Colorado State University
18
Linear Recurrence RelationsA linear homogeneous recurrence relation of
degree k with constant coefficients is a recurrence relation of a form
an = c1an-1 + c2an-2+ … + ckan-k
where, c1,c2 ,c3 …. ,ck are real numbers and ck is not 0.
![Page 19: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/19.jpg)
CS200 Algorithms and Data Structures Colorado State University
19
Is this linear homogeneous recurrence relation?
1. fn = fn-1 + fn-2
2. an = an-1 + a2n-2
3. Hn = 2Hn-1+1
• Modeling of problems
• They can be systematically solved.
Linear homogeneous recurrence relations?
A.1 B. 2 C. 3 D. 1,2,3
![Page 20: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/20.jpg)
CS200 Algorithms and Data Structures Colorado State University
20
Theorem 1Let c1 and c2 be real numbers. Suppose that r2 –c1r – c2 = 0 has two distinct roots r1 and r2 . Then the sequence {an} is a solution of the recurrence relation an = c1an-1 +c2an-2
if and only if
an = α1r1n + α2r2
n for n= 0,1,2,… where α1 and α2 are constants.
Rosen section8.2 theorem 1, pp.515
![Page 21: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/21.jpg)
CS200 Algorithms and Data Structures Colorado State University
21
What is the solution of the recurrence relation
• an = an-1 + 2an-2 with a0 = 2 and a1 = 7?
Rosen Section 8-2 Example 3 pp. 516
![Page 22: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/22.jpg)
CS200 Algorithms and Data Structures Colorado State University
22
Outline
• Recurrence Relations• Divide and Conquer• Understanding the Complexity of Algorithms
![Page 23: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/23.jpg)
CS200 Algorithms and Data Structures Colorado State University
23
Divide-and-Conquer
Basic idea: Take large problem and divide it into smaller problems until problem is trivial, then combine parts to make solution.
Recurrence relation for the number of steps required:
f(n) = a f(n / b) + g(n)
n/b : the size of the sub-problems solved
a : number of sub-problems
g(n) : steps necessary to combine solutions to sub-problems
![Page 24: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/24.jpg)
CS200 Algorithms and Data Structures Colorado State University
24
Example: Binary Searchpublic static int binSearch (int myArray[], int first,
int last, int value) {// returns the index of value or -1 if not in the arrayint index;if (first > last) { index = -1; }else { int mno = (first + last)/2; if (value == myArray[mno]) { index = mno; } else if (value < myArray[mno]) {
index = binSearch(myArray, first, mno-1, value); }
else { index = binSearch(myArray, mno+1, last, value); }
} return index;
}
What are a, b, and g(n)?
![Page 25: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/25.jpg)
CS200 Algorithms and Data Structures Colorado State University
25
Estimating big-O (Master Theorem)
From section 8.3 (for 6th Edition 7.3) in Rosen
![Page 26: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/26.jpg)
CS200 Algorithms and Data Structures Colorado State University
26
Example: Binary Search using the Master Theorem
f(n) = a f(n / b) + nd
for the binary search algorithm
f(n) = a f(n / b) + nd
= 1 f(n / 2) + 3
Therefore, d = 0 (to make nd a constant), b = 2, a = 1.
bd = 20 = 1,
It satisfies the second condition of the Master theorem.
So, f(n) = O(ndlog2n)
= O(n0log2n)
= O(log2n)
![Page 27: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/27.jpg)
CS200 Algorithms and Data Structures Colorado State University
27
Step-by-step Master Theorem• Modeling the divide-conquer with a recurrence
relation.
• f(n) is a recurrence function representing the number of operations with size of input n.
![Page 28: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/28.jpg)
CS200 Algorithms and Data Structures Colorado State University
28
Step-by-step Master Theorem
Method A
Method A Method A
Method A Method A Method A Method A
Method_A(array){Method_A(left_half);Method_A(right_half);Complete_the_Method(array);
}
Call Method_A(left_half) Call Method_A(right_half)
Method_A(array)
![Page 29: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/29.jpg)
CS200 Algorithms and Data Structures Colorado State University
29
Step-by-step Master Theorem
Method B
Method B
Method B
Method_B(array){Method_A(possible_half);Do_nothing_Method(array);
}
Call Method_B(left_half)
Method_A(array)
![Page 30: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/30.jpg)
CS200 Algorithms and Data Structures Colorado State University
30
Step-by-step Master Theorem
Method C
Method C Method C
Method C
Method_C(array){Method_C(first_one_third);Method_C(second_one_third);Method_C(last_one_third);Complete_method(array);
}
Call Method_A(right_half)
Method_C(array)
Method C
Method C Method
C
Method C Method
C Method C
Method C Method
C Method C
Call Method_C(second_one_thrid)
Call Method_C(last_one_thrid) Call Method_C(first_one_thrid)
![Page 31: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/31.jpg)
CS200 Algorithms and Data Structures Colorado State University
31
Method
Method Method Method
Method Method
Method Method
Method Method
Method Method
Method Method
Method Method
… LEVEL 1
LEVEL 2
LEVEL 3
LEVEL 4 …
Goal 1: What is the complexity per level? Goal 2: What is the total complexity?
![Page 32: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/32.jpg)
CS200 Algorithms and Data Structures Colorado State University
32
LEVEL 1 Method
Method Method Method
Method Method
Method Method
Method Method
Method Method
Method Method
Method Method
…
…“Method” calls “Method” recursively a times “Method” calls “Method” recursively with input size n/b
“Method” is called with input size n.
![Page 33: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/33.jpg)
CS200 Algorithms and Data Structures Colorado State University
33
LEVEL 1 Method
Method Method Method
Method Method
Method Method
Method Method
Method Method
Method Method
Method Method
…
…“Method” includes software fragment on top of calling methods recursively, and it takes g(n) operations.
![Page 34: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/34.jpg)
CS200 Algorithms and Data Structures Colorado State University
34
LEVEL 1 Method
Method Method Method
Method Method
Method Method
Method Method
Method Method
Method Method
Method Method
…
…Therefore, in LEVEL 1, “Method” requires, f(n) = a f(n / b) + g(n)
![Page 35: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/35.jpg)
CS200 Algorithms and Data Structures Colorado State University
35
Method
Method Method Method
Method Method
Method Method
Method Method
Method Method
Method Method
Method Method
…
…This is also true for LEVEL 2 f(n) = a f(n / b) + g(n)
LEVEL 2
![Page 36: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/36.jpg)
CS200 Algorithms and Data Structures Colorado State University
36
Method
Method Method Method
Method Method
Method Method
Method Method
Method Method
Method Method
Method Method
…
…This is also true for LEVEL i f(n) = a f(n / b) + g(n)
LEVEL i
![Page 37: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/37.jpg)
CS200 Algorithms and Data Structures Colorado State University
37
More generally,
O(nd) if a < bd
f(n) is O(ndlogn) if a = bd
O(nlogba) if a > bd
![Page 38: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/38.jpg)
CS200 Algorithms and Data Structures Colorado State University
38
Base case• We will set d = 1
– The bottom level of the tree is equally well computed.
– Base case– It is straightforward to extend the proof for the case
when d ≠ 1.
![Page 39: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/39.jpg)
CS200 Algorithms and Data Structures Colorado State University
39
Goal 1. complexity per level • Let’s think about the recursion
tree.
• There will be logbn levels.
• At each level, the number of subproblems will be multiplied by a.
• Therefore, the number of subproblems at level i will be ai.
• Each subproblem at level i is a problem of size (n/bi).
• A subproblem of size (n/bi) requires (n/bi)d additional work.
logbn levels
![Page 40: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/40.jpg)
CS200 Algorithms and Data Structures Colorado State University
40
The total number of units of work on level i
• ai(n/bi)d = nd (ai/bdi)
= nd (a/bd)i
For the level i , the work per level
is decreasing, constant, or
increasing exactly when (a/bd)i is
decreasing, constant, or increasing.
logbn levels
![Page 41: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/41.jpg)
CS200 Algorithms and Data Structures Colorado State University
41
Observation• Now, observe that,
(a/bd) = 1
a = bd
• Therefore, the relations,
(1)a < bd (2) a = bd, (3) a > bd
are the conditions deciding the types of the growth function
![Page 42: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/42.jpg)
CS200 Algorithms and Data Structures Colorado State University
42
Goal 2: Bounding f(n) in the different cases.
• In general, we have that the total work done is,
(1) a < bd
(2) a = bd
(3) a > bd €
nd
i=0
logb n
∑ (a /bd )i = nd (i=0
logb n
∑ a /bd )i
![Page 43: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/43.jpg)
CS200 Algorithms and Data Structures Colorado State University
43
Case 1. a < bd
• nd times a geometric series with a ratio of less than 1.
• First item is the biggest one.€
nd
i=0
logb n
∑ (a /bd )i = nd (i=0
logb n
∑ a /bd )i
€
nd (i=0
logb n
∑ a /bd )i = Ο(nd )
![Page 44: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/44.jpg)
CS200 Algorithms and Data Structures Colorado State University
44
Case 2. a = bd
• (a/bd) = 1
• nd(1+1+1+…+1) = nd(logbn)€
nd
i=0
logb n
∑ (a /bd )i = nd (i=0
logb n
∑ a /bd )i
€
nd (i=0
logb n
∑ a /bd )i = Ο(nd logb n)
![Page 45: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/45.jpg)
CS200 Algorithms and Data Structures Colorado State University
45
Case 3. a > bd
• (a/bd) > 1• Therefore the largest term is the last one.
€
nd
i=0
logb n
∑ (a /bd )i = nd (i=0
logb n
∑ a /bd )i
€
nd (a /bd )logb n = nd (alogb n /(bd )logb n )
= nd (n logb a /n logb b d
)
= nd (n logb a /nd )
= n logb a
nd (i=0
logb n
∑ a /bd )i = Ο(n logb a )
![Page 46: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/46.jpg)
CS200 Algorithms and Data Structures Colorado State University
46
Complexity of MergeSort with Master Theorem (1/2)
• Mergesort splits a list to be sorted twice per level.
• Uses fewer than n comparisons to merge the two sorted lists of n/2 items each into one sorted list.
• Function M(n) satisfying the divide-and-conquer recurrence relation
• M(n) = 2M(n/2) + n
![Page 47: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/47.jpg)
CS200 Algorithms and Data Structures Colorado State University
47
Complexity of MergeSort with Master Theorem (2/2)
M(n) = 2M(n/2) + n
for the mergesort algorithm
f(n) = a f(n / b) + nd
= 2 f(n / 2) + n1
Therefore, d = 1, b = 2, a = 2.
bd = 21 = 2
It satisfies the second condition of the Master theorem.
So, f(n) = O(ndlog2n)
= O(n1log2n)
= O(nlog2n)
![Page 48: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/48.jpg)
CS200 Algorithms and Data Structures Colorado State University
48
Quicksort?
![Page 49: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/49.jpg)
CS200 Algorithms and Data Structures Colorado State University
49
The Closest-Pair Problem• Consider the problem of determining the
closest pair of points in a set of n points (x1, y1)…(xn, yn) in the plain, where the distance
between two points (xi, yi) and (xj, yj) is the usual Euclidean distance,
________________ √(xi - xj)2 + (yi - yj)2
![Page 50: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/50.jpg)
CS200 Algorithms and Data Structures Colorado State University
50
Solution (1/5)
• Calculate the distance between every pair of points and then find the smallest of these distances– It will take O(n2)
![Page 51: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/51.jpg)
CS200 Algorithms and Data Structures Colorado State University
51
Solution (2/5)
• Assume that we have n points such that n = 2k.
• When n = 2, the distance between two points is the minimum distance.
• We sort the points in order of increasing x coordinates, and also sort the points in order of
increasing y coordinates using mergesort.
– It will take O(n log n)
• Using sorted list by x coordinate, divide the group of points into two (Right and Left)
![Page 52: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/52.jpg)
CS200 Algorithms and Data Structures Colorado State University
52
Solution (3/5)l
Closest pair
dR
dL
L Rdd
![Page 53: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/53.jpg)
CS200 Algorithms and Data Structures Colorado State University
53
Solution (4/5)• Find the closest pair within R and L.
• Merge the region R and L by examining the points on the boarder area.
– Find closest pair between pairs from R and L. (d = min(dr, dl))
– Check the points within the strip of the maximum distance of d from the border.
– Sort the points within the strip with associated y coordinate.
– Beginning with a point in the strip with the smallest y coordinate.
![Page 54: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/54.jpg)
CS200 Algorithms and Data Structures Colorado State University
54
Solution (5/5)
• We need at most 7 comparisons for each of the points in the strip.
d/2 d/2 d/2 d/2
d/2
d/2 d/√2
l
d d
![Page 55: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/55.jpg)
CS200 Algorithms and Data Structures Colorado State University
55
Analysis• We find the function f(n) satisfying the
recurrence relations
f(n) = 2 f(n / 2) + 7n where f(2) = 1
By the master theorem, f(n) =O(n logn)
![Page 56: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/56.jpg)
CS200 Algorithms and Data Structures Colorado State University
56
Outline
• Recurrence Relations• Divide and Conquer• Understanding the Complexity of
Algorithms
![Page 57: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/57.jpg)
CS200 Algorithms and Data Structures Colorado State University
57
Understanding the Complexity of Algorithm
• Commonly used terminology for the complexity of algorithms
O(1) : Constant complexity
O(logn) : Logarithmic complexity
O(n) : Linear complexity
O(n log n) : Linearithmic complexity
O(nb) : Polynomial complexityO(bn), where b > 1 : Exponential complexityO(n!) : Factorial complexity
![Page 58: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/58.jpg)
CS200 Algorithms and Data Structures Colorado State University
58
Tractability
• A problem that is solvable using an algorithm with polynomial worst-case complexity is called tractable.
• If estimation has high degree or if the coefficients are extremely large, the algorithm may take an extremely long time to solve the problem.
![Page 59: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/59.jpg)
CS200 Algorithms and Data Structures Colorado State University
59
Intractable problems and Unsolvable problems
• If the problem cannot be solved using an algorithm with worst-case polynomial time complexity, such problems are called intractable.
• If it can be shown that no algorithm exists for solving them, such problems are called unsolvable.
![Page 60: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/60.jpg)
CS200 Algorithms and Data Structures Colorado State University
60
Class NP and NP-Complete
• Problems for which a solution can be checked in polynomial time are said to belong to the class NP.– Tractable problems belong to class P.
• NP-complete problems are an important class of problems– If any of these problems can be solved by a polynomial
worst-case time algorithm then …• All problems in the class NP can be solved by polynomial worst-
case time algorithms.
![Page 61: CS200 Algorithms and Data StructuresColorado State University Part 5. Computational Complexity (3) CS 200 Algorithms and Data Structures 1.](https://reader035.fdocuments.in/reader035/viewer/2022062713/56649cef5503460f949bdc63/html5/thumbnails/61.jpg)
CS200 Algorithms and Data Structures Colorado State University
61
NP-Hard• NP-Hard problems are at least as hard as the
hardest problems in NP