Introduction to Computer Science Lecture 5:...
Transcript of Introduction to Computer Science Lecture 5:...
![Page 1: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/1.jpg)
Introduction to Computer ScienceLecture 5: Algorithms
Tian-Li Yu
Taiwan Evolutionary Intelligence Laboratory (TEIL)Department of Electrical Engineering
National Taiwan University
Slides made by Tian-Li Yu, Jie-Wei Wu, and Chu-Yu Hsu
Tian-Li Yu Algorithms 1 / 41
![Page 2: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/2.jpg)
Algorithms
Definitions
Algorithm: ordered set of unambiguous, executable steps that defines a terminatingprocess.
Program: formal representation of an algorithm.
Process: activity of executing a program.
Primitives, programming languages.
Abstraction
Tian-Li Yu Algorithms 2 / 41
![Page 3: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/3.jpg)
Algorithms
Folding a Bird
Refer to figure 5.2 in Computer Science An Overview 11th Edition.
Tian-Li Yu Algorithms 3 / 41
![Page 4: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/4.jpg)
Algorithms
Origami Primitives
Refer to figure 5.3 in Computer Science An Overview 11th Edition.
Tian-Li Yu Algorithms 4 / 41
![Page 5: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/5.jpg)
Algorithms
Algorithm Representation
Flowchart
Popular in 50s and 60sOverwhelming for complexalgorithms
Pseudocode
A loosen version of formalprogramming languages
Buy new lamp
Replace bulbBulb
burned out?
Lamp doesn’t work
Lamp
plugged in? Plug in lampNo
No
Yes
Tian-Li Yu Algorithms 5 / 41
![Page 6: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/6.jpg)
Algorithms
Pseudocode Primitives
Assignmentname ← expression
Conditional selectionif (condition) then (activity)
Repeated executionwhile (condition) do (activity)
Procedureprocedure name
procedure GreetingsCount ← 3while (Count > 0) do
(print the message “Hello” and
Count ← Count − 1)
Tian-Li Yu Algorithms 6 / 41
![Page 7: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/7.jpg)
Problem Solving
Polya’s Problem Solving Steps
How to Solve It by George Polya, 1945.
1 Understand the problem.
2 Devise a plan for solving the problem.
3 Carry out the plan.
4 Evaluate the solution for accuracy and its potential as a toolfor solving other problems.
Tian-Li Yu Algorithms 7 / 41
![Page 8: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/8.jpg)
Problem Solving
Problem Solving
Top-down
Stepwise refinementProblem decomposition
Bottom-up
Both methods often complement each other
Usually,
planning → top-downimplementation → bottom-up
Tian-Li Yu Algorithms 8 / 41
![Page 9: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/9.jpg)
Problem Solving Iterations
Iterations
Loop control
Initialize: Establish an initial state that will be modifiedtoward the termination condition
Test: Compare the current state to the terminationcondition and terminate the repetition if equal
Modify: Change the state in such a way that it movestoward the termination condition
Tian-Li Yu Algorithms 9 / 41
![Page 10: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/10.jpg)
Problem Solving Iterations
Loops
Pre-test(while. . . )
Activity
Test
condition
Condition
false
Condition
true
Post-test(do. . . while, repeat. . . until)
Activity
Test
conditionCondition
falseCondition
true
Tian-Li Yu Algorithms 10 / 41
![Page 11: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/11.jpg)
Problem Solving Insertion Sort
Insertion Sort
Fred
Alex
Diana
Byron
Carol
Fred
Alex
Diana
Byron
Carol
Fred Alex
Diana
Byron
Carol
Sorted
Initial list:
Fred
Alex
Diana
Byron
Carol
Sorted list:
Fred
Alex
Diana
Byron
Carol
Fred
Alex
Diana
Byron
Carol
Fred
Alex
Fred
Alex Alex
Diana
Byron
Carol
Diana
Byron
Carol
Sorted
Fred
Diana
Byron
Carol
Fred
Alex
Diana
Byron
Carol
Fred
Alex Alex
Diana
Fred
Alex
Diana
Fred
Diana
Fred
Diana
Byron
Carol
Byron
Carol
Sorted Byron
Byron
Carol
Alex
Carol
Fred
Alex Alex
Diana
Fred
Diana
Byron
Fred
Alex
Diana
Byron Byron
Carol
CarolSorted
Fred
Diana
Carol
Carol
Byron
Alex
Fred
Diana
Carol
Byron
AlexTian-Li Yu Algorithms 11 / 41
![Page 12: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/12.jpg)
Problem Solving Insertion Sort
Pseudocode for Insertion Sort
procedure InsertionSort (List)
1 N ← 22 while (the value of N does not exceed the length of List) do3 (Select the N-th entry in List as the pivot entry4 Move the pivot to a temporary location leaving a hole in List5 while (there is a name above the hole and that name is greater
than the pivot) do6 (move the name above the hole down into the hole leaving a
hole above the name)7 Move the pivot entry into the hole in List8 N ← N + 19 )
Tian-Li Yu Algorithms 12 / 41
![Page 13: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/13.jpg)
Problem Solving Recursion
Binary Search
Original list First sublist Second sublist
Alice
Bob
Carol
David
Elaine
Fred
George
HarryIrene
John
Kelly
Larry
Mary
Nancy
Oliver
Irene
John
Kelly
Larry
Mary
Nancy
Oliver
Irene
John
Kelly
Tian-Li Yu Algorithms 13 / 41
![Page 14: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/14.jpg)
Problem Solving Recursion
Pseudocode for Binary Search
procedure BinarySearch (List, TargetValue)
1 if (List empty) then2 (Report that the search failed.)3 else (4 Select the “middle” entry in List to be the TestEntry5 Execute the block of instructions below that is associated with the appropriate
case.6 case 1: TagetValue = TestEntry7 (Report that the search succeeded.)8 case 2: TagetValue < TestEntry9 (Search the portion of List preceding TestEntry for TargetValue, and
report the result of that search.)10 case 3: TagetValue > TestEntry11 (Search the portion of List succeeding TestEntry for TargetValue, and
report the result of that search.)12 ) end if
Tian-Li Yu Algorithms 14 / 41
![Page 15: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/15.jpg)
Problem Solving Recursion
Recursive Problem Solving (contd.)
Factorial int factorial (int x) if (x==0) return 1;return x ∗ factorial(x−1);
Do not abuseCalling functions takes a long timeAvoid tail recursions
int factorial (int x) int product = 1;for (int i=1; i<=x; ++i)
product ∗= i;return product;
int Fibonacci (int x) if (x==0) return 0;if (x==1) return 1;return Fibonacci(x−2) + Fibonacci(x−1);
Tian-Li Yu Algorithms 15 / 41
![Page 16: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/16.jpg)
Dynamic Programming
Divide and Conquer vs. Dynamic Programming
Divide and conquer (D&C):
SubproblemsTop-downBinary search, merge sort, ...
Dynamic programming (DP):
Subprograms share subsubproblemsBottom-upShortest path, matrix-chain multiplication, ...
Tian-Li Yu Algorithms 16 / 41
![Page 17: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/17.jpg)
Dynamic Programming Shortest Path
Shortest Path
ShortestAE = mini∈A,B,C ,D,E(ShortestAi + ShortestiE )
A
B C
D E
3
5
1 2
5
3
6
A:3
Tian-Li Yu Algorithms 17 / 41
![Page 18: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/18.jpg)
Dynamic Programming Shortest Path
Shortest Path (contd.)
ShortestAE = mini∈A,B,C ,D,E(ShortestAi + ShortestiE )
A
B C
D E
3
5
1 2
5
3
6
A:3
B:4
B:8
Tian-Li Yu Algorithms 18 / 41
![Page 19: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/19.jpg)
Dynamic Programming Shortest Path
Shortest Path (contd.)
ShortestAE = mini∈A,B,C ,D,E(ShortestAi + ShortestiE )
A
B C
D E
3
5
1 2
5
3
6
A:3
B:4
D:7
D:10
Tian-Li Yu Algorithms 19 / 41
![Page 20: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/20.jpg)
Dynamic Programming Shortest Path
Shortest Path (contd.)
ShortestAE = mini∈A,B,C ,D,E(ShortestAi + ShortestiE )
A
B C
D E
3
5
1 2
5
3
6
A:3
B:4
D:7
C:9
Tian-Li Yu Algorithms 20 / 41
![Page 21: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/21.jpg)
Dynamic Programming Shortest Path
Shortest Path (contd.)
ShortestAE = mini∈A,B,C ,D,E(ShortestAi + ShortestiE )
A
B C
D E
3
5
1 2
5
3
6
A:3
B:4
D:7
C:9
Tian-Li Yu Algorithms 21 / 41
![Page 22: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/22.jpg)
Dynamic Programming Matrix-Chain Multiplication
Matrix-Chain Multiplication
Matrices: A : p × q; B : q × r
Then C = A · B is a p × r matrix.
Ci,j =∑q
k=1 Ai,k · Bk,j
Time complexity: pqr scalar multiplications
The matrix-chain multiplication problem
Given a chain < A1, A2, ..., An > of n matrices, which Ai is of dimension pi−1 × pi ,parenthesize properly to minimize # of scalar multiplications.
Tian-Li Yu Algorithms 22 / 41
![Page 23: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/23.jpg)
Dynamic Programming Matrix-Chain Multiplication
Matrix-Chain Multiplication
(p × q) · (q × r) → (p × r)(pqr) scalar multiplications
A1, A2, A3 : (10× 100), (100× 5), (5× 50)
(A1A2)A3 → (10× 100× 5) + (10× 5× 50) = 7500
A1(A2A3) → (100× 1000× 50) + (1000× 50× 50) = 75000
4 matrices:((A1A2)A3)A4
A1(A2A3)A4
(A1A2)(A3A4)A1(A2(A3A4))
Tian-Li Yu Algorithms 23 / 41
![Page 24: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/24.jpg)
Dynamic Programming Matrix-Chain Multiplication
The Minimal # of Multiplications
m[i , j ]: minimal # of multiplications to compute matrix Ai ,j = AiAi+1...Aj , where1 ≤ i ≤ j ≤ n.
m[i , j ] =
0 , i = j
mink (m[i , k] + m[k + 1, j ] + pi−1pkpj) , i 6= j
Tian-Li Yu Algorithms 24 / 41
![Page 25: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/25.jpg)
Dynamic Programming Matrix-Chain Multiplication
Bottom-Up DP
A1 : 7× 3
A2 : 3× 1
A3 : 1× 2
A4 : 2× 4
p0 = 7
p1 = 3
p2 = 1
p3 = 2
p4 = 4
m[i , i ] = 0
m[1, 2] = 0 + 0 + 7× 3× 1 = 21
m[2, 3] = 6
m[3, 4] = 8
m[1, 3] = 35
min 21 + 0 + 7× 1× 2, 0 + 6 + 7× 3× 2
m[2, 4] = 20min 6 + 0 + 3× 2× 4, 0 + 8 + 3× 1× 4
Tian-Li Yu Algorithms 25 / 41
![Page 26: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/26.jpg)
Dynamic Programming Matrix-Chain Multiplication
Bottom-Up DP (contd.)
A1 : 7× 3
A2 : 3× 1
A3 : 1× 2
A4 : 2× 4
p0 = 7
p1 = 3
p2 = 1
p3 = 2
p4 = 4
m[1, 4] = minm[1, 1] + m[2, 4] + 7× 3× 4,m[1, 2] + m[3, 4] + 7× 1× 4,m[1, 3] + m[4, 4] + 7× 2× 4
= 57
Ans: (A1A2)(A3A4)
Tian-Li Yu Algorithms 26 / 41
![Page 27: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/27.jpg)
Dynamic Programming Matrix-Chain Multiplication
Table Filling
A1 : 7× 3
A2 : 3× 1
A3 : 1× 2
A4 : 2× 4
p0 = 7
p1 = 3
p2 = 1
p3 = 2
p4 = 4
A1
A2
A3
A4
0 0 0 0
21 6 8
35 20
57
Tian-Li Yu Algorithms 27 / 41
![Page 28: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/28.jpg)
Top-down vs. Botton-up
Top-Down Manner (Binary Search)
Full list
First half list Second half list
First quarter Second quarter
Tian-Li Yu Algorithms 28 / 41
![Page 29: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/29.jpg)
Top-down vs. Botton-up
Bottom-up Manner (Shortest Path)
Two adjacent nodes
Three adjacent nodes
Four adjacent nodes
Tian-Li Yu Algorithms 29 / 41
![Page 30: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/30.jpg)
Analyzing Algorithms Efficiency
Algorithm Efficiency
Number of instructions executed
Execution time
What about on different machines?
O, Ω, Θ notations
Pronunciations: big-o, big-omega, big-theta
Tian-Li Yu Algorithms 30 / 41
![Page 31: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/31.jpg)
Analyzing Algorithms Efficiency
Asymptotic Analysis
Exact analysis is often difficult and tedious.
Asymptotic analysis emphasizes the behavior of the algorithm when n tends to infinity.
Asymptotic
Upper boundLower boundTight bound
n
T(n) asymptotic
upper bond
asymptotic
lower bond
Tian-Li Yu Algorithms 31 / 41
![Page 32: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/32.jpg)
Analyzing Algorithms Efficiency
Big-O
O(g(n)) = f(n)| ∃c > 0, n0 > 0 s.t. ∀n ≥ n0, 0 ≤ f(n) ≤ cg(n)
Asymptotic upper bound
If f(n) is a member of the set ofO(g(n)), we write f(n) = O(g(n)).
Examples100n = O(n2)n100 = O(2n)2n + 100 = O(n)
nn0
f(n)
c g(n)
f(n) = O(g(n))
Tian-Li Yu Algorithms 32 / 41
![Page 33: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/33.jpg)
Analyzing Algorithms Efficiency
Big-Omega
Ω(g(n)) = f(n)| ∃c > 0, n0 > 0 s.t.∀n ≥ n0, 0 ≤ cg(n) ≤ f(n)
Asymptotic lower bound
If f(n) is a member of the set ofΩ(g(n)), we write f(n) = Ω(g(n)).
Examples0.01n2 = Ω(n)2n = Ω(n100)2n + 100 = Ω(n)
nn0
f(n)
c g(n)
f(n) = Ω(g(n))
Tian-Li Yu Algorithms 33 / 41
![Page 34: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/34.jpg)
Analyzing Algorithms Efficiency
Big-Theta
Θ(g(n)) = f(n)| ∃c1, c2, n0 > 0 s.t. ∀n ≥ n0, 0 ≤ c1g(n) ≤ f(n) ≤ c2g(n)
Asymptotic tight bound
If f(n) is a member of the set ofΘ(g(n)), we write f(n) = Θ(g(n)).
Examples0.01n2 = Θ(n2)2n + 100 = Θ(n)n + log n = Θ(n)
nn0
f(n)
c g(n)2
c g(n)1
f(n) = Θ(g(n))
Theorem
f(n) = Θ(g(n)) iff f(n) = O(g(n)) and f(n) = Ω(g(n))
Tian-Li Yu Algorithms 34 / 41
![Page 35: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/35.jpg)
Analyzing Algorithms Efficiency
Efficiency Analysis
Best, worst, average cases
Comparisons made for each pivot
4th pivot
Barbara
Carol
David
Elaine
Alfred7
8
9
10
3rd pivot
Carol
David
Elaine
Barbara
Alfred
4
5
6
2nd pivot
David
Elaine
Carol
Barbara
Alfred
2
3
Sorted list
Alfred
Barbara
Carol
Elaine
David
Initial list
Elaine
David
Carol
Barbara
Alfred
1st pivot
Elaine
David
Carol
Barbara
Alfred
1
Worst case for insertion sort
Worst: (n2 − n)/2, best: (n − 1), average: Θ(n2)
Tian-Li Yu Algorithms 35 / 41
![Page 36: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/36.jpg)
Analyzing Algorithms Correctness
Software Verification
Traveler’s gold chain
Cut
Cut Cut Cut
Tian-Li Yu Algorithms 36 / 41
![Page 37: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/37.jpg)
Analyzing Algorithms Correctness
Assertion for “While”
Body
Modifiy
Initialize
Test
condition
Condition
false
Condition
true Loop invariant
and terminate condition
Loop invariant
Precondition
Precondition
Loop invariant
Termination condition
Tian-Li Yu Algorithms 37 / 41
![Page 38: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/38.jpg)
Analyzing Algorithms Correctness
Correct or Not?
Count ← 0Remainder ← Dividendrepeat (Remainder ← Remainder − Divisor
Count ← Count + 1)until (Remainder < Divisor)Quotient ← Count
Problematic
Remainder > 0?
Preconditions:
Dividend > 0Divisor > 0Count = 0Remainder = Dividend
Loop invariants:
Dividend > 0Divisor > 0Dividend =Count · Divisor + Remainder
Termination condition:
Remainder < Divisor
Tian-Li Yu Algorithms 38 / 41
![Page 39: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/39.jpg)
Analyzing Algorithms Correctness
Verification of Insertion Sort
Loop invariant of the outer loop
Each time the test for termination is performed, the names preceding the N-th entry form asorted list
Termination condition
The value of N is greater than the length of the list.
If the loop terminates, the list is sorted
Tian-Li Yu Algorithms 39 / 41
![Page 40: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/40.jpg)
Analyzing Algorithms Correctness
Final Words for Software Verification
In general, not easy.
Need a formal PL with better properties.
Tian-Li Yu Algorithms 40 / 41
![Page 41: Introduction to Computer Science Lecture 5: Algorithmsocw.aca.ntu.edu.tw/ocw_files/101S210/101S210_CS10L01 .pdf · The matrix-chain multiplication problem Given a chain < A 1,](https://reader034.fdocuments.in/reader034/viewer/2022051603/5ff44c4724c6d94df53bf295/html5/thumbnails/41.jpg)
Analyzing Algorithms Correctness
License
Page File Licensing Source/ author
7
”George Plya ca 1973”.,Author: Thane Plambeck fromPalo Alto, California, Original ca 1973, scanned March14, 2007 Source: http://en.wikipedia.org/wiki/File:
George_P%C3%B3lya_ca_1973.jpg, Date:2013/05/14, Licensedunder the terms of the cc-by-2.0.
Tian-Li Yu Algorithms 41 / 41