CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic...
Transcript of CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic...
![Page 1: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/1.jpg)
CS141: Intermediate Data Structures
and Algorithms
Dynamic Programming
Amr Magdy
![Page 2: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/2.jpg)
Programming?
In this context, programming is a tabular method
Storing previously calculated results in a table, and look it up later
Other examples:
Linear programing
Integer programming
2
![Page 3: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/3.jpg)
Main idea
3
Fibonacci recursion tree
![Page 4: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/4.jpg)
Main idea
4
Fibonacci recursion tree
![Page 5: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/5.jpg)
Main idea
Do not repeat same work, store the result and look it up later
5
Fibonacci recursion tree
![Page 6: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/6.jpg)
Main idea: DP vs Divide & Conquer
Do not repeat same work, store the result and look it up later
Is MergeSort(A, 1, n/2) and MergeSort(A, n/2, n) the same?
Is Fib(n-2) and Fib(n-2) the same?
6
![Page 7: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/7.jpg)
Main idea: DP vs Divide & Conquer
Do not repeat same work, store the result and look it up later
Is MergeSort(A, 1, n/2) and MergeSort(A, n/2, n) the same?
No
Is Fib(n-2) and Fib(n-2) the same?
Yes
7
![Page 8: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/8.jpg)
Main idea: DP vs Divide & Conquer
Do not repeat same work, store the result and look it up later
Is MergeSort(A, 1, n/2) and MergeSort(A, n/2, n) the same?
No
Is Fib(n-2) and Fib(n-2) the same?
Yes
Same function + same input → same output (DP)
DC same function has different inputs
8
![Page 9: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/9.jpg)
Rod Cutting Problem
9
![Page 10: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/10.jpg)
Rod Cutting Problem
10
![Page 11: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/11.jpg)
Rod Cutting Problem
11
![Page 12: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/12.jpg)
Rod Cutting Problem
12
![Page 13: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/13.jpg)
Rod Cutting Problem
13
![Page 14: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/14.jpg)
Rod Cutting Problem
14
![Page 15: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/15.jpg)
Rod Cutting Problem
15
![Page 16: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/16.jpg)
Rod Cutting Problem
16
![Page 17: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/17.jpg)
Rod Cutting Problem
17
![Page 18: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/18.jpg)
Rod Cutting Problem
Given a rod of length n and prices pi, find the cutting
strategy that makes the maximum revenue
In the example: (2+2) cutting makes r=5+5=10
18
![Page 19: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/19.jpg)
Rod Cutting Problem
Naïve: try all combinations
19
![Page 20: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/20.jpg)
Rod Cutting Problem
Naïve: try all combinations
How many?
20
![Page 21: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/21.jpg)
Rod Cutting Problem
Naïve: try all combinations
How many?
0 cut: 1 1 cut: (n-1) 2 cuts: n-1C2 = Θ(n2)
3 cuts: n-1C3 = Θ(n3) …………… n cuts: n-1Cn-1 = Θ(1)
21
![Page 22: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/22.jpg)
Rod Cutting Problem
Naïve: try all combinations
How many?
0 cut: 1 1 cut: (n-1) 2 cuts: n-1C2 = Θ(n2)
3 cuts: n-1C3 = Θ(n3) …………… n cuts: n-1Cn-1 = Θ(1)
Total: Θ(1+n+n2+n3+…+nn/2+….+n3+n2+n+1)
Total: O(nn)
22
![Page 23: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/23.jpg)
Rod Cutting Problem
Naïve: try all combinations
How many?
0 cut: 1 1 cut: (n-1) 2 cuts: n-1C2 = Θ(n2)
3 cuts: n-1C3 = Θ(n3) …………… n cuts: n-1Cn-1 = Θ(1)
Total: Θ(1+n+n2+n3+…+nn/2+….+n3+n2+n+1)
Total: O(nn)
Better solution? Can I divide and conquer?
23
![Page 24: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/24.jpg)
Rod Cutting Problem
Naïve: try all combinations
How many?
0 cut: 1 1 cut: (n-1) 2 cuts: n-1C2 = Θ(n2)
3 cuts: n-1C3 = Θ(n3) …………… n cuts: n-1Cn-1 = Θ(1)
Total: Θ(1+n+n2+n3+…+nn/2+….+n3+n2+n+1)
Total: O(nn)
Better solution? Can I divide and conquer?
24
divide
![Page 25: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/25.jpg)
Rod Cutting Problem
Naïve: try all combinations
How many?
0 cut: 1 1 cut: (n-1) 2 cuts: n-1C2 = Θ(n2)
3 cuts: n-1C3 = Θ(n3) …………… n cuts: n-1Cn-1 = Θ(1)
Total: Θ(1+n+n2+n3+…+nn/2+….+n3+n2+n+1)
Total: O(nn)
Better solution? Can I divide and conquer?
25
divide
conquer
![Page 26: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/26.jpg)
Rod Cutting Problem
Naïve: try all combinations
How many?
0 cut: 1 1 cut: (n-1) 2 cuts: n-1C2 = Θ(n2)
3 cuts: n-1C3 = Θ(n3) …………… n cuts: n-1Cn-1 = Θ(1)
Total: Θ(1+n+n2+n3+…+nn/2+….+n3+n2+n+1)
Total: O(nn)
Better solution? Can I divide and conquer?
But I don’t really know the best way to divide
26
divide
conquer
![Page 27: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/27.jpg)
Rod Cutting Problem
Naïve: try all combinations
How many?
0 cut: 1 1 cut: (n-1) 2 cuts: n-1C2 = Θ(n2)
3 cuts: n-1C3 = Θ(n3) …………… n cuts: n-1Cn-1 = Θ(1)
Total: Θ(1+n+n2+n3+…+nn/2+….+n3+n2+n+1)
Total: O(nn)
Better solution? Can I divide and conquer?
But I don’t really know the best way to divide
27
![Page 28: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/28.jpg)
Rod Cutting Problem
Naïve: try all combinations
How many?
0 cut: 1 1 cut: (n-1) 2 cuts: n-1C2 = Θ(n2)
3 cuts: n-1C3 = Θ(n3) …………… n cuts: n-1Cn-1 = Θ(1)
Total: Θ(1+n+n2+n3+…+nn/2+….+n3+n2+n+1)
Total: O(nn)
Better solution? Can I divide and conquer?
But I don’t really know the best way to divide
28
![Page 29: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/29.jpg)
Rod Cutting Problem
Naïve: try all combinations
How many?
0 cut: 1 1 cut: (n-1) 2 cuts: n-1C2 = Θ(n2)
3 cuts: n-1C3 = Θ(n3) …………… n cuts: n-1Cn-1 = Θ(1)
Total: Θ(1+n+n2+n3+…+nn/2+….+n3+n2+n+1)
Total: O(nn)
Better solution? Can I divide and conquer?
But I don’t really know the best way to divide
29
![Page 30: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/30.jpg)
Rod Cutting Problem
Recursive top-down algorithm
30
![Page 31: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/31.jpg)
Rod Cutting Problem
Better solution? Can I divide and conquer?
But I don’t really know the best way to divide
How many subproblems (recursive calls)?
31
![Page 32: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/32.jpg)
Rod Cutting Problem
Better solution? Can I divide and conquer?
But I don’t really know the best way to divide
How many subproblems (recursive calls)?
32
![Page 33: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/33.jpg)
Rod Cutting Problem
Better solution? Can I divide and conquer?
But I don’t really know the best way to divide
How many subproblems (recursive calls)?
33
![Page 34: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/34.jpg)
Rod Cutting Problem
Better solution? Can I divide and conquer?
But I don’t really know the best way to divide
How many subproblems (recursive calls)?
34
(Prove by induction)
![Page 35: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/35.jpg)
Rod Cutting Recursive Complexity
Find the complexity of T n = 1 + σ𝑗=0𝑛−1𝑇(𝑗)
Proof by induction:
Assume the solution is some function X(n)
Show that X(n) is true for the smallest n (the base case), e.g., n=0
Prove that X(n+1) is a solution for T(n+1) given X(n)
You are done
Given T n = 1 + σ𝑗=0𝑛−1𝑇(𝑗)
Assume T n = 2𝑛
T 0 = 1 + σ𝑗=0−1 𝑇 𝑗 = 1 = 20 (base case)
T n + 1 = 1 + σ𝑗=0𝑛 𝑇 𝑗 = 1 +
σ𝑗=0𝑛−1𝑇 𝑗 + 𝑇 𝑛 = 𝑇 𝑛 + 𝑇 𝑛 = 2𝑇 𝑛 = 2 ∗ 2𝑛 = 2𝑛+1
Then, T n = 2𝑛
35
![Page 36: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/36.jpg)
Rod Cutting Problem
Better solution? Can I divide and conquer?
But I don’t really know the best way to divide
How many subproblems (recursive calls)?
Can we do better?36
(Prove by induction)
![Page 37: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/37.jpg)
Rod Cutting Problem
Better solution? Can I divide and conquer?
But I don’t really know the best way to divide
How many subproblems (recursive calls)?
Can we do better?37
![Page 38: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/38.jpg)
Rod Cutting Problem
Subproblem overlapping
No need to re-solve the same problem
38
![Page 39: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/39.jpg)
Rod Cutting Problem
Subproblem overlapping
No need to re-solve the same problem
Idea:
Solve each subproblem once
Write down the solution in a lookup table (array, hashtable,…etc)
When needed again, look it up in Θ(1)
39
![Page 40: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/40.jpg)
Rod Cutting Problem
Subproblem overlapping
No need to re-solve the same problem
Idea:
Solve each subproblem once
Write down the solution in a lookup table (array, hashtable,…etc)
When needed again, look it up in Θ(1)
40
Dynamic Programming
![Page 41: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/41.jpg)
Rod Cutting Problem
Recursive top-down dynamic programming algorithm
41
![Page 42: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/42.jpg)
Rod Cutting Problem
Recursive top-down dynamic programming algorithm
42
Θ(n2)
![Page 43: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/43.jpg)
Rod Cutting Problem
Bottom-up dynamic programming algorithm
I know I will need the smaller problems → solve them first
Solve problem of size 0, then 1, then 2, then 3, … then n
43
![Page 44: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/44.jpg)
Rod Cutting Problem
Bottom-up dynamic programming algorithm
I know I will need the smaller problems → solve them first
Solve problem of size 0, then 1, then 2, then 3, … then n
44
![Page 45: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/45.jpg)
Rod Cutting Problem
Bottom-up dynamic programming algorithm
I know I will need the smaller problems → solve them first
Solve problem of size 0, then 1, then 2, then 3, … then n
45
Θ(n2)
![Page 46: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/46.jpg)
Elements of a Dynamic Programming
Problem
Optimal substructure
Optimal solution of a larger problem comes from optimal
solutions of smaller problems
Subproblem overlapping
Same exact sub-problems are solved again and again
46
![Page 47: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/47.jpg)
Optimal Substructure
Longest path from A to F LP(A,F) includes node B
But it does not include LP(A,B) and LP(B,F)
i.e., optimal solutions for the subproblems A→B, and B→F
cannot be combined to find an optimal solution for A → F
47
![Page 48: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/48.jpg)
Dynamic Programming vs. D&C
How different?
48
![Page 49: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/49.jpg)
Dynamic Programming vs. D&C
How different?
No subproblem overlapping
Each subproblem with distinct input is a new problem
Not necessarily optimization problems, i.e., no objective function
49
![Page 50: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/50.jpg)
Reconstructing Solution
Rod cutting problem: What are the actual cuts?
Not only the best revenue (the optimal objective function value)
50
![Page 51: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/51.jpg)
Reconstructing Solution
Rod cutting problem: What are the actual cuts?
Not only the best revenue (the optimal objective function value)
51
![Page 52: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/52.jpg)
Reconstructing Solution
Rod cutting problem: What are the actual cuts?
Not only the best revenue (the optimal objective function value)
Let’s trace examples
52
![Page 53: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/53.jpg)
Matrix Chain Multiplication
How to multiply a chain of four matrices ?
53
![Page 54: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/54.jpg)
Matrix Chain Multiplication
How to multiply a chain of four matrices ?
54
![Page 55: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/55.jpg)
Matrix Chain Multiplication
How to multiply a chain of four matrices ?
Does it really make
a difference?
55
![Page 56: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/56.jpg)
Matrix Chain Multiplication
How to multiply a chain of four matrices ?
Does it really make
a difference?
# of multiplications:
A.rows*B.cols*A.cols
56
![Page 57: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/57.jpg)
Matrix Chain Multiplication
Does it really make
a difference?
# of multiplications:
A.rows*B.cols*A.cols
Example:
A1*A2*A3
Dimensions:
10x100x5x50
# of multiplications in ((A1*A2)*A3)=10*100*5+10*5*50=7.5K
# of multiplications in (A1*(A2*A3))=100*5*50+10*100*50=75K
57
![Page 58: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/58.jpg)
Matrix Chain Multiplication
Given n matrices A1 A2 … An of dimensions p0 p1 … pn,
find the optimal parentheses to multiply the matrix chain
58
![Page 59: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/59.jpg)
Matrix Chain Multiplication
Given n matrices A1 A2 … An of dimensions p0 p1 … pn,
find the optimal parentheses to multiply the matrix chain
A1 A2 A3 A4 A5 … An
59
![Page 60: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/60.jpg)
Matrix Chain Multiplication
Given n matrices A1 A2 … An of dimensions p0 p1 … pn,
find the optimal parentheses to multiply the matrix chain
A1 A2 A3 A4 A5 … An
(A1 A2 A3)(A4 A5 … An)
60
![Page 61: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/61.jpg)
Matrix Chain Multiplication
Given n matrices A1 A2 … An of dimensions p0 p1 … pn,
find the optimal parentheses to multiply the matrix chain
A1 A2 A3 A4 A5 … An
(A1 A2 A3)(A4 A5 … An)
Sub-chains C1 = (A1 A2 A3), C2 = (A4 A5 … An)
61
![Page 62: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/62.jpg)
Matrix Chain Multiplication
Given n matrices A1 A2 … An of dimensions p0 p1 … pn,
find the optimal parentheses to multiply the matrix chain
A1 A2 A3 A4 A5 … An
(A1 A2 A3)(A4 A5 … An)
Sub-chains C1 = (A1 A2 A3), C2 = (A4 A5 … An)
Total Cost C = cost(C1)+cost(C2)+p0p3pn
62
![Page 63: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/63.jpg)
Matrix Chain Multiplication
Given n matrices A1 A2 … An of dimensions p0 p1 … pn,
find the optimal parentheses to multiply the matrix chain
A1 A2 A3 A4 A5 … An
(A1 A2 A3)(A4 A5 … An)
Sub-chains C1 = (A1 A2 A3), C2 = (A4 A5 … An)
Total Cost C = cost(C1)+cost(C2)+p0p3pn
Then, if cost(C1) and cost(C2) are minimal (i.e., optimal),
then C is optimal (optimal substructure holds)
63
![Page 64: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/64.jpg)
Matrix Chain Multiplication
Given n matrices A1 A2 … An of dimensions p0 p1 … pn, find
the optimal parentheses to multiply the matrix chain
A1 A2 A3 A4 A5 … An
(A1 A2 A3)(A4 A5 … An)
Sub-chains C1 = (A1 A2 A3), C2 = (A4 A5 … An)
Total Cost C = cost(C1)+cost(C2)+p0p3pn
Then, if cost(C1) and cost(C2) are minimal (i.e., optimal),
then C is optimal (optimal substructure holds)
Proof by contradiction:
Given C is optimal, are cost(C1)=c1 and cost(C2)=c2 optimal?
Assume c1 is NOT optimal, then Ǝ an optimal solution of cost c1’ < c1
Then c1’+c2+p < c1+c2+p → C’ < C
Then C is not optimal → contradiction!
Then C1 has to be optimal → optimal substructure holds64
![Page 65: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/65.jpg)
Matrix Chain Multiplication
Given n matrices A1 A2 … An of dimensions p0 p1 … pn,
find the optimal parentheses to multiply the matrix chain
A1 A2 A3 A4 A5 … An
(A1 A2 A3)(A4 A5 … An)
Sub-chains C1 = (A1 A2 A3), C2 = (A4 A5 … An)
Total Cost C = cost(C1)+cost(C2)+p0p3pn
Then, if cost(C1) and cost(C2) are minimal (i.e., optimal),
then C is optimal (optimal substructure holds)
Optimal C1, C2 might be one of different options
C1 = (A1 A2), C2 = (A3 A4 A5 … An)
C1 = (A1), C2 = (A2 A3 A4 A5 … An)
C1 = (A1 A2 A3 A4), C2 = (A5 … An)
…….65
![Page 66: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/66.jpg)
Matrix Chain Multiplication
Assume k is length of first sub-chain C1
66
A1 A2 A3 A4 A5
(A1) (A2 A3 A4 A5)
k=1
(A1A2 A3 A4)(A5)
k=4
(A1A2) ( A3 A4 A5)
k=2
(A1A2 A3)(A4A5)
k=3
k=1 k=2 k=3
(A2)(A3 A4 A5) (A2A3)(A4 A5) (A2 A3 A4)(A5)
k=1 k=2
(A1)(A2 A3) (A1A2)(A3)
…….…….
…….
k=1 k=2
(A2)(A3 A4) (A2A3)(A4)
![Page 67: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/67.jpg)
Matrix Chain Multiplication
Assume k is length of first sub-chain C1
67
A1 A2 A3 A4 A5
(A1) (A2 A3 A4 A5)
k=1
(A1A2 A3 A4)(A5)
k=4
(A1A2) ( A3 A4 A5)
k=2
(A1A2 A3)(A4A5)
k=3
k=1 k=2 k=3
(A2)(A3 A4 A5) (A2A3)(A4 A5) (A2 A3 A4)(A5)
k=1 k=2
(A1)(A2 A3) (A1A2)(A3)
…….…….
…….
k=1 k=2
(A2)(A3 A4) (A2A3)(A4)
![Page 68: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/68.jpg)
Matrix Chain Multiplication
Assume k is length of first sub-chain C1
Obviously, a lot of overlapping subproblems appear
68
A1 A2 A3 A4 A5
(A1) (A2 A3 A4 A5)
k=1
(A1A2 A3 A4)(A5)
k=4
(A1A2) ( A3 A4 A5)
k=2
(A1A2 A3)(A4A5)
k=3
k=1 k=2 k=3
(A2)(A3 A4 A5) (A2A3)(A4 A5) (A2 A3 A4)(A5)
k=1 k=2
(A1)(A2 A3) (A1A2)(A3)
…….…….
…….
k=1 k=2
(A2)(A3 A4) (A2A3)(A4)
![Page 69: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/69.jpg)
Matrix Chain Multiplication
Assume k is length of first sub-chain C1
Obviously, a lot of overlapping subproblems appear
Optimal substructure + subproblem overlapping =
dynamic programming69
A1 A2 A3 A4 A5
(A1) (A2 A3 A4 A5)
k=1
(A1A2 A3 A4)(A5)
k=4
(A1A2) ( A3 A4 A5)
k=2
(A1A2 A3)(A4A5)
k=3
k=1 k=2 k=3
(A2)(A3 A4 A5) (A2A3)(A4 A5) (A2 A3 A4)(A5)
k=1 k=2
(A1)(A2 A3) (A1A2)(A3)
…….…….
…….
k=1 k=2
(A2)(A3 A4) (A2A3)(A4)
![Page 70: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/70.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
70
![Page 71: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/71.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
71
![Page 72: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/72.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
72
![Page 73: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/73.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
73
![Page 74: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/74.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 2:
(A1 A2)
(A2 A3)
(A3 A4)
(A4 A5)
(A5 A6)
74
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 75: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/75.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 2:
(A1 A2) = 30*35*15 = 15750
(A2 A3)
(A3 A4)
(A4 A5)
(A5 A6)
75
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 76: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/76.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 2:
(A1 A2) = 30*35*15 = 15750
(A2 A3) = 35*15*5 = 2625
(A3 A4)
(A4 A5)
(A5 A6)
76
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 77: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/77.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 2:
(A1 A2) = 30*35*15 = 15750
(A2 A3) = 35*15*5 = 2625
(A3 A4) = 15*5*10 = 750
(A4 A5) = 5*10*20 = 1000
(A5 A6) = 10*20*25 = 5000
77
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 78: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/78.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 3:
(A1 A2 A3)
(A2 A3 A4)
(A3 A4 A5)
(A4 A5 A6)
78
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 79: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/79.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 3:
(A1 A2 A3) = (A1 A2) A3
or A1 (A2 A3)
(A2 A3 A4)
(A3 A4 A5)
(A4 A5 A6)
79
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 80: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/80.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 3:
(A1 A2 A3) = (A1 A2) A3 [15750+30*15*5]
or A1 (A2 A3) [30*35*5+2625]
= 7875
(A2 A3 A4)
(A3 A4 A5)
(A4 A5 A6)
80
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 81: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/81.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 3:
(A1 A2 A3) = 7875
(A2 A3 A4) = (A2 A3) A4
or A2 (A3 A4)
(A3 A4 A5)
(A4 A5 A6)
81
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 82: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/82.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 3:
(A1 A2 A3) = 7875
(A2 A3 A4) = (A2 A3) A4 [2625+35*5*10]
or A2 (A3 A4) [35*15*10+750]
= 4375
(A3 A4 A5)
(A4 A5 A6)
82
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 83: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/83.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 3:
(A1 A2 A3) = 7875
(A2 A3 A4) = 4375
(A3 A4 A5) = 2500
(A4 A5 A6) = 3500
83
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 84: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/84.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 4:
(A1 A2 A3 A4) = A1 (A2 A3 A4)
Or (A1 A2)(A3 A4)
Or (A1 A2 A3) A4
(A2 A3 A4 A5)
(A3 A4 A5 A6)
84
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 85: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/85.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 4:
(A1 A2 A3 A4) = A1 (A2 A3 A4)
Or (A1 A2)(A3 A4)
Or (A1 A2 A3) A4
= 9375
(A2 A3 A4 A5)
(A3 A4 A5 A6)
85
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 86: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/86.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 4:
(A1 A2 A3 A4) = 9375
(A2 A3 A4 A5) = 7125
(A3 A4 A5 A6) = 5375
86
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 87: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/87.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 5:
(A1 A2 A3 A4 A5) = 11875
(A2 A3 A4 A5 A6) = 10500
87
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 88: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/88.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
Example: n=6,
Chains of length 6:
(A1 A2 A3 A4 A5 A6) = 15125
88
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 89: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/89.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
89
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 90: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/90.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
90
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 91: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/91.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
91
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 92: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/92.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
92
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
A1A2A3 = (A1A2)A3
Or A1(A2A3)
![Page 93: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/93.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
93
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
A2A3A4 = (A2A3)A4
Or A2(A3A4)
![Page 94: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/94.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
94
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
A3A4A5 = (A3A4)A5
Or A3(A4A5)
![Page 95: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/95.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
95
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
A4A5A6 = (A4A5)A6
Or A4(A5A6)
![Page 96: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/96.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
96
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
A1A2A3A4 = A1(A2A3A4)
Or (A1A2)(A3A4)
Or (A1A2A3) A4
![Page 97: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/97.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
97
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 98: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/98.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
98
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 99: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/99.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
99
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
![Page 100: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/100.jpg)
Matrix Chain Multiplication:
Designing Algorithm
What is the smallest subproblem?
A chain of length 2
Solve all chains of length 2, then 3, then 4, …n
100
A1: 30x35
A2: 35x15
A3: 15x5
A4: 5x10
A5: 10x20
A6: 20x25
0 15750 7875 9375 11875 15125
0 2625 4375 7125 10500
0 750 2500 5375
0 1000 3500
0 5000
0
1 2 3 4 5 6
1
2
3
4
5
6
![Page 101: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/101.jpg)
Matrix Chain Multiplication
Generally: Ai … Ak … Aj of dimensions pi … pk … pj
(Ai … Ak)(Ak+1 … Aj), where k=i,i+1,…j-1
Then, solve each sub-chains recursively
101
![Page 102: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/102.jpg)
Matrix Chain Multiplication
Generally: Ai … Ak … Aj of dimensions pi … pk … pj
(Ai … Ak)(Ak+1 … Aj), where k=i,i+1,…j-1
Then, solve each sub-chains recursively
102
![Page 103: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/103.jpg)
Matrix Chain Multiplication
103
![Page 104: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/104.jpg)
Longest Common Subsequence
104
A string subsequence is an ordered set of characters
(not necessarily consecutive)
A common subsequence of two strings is a subsequence
that exist in both strings.
The longest common subsequence is the common
subsequence of the maximum length.
![Page 105: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/105.jpg)
Longest Common Subsequence
Given two strings:
Find the longest common subsequence of X and Y
LCS(X,Y)
105
![Page 106: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/106.jpg)
Longest Common Subsequence
Given two strings:
Find the longest common subsequence of X and Y
LCS(X,Y)
Brute force?
106
![Page 107: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/107.jpg)
Longest Common Subsequence
Given two strings:
Find the longest common subsequence of X and Y
LCS(X,Y)
Brute force? O(n*2m) or O(m*2n) [enumerate all subsequences of X
and check in Y, or vice versa]
107
![Page 108: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/108.jpg)
Longest Common Subsequence
Given two strings:
Find the longest common subsequence of X and Y
LCS(X,Y)
Brute force? O(n*2m) or O(m*2n) [enumerate all subsequences of X
and check in Y, or vice versa]
Are smaller problems simpler?
108
![Page 109: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/109.jpg)
Longest Common Subsequence
Given two strings:
Find the longest common subsequence of X and Y
LCS(X,Y)
Brute force? O(n*2m) or O(m*2n) [enumerate all subsequences of X
and check in Y, or vice versa]
Are smaller problems simpler?
If xm = yn, then this character can be included in LCS, and then we
get LCS of X without xm and Y without yn (prefix Xm-1 and Yn-1)
109
![Page 110: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/110.jpg)
Longest Common Subsequence
Given two strings:
Find the longest common subsequence of X and Y
LCS(X,Y)
Brute force? O(n*2m) or O(m*2n) [enumerate all subsequences of X
and check in Y, or vice versa]
Are smaller problems simpler?
If xm = yn, then this character can be included in LCS, and then we
get LCS of X without xm and Y without yn (prefix Xm-1 and Yn-1)
Let’s define string prefixes
and same for Yj for j = 0,1,…., n110
![Page 111: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/111.jpg)
Longest Common Subsequence
Prove by contradiction
111
![Page 112: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/112.jpg)
Longest Common Subsequence
Prove by contradiction
112
Xm Yn
Xm-1 Yn-1
Xm Yn-1
Xm-1 Yn
Xm Yn-2
Xm-1 Yn-1Xm-1 Yn-2 Xm Yn-2
Xm-1 Yn-1
Xm-1 Yn-1
Xm-2 Yn-1
Xm-2 Yn
![Page 113: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/113.jpg)
Longest Common Subsequence
Prove by contradiction
113
Xm Yn
Xm-1 Yn-1
Xm Yn-1
Xm-1 Yn
Xm Yn-2
Xm-1 Yn-1Xm-1 Yn-2 Xm Yn-2
Xm-1 Yn-1
Xm-1 Yn-1
Xm-2 YnXm-2 Yn-1
![Page 114: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/114.jpg)
Longest Common Subsequence
Let c[i,j] is LCS length of Xi and Yj
114
![Page 115: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/115.jpg)
Longest Common Subsequence
Example: X=“CS141” Y=“CS111”
115
“” C S 1 1 1
“”
C
S
1
4
1
![Page 116: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/116.jpg)
Longest Common Subsequence
Example: X=“CS141” Y=“CS111”
116
0 0 0 0 0 0
0
0
0
0
0
“” C S 1 1 1
“”
C
S
1
4
1
![Page 117: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/117.jpg)
Longest Common Subsequence
Example: X=“CS141” Y=“CS111”
117
0 0 0 0 0 0
0 1 1 1 1 1
0 1 2 2 2 2
0 1 2 3 3 3
0 1 2 3 3 3
0 1 2 3 4 4
“” C S 1 1 1
“”
C
S
1
4
1
![Page 118: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/118.jpg)
Longest Common Subsequence
118
![Page 119: CS141: Intermediate Data Structures and Algorithms Dynamic ...amr/courses/19FCS141/...Dynamic Programming Amr Magdy. Programming? In this context, programming is a tabular method Storing](https://reader036.fdocuments.in/reader036/viewer/2022071216/60470dff36ec924a9828deb8/html5/thumbnails/119.jpg)
Book Readings
Ch. 15: 15.1-15.4
119