CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray...
Transcript of CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray...
![Page 1: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/1.jpg)
CS 4102: AlgorithmsLecture 11: Dynamic Programming
David WuFall 2019
![Page 2: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/2.jpg)
2
Spring 2019 Warm Up
How many ways are there to tile a 2×𝑛board with dominoes?
How many ways to tile a 2×7 board With these?
![Page 3: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/3.jpg)
3
Tiling Dominoes
3
𝑛 − 1Tile 𝑛 = Tile 𝑛 − 1 + Tile(𝑛 − 2)
Two ways to fill the final column:
𝑛 − 2
Tile 0 = Tile 1 = 1
1 1 2 3 5 8 13 21
0 1 2 3 4 5 6 7
![Page 4: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/4.jpg)
Today’s Keywords
Dynamic programmingMaximum sum contiguous subarrayTiling dominoesLog cuttingMatrix chaining
4
CLRS Readings: Chapter 14
![Page 5: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/5.jpg)
Homework
• HW3 due Tuesday, October 1, 11pm Wednesday, October 2, 11pm• Divide and conquer algorithms• Written (use LaTeX!) – Submit both zip and pdf!
• Regrade office hours:• Thursday 11am-12pm (Rice 210)• Thursday 4pm-5pm (Rice 501)
5
![Page 6: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/6.jpg)
Maximum Sum Subarray Problem
Maximum sum contiguous subarray (MSCS) problem:find the largest contiguous subarray that
maximizes the sum of the values
6
5
0
-4
2
8
1
3
3
7
4
-15
5
2
6
8
7
-20
8
17
9
8
10
-50
11
-5
12
22
13
![Page 7: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/7.jpg)
Maximum Sum Subarray Problem
Maximum sum contiguous subarray (MSCS) problem:find the largest contiguous subarray that
maximizes the sum of the values
7
5
0
-4
2
8
1
3
3
7
4
-15
5
2
6
8
7
-20
8
17
9
8
10
-50
11
-5
12
22
13
![Page 8: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/8.jpg)
5
0
-4
2
8
1
3
3
7
4
-15
5
2
6
8
7
-20
8
17
9
8
10
-50
11
-5
12
22
13
Divide and Conquer Θ(𝑛 log 𝑛)
8
Divide in halfRecursively solve on left
Recursively solve on right
19 25
![Page 9: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/9.jpg)
5
0
-4
2
8
1
3
3
7
4
-15
5
2
6
8
7
-20
8
17
9
8
10
-50
11
-5
12
22
13
Divide and Conquer Θ(𝑛 log 𝑛)
9
Divide in halfRecursively solve on left
Recursively solve on right
Combine: Find largest sum that spans the cut
Largest sum that ends here
+ Largest sum that starts here
2-13-6-3-716 -20-42-37135-128
19 25
![Page 10: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/10.jpg)
5
0
-4
2
8
1
3
3
7
4
-15
5
2
6
8
7
-20
8
17
9
8
10
-50
11
-5
12
22
13
Divide and Conquer Θ(𝑛 log 𝑛)
10
Divide in halfRecursively solve on left
Recursively solve on right
Combine: Find largest sum that spans the cut
Largest sum that ends here
+ Largest sum that starts here
2-13-6-3-716 -20-42-37135-128
19 25
19𝑇 𝑛 = 2𝑇 ⁄𝑛 2 + Θ 𝑛 ∈ Θ 𝑛 log 𝑛
![Page 11: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/11.jpg)
Unbalanced Divide and Conquer
Divide• Make a subproblem of all but the last element
5
0
-4
2
8
1
3
3
7
4
-15
5
2
6
8
7
-20
8
17
9
8
10
-50
11
-5
12
22
13
![Page 12: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/12.jpg)
Unbalanced Divide and Conquer
Divide• Make a subproblem of all but the last element
Conquer• Find best subarray on the left (𝐵𝑆𝐿(𝑛 − 1))• Find the best subarray ending at the divide (𝐵𝐸𝐷(𝑛 − 1))
5
0
-4
2
8
1
3
3
7
4
-15
5
2
6
8
7
-20
8
17
9
8
10
-50
11
-5
12
22
13
Best subarray ending at the divide is empty
![Page 13: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/13.jpg)
Unbalanced Divide and Conquer
Divide• Make a subproblem of all but the last element
Conquer• Find best subarray on the left (𝐵𝑆𝐿(𝑛 − 1))• Find the best subarray ending at the divide (𝐵𝐸𝐷(𝑛 − 1))
5
0
-4
2
8
1
3
3
7
4
-15
5
2
6
8
7
-20
8
17
9
8
10
-50
11
-5
12
22
13
Combine• Find the best subarray that “spans the divide” and output best among all
candidates
![Page 14: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/14.jpg)
Unbalanced Divide and Conquer
Best subarray that spans divide must include last element: 𝐵𝐸𝐷 𝑛• 𝐵𝐸𝐷 𝑛 = max 𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0
Best subarray must either include or exclude the last element• 𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
5
0
-4
2
8
1
3
3
7
4
-15
5
2
6
8
7
-20
8
17
9
8
10
-50
11
-5
12
22
13
Combine• Find the best subarray that “spans the divide” and output best among all
candidates
![Page 15: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/15.jpg)
Unbalanced Divide and Conquer
Divide• Make a subproblem of all but the last element
Conquer• Find best subarray on the left (𝐵𝑆𝐿(𝑛 − 1))• Find the best subarray ending at the divide (𝐵𝐸𝐷(𝑛 − 1))
Combine• New best subarray ending at the divide:
• 𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)• New best on the left:
• 𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
If we compute 𝐵𝐸𝐷(𝑛 − 1) and 𝐵𝑆𝐿(𝑛 − 1), then Combine is
constant-time!
![Page 16: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/16.jpg)
Unbalanced Divide and Conquer
8 -4 3 7 -15 2 8 -20 17 8 -50 -5 220 21 3 4 5 6 7 8 9 10 11 12 13
Divide
5
𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
![Page 17: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/17.jpg)
Unbalanced Divide and Conquer
8 -4 3 7 -15 2 8 -20 17 8 -50 220 21 3 4 5 6 7 8 9 10 11 12 13
Divide
5 -5
𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
![Page 18: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/18.jpg)
Unbalanced Divide and Conquer
8 -4 3 7 -15 2 8 -20 17 8 220 21 3 4 5 6 7 8 9 10 11 12 13
Divide
5 -5-50
𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
![Page 19: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/19.jpg)
Unbalanced Divide and Conquer
8 -4 3 7 -15 2 8 -20 17 8 220 21 3 4 5 6 7 8 9 10 11 12 13
Divide
5 -5-50
𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
![Page 20: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/20.jpg)
Unbalanced Divide and Conquer
8 -4 3 7 -15 2 8 -20 17 8 220 21 3 4 5 6 7 8 9 10 11 12 13
Divide
5 -5-50
55
Recursively solve on left
5
Find largest sum ending at the cut
5
𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
![Page 21: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/21.jpg)
Unbalanced Divide and Conquer
8 -4 3 7 -15 2 8 -20 17 8 220 21 3 4 5 6 7 8 9 10 11 12 13
Divide
5 -5-50
5
5
𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
13
13
Recursively solve on left
13
Find largest sum ending at the cut
13
![Page 22: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/22.jpg)
Unbalanced Divide and Conquer
8 -4 3 7 -15 2 8 -20 17 8 220 21 3 4 5 6 7 8 9 10 11 12 13
Divide
5 -5-50
5
5
𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
13
13
9
13
Recursively solve on left
13
Find largest sum ending at the cut
9
![Page 23: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/23.jpg)
-4
Unbalanced Divide and Conquer
8 3 7 -15 2 8 -20 17 8 220 21 3 4 5 6 7 8 9 10 11 12 13
5 -5-50
5
5
𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
13
13
9
13
Divide
12 19 4 6 14 0 17 25 0 0 22
13 19 19 19 19 19 19 25 25 25 25
𝑇 𝑛 = 𝑇 𝑛 − 1 + Θ 1 ∈ Θ 𝑛
Recursively solve on left
25
Find largest sum ending at the cut
22
![Page 24: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/24.jpg)
Was Unbalanced Better?
Old:• We split into 2 problems of size 𝑛/2• Linear time combine (to find arrays that span the cut)
New:• We split into 2 problems of size 𝑛 − 1 and 1• Constant time combine
𝑇 𝑛 = 2𝑇 ⁄𝑛 2 + Θ(𝑛) ∈ Θ 𝑛 log 𝑛
𝑇 𝑛 = 𝑇 𝑛 − 1 + 𝑇 1 + Θ 1 ∈ Θ(𝑛)
Θ 1
![Page 25: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/25.jpg)
Another Look at the Recursion
8 -4 3 7 -15 2 8 -20 17 8 220 21 3 4 5 6 7 8 9 10 11 12 13
Divide
5 -5-50
55
Recursively solve on left
5
Find largest sum ending at the cut
5
𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
![Page 26: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/26.jpg)
Another Look at the Recursion
8 -4 3 7 -15 2 8 -20 17 8 220 21 3 4 5 6 7 8 9 10 11 12 13
Divide
5 -5-50
5
5
𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
13
13
Recursively solve on left
13
Find largest sum ending at the cut
13
![Page 27: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/27.jpg)
Another Look at the Recursion
8 -4 3 7 -15 2 8 -20 17 8 220 21 3 4 5 6 7 8 9 10 11 12 13
Divide
5 -5-50
5
5
𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
13
13
9
13
Recursively solve on left
13
Find largest sum ending at the cut
9
![Page 28: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/28.jpg)
-4
Another Look at the Recursion
8 3 7 -15 2 8 -20 17 8 220 21 3 4 5 6 7 8 9 10 11 12 13
5 -5-50
5
5 13
13
9
13
Divide
12 19 4 6 14 0 17 25 0 0 22
13 19 19 19 19 19 19 25 25 25 25
Observation: No need to recurse! Just maintain two numbers and iterate from 1 to 𝑛: best value so far, best value ending at current position
𝐵𝐸𝐷 𝑛 = max(𝐵𝐸𝐷 𝑛 − 1 + 𝑎𝑟𝑟 𝑛 , 0)𝐵𝑆𝐿 𝑛 = max 𝐵𝑆𝐿 𝑛 − 1 , 𝐵𝐸𝐷 𝑛
![Page 29: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/29.jpg)
End of Midterm Exam Materials!
29
![Page 30: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/30.jpg)
30
Spring 2019 Tiling Dominoes
How many ways are there to tile a 2×𝑛board with dominoes?
How many ways to tile a 2×7 board With these?
![Page 31: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/31.jpg)
31
Tiling Dominoes
31
𝑛 − 1Tile 𝑛 = Tile 𝑛 − 1 + Tile(𝑛 − 2)
Two ways to fill the final column:
𝑛 − 2
Tile 0 = Tile 1 = 1
1 1 2 3 5 8 13 21
0 1 2 3 4 5 6 7
![Page 32: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/32.jpg)
How to compute Tile(𝑛)?
32
def tile(n):if n < 2:
return 1return tile(n-1) + tile(n-2)
Problem?
![Page 33: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/33.jpg)
Recursion Tree
33
Tile(5)
Tile(4) Tile(3)
Tile(3) Tile(2) Tile(2) Tile(1)
Tile(0)Tile(1)Tile(0)Tile(1)Tile(1)Tile(2)
Tile(0)Tile(1)
Runtime: Ω(2C)
![Page 34: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/34.jpg)
Recursion Tree
34
Tile(5)
Tile(4) Tile(3)
Tile(3) Tile(2) Tile(2) Tile(1)
Tile(0)Tile(1)Tile(0)Tile(1)Tile(1)Tile(2)
Tile(0)Tile(1)
But lots of redundant calls…Runtime: Ω(2C) We only computed
𝑛 distinct values
![Page 35: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/35.jpg)
Computing Tile(𝑛) with Memory(“Top Down”)
35
initialize array M of size ntile(n):
if n < 2:return 1
if M[n] is filled:return M[n]
M[n] = tile(n-1) + tile(n-2)return M[n]
M
0
1
2
3
4
5
6
![Page 36: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/36.jpg)
Computing Tile(𝑛) with Memory(“Top Down”)
36
initialize array M of size ntile(n):
if n < 2:return 1
if M[n] is filled:return M[n]
M[n] = tile(n-1) + tile(n-2)return M[n]
1
1
2
3
5
8
13
M
0
1
2
3
4
5
6
Bottom-Up: Can also iterate through 𝑀 and fill in entries sequentiallyRuntime: Θ(𝑛)
Bottom-Up:Fill in entries from small instances to
large instances
![Page 37: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/37.jpg)
Dynamic Programming
Requires optimal substructure• Solution to larger problem contains the solutions to smaller ones
(“overlapping subproblems”)
General idea: Identify recursive structure of the problem and express solution to larger instances in terms of solutions to smaller instances
37𝑛 − 1 𝑛 − 2
![Page 38: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/38.jpg)
Generic Divide and Conquer
def myDCalgo(problem):
if baseCase(problem):solution = solve(problem)
return solutionfor subproblem of problem: # After dividing
subsolutions.append(myDCalgo(subproblem))solution = Combine(subsolutions)
return solution38
![Page 39: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/39.jpg)
Generic Top-Down Dynamic Programmingmem = {}def myDPalgo(problem):
if mem[problem] not empty:return mem[problem]
if baseCase(problem):solution = solve(problem)mem[problem] = solutionreturn solution
for subproblem of problem:subsolutions.append(myDPalgo(subproblem))
solution = OptimalSubstructure(subsolutions)mem[problem] = solutionreturn solution
39
Also called “memoization”
![Page 40: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/40.jpg)
Dynamic Programming
Requires optimal substructure• Solution to larger problem contains the solutions to smaller ones
General Blueprint:1. Identify recursive structure of the problem
• What is the “last thing” done?2. Select a good order for solving subproblems
• “Top Down:” Solve each problem recursively• “Bottom Up:” Iteratively solve each problem from smallest to largest
40
![Page 41: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/41.jpg)
Log Cutting
41
Given: a log of length 𝑛, a list (of length 𝑛) of prices 𝑃Problem: Find the best way to cut the log
1 5 8 9 10 17 17 20 24 30
10987654321Length:
Price:
Problem formulation: Find lengths ℓG, … , ℓI that maximizes ∑K∈ I 𝑃[ℓK] and such that ∑K∈ I ℓK = 𝑛
Brute Force: 𝑂(2C)
𝑃[𝑖] is the price of a cut of size 𝑖
![Page 42: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/42.jpg)
A “Greedy” Approach
Greedy algorithms (next unit) build a solution by picking the best option “right now”• Possible strategy: choose the most profitable cut first
42
Greedy: Lengths: 5, 1Profit: 51
Better: Lengths: 2, 4Profit: 54
1 18 24 36 50
54321Length:
Price: 50
6
![Page 43: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/43.jpg)
A “Greedy” Approach
Greedy algorithms (next unit) build a solution by picking the best option “right now”• Possible strategy: select the “most bang for your buck” (best price/length ratio)
43
Greedy: Lengths: 5, 1Profit: 51
Better: Lengths: 2, 4Profit: 54
1 18 24 36 50
54321Length:
Price: 50
6
1 9 8 9 10 8.3Ratio:
Greedy solution is suboptimal
![Page 44: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/44.jpg)
Dynamic Programming
Requires optimal substructure• Solution to larger problem contains the solutions to smaller ones
General Blueprint:1. Identify recursive structure of the problem
• What is the “last thing” done?2. Select a good order for solving subproblems
• “Top Down:” Solve each problem recursively• “Bottom Up:” Iteratively solve each problem from smallest to largest
44
![Page 45: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/45.jpg)
Dynamic Programming
Requires optimal substructure• Solution to larger problem contains the solutions to smaller ones
General Blueprint:1. Identify recursive structure of the problem
• What is the “last thing” done?2. Select a good order for solving subproblems
• “Top Down:” Solve each problem recursively• “Bottom Up:” Iteratively solve each problem from smallest to largest
45
![Page 46: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/46.jpg)
Step 1: Identify Recursive Structure
46
Cut(𝑛) = value of best way to cut a log of length 𝑛
ℓC𝐶𝑢𝑡(𝑛 − ℓC)
Cut 𝑛 = max
Cut 𝑛 − 1 + 𝑃 1Cut 𝑛 − 2 + 𝑃 2
⋮Cut 0 + 𝑃[𝑛]
Last Cutbest way to cut a log of length 𝒏 − ℓ𝒏
𝑃 𝑖 = value of a cut of length 𝑖
![Page 47: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/47.jpg)
Dynamic Programming
Requires optimal substructure• Solution to larger problem contains the solutions to smaller ones
General Blueprint:1. Identify recursive structure of the problem
• What is the “last thing” done?2. Select a good order for solving subproblems
• “Top Down:” Solve each problem recursively• “Bottom Up:” Iteratively solve each problem from smallest to largest
47
![Page 48: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/48.jpg)
2. Select a Good Order for Solving Subproblems
48
654321Length:
Cut(𝑖): 0
0
Solve smallest subproblem first
Cut 0 = 0
1 18 24 36 50Price: 50
54321Length: 60
![Page 49: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/49.jpg)
2. Select a Good Order for Solving Subproblems
49
Solve smallest subproblem first
Cut 1 = Cut 0 + 𝑃[1]
1Cut(𝑖): 0
654321Length: 0
1 18 24 36 50Price: 50
54321Length: 61
![Page 50: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/50.jpg)
2. Select a Good Order for Solving Subproblems
50
Solve smallest subproblem first
1 18Cut(𝑖): 0
654321Length: 0
1 18 24 36 50Price: 50
54321Length: 6
Cut 2 = max XCut 1 + 𝑃 1Cut 0 + 𝑃 2
2
![Page 51: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/51.jpg)
2. Select a Good Order for Solving Subproblems
51
Solve smallest subproblem first
1 18 24Cut(𝑖): 0
654321Length: 0
1 18 24 36 50Price: 50
54321Length: 6
Cut 3 = maxZCut 2 + 𝑃 1Cut 1 + 𝑃 2Cut 0 + 𝑃 3
3
![Page 52: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/52.jpg)
2. Select a Good Order for Solving Subproblems
52
Solve smallest subproblem first
1 18 24 36 50 54Cut(𝑖): 0
654321Length: 0
1 18 24 36 50Price: 50
54321Length: 66
Cut 𝑛 = max
Cut 𝑛 − 1 + 𝑃 1Cut 𝑛 − 2 + 𝑃 2
⋮Cut 0 + 𝑃[𝑛]
![Page 53: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/53.jpg)
Log Cutting Pseudocode
53
initialize memory Ccut(n):
C[0] = 0for i = 1 to n:
best = 0for j = 1 to i:
best = max(best, C[i-j] + P[j])C[i] = best
return C[n]
Run Time: 𝑂(𝑛[)
![Page 54: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/54.jpg)
Finding the Cuts
This procedure told us the profit, but not the cuts themselvesIdea: remember the choice that you made, then backtrack
54
![Page 55: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/55.jpg)
Remembering the Choices
55
initialize memory C, choicescut(n):
C[0] = 0for i = 1 to n:
best = 0for j = 1 to i:
if best < C[i-j] + P[j]:best = C[i-j] + P[j]choices[i] = j
C[i] = bestreturn C[n], choices
size of the last cut
![Page 56: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/56.jpg)
Reconstruct the Cuts
56
1 1 2 4 3 4 1 2 4 3
10987654321Length:
Choices: 0
0
Backtrack through the choices:
7621
Optimal cut for log of length 10 is to first cut
segment of length 3
![Page 57: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/57.jpg)
Backtracking Pseudocode
i = n
while i > 0:
print choices[i]i = i – choices[i]
57
![Page 58: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/58.jpg)
Dynamic Programming
Requires optimal substructure• Solution to larger problem contains the solutions to smaller ones
General Blueprint:1. Identify recursive structure of the problem
• What is the “last thing” done?2. Select a good order for solving subproblems
• “Top Down:” Solve each problem recursively• “Bottom Up:” Iteratively solve each problem from smallest to largest
3. Save solution to each subproblem in memory
58
![Page 59: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/59.jpg)
Matrix Chaining
59
𝑀G × 𝑀\× ×
Problem: Given a sequence of matrices 𝑀G,… ,𝑀C, what is the most efficient way to multiply them?
𝑀[ 𝑀]
𝑛G×𝑛[ 𝑛[×𝑛\ 𝑛\×𝑛] 𝑛]×𝑛^
Remember: matrix multiplication is associative
![Page 60: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/60.jpg)
Order Matters!
𝑀G×𝑀[ ×𝑀\• requires 𝑛G𝑛[𝑛\ + 𝑛G𝑛\𝑛] operations
60
𝑀G × 𝑀\×𝑀[
𝑛G×𝑛[ 𝑛[×𝑛\ 𝑛\×𝑛]
𝑛G×𝑛\
𝑛G = 7 𝑛[ = 10𝑛\ = 20 𝑛] = 8
Total operations:2520
![Page 61: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/61.jpg)
Order Matters!
𝑀G× 𝑀[ ×𝑀\• requires 𝑛G𝑛[𝑛] + 𝑛[𝑛\𝑛] operations
61
𝑀G × 𝑀\×𝑀[
𝑛G×𝑛[ 𝑛[×𝑛\ 𝑛\×𝑛]
𝑛[×𝑛]
𝑛G = 7 𝑛[ = 10𝑛\ = 20 𝑛] = 8
Total operations:2160
Much better than 2520 from before
![Page 62: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/62.jpg)
Dynamic Programming
Requires optimal substructure• Solution to larger problem contains the solutions to smaller ones
General Blueprint:1. Identify recursive structure of the problem
• What is the “last thing” done?2. Select a good order for solving subproblems
• “Top Down:” Solve each problem recursively• “Bottom Up:” Iteratively solve each problem from smallest to largest
3. Save solution to each subproblem in memory
62
![Page 63: CS 4102: Algorithmsjh2jf/courses/fall2019/cs4102/lectures/... · Maximum sum contiguous subarray Tiling dominoes Log cutting Matrix chaining 4 CLRS Readings: Chapter 14. Homework](https://reader036.fdocuments.in/reader036/viewer/2022081620/610b812b1b27231d361b0fb1/html5/thumbnails/63.jpg)
Dynamic Programming
Requires optimal substructure• Solution to larger problem contains the solutions to smaller ones
General Blueprint:1. Identify recursive structure of the problem
• What is the “last thing” done?2. Select a good order for solving subproblems
• “Top Down:” Solve each problem recursively• “Bottom Up:” Iteratively solve each problem from smallest to largest
3. Save solution to each subproblem in memory
63