Algorithm Design and Analysis (ADA)
description
Transcript of Algorithm Design and Analysis (ADA)
![Page 1: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/1.jpg)
242-535 ADA: 7. Dynamic Prog. 1
• Objectiveo introduce DP, its two hallmarks, and two major
programming techniqueso look at two examples: the fibonacci series and
LCS
Algorithm Design and Analysis (ADA)
242-535, Semester 1 2014-2015
7. Dynamic Programming
![Page 2: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/2.jpg)
Overview7. Is the c[] Algorithm
Optimal?8. Repeated
Subproblems?9. LCS() as a DP
Problem10.Bottom-up Examples11.Finding the LCS12.Space and Bottom-up
1. Dynamic Programming (DP)
2. Fibonacci Series3. Features of DP Code4. Longest Common
Subsequence (LCS)5. Towards a Better LCS
Algorithm6. Recursive Definition
of c[]
![Page 3: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/3.jpg)
242-535 ADA: 7. Dynamic Prog. 3
• The "programming" in DP predates computing, and means "using tables to store information".
• DP is a recursive programming technique:o the problem is recursively described in terms of
smilar but smaller subproblems
1. Dynamic Programming (DP)
![Page 4: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/4.jpg)
242-535 ADA: 7. Dynamic Prog. 4
• Optimal solution from optimal partso An optimal (best) solution to the problem is a
composition of optimal (best) subproblem solutions
• Repeated (overlapping) sub-problemso the problem contains the same subproblems,
repeated many times
What makes DP Different?Hallmark #1
Hallmark #2
![Page 5: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/5.jpg)
242-535 ADA: 7. Dynamic Prog. 5
• Unix diff for comparing two files
• Smith-Waterman for genetic sequence alignmento determines similar regions between two protein
sequences (strings)
• Bellman-Ford for shortest path routing in networks
• Cocke-Kasami-Younger for parsing context free grammars
Some Famous DP Examples
![Page 6: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/6.jpg)
242-535 ADA: 7. Dynamic Prog. 6
• Series defined by • a0 = 1 • a1 = 1• an = an-1 + an-2
• Recursive algorithm:
• Running Time? O(2n)
2. Fibonacci Series1, 1, 2, 3, 5, 8, 13, 21, 34, …
fib(n)1. if n == 0 or n == 1, then 2. return 13. else4. a = fib(n-1)5. b = fib(n-2)6. return a+b
![Page 7: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/7.jpg)
242-535 ADA: 7. Dynamic Prog. 7
• Fibonacci can be viewed as a DP problem:o the optimal solution (fib(n)) is a combination of
optimal sub-solutions (fib(n-1) and fib(n-2))
o There are a lot of repeated subproblems • look at the execution graph (next slide)• 2n subproblems, but only n are different
Computing Fibonacci Faster
7
![Page 8: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/8.jpg)
242-535 ADA: 7. Dynamic Prog. 8
• Execution of fib(5):lots of repeated work,that only gets worsefor bigger n
![Page 9: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/9.jpg)
242-535 ADA: 7. Dynamic Prog. 9
• Memoization: values returned by recursive calls are stored in a table (array) so they do not need to be recalculatedo this can save enormous amount of running time
• Bottom-up algorithms: simple solutions are computed first (e.g. fib(0), fib(1), ...), leading to larger solutions (e.g. fib(23), fib(24), ...) o this means that the solutions are added to the table in a
fixed order which may allow them to be calculated quicker and the table to be less large
3. Features of DP Code
![Page 10: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/10.jpg)
242-535 ADA: 7. Dynamic Prog. 10
• Running time is linear = O(n)• Requires extra space for the m[] table =
O(n)
Memoization fib() Algorithm
fib(n)1.if (m[n] == 0) then2. m[n] = fib(n − 1) +
fib(n − 2) 3.return m[n] :
:
m[]0123
n-1
1100
0
::
![Page 11: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/11.jpg)
Bottom-up fib() Algorithmint fib(int n){ if (n == 0) return 1; else { int prev = 1; int curr = 1; int temp; for (int i=1; i < n; i++) { temp = prev + curr; prev = curr; curr = temp; } return curr; }}
• Running time = O(n)
• Space requirement is 3 variables = O(1) !o this has nothing to do with
changing from recursion to a loop, but with changing from top-down to bottom-up execution, which in this case is easier to write as a loop
![Page 12: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/12.jpg)
4. Longest Common Subsequence (LCS)
• Given two sequences x[1 . . m] and y[1 . . n], find a longest subsequence common to them both.
x: A B C B D A B
y: B D C A B A
BCBA = LCS(x, y)
andBDABBCAB
![Page 13: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/13.jpg)
242-535 ADA: 7. Dynamic Prog. 13
Check every subsequence of x[1 . . m] to see if it is also a subsequence of y[1 . . n].
Analysis• Checking time for each subsequence is O(n).• 2m subsequences of x[] (can use or not use each
element in x).Worst-case running time = O(n*2m), exponential time.
Brute-force LCS Algorithm
SLOW == BAD
![Page 14: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/14.jpg)
242-535 ADA: 7. Dynamic Prog. 14
Simplify the problem:1. Find the length of a LCS
2. We'll extend the algorithm later to find the LCS.
5. Towards a Better LCS Algorithm
![Page 15: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/15.jpg)
242-535 ADA: 7. Dynamic Prog. 15
• If X = < A, B, C, B, D, A, B > then
• A prefix is x[1 .. 4] == < A, B, C, B >o we abbreviate this as x4
• Also x0 is the empty sequence
Prefixes
![Page 16: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/16.jpg)
242-535 ADA: 7. Dynamic Prog. 16
• c[] is a table (2D array) for storing LCS lengths:c[i, j] = | LCS(x[1. . i], y[1. . j]) |
• | s | is the length of a sequence s
• Since x is of length m, and y is of length n, theno c[m, n] = | LCS(x, y) |
Creating a Table of Lengths
![Page 17: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/17.jpg)
242-535 ADA: 7. Dynamic Prog. 17
• Since X0 and Y0 are empty strings, their LCS is always empty (i.e. c[0, 0] == 0)
• The LCS of an empty string and any other string is empty, so for every i and j: c[0, j] == c[i, 0] == 0
Calculating LCS Lengths
![Page 18: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/18.jpg)
242-535 ADA: 7. Dynamic Prog. 18
Initial c[] 0 1 2 3 4 5
0
1
2
3
4
0
0
00000
0
0
0
![Page 19: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/19.jpg)
242-535 ADA: 7. Dynamic Prog. 19
• The first line of this definition fills the top row and first column of c[] with 0's, as in the non-recursive approach.
6. Recursive Definition of c[]
![Page 20: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/20.jpg)
242-535 ADA: 7. Dynamic Prog. 20
• When we calculate c[i, j], there are two cases:• First case: x[i] == y[j]: one more symbol in
strings X and Y matches, so the length of LCS Xi and Yj equals the length of LCS of smaller strings Xi-1 and Yi-1 , plus 1
otherwise]),1[],1,[max(
],[][ if1]1,1[],[
jicjicjyixjic
jic
![Page 21: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/21.jpg)
242-535 ADA: 7. Dynamic Prog. 21
• Second case: x[i] != y[j]• As symbols don’t match, our solution is not
improved, and the length of LCS(Xi , Yj) is the same as the biggest from before (i.e. max of LCS(Xi, Yj-1) and LCS(Xi-1,Yj)
otherwise]),1[],1,[max(
],[][ if1]1,1[],[
jicjicjyixjic
jic
![Page 22: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/22.jpg)
242-535 ADA: 7. Dynamic Prog. 22
• One advantage of a recursive definition for c[] is that it makes it easy to show that c is optimal by inductiono c[i, j] increases the size of a sub-solution (line
2) or uses the bigger of two sub-solutions (line 3)
o assuming that a smaller c[] entry is optimal, then a larger c[] entry is optimal.
o when combined with the base cases, which are optimal, then c[] is an optimal solution for all entries
7. Is c[] Algorithm Optimal?
![Page 23: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/23.jpg)
242-535 ADA: 7. Dynamic Prog. 23
• c[] is an optimal way to calculate the length of a LCS using smaller optimal solutions (Hallmark #1)
• The c[] algorithm can be used to return the LCS (see later), so LCS also has Hallmark #1
Is LCS() Optimal?
![Page 24: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/24.jpg)
242-535 ADA: 7. Dynamic Prog. 24
LCS(x, y, i, j) if i == 0 or j == 0 then c[i, j] ← 0 else if x[i] == y[ j]
then c[i, j] ← LCS(x, y, i–1, j–1) + 1else c[i, j] ← max( LCS(x, y, i–1, j),
LCS(x, y, i, j–1) ) return c[i, j]
• The recursive definition of c[] has been changed into a LCS() function which returns c[]
LCS Length as Recursive Code
![Page 25: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/25.jpg)
242-535 ADA: 7. Dynamic Prog. 25
• Does the LCS() algorithm have many repeating (overlapping) subproblems?o i.e. does it have DP Hallmark #2?
• Consider the worst case executiono x[i] ≠ y[ j], in which case the algorithm evaluates two
subproblems, each with only one parameter decremented
8. Repeated Subproblems?
![Page 26: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/26.jpg)
242-535 ADA: 7. Dynamic Prog. 26
Height = m + n. The total work is exponential, butwe’re repeating lots of subproblems.
Recursion Tree (in worst cases)
![Page 27: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/27.jpg)
242-535 ADA: 7. Dynamic Prog. 27
• The number of distinct LCS subproblems for two strings of lengths m and n is only m*n.o a lot less than 2m+n total no. of problems
Dynamic Programming Hallmark #2
![Page 28: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/28.jpg)
242-535 ADA: 7. Dynamic Prog. 28
• LCS has both DP hallmarks, and so will benefit from the DP programming techniques:o recursiono memoizationo bottom-up execution
9. LCS() as a DP Problem
![Page 29: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/29.jpg)
242-535 ADA: 7. Dynamic Prog. 29
LCS(x, y, i, j) if c[i, j] is empty then // calculate if not already in c[i, j] if i == 0 or j == 0 then c[i, j] ← 0 else if x[i] == y[ j]
then c[i, j] ← LCS(x, y, i–1, j–1) + 1else c[i, j] ← max( LCS(x, y, i–1, j),
LCS(x, y, i, j–1) ) return c[i, j]
9.1. Memoization
Time = Θ(m*n) == constant work per table entrySpace = Θ(m*n)
![Page 30: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/30.jpg)
242-535 ADA: 7. Dynamic Prog. 30
• This algorithm works top-downo start with large subsequences, and calculate the smaller
subsequences
• Let's switch to bottom-up executiono calculate the small subsequences first, then move to
larger ones
9.2. Bottom-up Execution
![Page 31: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/31.jpg)
242-535 ADA: 7. Dynamic Prog. 31
LCS-Length(X, Y)1. m = length(X) // get the # of symbols in X2. n = length(Y) // get the # of symbols in Y
3. for i = 1 to m c[i,0] = 0 // special case: Y0
4. for j = 1 to n c[0,j] = 0 // special case: X0
5. for i = 1 to m // for all Xi 6. for j = 1 to n // for all Yj
7. if ( Xi == Yj )8. c[i,j] = c[i-1,j-1] + 19. else
c[i,j] = max( c[i-1,j], c[i,j-1] )10. return c
LCS Length Bottom-up
the same recursivedefinition of c[] asbefore
![Page 32: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/32.jpg)
242-535 ADA: 7. Dynamic Prog. 32
We’ll see how a bottom-up LCS works on:• X = ABCB• Y = BDCAB
10. Bottom-up Examples
LCS(X, Y) = BCBX = A B C BY = B D C A B
LCS-length(X, Y) = 3
![Page 33: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/33.jpg)
242-535 ADA: 7. Dynamic Prog. 33
LCS Example 1j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
X = ABCB; m = |X| = 4Y = BDCAB; n = |Y| = 5Allocate array c[5,4]
ABCBBDCAB
![Page 34: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/34.jpg)
242-535 ADA: 7. Dynamic Prog. 34
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
ABCBBDCAB
for i = 1 to m c[i,0] = 0 for j = 1 to n c[0,j] = 0
![Page 35: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/35.jpg)
242-535 ADA: 7. Dynamic Prog. 35
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0
ABCBBDCAB
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
![Page 36: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/36.jpg)
242-535 ADA: 7. Dynamic Prog. 36
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 0
ABCBBDCAB
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
![Page 37: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/37.jpg)
242-535 ADA: 7. Dynamic Prog. 37
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 0 1
ABCBBDCAB
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
![Page 38: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/38.jpg)
242-535 ADA: 7. Dynamic Prog. 38
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
000 1 1
ABCBBDCAB
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
![Page 39: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/39.jpg)
242-535 ADA: 7. Dynamic Prog. 39
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
ABCBBDCAB
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
![Page 40: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/40.jpg)
242-535 ADA: 7. Dynamic Prog. 40
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 1 11
ABCBBDCAB
![Page 41: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/41.jpg)
242-535 ADA: 7. Dynamic Prog. 41
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 1 1 1 2
ABCBBDCAB
![Page 42: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/42.jpg)
242-535 ADA: 7. Dynamic Prog. 42
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj )c[i,j] = c[i-1,j-1] + 1
else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
21 1 11
1 1
ABCBBDCAB
![Page 43: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/43.jpg)
242-535 ADA: 7. Dynamic Prog. 43
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 21 11
1 1 2
ABCBBDCAB
![Page 44: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/44.jpg)
242-535 ADA: 7. Dynamic Prog. 44
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj )c[i,j] = c[i-1,j-1] + 1
else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 21 1
1 1 2
1
22
ABCBBDCAB
![Page 45: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/45.jpg)
242-535 ADA: 7. Dynamic Prog. 45
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 21 1
1 1 2
1
22
1
ABCBBDCAB
![Page 46: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/46.jpg)
242-535 ADA: 7. Dynamic Prog. 46
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj )c[i,j] = c[i-1,j-1] + 1
else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 21 1
1 1 2
1
22
1 1 2 2
ABCBBDCAB
![Page 47: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/47.jpg)
242-535 ADA: 7. Dynamic Prog. 47
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] )
1000 1
1 21 1
1 1 2
1
22
1 1 2 2 3
ABCBBDCAB
![Page 48: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/48.jpg)
242-535 ADA: 7. Dynamic Prog. 48
• The bottom-up LCS algorithm calculates the values of each entry of the array c[m, n]
• So what is the running time? O(m*n)
• Since each c[i, j] is calculated in constant time, and there are m*n elements in the array
Running Time
![Page 49: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/49.jpg)
242-535 ADA: 7. Dynamic Prog. 49
Example 2
m elementsin x[1..m]
n elements in y[1..n]
B == B, somax + 1
![Page 50: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/50.jpg)
242-535 ADA: 7. Dynamic Prog. 50
• So far, we have found the length of LCS.• We want to modify this algorithm to have it
calculate LCS of X and Y
Each c[i, j] depends on c[i-1, j] and c[i, j-1] or c[i-1, j-1]For each c[i, j] we can trace back how it was
calculated:
11. Finding the LCS
2
2 3
2 For example, here c[i, j] = c[i-1, j-1] +1 = 2+1=3
![Page 51: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/51.jpg)
242-535 ADA: 7. Dynamic Prog. 51
• So we can start from c[m,n] (bottom right of c[]) and move backwards
• Whenever c[i,j] = c[i-1, j-1]+1, record x[i] as part of the LCS
• When i=0 or j=0, we have reached the beginning, so can stop.
Remember that:
otherwise]),1[],1,[max(
],[][ if1]1,1[],[
jicjicjyixjic
jic
![Page 52: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/52.jpg)
242-535 ADA: 7. Dynamic Prog. 52
Finding LCS Example 1j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
Yj BB ACD
0
0
00000
0
0
0
1000 1
1 21 1
1 1 2
1
22
1 1 2 2 3B
![Page 53: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/53.jpg)
242-535 ADA: 7. Dynamic Prog. 53
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
Yj BB ACD
0
0
00000
0
0
0
1000 1
1 21 1
1 1 2
1
22
1 1 2 2 3B
![Page 54: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/54.jpg)
242-535 ADA: 7. Dynamic Prog. 54
j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
Yj BB ACD
0
0
00000
0
0
0
1000 1
1 21 1
1 1 2
1
22
1 1 2 2 3B
B C BLCS:
![Page 55: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/55.jpg)
242-535 ADA: 7. Dynamic Prog. 55
Reconstruct LCS by tracing backwards (start at bottom-right, follow numbers and the lines back to top-left).
LCS() = BCBA
Finding LCS for Example 2
![Page 56: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/56.jpg)
242-535 ADA: 7. Dynamic Prog. 56
• There may be several paths through the table, which represent different answers for LCS()
• LCS() = BDAB
• All of them have the LCS-length of 4
LCS() Offers Choices
0 0 0 0 0 0 0 00 0 1 1 1 1 1 10 0 1 1 1 2 2D 20 0 1 2 2 2 2C 20 1 1 2 2 2 3A 30 1 2 2 3 3 3B 40 1 2 2 3 3
AA B C B D B
B
A 4 4
0
4
0
1
23
1
2
34
![Page 57: Algorithm Design and Analysis (ADA)](https://reader036.fdocuments.in/reader036/viewer/2022062315/56816356550346895dd403af/html5/thumbnails/57.jpg)
242-535 ADA: 7. Dynamic Prog. 57
• With this bottom-up approach, the space requirements can be reduced to O(min(m, n)) from O(m*n). Why?
• The calculation of each element in the current row only depends on 3 elements (2 from the previous row).
• Alternatively, a column can be calculated using the previous column.
• So the calculation space only requires 1 row or 1 column, which ever is smaller.
12. Space and Bottom-up