P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without...
-
Upload
aubrey-simon -
Category
Documents
-
view
223 -
download
1
Transcript of P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without...
![Page 1: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/1.jpg)
![Page 2: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/2.jpg)
PROBLEM
A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining
elements.
Using code or pseudocode, write an algorithm that determines the longest common
subsequence of two strings.
![Page 3: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/3.jpg)
INTRODUCTION TO ALGORITHMS
DYNAMIC PROGRAMMING
An introduction
Simon Ellis
27th March, 2014
![Page 4: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/4.jpg)
Fibonacci numbers
Iterative solution
![Page 5: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/5.jpg)
Fibonacci numbers (an aside)
Closed form solution exists Binet’s formula
To calculate any Fibonacci number Fn
![Page 6: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/6.jpg)
Pascal’s Triangle
Iterative solution Begin with 1 on row 0 Each new row is offset to the left by the width of one
number Construct the elements of rows as follows:
For each new value k, sum the value above and left with the value above and right to find k
If there is no number, substitute zero in its place
![Page 7: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/7.jpg)
Pascal’s Triangle
![Page 8: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/8.jpg)
Pascal’s Triangle (an aside)
Closed form solution exists
To calculate values in row n
Use symmetry to derive right-hand side of row (or calculate)
![Page 9: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/9.jpg)
Pascal’s Triangle (another aside)
1
1
2
3
58
1321
34
![Page 10: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/10.jpg)
Calculating prime numbers
Iterative solution Let P be the set of prime numbers; at start, P = ∅ For each value k = 2…n
For each value j = 2…k – 1
• If k/j is not an integer for all j, P = P ∪ { k }
Output P
Naïve, slow solution
![Page 11: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/11.jpg)
Dynamic programming
May refer to two things A mathematical optimisation method An approach to computer programming and algorithm
design
A method for solving complex problems using recursion Problems may be solved by solving smaller parts
Keep addressing smaller parts until each subproblem becomes tractable
Problem must possess both of the following traits Optimal substructure Overlapping subproblems
![Page 12: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/12.jpg)
Optimal substructure
A problem has optimal substructure if an optimal solution can be constructed efficiently from optimal solutions of its subproblems
May be solved using a greedy algorithm Iff it can be proved by induction that this is optimal for
all steps Dynamic programming may be used if subproblems
exist Otherwise a brute-force search is required
![Page 13: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/13.jpg)
Optimal substructure
Optimal substructure Finding the shortest path between two cities by car
e.g. if the shortest route from Seattle to Los Angeles passes through Portland and Sacramento, the shortest route from Portland to Los Angeles passes through Sacramento
Not optimal substructure Buying the cheapest ticket from Sydney to New York
Even if a ticket has stops in Dubai and London we cannot conclude that the cheapest ticket will stop in London as the price at which an airline sells multi-flight tickets is not the sum of the prices for which it would sell individual flights on the same trip
![Page 14: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/14.jpg)
Overlapping subproblems
A problem has overlapping subproblems if either of the following is true The problem can be broken down into subproblems
which are reused several times A recursive algorithm solves the same subproblem
over and over instead of generating new subproblems
Examples Fibonacci numbers Pascal’s Triangle …
![Page 15: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/15.jpg)
Examples of dynamic programming
Fibonacci numbers Pascal’s Triangle Prime number generation Dijkstra’s algorithm/A* algorithm Towers of Hanoi game Matrix chain multiplication Beat tracking in music information retrieval
software Neural networks Word-wrapping in word processors (especially
LATEX)
Transposition and refutation tables in computer chess games
![Page 16: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/16.jpg)
Recursion
Recursion is a powerful tool for solving dynamic programming problems
Not always an optimal solution on its own Naïve Wasteful
e.g. to calculate F100, a lot of work shall be wasted
Two standard approaches to improving performance “Top-down” “Bottom-up”
![Page 17: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/17.jpg)
“Top-down” approach
In software design Developing a program by starting with a large concept
and adding increasing layers of specialisation
In dynamic programming A solution which derives directly from the recursive
solution
Performance is improved by memoisation Solutions to subproblems are stored in a data structure If a subproblem is solved, we read from the table, else
we calculate and add it
![Page 18: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/18.jpg)
“Bottom-up” approach
In software design Developing a program by starting with many small
objects and functions and building on them to create more functionality
In dynamic programming Solving the subproblems first and aggregating them
Performance is improved using stored data Memoisation may be used
![Page 19: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/19.jpg)
Memoisation
Optimisation technique
Reduces need to repeat function calls, which may be expensive
“Memoised” function stores results of previous calls with a specific set of inputs
Function must be ‘referentially transparent’ i.e. calling the function must have exactly the same
outcome as returning the value that would be produced by calling the function
![Page 20: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/20.jpg)
Memoisation
Not the same as using a lookup table Lookup tables are precalculated before use Memoised tables are calculated transparently as
required
Memoisation optimises for time over space Time/space trade-off in all algorithms ‘Computational complexity’
Complexity in time (how much time is required) Complexity in space (how much memory is required)
Cannot reduce one without increasing the other
![Page 21: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/21.jpg)
Computational complexity
Roughly, the quantity of resources required to solve a problem computationally
Resources are whatever is appropriate to the situation Time Memory Logic gates on a circuit board Number of processors in a supercomputer
e.g. “Big-O” notation is a measure of complexity in time
![Page 22: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/22.jpg)
Computational complexity
Cannot reduce all complexity values simultaneously
All software design requires a trade-off Primarily between time (runtime) and space (memory
requirements)
Memoisation optimises for time over space Calculations are performed more quickly… … but memory is required to store precalculated data
![Page 23: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/23.jpg)
Longest common subsequence
Assume two strings, S and T
Solution? Generate all possible subsequences of both sequences, the set
Q
Return longest subsequence in Q
int lcs(char *S, char *T, int m, int n){ if (m == 0 || n == 0) return 0; if (S[m-1] == T[n-1]) return 1 + lcs(S, T, m-1, n-1); else return max(lcs(S, T, m, n-1), lcs(S, T, m-1, n));}
![Page 24: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/24.jpg)
Longest common subsequence
Time: O(2n) Why?
Is this an efficient algorithm for this problem?
int lcs(char *S, char *T, int m, int n){ if (m == 0 || n == 0) return 0; if (S[m-1] == T[n-1]) return 1 + lcs(S, T, m-1, n-1); else return max(lcs(S, T, m, n-1), lcs(S, T, m-1, n));}
![Page 25: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/25.jpg)
Longest common subsequence
Can we use dynamic programming to improve performance?
Does the problem meet the requirements? Does it have optimal substructure? Does it have overlapping subproblems?
![Page 26: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/26.jpg)
Optimal substructure of LCS
Assume Input sequences S[0..m-1] and T [0..n-1] with lengths m
and n L(S[0..m-1], T [0..n-1]) is the length of the LCS of S and T
L(S[0..m-1], T [0..n-1]) is defined recursively as follows If last characters of both sequences match
L(S [0..m–1], T [0..n–1])=1 + L(S [0..m–2], T [0..n–2])
If last characters of both sequences do not match L(S [0..m–1], T [0..n–1]) = MAX( L(S [0..m–2], T [0..n–1]),
L(S [0..m–1], T [0..n–2]) )
![Page 27: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/27.jpg)
Optimal substructure of LCS
Examples Last characters match
Consider the input strings “AGGTAB” and “GXTXAYB” Length of LCS can be written as
• L(“AGGTAB”, “GXTXAYB”) = 1 + L(“AGGTA”, “GXTXAY”)
Last characters do not match Consider the input strings “ABCDGH” and “AEDFHR” Last characters do not match for the strings. Length of LCS can be written as
• L(“ABCDGH”, “AEDFHR”) = MAX ( L(“ABCDG”, “AEDFHR”),
L(“ABCDGH”, “AEDFH”) )
![Page 28: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/28.jpg)
Overlapping subproblems in LCS
Consider the original code Many of the same problems being solved repeatedly
Subproblems overlap
Performance can be improved by memoisation
lcs("AXYT", "AYZX”)/ \
lcs("AXY", "AYZX") lcs("AXYT", "AYZ") / \ / \
lcs("AX", "AYZX") lcs("AXY", "AYZ") lcs("AXY", "AYZ") lcs("AXYT", "AY")
![Page 29: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/29.jpg)
Example of LCS of two strings
∅ A G C A T
∅ ∅ ∅ ∅ ∅ ∅ ∅
G ∅
A ∅
C ∅
![Page 30: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/30.jpg)
Example of LCS of two strings
∅ A G C A T
∅ ∅ ∅ ∅ ∅ ∅ ∅
G ∅
∅ (G)
(G)
(G)
(G)
A ∅
C ∅
![Page 31: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/31.jpg)
Example of LCS of two strings
∅ A G C A T
∅ ∅ ∅ ∅ ∅ ∅ ∅
G ∅
∅ (G)
(G)
(G)
(G)
A ∅
(A) (A)
(G)
(A)(G) (GA)
(GA)
C ∅
![Page 32: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/32.jpg)
Example of LCS of two strings
∅ A G C A T
∅ ∅ ∅ ∅ ∅ ∅ ∅
G ∅
∅ (G)
(G)
(G)
(G)
A ∅
(A) (A)
(G)
(A)(G) (GA)
(GA)
C ∅
(A) (A)
(G)(AC)(GC)
(AC)(GC)(GA)
(AC)(GC)(GA)
![Page 33: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/33.jpg)
Example of LCS of two strings
New runtime O(n ⋅ m) Much less than previously: O(2n)
But increased space requirement to store working data
![Page 34: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/34.jpg)
ANY QUESTIONS?
![Page 35: P ROBLEM A subsequence is a sequence derived from another sequence by deleting some elements without changing the order of the remaining elements. Using.](https://reader035.fdocuments.in/reader035/viewer/2022062407/56649ce65503460f949b40d1/html5/thumbnails/35.jpg)
References
http://faculty.ycp.edu/~dbabcock/cs360/lectures/lecture13.html
http://www.algorithmist.com/index.php/Longest_Common_Subsequence