Matrix Parenthesizing Longest Common Subsequence Activity...
Transcript of Matrix Parenthesizing Longest Common Subsequence Activity...
UMass Lowell Computer Science 91.503Analysis of Algorithms
Prof. Karen Daniels
Design Patterns for Optimization Problems
Dynamic ProgrammingMatrix Parenthesizing
Longest Common SubsequenceActivity Selection
Algorithmic Paradigm Context
Divide & Conquer
Dynamic Programming
Greedy Algorithm
View problem as collection of subproblems
“Recursive” nature Independent subproblems overlapping typically
sequential dependence
Number of subproblems depends on partitioning
factors
typically small
Preprocessing typically sort Characteristic running time typically log
function of n depends on number and difficulty of subproblems
often dominated by nlogn sort
Primarily for optimization problems (find an optimal solution)
Optimal substructure: optimal solution to problem contains within it optimal solutions to subproblems
Greedy choice property: locally optimal produces globally optimal
Heuristic version useful for bounding optimal value
Subproblem solution orderMake choice, then solve subproblem(s)
Solve subproblem(s), then make choice
Dynamic Programming Approach to Optimization Problems
1. Characterize structure of an optimal solution.
2. Recursively define value of an optimal solution.
3. Compute value of an optimal solution, typically in bottom-up fashion.
4. Construct an optimal solution from computed information.(separate slides for rod cutting)
source: 91.503 textbook Cormen, et al.
Example: Matrix Parenthesization Definitions
Given “chain” of n matrices: <A1, A2, … An, >
Compute product A1A2… An efficiently
Multiplication order matters!
Matrix multiplication is associative
Minimize “cost” = number of scalar multiplications
source: 91.503 textbook Cormen, et al.
Example: Matrix Parenthesization Step 1: Characterizing an Optimal Solution
Observation: Any parenthesization of AiAi+1… Aj must split it between Ak and Ak+1 for some k.
THM: Optimal Matrix Parenthesization:If an optimal parenthesization of AiAi+1… Aj splits at k, then parenthesization of prefix AiAi+1… Ak must be an optimal parenthesization.
Why?If existed less costly way to parenthesize prefix, then substituting that
parenthesization would yield less costly way to parenthesize AiAi+1… Aj , contradicting optimality of that parenthesization.
source: 91.503 textbook Cormen, et al.
common DP proof technique: “cut-and-paste” proof by contradiction
Example: Matrix Parenthesization Step 2: A Recursive Solution
Recursive definition of minimum parenthesizationcost:
m[i,j]= min{m[i,k] + m[k+1,j] + pi-1pkpj} if i < j 0 if i = j
How many distinct subproblems?
i <= k < j
each matrix Ai has dimensions pi-1 x pi
source: 91.503 textbook Cormen, et al.
Example: Matrix Parenthesization Step 3: Computing Optimal Costs
0
2,625
2,500
1,000
s: value of k that achieves optimal cost in computing m[i, j]
source: 91.503 textbook Cormen, et al.
Example: Matrix Parenthesization Step 4: Constructing an Optimal Solution
PRINT-OPTIMAL-PARENS(s, i, j)if i == j
print “A”ielse print “(“
PRINT-OPTIMAL-PARENS(s, i, s[i, j])PRINT-OPTIMAL-PARENS(s, s[i, j]+1, j)print “)“
source: 91.503 textbook Cormen, et al.
Example: Matrix ParenthesizationMemoization
• Provide Dynamic Programming efficiency
• But with top-down strategy• Use recursion• Fill in m table “on demand”
• (can modify to fill in s table)
source: 91.503 textbook Cormen, et al.
MEMOIZED-MATRIX-CHAIN(p)1 n = p.length – 12 let m[1…n,1…n] be a new table.3 for i = 1 to n4 for j = i to n5 m[i,j] = 6 return LOOKUP-CHAIN(m, p,1,n)
∞
LOOKUP-CHAIN(m,p,i,j) 1 if m[i,j] < 2 return m[i,j]3 if i==j 4 m[i,j] = 05 else for k = i to j-16 q = LOOKUP-CHAIN(m,p,i,k)
+ LOOKUP-CHAIN(m,p,k+1,j)+ pi-1 pk pj
7 if q < m[i,j] 8 m[i,j] = q9 return m[i,j]
∞
Example: Longest Common Subsequence (LCS): Motivation
• Strand of DNA: string over finite set {A,C,G,T}• each element of set is a base: adenine, guanine, cytosine or thymine
• Compare DNA similarities• S1 = ACCGGTCGAGTGCGCGGAAGCCGGCCGAA
• S2 = GTCGTTCGGAATGCCGTTGCTCTGTAAA• One measure of similarity:
• find the longest string S3 containing bases that also appear (not necessarily consecutively) in S1 and S2
• S3 = GTCGTCGGAAGCCGGCCGAA
source: 91.503 textbook Cormen, et al.
Example: LCS Definitions
• Sequence is a subsequence of if (strictly increasing
indices of X) such that• example: is subsequence of
with index sequence
• Z is common subsequence of X and Y if Z is subsequence of both X and Y• example:
• common subsequence but not longest• common subsequence. Longest?
Longest Common Subsequence Problem: Given 2 sequences X, Y, find maximum-length common subsequence Z.
source: 91.503 textbook Cormen, et al.
Example: LCS Step 1: Characterize an LCS
THM 15.1: Optimal LCS Substructure Given sequences:For any LCS of X and Y:
1 if then and Zk-1 is an LCS of Xm-1 and Yn-1
2 if then Z is an LCS of Xm-1 and Y 3 if then Z is an LCS of X and Yn-1
PROOF: based on producing contradictions
1 a) Suppose . Appending to Z contradicts longest nature of Z.b) To establish longest nature of Zk-1, suppose common subsequence W of Xm-1 and Yn-1
has length > k-1. Appending to W yields common subsequence of length > k = contradiction.
2 To establish optimality (longest nature), common subsequence W of Xm-1 and Y of length > k would also be common subsequence of Xm, Y, contradicting longest nature of Z.
3 Similar to proof of (2)source: 91.503 textbook Cormen, et al.
(using prefix notation)
Example: LCS Step 2: A Recursive Solution
Implications of Theorem 15.1:
?yes no
Find LCS(Xm-1, Yn-1) Find LCS(Xm-1, Y) Find LCS(X, Yn-1)
LCS1(X, Y) = LCS(Xm-1, Yn-1) + xm LCS2(X, Y) = max(LCS(Xm-1, Y), LCS(X, Yn-1))
Example: LCS Step 2: A Recursive Solution (continued)
• Overlapping subproblem structure:
• Recurrence for length of optimal solution:
Conditions of problem can exclude some subproblems!
),(),( 111 YXLCSYXLCS mnm −−− ⊆
),(),( 111 −−− ⊆ nnm YXLCSYXLCS),( YXLCS⊆
c[i,j]= c[i-1,j-1]+1 if i,j > 0 and xi=yj
max(c[i,j-1], c[i-1,j]) if i,j > 0 and xi=yj
0 if i=0 or j=0
Θ(mn) distinct subproblems
source: 91.503 textbook Cormen, et al.
Example: LCS Step 3: Compute Length of an LCS
source: 91.503 textbook Cormen, et al.
c table(represent b table)
0
12
3
4
What is the asymptotic worst-
case time complexity?
Activity SelectionOptimization Problem
• Problem Instance:• Set S = {a1,a2,...,an} of n activities• Each activity i has:
• start time: si
• finish time: fi
• Activities require exclusive use of a common resource.• Activities i, j are compatible iff non-overlapping:
• Objective:• select a maximum-sized set of mutually compatible activities
ii fs ≤
)[ ii fs ≤ )[ jj fs ≤
source: 91.404 textbook Cormen, et al.
Activity Selection
1 2 3 4 5 6 7 8 9 10 1211 13 14 15 16
1
2
3
4
8
7
6
5
Activity Time Duration
Activity Number
What is answer in this case?
Activity Selection
}:{ jkkikij sfsfSaS ≤<≤∈=Solution to Sij including ak produces 2 subproblems:1) Sik (start after ai finishes; finish before ak starts)2) Skj (start after ak finishes; finish before aj starts)
source: 91.404 textbook Cormen, et al.
⎪⎭
⎪⎬⎫
⎪⎩
⎪⎨⎧
/≠++/=
=∈<< 0 if}1],[],[{max
0 if0],[
; ijSajki
ij
SjkckicS
jicijk
c[i,j]=size of maximum-size subset of mutually compatible activities in Sij.