Post on 31-Mar-2015
A New Recombination Lower Bound and The Minimum Perfect
Phylogenetic Forest Problem
Yufeng Wu and Dan Gusfield
UC Davis
COCOON’07 July 16, 2007
2
Recombination
• Recombination: one of the principle genetic forces shaping sequence variations within species
• Two equal length sequences generate a third new equal length sequence during meiosis.
110001111111001
000110000001111
Prefix
Suffix
11000 0000001111
Breakpoint
3
Ancestral Recombination Graph (ARG)
10 01 0011
10
00
00
01
1 0 0 1
1 1
Network, not tree!
Assumption: at most one mutation per site
Mutations
Recombination
4
A Min ARG for Kreitman’s data
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
ARG created by SHRUB
5
Minimizing Recombination• Given enough recombinations, any set of
sequences can be trivially derived on an ARG.
• Problem: Given a set of sequences M, construct an ARG that derives M using one mutation per site, and the minimum number of recombinations (Rmin).
• NP-hard (Wang, et al 2001, Semple et al.).– Efficiently computed Lower bounds on Rmin
exist.
History Bound (Myers & Griffiths 2003)
000
100
010
011
111
Iterate the following operations1. Remove a column with a single 0 or 12. Remove a duplicate row3. Remove any row
History bound: the minimum number of type-3 operations needed to reduce the matrix to empty
000
100
010
011
00
10
01
01
Empty.
One type-3 operation
00
10
01
M
7
Graphical interpretation of history bound (HistB)
• Each operation corresponds to an operation that decomposes the optimal, but unknown ARG.
• Removing an exposed recombination node in the ARG corresponds to a single type-3 operation. So when decomposing the optimal ARG, the number of recombination nodes => number of corresponding type-3 operations.
• However, not all type-3 operations correspond to removing a recombination node.• Since the optimal ARG is unknown, the history bound is the
minimum number of type-3 operations needed to make the matrix empty.
8
4
1
3
2 5
a: 00010
b: 10010
d: 10100
c: 00100
e: 01100
f: 00101g: 00101
2p s
a: 00010b: 10010c: 00100d: 10100
e: 01100
f: 00101g: 00101
Operations on M correspond to operations on the optimal ARG
M
9
4
1
3
2 5
a: 00010
b: 10010
d: 10100
c: 00100
e: 01100
f: 00101
2p s
a: 00010b: 10010c: 00100d: 10100
e: 01100
f: 00101
12345
Type-2 operation
10
4
1
3
a: 001
b: 101
d: 110
c: 010
e: 010
f: 010
2p s
a: 001b: 101c: 010d: 110
e: 010
f: 010
134
Type-1 operations
11
4
1
3
a: 001
b: 101
d: 110
c: 010
2p s
a: 001b: 101c: 010d: 110
134
Type-2 operations
12
4
1
3
a: 001
b: 101 c: 010
a: 001b: 101c: 010
134
Type-3 operation
Then three more Type-1 operations fully reduce M and the ARG.
13
History bound
• Initially required trying all n! permutations of the rows to choose the type-3 operations.
• The bound can be computed by DP in O(2n) time (Bafna, Bansal).
• On datasets where it can be computed, the history bound is observed to be higher than (or equal to) all studied lower bounds (about ten of them).
• There is no static definition for what the history bound is -- it is only defined by the algorithms that compute it! The work in this paper comes out of an attempt to find a simple static definition.
14
Why a static definition matters
• We want a definition of what is being computed, independent of how it is computed, so that we can reason about it and find alternative ways to compute or approximate it.
• For example, with no static definition of the history bound, we don’t know how to formulate an integer linear program to compute it.
15
00000
1
2
4
3
510100
1000001011
00010
01010
12345sites
Site mutations on edgesThe tree derives the set M:1010010000010110101000010 starting from 00000
Only one mutation per siteallowed.
Perfect Phylogeny
Intro. to Forest Bound: Decompose an Optimal ARG to A Forest of Trees,
removing recombination edges
An ARG with three recombinations
After removing recombination edges, four trees result.
The number of trees is precisely the number of recombinations plus one
17
Idea behind the Forest Bound (FB)
Each tree created in this way contains at mostone occurrence of any site, and each site occursin at most one of the trees. So the trees form aforest of related perfect phylogenies.
18
Forest Bound
Given a set of sequences M, partition M intothe fewest subsets so that each subset of sequences can be derived on a tree, whereeach site occurs at most once in the forest oftrees. The number of trees, minus one, is a validlower bound on Rmin.
22
Comparing the Forest Bound (FB) to:
• History Bound (HistB)
• Optimal Haplotype Bound (OhapB): The currently best lower bound that can be computed in practice for biological data.
• Theorem: On any data, OhapB <= FB <= HistB On some data, OhapB < FB < HistBThus the FB is the highest lower bound with a static
definition.
23
First, define the Haplotype Lower Bound (S. Myers, 2003)
• Rh = Number of distinct sequences (rows) - Number of distinct sites (columns) -1 <= minimum number of recombinations needed (folklore)
• Before computing Rh, remove any site that is compatible with all other sites. A valid lower bound results - generally increases the bound.
• Generally Rh is really bad bound, often negative, when used on large intervals, but Very Good when used as local bounds in the Composite Method. Myers implemented the method in a program called RecMin, which was a huge advance, generally three times higher than the prior best lower bound method.
• The composite method can be used with any lower bound method and the better the initial lower bounds, the better the composite result.
24
Then, the Subset Bound (Myers)
• Let S be a subset of sites, and Rh(S) be the haplotype bound computed on the sequences restricted to S. Rh(S) is a valid lower bound on Rmin.
• Optimal Haplotype Bound (OhapB) is the maximum Rh(S) over all subsets of sites.
• Practical computation of OhapB via ILP was studied in (SWG 2005) and exploited in the program Hapbound. Hapbound gives provably better bounds than RecMin.
25
Now, the Optimal Haplotype Bound (OhapB)
• OhapB is the maximum haplotype bound over any subset of columns.
• NP-hard (Bafna & Bansal, 2005)– Efficiently computed in
practice (Song, Wu, Gusfield 2005)
0 0 0
0 0 1
1 0 0
1 1 1
Rh = 4 – 3 – 1
= 0
11
01
10
00Rh = 4 – 2 – 1
= 1
Forest Bound (FB) is Higher than Haplotype Bound (Rh)
10001 00010
00100
1101101101
s2
s1s4
Steiner nodes
Rh = number distinct rows – number distinct columns – 1 = 5 -- 5 --1 = --1
s5
s3
FB = 2
27
FB >= Rh
FB obtained from all the data M is >= FB obtained from a subset of the columns, so assume all columns in M are distinct.
FB = # trees in the FB -- 1 = # nodes -- # edges -- 1 in the forest= # leaves + # Steiner nodes -- # columns -- 1= # rows + # Steiner nodes -- # columns -- 1>= # distinct rows -- # columns-- 1 = Rh
28
Forest Bound is Higher than Optimal Haplotype Bound
F(Ms) Rh(Ms) (i.e. the optimal haplotype bound).
s2 s3 s5
Optimal subset of columns Mss2 s3 s5
Input matrix M
Number of Trees after Taking Subset of Columns
s3
s1
s2
s5
s6s4
Minimum forest with 3 trees for entire data
s3
A legal forest for the subset data!
s2, s3, s5.
s2
s5
s3s2
s5
A legal forest with 3 trees for the subset
Cleanup
Also, taking subsets can not increase the number of trees, and so FB(M) FB(Ms).
So, FB(M) FB(Ms) Rh(Ms), so FB OhapB
30
FB <= HistB
The decomposition of the optimal ARG, directedby the operations of computing the history bound, creates a forest of HistB + 1 trees, where each site occurs at most once, in at most one tree. So FB <= HistB.
31
Computing the Forest Bound is NP-Hard
• Optimal haplotype bound is quite good, but NP-hard to compute.
• If the forest bound can be efficiently computable, we do not need to use optimal haplotype bound at all.
• Unfortunately, the forest bound is NP-hard to compute.
• Reduction from Exact-cover-by-3 sets.
NP-hardness Proof for the Minimum Perfect Phylogenetic Forest Problem
{1,3,5}
{1,2,4}
{2,4,6}
3-SetsBinary sequences on a hypercube
Sequences corresponding to the same set form a perfect phylogeny with a single novel sequence (not in input)
Two sequences from different sets are far apart, and would need two many mutations to connect, thus can not belong to the same tree.
Sequences corresponding to same element in two sets need same mutation and thus can not be both chosen.
33
Integer Programming Formulation for the Forest Bound
• For sequences with m sites, consider the hypercube all possible 2m sequences.
• Minimizing F is equivalent to reducing the number of Steiner nodes in the forests.
• We also need to ensure the edge linking two nodes in a tree is only labeled with columns that do not appear in other trees.
• Can easily incorporate the missing data in the input.• The IP formulation has exponential size, but practical
when the number of columns is relatively small.
34
Empirical Results
• On random generated dataset with 15 rows and 7 columns, FB > OhapB on 10% of the data. On more biological meaningful data (generated with simulation program ms), however, OhapB= FB more often.
• On dataset generated by ms with missing entries, FB is more often outperforms an approximate optimal Rh bound:– 30 rows and 7 columns and 30% missing entries: FB was
strictly larger in 8% of the data.– When the level of missing entries is lower, the approx.
OhapB matches the FB more often.