Reverse Engineering Gene Networks Using...

Introduction ABC-MCMC for Networks Simulations Data Analysis Discussion

Reverse Engineering Gene Networks Using Approximate

Bayesian Computation (ABC)Rencontre de statistique autour des modeles hierarchiques:

Universite de Strasbourg

Andrea Rau

January 14, 2011

1 / 34


Outline

1. Introduction• Gene regulatory networks

2. Approximate Bayesian Computation• Background and motivation• Monte Carlo approaches

3. ABC-MCMC for Networks• Simulation studies• Real data analysis: SOS DNA repair system in E. coli

4. Discussion

2 / 34


Gene Expression

• Genes: Functional regions of DNA that encode proteins and RNAmolecules

3 / 34


Gene Expression

• Genes: Functional regions of DNA that encode proteins and RNAmolecules

• Expression levels of thousands of genes can be measured using“high-throughput” technologies (e.g., microarrays, serial analysis ofgene expression, next-generation sequencing)

AAAAA

AAAAA

AAAAA

AAAAA

AAAAA

AAAAA

AAAAA

A B C D E F G

3 / 34


Time-Course Gene Expression

• Time-course gene expression data can elucidate information aboutpatterns of relationships of gene expression in a cell

• Large number of genes, few biological replicates or time points...N � P paradigm

4 / 34


Gene Regulatory Networks• Gene regulatory networks: set of genes that interact indirectly withone another through proteins called transcription factors (TF)

DNA

Gene 1 Gene 2 Gene 3 Gene 4

TF binding site in promoter region

Coding DNA (gene)

Transcrip!on factor

5 / 34



DNA



Coding DNA (gene)

Transcrip!on factor

• Abundance of TF is difficult to measure ⇒ expression levels ofcorresponding genes usually used as proxy

5 / 34



DNA



Coding DNA (gene)

Transcrip!on factor

1

2

3

4

• Abundance of TF is difficult to measure ⇒ expression levels ofcorresponding genes usually used as proxy

5 / 34


Reverse Engineering Gene Regulatory Networks

• Expression levels of thousands of genescan be measured using “high-throughput”technologies

T = 1

T = 2 T = 3

T = 4 …

6 / 34


Reverse Engineering Gene Regulatory Networks

• Expression levels of thousands of genescan be measured using “high-throughput”technologies

• Time-course gene expression data canelucidate information about patterns ofrelationships of gene expression

T = 1

T = 2 T = 3

T = 4 …

A B C D

A 1 1 0 0

B 0 0 0 0

C 0 0 0 1

D 0 1 0 0

A B

C D

A B C D

A 1 2 0 0

B 0 0 0 0

C 0 0 0 2

D 0 1 0 0

−

−

A B

C D

G = Θ =

Adjacency matrix Parameter matrix

6 / 34


Bayesian Framework

• Let observed time-course gene expression data bey = {yt : t = 1, . . . ,T}, where yt = (yt1, ..., ytP )

T .

7 / 34


Bayesian Framework


T .

• Reverse engineering gene networks is a high dimensional problem:many possible gene-to-gene interactions, few time points andreplicates (P � N)

7 / 34


Bayesian Framework


T .


• Many network structures may yield similarly high likelihoods, soposterior distributions may be more informative about particulargene-to-gene interactions

7 / 34


Bayesian Framework


T .



• A priori biological information may be encoded into the priordistributions (network topology, sparsity, information about pathwaysfrom bioinformatics databases, ...)

7 / 34


Bayesian Framework


T .



• A priori biological information may be encoded into the priordistributions (network topology, sparsity, information about pathwaysfrom bioinformatics databases, ...)

• ⇒ Fit model f (y|θ) to observed data y, where parameters are alsorandom variables following π(θ).

• Conditional distribution of network edges given observed data isπ(θ|y) ∝ f (y|θ)π(θ)

7 / 34


Approximate Bayesan Computation (ABC)

• Without restrictive distributional assumptions on model parameters,likelihood may be difficult to calculate

8 / 34




• Approximate Bayesian Computation: Sampling-based Bayesianapproach to infer approximate posterior distribution π(Θ|ρ(y?, y) ≤ ε)using simulated data y?, a distance function ρ, and tolerance ε

8 / 34





• First applied in population genetics problems (e.g., Pritchard et al.,1999; Beaumont et al., 2002)

• Some approaches for biological networks (Ratmann et al., 2007; Toniet al., 2010)

• Novel (and non-standard?) adaptation to reverse engineering generegulatory networks

8 / 34





• First applied in population genetics problems (e.g., Pritchard et al.,1999; Beaumont et al., 2002)

• Some approaches for biological networks (Ratmann et al., 2007; Toniet al., 2010)

• Novel (and non-standard?) adaptation to reverse engineering generegulatory networks

• Approximate when ε > 0 and equivalent to simulating from the priorwhen ε → ∞

8 / 34


ABC Motivation

Prior: P(Θ)

Posterior: P(Θ|y) Time

x

x

x

x

x x

Rejec!on sampling (Pritchard et al., 1999)

x = Observed data

(Slide idea from Tina Toni)

9 / 34


ABC Motivation

Prior: P(Θ)


x

x

x

x

x x

x = Observed data


• Tolerance/distance criteria

• Summary sta"s"cs (Beaumont et al., 2002)


9 / 34


ABC Motivation

Prior: P(Θ)


x

x = Observed data

x

x

x

x x


x = Observed data



• Post-sampling regression (Leuenberger and Weimann, 2009)

• Nonparametric es"ma"on (Beaumont et al., 2002)


9 / 34


ABC Motivation

Prior: P(Θ)


x

x = Observed data

x

x

x

x x

• Sequen"al methods (Sisson et al., 2007)

• Markov Chain Monte Carlo (Marjoram et al., 2003; Bortot et al., 2007)

• Post-sampling regression (Leuenberger and Weimann, 2009)

• Nonparametric es"ma"on (Beaumont et al., 2002)





9 / 34


ABC-MCMC (Marjoram et al., 2003)

• ABC-Markov chain Monte Carlo (MCMC): Construct a Markov chain(e.g., using Metropolis-Hastings algorithm) with approximateposterior distribution π(Θ|ρ(y?, y) ≤ ε) as equilibrium distribution(Marjoram et al., 2003) Details

10 / 34


ABC-MCMC (Marjoram et al., 2003)

• ABC-Markov chain Monte Carlo (MCMC): Construct a Markov chain(e.g., using Metropolis-Hastings algorithm) with approximateposterior distribution π(Θ|ρ(y?, y) ≤ ε) as equilibrium distribution(Marjoram et al., 2003) Details

• Let q(·|·) and π(·) be the transition and prior distributions,respectively.

• Given previous Θi , a proposed Θ? is accepted at the iterations with(i + 1)st probability

α = min

{

1,π(Θ?)q(Θi |Θ?)

π(Θi )q(Θ?|Θi )1(ρ(y?, y) < ε)

}

10 / 34


Adapting ABC-MCMC to Networks

• Several adaptations must be made to the ABC-MCMC method ofMarjoram et al. (2003) for reverse engineering gene regulatorynetworks:

11 / 34




1. Computationally efficient way to simulate expression data y? from aknown regulatory network Θ?

11 / 34





2. Appropriate distance function ρ and tolerance ε to compare simulated(y?) and observed (y) data

11 / 34





2. Appropriate distance function ρ and tolerance ε to compare simulated(y?) and observed (y) data

3. Prior and proposal distributions for network structures

11 / 34


Simulating y? for Network Θ? (continuous)

Generally, we simulate gene expression at time t as a function of the geneexpression at the previous time point:

y?t = ft(yt−1,Θ?)

12 / 34


Simulating y? for Network Θ? (continuous)

Generally, we simulate gene expression at time t as a function of the geneexpression at the previous time point:

y?t = ft(yt−1,Θ?)

In practice, for continuous data (e.g., microarrays):

• Set y?1 = y1.

• Generate one-step-ahead predictors based on first-order VAR modelon gene expression for t = 2, . . . ,T :

y?t = Θ?yt−1

Details

12 / 34


Simulating y? for Network Θ? (discrete)

For count data (e.g., serial analysis of gene expression, RNA sequencing):

• Set y?1 = y1.

• yt ∼ Poisson(λt), where λt = πty·t ,∑P

i=1 πit = 1, and

y·t =∑P

i=1 yit .

13 / 34


Simulating y? for Network Θ? (discrete)

For count data (e.g., serial analysis of gene expression, RNA sequencing):

• Set y?1 = y1.

• yt ∼ Poisson(λt), where λt = πty·t ,∑P

i=1 πit = 1, and

y·t =∑P

i=1 yit .

• Generate one-step-ahead predictors based on first-order VAR modelon the level of gene expression for t = 2, . . . ,T :

π?

t = exp{

1y·t−1

Θ?yt−1

}

and π?t =

1∑Pi=1 π

?

it

π?

t

y?t ∼ Poisson(π?

t y·t)

Details

13 / 34


Distance Function and ToleranceDistance functions (ρ):

• Canberra: ρ(y?, y) =

T∑

t=1

P∑

i=1

|y?it − yit |

|y?it + yit |

• Euclidean: ρ(y?, y) =

√

√

√

√

T∑

t=1

P∑

i=1

(y?it − yit)2

• Manhattan: ρ(y?, y) =

T∑

t=1

P∑

i=1

|y?it − yit |

• Multivariate Time Series (MVT):

ρ(y?, y) = 1T

T∑

t=1

[(yt − y?t )− (yt − y?t )]′ Σ−1 [(yt − y?t )− (yt − y?t )]

14 / 34


Distance Function and ToleranceDistance functions (ρ):

• Canberra: ρ(y?, y) =

T∑

t=1

P∑

i=1

|y?it − yit |

|y?it + yit |

• Euclidean: ρ(y?, y) =

√

√

√

√

T∑

t=1

P∑

i=1

(y?it − yit)2

• Manhattan: ρ(y?, y) =

T∑

t=1

P∑

i=1

|y?it − yit |

• Multivariate Time Series (MVT):

ρ(y?, y) = 1T

T∑

t=1

[(yt − y?t )− (yt − y?t )]′ Σ−1 [(yt − y?t )− (yt − y?t )]

Tolerance (ε):

• ε = 1% quantile of distances ρ from 5000 random networks

14 / 34


Network Proposals• With networks, we must propose both a new structure and a new setof parameters

15 / 34



• To facilitate simulation, we introduce as an auxiliary variable a P × P

adjacency matrix G , where Gij = 1 if gene j regulates gene i , andGij = 0 otherwise.

• Note that Gij = 0 ⇔ Θij = 0 and Gij = 1 ⇔ Θij 6= 0

A B C D

A 1 1 0 0

B 0 0 0 0

C 0 0 0 1

D 0 1 0 0

A B

C D

A B C D

A 1 2 0 0

B 0 0 0 0

C 0 0 0 2

D 0 1 0 0

−

−

A B

C D

G = Θ =

15 / 34



• To facilitate simulation, we introduce as an auxiliary variable a P × P

adjacency matrix G , where Gij = 1 if gene j regulates gene i , andGij = 0 otherwise.

• Note that Gij = 0 ⇔ Θij = 0 and Gij = 1 ⇔ Θij 6= 0

A B C D

A 1 1 0 0

B 0 0 0 0

C 0 0 0 1

D 0 1 0 0

A B

C D

A B C D

A 1 2 0 0

B 0 0 0 0

C 0 0 0 2

D 0 1 0 0

−

−

A B

C D

G = Θ =

• Joint distribution of G and Θ may be seen as a completion to themarginal density of Θ

15 / 34


Two-Step Proposal Distribution

• Two-step proposal distribution: q(G ?|G i) and q(Θ?|Θi ,G ?):

A

B C

0 0 0

0 0 2

3 0 0

−

C Θ

i =

A A

B

B C

Parameter matrix

A

B C

0 0 0

0 0 1

1 0 0

A

A

B

B C

C

Gi =

A

B C

Adjacency matrix

16 / 34




A

B C

0 0 0

0 0 2

3 0 0

−

C Θ

i =

A A

B

B C

Parameter matrix

Add, delete, reverse edge

(Husmeier, 2003)

A

B C

0 0 0

0 0 1

1 0 0

A

A

B

B C

C

Gi =

A

B C

Adjacency matrix

A

B C

A

A

B

B C

G* =

A

B C

0 0 0

1 0 1

1 0 0

C

16 / 34




A

B C

0 0 0

0 0 2

3 0 0

−

C Θ

i =

A A

B

B C

Parameter matrix

A

B C Θ

* =

A

A

B

B C 0 0 0

0.4 0 0.3

2.7 0 0

−

C

Add, delete, reverse edge

(Husmeier, 2003)

A

B C

0 0 0

0 0 1

1 0 0

A

A

B

B C

C

Gi =

A

B C

Adjacency matrix

A

B C

A

A

B

B C

G* =

A

B C

0 0 0

1 0 1

1 0 0

C

Gaussian proposal

16 / 34


Prior Distributions

• Gene regulatory networks typically sparse with spoke-and-hubstructure and few regulators per gene (fan-in)

Spoke-

and-hub

A

B F C

D E A

B C

D

Gene A

fan-in = 3

17 / 34


Prior Distributions

• Gene regulatory networks typically sparse with spoke-and-hubstructure and few regulators per gene (fan-in)

Spoke-

and-hub

A

B F C

D E A

B C

D

Gene A

fan-in = 3

Prior distributions:

• π(G ) is uniform over all structures, with maximum fan-in of 5 or less

• π(Θ|G ) is uniform

17 / 34


ABC-MCMC Network Method

ABC-Net Algorithm:

0. Initialize Θi , G i , i = 0.

18 / 34



ABC-Net Algorithm:

0. Initialize Θi , G i , i = 0.1. (a) Propose G ? according to q(G |G i).

(b) Propose Θ? according to q(Θ|Θi ,G ?).

18 / 34



ABC-Net Algorithm:


(b) Propose Θ? according to q(Θ|Θi ,G ?).2. Simulate y? from f (·|Θ?,G ?).

18 / 34



ABC-Net Algorithm:


(b) Propose Θ? according to q(Θ|Θi ,G ?).2. Simulate y? from f (·|Θ?,G ?).3. Set {G i+1, Θi+1} = {G ?, Θ?} with probability

α = min{1, π(G?)π(Θ?|G?)q(G i |G?)q(Θi |Θ?)

π(G i )π(Θi |G i )q(G?|G i )q(Θ?|Θi )1 [ρ(y?, y) ≤ ε]}

and {G i+1,Θi+1} = {G i ,Θi} with probability 1− α.

18 / 34



ABC-Net Algorithm:





and {G i+1,Θi+1} = {G i ,Θi} with probability 1− α.4. Set i = i + 1. If i < N (a pre-set number of iterations), return to 1.

18 / 34



ABC-Net Algorithm:





and {G i+1,Θi+1} = {G i ,Θi} with probability 1− α.4. Set i = i + 1. If i < N (a pre-set number of iterations), return to 1.

• Burn-in period, number of iterations, chain thinning, ... Details

18 / 34


Simulations: Raf Signalling Protein Pathway• Simulations based on currently accepted gold-standard Raf signallingpathway (Sachs et al., 2005) in human immune system cells for 11genes (20 total edges)

19 / 34


Simulations: Raf Signalling Protein Pathway• Simulations based on currently accepted gold-standard Raf signallingpathway (Sachs et al., 2005) in human immune system cells for 11genes (20 total edges)

• Simulate T = 20 time points, R = 1 replicate using VAR model• Run ABC-Net algorithm for 10 independent chains of length 1× 106

with thinning interval of 50• Use Gelman-Rubin statistic to assess convergence across chains

19 / 34


ABC-Net Simulations

1. Choice of distance function ρ and tolerance ε

2. Suitability of VAR simulator for data generated with alternativemodels (nonlinear models, second-order models, and ordinarydifferential equations)

3. Sensitivity to prior distribution bounds

20 / 34


Simulations I: Choice of ρ and ε

• Set ε to be the 1%, 5%, or 10% quantile of distances ρ from 5000random networks

0.50

0.55

0.60

0.65

0.70

0.75

0.80

AU

C

1% 5% 10% 1% 5% 10% 1% 5% 10% 1% 5% 10%

Canberra Euclidean Manhattan MVT

Distance Function and Threshold

Area Under the Curve (By Distance Function and Threshold)

21 / 34


Simulations II: Suitability of VAR Simulator• Alternative models: first-order nonlinear VAR (VAR-NL(1)),second-order VAR (VAR(2)), second-order nonlinear VAR(VAR-NL(2)), and ordinary differential equation (ODE)

0.40

0.45

0.50

0.55

0.60

0.65

0.70

0.75

AU

C

VAR(1) VAR−NL(1) VAR(2) VAR−NL(2) ODE

Model Type

Area Under the Curve (By Model)

22 / 34


Simulations III: Sensitivity to prior distribution bounds• Vary prior bounds π(Θ|G ) between (-2,2), (-3,3), (-5,5) and (-10,10)

1 2 3 4 5

12

34

5

Gel

man

−R

ubin

Sta

tistic

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

Convergence Assessment by Prior Bounds

(−2,2) (−3,3) (−5,5) (−10,10)

Prior Bounds and Replicate Number

Convergence cutoff (= 1.2)

23 / 34


Simulations III: Prior bounds (-2,2)

Pip3 Plcγ

Pip2

Pkc

Jnk

P38

Pka

Mek

Raf

Erk

Akt

24 / 34



Pip3 Plcγ

Pip2

Pkc

P

24 / 34



Pip3 Plcγ

Pip2

Pkc

Jnk

P38

Pka

Mek

Raf

Erk

Akt

24 / 34



Raf

Erk

Akt

24 / 34



Pip3 Plcγ

Pip2

Pkc

Jnk

P38

Pka

Mek

Raf

Erk

Akt

24 / 34


Pip3 Plcγ Pip2 Pkc P38 Pka Jnk Raf Mek Erk Akt

−2 0 2

Pkc

−2 0 2

Raf

−2 0 2

Mek

−2 0 2

Erk

−2 0 2

Pka

Pk

cR

af

Me

kE

rkP

ka

−2 0 2

Ak

tP

38

Jnk

Plcγ

Akt

Pip

3P

ip2

−2 0 2

P38

−2 0 2

Jnk

−2 0 2

Plc γ0

10

10

10

10

10

1

−2 0 2

01

01

01

01

Pip30

1

−2 0 2

Pip2

Parameter value

De

nsi

tyApproximate Posterior Distributions

25 / 34


Simulations: Discussion

• “Flexible” and “rigid” edges yield additional information about thedynamics of the network

• Rigidity and flexibility are closely linked to the network dynamics,robustness, and sensitivity

26 / 34





• Canberra, Euclidean, and Manhattan distances perform similarly interms of AUC; MVT distance does not perform as well

26 / 34






• Performance of ABC-Net deteriorates for alternative models when aVAR simulator is used

• Alternative simulators may be used in situations where other modelsare known to be more appropriate

26 / 34






• Performance of ABC-Net deteriorates for alternative models when aVAR simulator is used

• Alternative simulators may be used in situations where other modelsare known to be more appropriate

• Wider prior bounds lead to convergence problems and may requiremore iterations

26 / 34


Data Analysis

• Different inference methods are better suited to different tasks:• Empirical Bayes Dynamic Bayesian Network (EBDBN) method (Rau et

al. (2010)) is a hierarchical (empirical) Bayesian method formoderately sized networks (e.g., 50 - 100 genes):

π(Θ|y, ψ) ∝ f (y|Θ)π(Θ|ψ)π(ψ)

Details

• ABC-Net algorithm may be used for detailed analyses of small,well-characterized networks (e.g., 10 - 20 genes)

27 / 34


Data Analysis

• Different inference methods are better suited to different tasks:• Empirical Bayes Dynamic Bayesian Network (EBDBN) method (Rau et

al. (2010)) is a hierarchical (empirical) Bayesian method formoderately sized networks (e.g., 50 - 100 genes):

π(Θ|y, ψ) ∝ f (y|Θ)π(Θ|ψ)π(ψ)

Details

• ABC-Net algorithm may be used for detailed analyses of small,well-characterized networks (e.g., 10 - 20 genes)

• Using two algorithms on a common task can help elucidate thestrengths and weaknesses of each one:

• S.O.S. DNA repair system in Escherichia coli

27 / 34


Data Analysis: S.O.S. DNA Repair System in E. coli

• S.O.S. DNA repair system of Escherichia coli (Ronen et al., 2002)• 8 genes, with lexA as a master regulator that inhibits S.O.S. genesunder normal conditions but activates them when DNA damage issensed by recA (“single-input” module architecture)

• 50 time points, 1 replicate• Maximum fan-in for ABC-Net method constrained to 2

recA

lexA

uvrD umuD uvrA uvrY ruvA polB

28 / 34


Results: S.O.S. DNA Repair System

recA

lexA


EBDBN Results:

True Posi!ve

False Posi!ve

False Nega!ve

29 / 34



recA

lexA


EBDBN Results:

True Posi!ve

False Posi!ve

False Nega!ve

ABC-Net Results:

Approx. Posterior

Distribu!ons

29 / 34



recA

lexA


29 / 34


Discussion: S.O.S. DNA Repair System

• In S.O.S. system, lexA decreases very rapidly, so S.O.S. genes turn onat about the same time

• Time-delay models (e.g., autoregressive models) show stronger linkbetween recA and S.O.S. genes

30 / 34





• S.O.S. DNA repair is a simple, yet sophisticated network ⇒ networkis reacting to conditions within the cell

30 / 34





• S.O.S. DNA repair is a simple, yet sophisticated network ⇒ networkis reacting to conditions within the cell

• “Rigid” and “flexible” edges identified by the ABC-Net algorithm canhelp clarify results from other inference methods

30 / 34


Summary

• Inferring gene regulatory networks is intrinsically difficult: complexnetwork topology, small number of replicates and time points, noise inexpression measurements

31 / 34


Summary


• Approximate Bayesian Computation methods can reveal informationabout the dynamics of biological systems from time-series geneexpression data

31 / 34


Summary


• Approximate Bayesian Computation methods can reveal informationabout the dynamics of biological systems from time-series geneexpression data

• ABC-MCMC Network (ABC-Net) approach uses a simulation-basedBayesian method with few distributional assumptions to inferapproximate posterior distributions in small networks

31 / 34


Future Work

• Further examine components of ABC-Net method:• More sophisticated data simulators and techniques to identify optimal

simulators for real data• Alternative and efficient network structure proposal schemes• Objective criterion to characterize approximate posterior distributions

(e.g., introduce hierarchical prior on latent indicator variable G inABC-Net method, and use local Bayes factor to quantitatively examineevidence of network edges)

32 / 34


Future Work




• Examine alternative simulators and distance functions for time seriesdigital gene expression measures (e.g., RNA sequencing data)

32 / 34


Future Work




• Examine alternative simulators and distance functions for time seriesdigital gene expression measures (e.g., RNA sequencing data)

• Develop statistical methods to combine results from multipleinference methods (i.e., consensus networks or model averaging)

32 / 34


Acknowledgements

Ph.D. advisor:

Rebecca W. Doerge

Ph.D. committee members:Bruce CraigJayanta GhoshAlan QiFlorence Jaffrezic (INRA-GABI)

Jean-Louis FoulleyRWD research groupMy TruongDoug Crabill

33 / 34


References• Beaumont, M. et al. (2002) Approximate Bayesian computation in population genetics. Genetics 162, 2025-2035.

• Bortot, P. et al. (2007) Inference for stereological extremes. JASA March 2007, 84-92.

• Husmeier, D. (2003) Sensitivity and specificity of inferring genetic regulatory interactions from microarray experimentswith dynamic Bayesian networks. Bioinformatics 19, 2271-2282.

• Leuenberger, C. and Wegmann, D. (2009) Bayesian computation and model selection without likelihoods. Genetics 183,1-10.

• Marjoram, P. et al. (2003) Markov chain Monte Carlo without likelihoods. PNAS 100, 15324-15328.

• Pritchard, J. et al. (1999) Population growth of human Y chromosomes: a study of Y chromosome microsatellites.Molecular Biology and Evolution 16, 1791-1798.

• Ratmann, O. et al. (2007) Using likelihood-free inference to compare evolutionary dynamics of the protein networks ofH. pylorii and P. falciparum. PLoS Computational Biology 3, 2266-2278.

• Rau, A. et al. (2010) An empirical Bayesian method for estimating biological networks from temporal microarray data.SAGMB 9:1, Article 9.

• Ronen, M. et al. (2002) Assigning numbers to the arrows: parameterizing a gene regulation network by using accurateexpression kinetics.PNAS 99, 10555-10560.

• Sisson, S. et al. (2007) Sequential Monte Carlo without likelihoods. PNAS 104, 1760-1765.

• Toni, T. and Stumpf, M. (2010) Simulation-based model selection for dynamical systems in systems and populationbiology. Bioinformatics 26, 104-110.

Recent reviews on ABC:• Beaumont, M. et al. (2010) Approximate Bayesian computation in evolution and ecology. Annual Review of Ecology,

Evolution, and Sytematics 41, 379-405.

• Csillery, K. et al. (2010) Approximate Bayesian computation (ABC) in practice. Trends in Ecology and Evolution, 25,410-418.

34 / 34

Appendix

Likelihood-Free MCMC

TheoremUnder suitable regularity conditions, π(Θ|ρ(y, y?) ≤ ε) is the stationary

distribution of the chain.

Appendix

Likelihood-Free MCMC

TheoremUnder suitable regularity conditions, π(Θ|ρ(y, y?) ≤ ε) is the stationary

distribution of the chain.

Let r(Θ → Θ?) be the transition mechanism of the chain. We must checkwhether f (Θ|ρ(y?, y) ≤ ε)r(Θ → Θ?) = f (Θ?|ρ(y?, y) ≤ ε)r(Θ? → Θ).

Appendix

Proof.Without loss of generality, choose Θ? 6= Θ such that

π(Θ?)q(Θ|Θ?)

π(Θ)q(Θ?|Θ)≤ 1

Detailed balance equation:

Back

Appendix


π(Θ?)q(Θ|Θ?)

π(Θ)q(Θ?|Θ)≤ 1


f (Θ|ρ(y?, y) ≤ ε)r(Θ → Θ?) =

Back

Appendix


π(Θ?)q(Θ|Θ?)

π(Θ)q(Θ?|Θ)≤ 1


f (Θ|ρ(y?, y) ≤ ε)r(Θ → Θ?) =


=P [ρ(y?, y) ≤ ε|Θ]π(Θ)

P [ρ(y?, y) ≤ ε]

{

q(Θ?|Θ)P [ρ(y?, y) ≤ ε|Θ?]×π(Θ?)q(Θ|Θ?)

π(Θ)q(Θ?|Θ)

}

Back

Appendix


π(Θ?)q(Θ|Θ?)

π(Θ)q(Θ?|Θ)≤ 1


f (Θ|ρ(y?, y) ≤ ε)r(Θ → Θ?) =


=P [ρ(y?, y) ≤ ε|Θ]π(Θ)

P [ρ(y?, y) ≤ ε]

{

q(Θ?|Θ)P [ρ(y?, y) ≤ ε|Θ?]×π(Θ?)q(Θ|Θ?)

π(Θ)q(Θ?|Θ)

}

=P [ρ(y?, y) ≤ ε|Θ?]π(Θ?)

P [ρ(y?, y) ≤ ε]{q(Θ|Θ?)P [ρ(y?, y) ≤ ε|Θ]}

Back

Appendix


π(Θ?)q(Θ|Θ?)

π(Θ)q(Θ?|Θ)≤ 1


f (Θ|ρ(y?, y) ≤ ε)r(Θ → Θ?) =


=P [ρ(y?, y) ≤ ε|Θ]π(Θ)

P [ρ(y?, y) ≤ ε]

{

q(Θ?|Θ)P [ρ(y?, y) ≤ ε|Θ?]×π(Θ?)q(Θ|Θ?)

π(Θ)q(Θ?|Θ)

}

=P [ρ(y?, y) ≤ ε|Θ?]π(Θ?)

P [ρ(y?, y) ≤ ε]{q(Θ|Θ?)P [ρ(y?, y) ≤ ε|Θ]}

= f (Θ?|ρ(y?, y) ≤ ε)q(Θ|Θ?)P [ρ(y?, y) ≤ ε|Θ]α(Θ?,Θ)

Back

Appendix


π(Θ?)q(Θ|Θ?)

π(Θ)q(Θ?|Θ)≤ 1


f (Θ|ρ(y?, y) ≤ ε)r(Θ → Θ?) =


=P [ρ(y?, y) ≤ ε|Θ]π(Θ)

P [ρ(y?, y) ≤ ε]

{

q(Θ?|Θ)P [ρ(y?, y) ≤ ε|Θ?]×π(Θ?)q(Θ|Θ?)

π(Θ)q(Θ?|Θ)

}

=P [ρ(y?, y) ≤ ε|Θ?]π(Θ?)

P [ρ(y?, y) ≤ ε]{q(Θ|Θ?)P [ρ(y?, y) ≤ ε|Θ]}

= f (Θ?|ρ(y?, y) ≤ ε)q(Θ|Θ?)P [ρ(y?, y) ≤ ε|Θ]α(Θ?,Θ)

= f (Θ?|ρ(y?, y) ≤ ε)r(Θ? → Θ)

Back

Appendix

Simulation Procedure

• One-step-ahead predictors: y? = Θ?yt−1

• Suppose we have complete knowledge from the past (yt−1) and wewant to predict (simulate) expression at time t based on the currentnetwork Θ?: y?t = yt = E (yt |yt−1,Θ

?).• Data from previous time point used to determine forecasted

(simulated) values at current time point

Appendix






• Toni and Stumpf (2010): Numerically solved ordinary differentialequations and added noise to obtain simulated time-course data

Appendix







• Ratmann et al. (2007): Evolutionary history of protein interactionnetworks simulated using mixture evolution model (average summarystatistics over 50 generated networks)

Appendix







• Ratmann et al. (2007): Evolutionary history of protein interactionnetworks simulated using mixture evolution model (average summarystatistics over 50 generated networks)

• Marjoram et al. (2003): DNA sequences simulated using coalescenttrees

Back

Appendix

LFN Implementation Details

• Burn-in period• Cooling procedure: Temper acceptance with exponential cooling

scheme, starting at some initial temperature ε0 and cooling toεi+1 = λεi until the minimal temperature εmin = ε is reached. We useλ = 0.90 and set ε0 = ελ−10.

• Use each εi for 200 iterations, then cool to next value.• If εmin is reached and the acceptance rate for the chain ≤ 1%, the

burn-in period is reinitialized.

Appendix

LFN Implementation Details

• Burn-in period• Cooling procedure: Temper acceptance with exponential cooling

scheme, starting at some initial temperature ε0 and cooling toεi+1 = λεi until the minimal temperature εmin = ε is reached. We useλ = 0.90 and set ε0 = ελ−10.

• Use each εi for 200 iterations, then cool to next value.• If εmin is reached and the acceptance rate for the chain ≤ 1%, the

burn-in period is reinitialized.

• Chain length:• 10 chains for 1× 106 iterations each (1 × 107 iterations total)• Thinning interval of 50 (2 × 105 remaining iterations)• Inference made on samples corresponding to smallest 1% of ρ(y?, y)

(2000 iterations)

Back

Appendix

Approximate Bayesian Methods IEmpirical Bayes Dynamic Bayesian Network (EBDBN) Algorithm

• Parameters Θ (i.e., network edges) may be related to one another

Appendix


• Parameters Θ (i.e., network edges) may be related to one another ⇒Hierarchical Bayes model samples parameters from a commondistribution: π(Θ|y,ψ) ∝ f (y|Θ)π(Θ|ψ)π(ψ)

Appendix



• Idea: Use observed data y to estimate ψ

• Common parametric EB models (based on conjugate distributions)include Poisson-Gamma, Beta-binomial, multinomial-Dirichlet, andGaussian-Gaussian

Appendix



• Idea: Use observed data y to estimate ψ

• Common parametric EB models (based on conjugate distributions)include Poisson-Gamma, Beta-binomial, multinomial-Dirichlet, andGaussian-Gaussian

• Let yt and xt be observed gene expression and unobserved hiddenstates measured at time t:

xt = Axt−1 + Byt−1 + wt

yt = Cxt +Θyt−1 + vtwt ∼ N(0, I ), vt ∼ N(0,V−1))

Appendix

Empirical Bayes Dynamic Bayesian Network (EBDBN):

Rau et al., 2010Let y = {yt}t=1,...,T be observed gene expression and x = {xt}t=1,...,T beunobserved hidden states measured at the same time points.

Appendix



• Linear Gaussian state space model:



Appendix






Arows ∼ N(0, α−1) Brows ∼ N(0, β−1)Crows ∼ N(0, (V γ)−1) Θrows ∼ N(0, (V δ)−1)

Appendix






Arows ∼ N(0, α−1) Brows ∼ N(0, β−1)Crows ∼ N(0, (V γ)−1) Θrows ∼ N(0, (V δ)−1)

Choose hidden

state dimension,

Ini!alize x, α, β, γ, δ

EM algorithm to

update α, β, γ, δ

Kalman filter and

smoother to

update x

Convergence check

Final network

based on

π(Θ|y, x, α, β, γ, δ)

Back

Reverse Engineering Gene Networks Using...

Documents

Transcript of Reverse Engineering Gene Networks Using...