UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Model Reduction for Parameter Estimation Eric...

Manchester PESB Workshop 28/3/07

UCI ICS IGB SISL

Model Reduction for Parameter Estimation

Eric MjolsnessScientific Inference Systems Laboratory (SISL)

University of California, Irvine

www.ics.uci/edu/~emj

and Caltech Biological Network Modeling Center (BNMC)

in collaboration with Rebecca Castaño, Dasha Chudova, Michael Duff, Victoria Gor, Henrik Jönsson, Tobias Mann, George Marnellos, Elliot Meyerowitz, John Reinitz, Bruce Shapiro, David Sharp, Padhraic Smyth, Yuanfeng Wang, Barbara Wold, Guy

Yosiphon, Li Zhang

Parameter Estimation In Systems Biology (PESB)

Pascal Workshop, Manchester, UK

March 28, 2007


UCI ICS IGB SISL

Topics

• A long-running thread in parameter estimation• Biological applications:

– transcriptional regulation

– development

• Perspectives:– a near-universal bio. modeling language and semantics

and its implications for …

– parameter estimation and model reduction


UCI ICS IGB SISL

Transcriptional Gene Regulation Networks

• Gene Regulation Network [MSR’91] model

τ i &vi =g Tijvj + hij∑( )−λivi

Drosophila gap gene expression patterns. Reinitz, Mjolsness, Sharp, Journal Experimental Zoology 271(47-56) 1995. Fitting method demonstrated in Mittenthal and Baskin, The Principles of Organization of Organisms, Addison Wesley 1992.

[Mjolsness et al. J. Theor. Biol. 152: 429-453, 1991]

E.g. Drosophila A-P axis:


UCI ICS IGB SISL

GRN Parameter Optimization• Simulated Annealing [1990, ’92]

– Lam/Delosme SA for real-valued params– Gap genes [JEZ 271(47-56) 1995]:

• 33 real-valued parameters

• Genetic Algorithm– Distributed over islands with migration, for diversity

• SA, GA compared in G. Marnellos thesis [1997] – GA won on evolution (life history) problems– SA won on development problems

• Other apps to GRN’s and signaling [Gor, Zhang]• Then many others. Recently:

– Kozlov BGRS 2006: differential evolution– Tomlin 2006: Adjoint method ~BP/cont. time


UCI ICS IGB SISL

GRN ANN Equations ’91

Model statement and its derivation from stat mech: [Mjolsness Sharp and Reinitz, J. Theor. Biol. 152: 429-453, 1991]

Key properties: (1) additivity, (2) saturation above and below, (3) monotonicity.


UCI ICS IGB SISL

[MSR91] equations are no longer just “phenomenological”.

[J. Theor. Biol. 152: 429-453, 1991]

τ i &vi =g Tijvj + hij∑( )−λivi

Model Reduction Example:

Gene Regulation NetworkDerived from Stat Mech

[J.Bioinformatics & Comp. Biology, in press 2007]


UCI ICS IGB SISL

Dynamical Model Reduction via Clustering

– Core/Halo Models: • “From Coexpression to Coregulation …”

[NIPS 1999 p.928-34]

• Identifiability by Gibbs sampling[Duff et al., ICSB 2005]

– Functional Mixture Models [Chudova et al. NIPS 2003]

Cluster

MT

v


UCI ICS IGB SISL

Core/Leaf Model Inference

• 3-node oscillator + leaves • Modeled by SE

• topologies

• Identifiability:• x25 time points: identifiable• x10 points: not identifiable• x10 points x2 genotypes:

~identifiable (ranked #3)

• [Duff et al. ICSB2005]

€

6

3

⎛

⎝ ⎜

⎞

⎠ ⎟× 33 = 540

X jt+1 =Xj

t +α jg wij Xit +hj

i∑⎛⎝⎜

⎞⎠⎟−λ j Xj

t + ε jt+1


UCI ICS IGB SISL

SDE Advantages

• Intermediate cost for stochastic simulation

• Relationship to stochastic optimization

• Derivation from Fokker-Planck equation• Eg. for GRN, HCA: [JBCB in press 2007]:


UCI ICS IGB SISL

Hierarchical Cooperative Activation: Alternative diagram notations

• Bio-like:

• Machine learning:


UCI ICS IGB SISL

Hierarchical Cooperative Activation Model (HCA)

Transcription factor inputs

Binding site occupation -dimerization, competitive binding

Binding site activation

Promoter element activation

Transcription output

network

τ i

dvi

dt= [transcribing]i − λ ivi

[transcribing]i = g ui( )=Jui

1 + Jui

ui =1 + Jα Pα

1 + ˆ J α Pα

⎛

⎝ ⎜ ⎞

⎠ ⎟

α∈i∏

In: Computational Methods in Molecular Biology, eds. J. M. Bower and H. Bolouri, MIT Press 2001


UCI ICS IGB SISL

How to model transcriptional regulation?

[Robert P. Zinzen, Kate Senger, Mike Levine, and Dmitri Papatsenko. Current Biology 16, 1–8, July 11, 2006]

E.g. Drosophila D-V axis:


UCI ICS IGB SISL

Hard vs. Soft Logic

HierarchicalCooperativeActivation (HCA)

Zinzen et al. modification

Experiment: Yuanfeng Wang, UCI Physics


UCI ICS IGB SISL

HCA- Z and ANN-like Equations• Assume many binding sites per module

• Assume extreme (usually low) occupancy per site

where

A model reduction:


UCI ICS IGB SISLGRSN: Gene Regulation

+ Signal Transduction Network

transcriptional regulation targets

receptors

ligands

cell

nucleus

d

dtv

a(t) =

1τ a

g(ua + ha) −λava⎡⎣ ⎤⎦,

where

ua(t) = Tabvb(t) +b∑ I T̂abvb

I (t)b∑

I∈Nbrs∑ + I %Tac

1 %Tcb2vc(t)vb

I

c∑

b∑

I∈Nbrs∑ (t)

T

[Marnellos, Mjolsness, Shapiro]

+ …

Drosophila neurogenesis[Marnellos, Mjolsness PSB ’98]Xenopus ciliated cells [PSB ’00]

Arabidopsis SAM [Gor, Mjolsness,Meyerowitz, NASA Evolvable Hardware ’99]


UCI ICS IGB SISL

Arabidopsis Shoot Apical Meristem (SAM)


UCI ICS IGB SISL

CLV3CLV1

?

Fletcher et al., Science v. 283, 1999

Brand et. al., Science 289, 617-619, (2000)

WUS


UCI ICS IGB SISL

SAM growth imageryH2B cell

nuclei

V. Reddy,Caltech

QuickTime™ and aTIFF decompressor

are needed to see this picture.


UCI ICS IGB SISL

CLV3/WUS networks

V. Agrawal, B. Shapiro, Caltech

Z

wusclv1

clv3Xdiffusive

YL1diffusive


UCI ICS IGB SISL

CLV/WUS model behavior

Activation domains in Cellerator model: WUS (yellow), CLV3I1 (green), CLV3 (blue and purple), CLV1 (red and purple).

B. Shapiro, JPL/Caltech

QuickTime™ and aTIFF (Uncompressed) decompressor



UCI ICS IGB SISL

CLV/WUS Parameter Optimization by SA

Courtesy H. Jönsson 2007;cf. ICSB 2006

QuickTime™ and aH.264 decompressor


14 parameters


UCI ICS IGB SISL

Biological scale hierarchiesBiology, networks, & models:

Objects(L)

Processes(L)

Objects(L+1)

Processes(L+1)

Objects(L-1)

Processes(L-1)

Processes(L-1)

Objects(L+2)

…

…

Noun and verb hierarchies:

mutant

wild type

Perspective …


UCI ICS IGB SISL

Dynamical Grammar Aims

• Biology: Model complex systems– developmental biology (fly embryo, plant shoot/root)

– molecular complexes

– multiple-scale, heterogeneous, variable-structure systems

• Mathematics: Capture, unify, extend techniques– Generalized reactions cover all processes

– Operator algebra, perturbation theory, …

[Annals of Math. and A. I., 47(3-4), January 2007]


UCI ICS IGB SISL

Elementary Reactions

• A B+ C with rate kf

• B+ C A with rate kr

• Effective conservation lawsE.g. NA+ NB, NA+ NC


UCI ICS IGB SISL

Elementary Processes• A(x) B(y) + C(z) with f (x, y, z)

• B(y) + C(z) A(x) with r (y, z, x)

• Examples– Chemical reaction networks w/o params– .

– XXX from paper

• Effective conservation laws– E.g. ∫ NA(x) dx + ∫ NB(y) dy ,

∫ NA(x) dx + ∫ NC(z) dz


UCI ICS IGB SISL

Elementary process models

• Composition is by independent parallelism • Create elementary processes from yet more elementary “Basis operators”

– Term creation/annihilation operators: for each parm value,

– Obeying Heisenberg algebra

– Yet classical, not quantum, probabilities


UCI ICS IGB SISL

SPG Modeling Language: Semantics Semantic map

from Grammar to Stochastic Process

• Commutative diagrams for composition operations

• Translation of a Rule

’

H, dp/dt H’, dp’/dt


UCI ICS IGB SISL

• Time Ordered Product Expansion (TOPE) formula:

– H0 = the easy part (if only recursively)

– Feynman diagrams result (QFT: Perturbation theory, Wick’s theorem)

• Gillespie stochastic simulation algorithm– H0 = diag( 1· H´) ; H1 = H´

– Mixed (heterogeneous) ODE/SSA algorithm (novel)

• Implemented in “Plenum” (Yosiphon)[Annals of Math. and A. I., 47(3-4), January 2007]


UCI ICS IGB SISL

Model Reduction for Dynamical Systems

• Diagram:

• Objectives:Thus, parameter estimation can aid model reduction

• Uses of diagram:

[UCI ICS TR #05-09]


UCI ICS IGB SISL

Composition vs. Specializationin a Lattice of Models

• Orthogonal kinds of model reduction/expansion (PartOf~InA, IsA)

• Commutative diagram for model lattice:

• Specialization: eg. discretized (DBN) vs. continuous (ODE) vs. quantized (stochastic) vbls, time, space - heterogeneous dynamics

• Initialize param search in specialized model

• high-level vision app [NIPS 1990]:

• Thus, model reduction can aid parameter estimation

INA

INA ISA

ISA


UCI ICS IGB SISL

A Parameter Estimation Future• Parameter estimation model reduction

– Multiscale, heterogeneous, variable-structure, … models all incorporated in a lattice

– Common (operator algebra) semantics

• Perpetual data assimilation– Continual influx of data

– Perpetual fitting to an expanding lattice of models• Specialize to the limit of identifiability

• Model analyses to explain the “hits”


UCI ICS IGB SISL

Conclusions• Model reductions for transcriptional regulation:

– GRN’91, HCA

• Model reduction for large-scale data: – Core/Halo, Functional Mixture, … models

• Common framework: generalized reactions– Dynamical grammars operator algebra

• Parameter estimation model reduction– Mutually enhancing interaction

UCI ICS IGB SISL


For further information:–www.ics.uci.edu/~emj–www.computableplant.org

Funding: US National Science Foundation FIBR program,NIH BISTI program

Invitation…


UCI ICS IGB SISL

UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Model Reduction for Parameter Estimation Eric...

Documents

Transcript of UCI ICS IGB SISL Manchester PESB Workshop 28/3/07 Model Reduction for Parameter Estimation Eric...