Statistical Inference for dynamic Models with the ...Numerical methods Given ODE parameter βand...
49
Statistical Inference for dynamic Models with the Generalized Profiling Method Jiguo Cao, Simon Fraser University Department Seminars Department of Statistics University of British Columbia October 30, 2007
Transcript of Statistical Inference for dynamic Models with the ...Numerical methods Given ODE parameter βand...
Statistical Inference for dynamic Modelswith the Generalized Profiling Method
Jiguo Cao, Simon Fraser University
Department SeminarsDepartment of Statistics
University of British ColumbiaOctober 30, 2007
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Outline
1 Introduction for dynamic Models
2 Estimating dynamic models
3 The Generalized Profiling Method
4 Summary and Ongoing work
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Outline
1 Introduction for dynamic Models
2 Estimating dynamic models
3 The Generalized Profiling Method
4 Summary and Ongoing work
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
What are Ordinary Differential Equations (ODEs)?
A general form for ODEs:
ddt
x(t) = g(x|β)
Variable: x(t);ODE parameter: β;ODEs relate the rate of change (dx
dt ) of a process to itscurrent state (x)
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Why Using Ordinary Differential Equations(ODEs)?
ODEs model the rate of change (dxdt ) of a process
Many models are given directly in ODE forms inEngineering, Physics, Biology, · · ·Newton’s Second Law: F = m ∗ d2
dt2 x(t)Nonlinear ODEs can provide simple models forcomplex behavior
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
A Predator-prey system
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
A Predator-prey dynamic Model
Fussmann et al. (2000). Science 290.
dNdt
= −δN − FC(N)C + δN∗
N : concentration of nitrogenδ : the fraction of the volume of the system replaceddailyN∗ : concentration of nitrogen in the inflow.FC(N) = bCN/(KC + N)
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
A Predator-prey dynamic Model
dCdt
= −δC + FC(N)C − FB(C)B/ε
C : concentration of Cδ : fraction of the volume of the system replaced dailyε : assimilation efficiency of BFC(N) = bCN/(KC + N)
FB(C) = bBC/(KB + C)
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
A Predator-prey dynamic Model
dRdt
= −(δ + m + λ)R + FB(C)R
R : concentration of B with reproducing abilityδ : fraction of the volume of the system replaced dailyλ : decay of fecundity of Rm : mortality of RFB(C) = bBC/(KB + C)
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
A Predator-prey dynamic Model
dBdt
= FB(C)R − (δ + m)B
B : concentration of total Bδ : fraction of the volume of the system replaced dailym : mortality of BFB(C) = bBC/(KB + C)
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
A Predator-prey dynamic Model
Fussmann et al. (2000). Science 290.
dNdt
= δ(N∗ − N) − FC(N)C
dCdt
= FC(N)C − FB(C)B/ε− δC
dRdt
= FB(C)R − (δ + m + α)R
dBdt
= FB(C)R − (δ + m)B .
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Numerical methods
Given ODE parameter β and initial values x(t0)Finding ODE solutions and properties
Euler MethodMidpoint MethodRunge-Kutta Method
Warning: ODE solutions are sensitive to parameters andinitial values
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Experimental Data
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Estimating ODEs
ODEs: ddt x(t) = g(x|β)
Observations: y(t1), · · · ,y(tn)
Question 1: If g(·) is known, Estimating β
Question 2: If g(·) is unknown, Estimating g(·)
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Outline
1 Introduction for dynamic Models
2 Estimating dynamic models
3 The Generalized Profiling Method
4 Summary and Ongoing work
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Current Methods
ddt
x(t) = g(x|β)
Simulated Annealing method
Bayesian MCMC method (Huang, Liu, and Wu 2005)
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Current Methods
ddt
x(t) = g(x|β)
Multiple shooting method (Bock 1981, 1983).
Integrating ODEs with numerical quadrature(Himmelblau et al. 1967): x(t) =
∫g(x|β)dt
Estimating the derivative ddt x(t) (Ramsay and
Silverman 2005)
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Generalized Profiling Method
Allowing for missing variables
Computation is fast and stable
Interval estimation
User-friendly program available
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Nonparametric estimation
Estimate a smooth function to approximate ODE solutionsby a linear combination of basis functions:
f(t) =K∑
k=1
ckφk (t) = φ(t)′c
φ(t) = (φ1(t), · · · , φK (t)) is a vector of basis functions,for example, B-spline (Fixed and Known).c = (c1, · · · , cK ) is the basis coefficient.
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Cubic B-spline Basis
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Three kinds of parameters
The basis coefficient c
The ODE parameter β
Smoothing parameter: λ
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Inner level of optimization criterion
Smooth function: f(t) = φ(t)′c
Fitting to dataObservations: y(ti)Fitting to data: C1 =
∑ni=1[y(ti) − f(ti)]2
Fidelity to ODE ddt x(t) = g(x|β)
Difference between two sides of ODE:Lf(t) = d
dt f(t) − g(f(t)|β)
Fidelity to ODE: C2 =∫
[Lf(t)]2dt
Criterion: J(c|β, λ,y) = C1 + λC2
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Median and outer level of optimization criterion
Median level: estimating ODE parameter β
Criterion: H(β|λ,y) =∑n
i=1[y(ti) − φ(ti)′c(β)]2
Outer level: estimating smoothing parameter λCriterion: generalized cross-validationGCV (c(β(λ), λ), β(λ), λ|y)
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Three nested levels of optimization
Inner level: J(c|β,λ,y)⇒c is a function of β and λ: c(β,λ).
Median level: H(c(β,λ),β|λ,y)⇒ β is a function of λ: β(λ).
Outer level: F (c(β(λ),λ), β(λ),λ|y) ⇒λ
c: basis coefficients; β: ODE parameter;λ: smoothing parameters;
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Byproduct: estimating initial values
Fitted curve: x(t) = φ(t)′c
Estimating initial values: x(t0) = φ(t0)′c
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Estimating Missing Variables
ddt
y1(t) = f1(y1(t), y2(t),β) ;ddt
y2(t) = f2(y1(t), y2(t),β) .
f1(·) and f2(·) are known.Observations for y1(t): y1 = (y1(t1), · · · , y1(tn))No observations for y2(t)yi(t) =
∑Kik=1 cikφik (t) = c′iφi(t)
J(c1,c2|β,λ,y1) = −logLikelihood(c1|y1)
+λ{2∑
i=1
wi
∫[c′i φi(t) − fi(c′1φ1(t),c
′2φ2(t),β)]2dt} ,
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Estimating Differential Equations
ddt
y(t) = g(y(t))
g(·) is a unknown function of y(t).Observations for y(t): y = (y(t1), · · · , y(tn))y(t) =
∑Kk=1 ckφk (t) = c′φ(t)
g(y) =∑K
k=1 βkψk (y) = β′ψ(y)
Parametersbasis coefficients: cODE Parameter: β
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
A Predator-prey dynamic Model
dNdt
= δ(N∗ − N) − FC(N)C
dCdt
= FC(N)C − FB(C)B/ε− δC
dRdt
= FB(C)R − (δ + m + α)R
dBdt
= FB(C)R − (δ + m)B .
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Estimating Results
Estimates ε α m bC MSEFussmann 0.25 0.40 0.055 3.3 1.96Profiling 0.11 0.01 0.152 3.9 0.34SEs 0.020 0.14 0.073 0.47
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Fitting ODEs to data
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
A Predator-prey dynamic Model
dNdt
= δ(N∗ − N) − FC(N)C
dCdt
= FC(N)C − FB(C)B/ε− δC
dRdt
= FB(C)R − (δ + m + α)R
dBdt
= FB(C)R − (δ + m)B .
Two functional responses
FC(N) = bCNKC+N
FB(C) = bBNKB+N
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Estimating Functional Responses
Nonparametric functional responses
FC(N) =∑
(c1i ψ
1i (N))
FB(C) =∑
(c2i ψ
2i (C))
ψ1i (N) and ψ2
i (C) are basis functionsc1
i and c2i are basis coefficients
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Estimating Functional Responses
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Outline
1 Introduction for dynamic Models
2 Estimating dynamic models
3 The Generalized Profiling Method
4 Summary and Ongoing work
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
The Generalized Profiling Method
Frequentist version of Bayesian Multilevel ModelingApproach
AdvantageFast and stable computationUnconditional variance estimationUser-friendly program
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Designed to estimate three kinds of parameters
Nuisance (local) parameters: cParameters not of primary interest.The dimension is large and increases with the samplesize.
Structural (global) parameters: βParameters of primary interest.The dimension is small and fixed with the size of data.
Complexity (smoothing) parameters: λControl the complexity (effective degrees of freedom)of models.
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Three nested levels of optimizationInner level: J(c|β,λ,y)⇒c is a function of β and λ: c(β,λ).Median level: H(c(β,λ),β|λ,y)⇒ β is a function of λ: β(λ).Outer level: F (c(β,λ), β(λ),λ|y) ⇒λ
Unique aspects of Generalized Profiling MethodDifferent optimization criterion in each levelFunctional relationships among parameters
c: nuisance parameters; β: structural parameters;λ: complexity parameters;
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Low computation load
Newton-Raphson algorithm is applied.Gradients and Hessian matrices worked outanalytically
Inner level: J(c|β,λ,y)
Median level: H(β|λ,y)
Gradient: dHdβ
= ∂H∂β
+ ∂H∂c
∂c∂β
c: nuisance parameters; β: structural parameters;λ: complexity parameters;
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Low computation load
Newton-Raphson algorithm is applied.Gradients and Hessian matrices worked outanalytically
Inner level: J(c|β,λ,y)
Median level: H(β|λ,y)
Gradient: dHdβ
= ∂H∂β
+ ∂H∂c
∂c∂β
Implicit Function Theorem
Inner level: J(c|β,λ,y) ⇒ c(β,λ)
∂c∂β
= −[
∂2J∂c2
]−1[ ∂2J∂c∂β
]c: nuisance parameters; β: structural parameters;λ: complexity parameters;
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Unconditionally variance estimate
Unconditional variance (modified delta method):
Var(c) =[d cdy
]Var(y)
[d cdy
]′dcdy = ∂c
∂y + ∂c∂
ˆβd ˆβdy + ∂c
∂ˆλ
d ˆλdy
c: nuisance parameters; β: structural parameters;λ: complexity parameters;
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Unconditionally variance estimate
Unconditional variance (modified delta method):
Var(c) =[d cdy
]Var(y)
[d cdy
]′dcdy = ∂c
∂y + ∂c∂
ˆβd ˆβdy + ∂c
∂ˆλ
d ˆλdy
Conditional variance (other methods): ignoringuncertainty coming from β and λ
Var(c|β, λ) =[∂c∂y
]Var(y)
[∂c∂y
]′c: nuisance parameters; β: structural parameters;λ: complexity parameters;
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
User-Friendly program
Users doProvide the appropriate optimization criteria for the threelevels of optimization: J,H,F
Users don’t doAll Mathematical Details: 41 complicated formulas
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Outline
1 Introduction for dynamic Models
2 Estimating dynamic models
3 The Generalized Profiling Method
4 Summary and Ongoing work
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Estimating ODEs from noisy data
Allowing for missing variables
Computation is fast and stable
Interval estimation
User-friendly program available
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Generalized Profiling Method
Estimate nuisance, structural, and complexityparameters
Three nested levels of optimization
Low computation load.
Unconditional variance estimate.
User-friendly program
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Ongoing Work
Estimating ODEsRandom and fixed effects of ODE parametersSmoothing parameter selectionStochastic differential equationsAsymptotic properties
Generalized Profiling methodMore applications
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
Collaborators
James O. Ramsay, McGill University
Gregor Fussmann, McGill University
Giles Hooker, Cornell University
David Campbell, Simon Fraser University
Liangliang Wang, University of British Columbia
Introduction for dynamic Models Estimating ODEs Generalized Profiling Method Summary
If you have interesting projects,
Email: jiguo [email protected]: 778-782-7600Office: K10556, Department of Statistics and ActuarialScience, SFUCourse: Stat 890, Functional Data Analysis, SFU,Spring, 2008.
