Public Health, Introduction to Thinking Epidemiology, and ...bellanlab.publichealth.uga.edu ›...
47
Consequences of Heterogeneity Public Health, Epidemiology, and Models Introduction to Dynamics of Infectious Diseases Foundations of Dynamic Modeling (Hidden) Assumptions of Simple ODE’s Breaking Assumptions! Difference Equations and Discrete Time Intervals Discrete Individuals and Finite Populations Non-exponential Waiting Times Heterogeneity tutorial Epidemic Cards Introduction to Thinking about Data Introduction to Infectious Disease Data Thinking about Data II: Confounding, bias, and noise Working with Data and Databases Formulating Research Questions Creating a Model World Study Design and Analysis: Epi methods & RCTs Introduction to Statistical Philosophy Variability, Sampling Distributions, and Simulation HIV in Harare tutorial Integration! Introduction to Likelihood Demographic Stochasticity and Ensemble Models Fitting Dynamic Models I - III
Transcript of Public Health, Introduction to Thinking Epidemiology, and ...bellanlab.publichealth.uga.edu ›...
Consequences of Heterogeneity
Public Health, Epidemiology, and Models
Introduction to Dynamics of Infectious Diseases
Foundations of Dynamic Modeling
(Hidden) Assumptions of Simple ODE’s
Breaking Assumptions!
Difference Equations and Discrete Time Intervals
Discrete Individuals and Finite Populations
Non-exponential Waiting Times
Heterogeneity tutorial
Epidemic Cards
Introduction to Thinking about Data
Introduction to Infectious Disease Data
Thinking about Data II: Confounding, bias, and
noise
Working with Data and Databases
Formulating Research Questions
Creating a Model World
Study Design and Analysis: Epi methods & RCTs
Introduction to Statistical Philosophy
Variability, Sampling Distributions, and
Simulation
HIV in Harare tutorial
Integration!
Introduction to Likelihood
Demographic Stochasticity and Ensemble Models
Fitting Dynamic Models I - III
Consequences of Heterogeneity
Public Health, Epidemiology, and Models
Introduction to Dynamics of Infectious Diseases
Foundations of Dynamic Modeling
(Hidden) Assumptions of Simple ODE’s
Breaking Assumptions!
Difference Equations and Discrete Time Intervals
Discrete Individuals and Finite Populations
Non-exponential Waiting Times
Heterogeneity tutorial
Epidemic Cards
Introduction to Thinking about Data
Introduction to Infectious Disease Data
Thinking about Data II: Confounding, bias, and
noise
Working with Data and Databases
Formulating Research Questions
Creating a Model World
Study Design and Analysis: Epi methods & RCTs
Introduction to Statistical Philosophy
Variability, Sampling Distributions, and
Simulation
HIV in Harare tutorial
Integration!
Introduction to Likelihood
Demographic Stochasticity and Ensemble Models
Fitting Dynamic Models I - III
Modeling for Policy
Model Assessment
Likelihood fitting and dynamic
models: Part 1
Steve Bellan, PhD, MPH Center for Computational Biology and Bioinformatics
The University of Texas at Austin
Clinic on the Meaningful Modeling of Epidemiological Data ICI3D Program and AIMS - South Arica June 3, 2016
Integra(on Is Hard If true prevalence were 30%, then p(28 or more extreme) is
prob
abilit
y
2X
0 10 20 30 40 50 60 70 80 90 1000.00
0.05
0.10
number HIV+
2*pbinom(28, 100, 0.3, lower.tail = TRUE)p = 0.754
hypothetical prevalence: 30 %pr
obab
ility
0 10 20 30 40 50 60 70 80 90 1000.00
0.05
0.10
number HIV+
dbinom(28, 100, 0.3) = 0.0804
Do we want to sum or integrate the area under a curve?
Or just evaluate a function at one point?
P Values
Likelihoods
hypothetical prevalence
−log
(like
lihoo
d)
0.0 0.2 0.4 0.6 0.8 1.00
100
200
300
400
we usually minimize the −log(likelihood)
Let’s zoom in…
Building Confidence Intervals Likelihood Ra(o Test
Maximum Likelihood Estimate ! = !
!!
!! = !
! =28100 = 0.28!!!!
hypothetical prevalence
−log
(like
lihoo
d)
0.0 0.2 0.4 0.6 0.8 1.02
4
6
Building Confidence Intervals Likelihood Ra(o Test
! = !!!
!! = !
! =28100 = 0.28!!!!
Maximum Likelihood Estimate
Building Confidence Intervals Likelihood Ra(o Test
! = !!!
!! = !
! =28100 = 0.28!!
2 log L(alternative*hypothesis)L(null%hypothesis) ~!!"!!! !
!!
2log!(!!"#$%&!#'($)− 2 log !!"## ~!!"!!! !!−2!!"# + 2!!"## > !!"!!,!!.!"! = 3.84!!!!"## − !!"# > 1.92!
hypothetical prevalence
−log
(like
lihoo
d)
0.0 0.2 0.4 0.6 0.8 1.02
4
6
1.92
Building Confidence Intervals Likelihood Ra(o Test
hypothetical prevalence
−log
(like
lihoo
d)
0.0 0.2 0.4 0.6 0.8 1.02
4
6
95% CI includes HIV prevalences of 19.9% to 37.2%
0.199 0.372
0.0 0.2 0.4 0.6 0.8 1.0
hypothetical prevalence (null hypothesis)
p−va
lue
0.0
0.5
1.0
0.195 0.37895% CI includes 19.5% to 37.8%
0.0 0.2 0.4 0.6 0.8 1.0
potential prevalences (our models)
−log
(like
lihoo
d)
2
4
695% CI includes 19.9% to 37.2%
0.199 0.372
Comparing Confidence Intervals
Process Model
S I
Process Model
S I
Parameters
Process Model
S I
Parameters
Where do parameters come
from?
A priori parameterization
¨ Use external data to determine values for the parameters in your model
CASCADE study
A priori parameterization
¨ Use external data to determine values for the parameters in your model ¤ eg, time from seroconversion to death
¨ Plug estimates into models to determine expected dynamics
A priori parameterization
¤ Long-term time series are not available ¤ Designing a new study ¤ Data are limited and your goal is to estimate a particular quantity that has not been directly measured ¤ Comparing model structures, especially when multiple long-term time series are not available for validation
Fitting models to data
¨ A priori parameterization ¤ Use external data to determine values for the
parameters in your model ¤ Rarely possible for all model parameters
Fitting models to data
¨ A priori parameterization ¤ Use external data to determine values for the
parameters in your model ¤ Rarely possible for all model parameters
¨ Trajectory matching
¨ Feature matching
Process Model
S I
Parameters
some (possibly) fixed and others to be fitted
Time series
Process Model
S I
Parameters
some (possibly) fixed and others to be fitted
Process Model
S I
Parameters
some (possibly) fixed and others to be fitted
Time series expectation
or distribution of latent variables
Process Model
S I
Parameters
some (possibly) fixed and others to be fitted
Time series expectation
or distribution of latent variables
Deterministic models
Process Model
S I
Parameters
some (possibly) fixed and others to be fitted
Time series expectation
or distribution of latent variables
Deterministic models
Stochastic models
Data
Data n = 3092
Data
Observation model
n = 3092
€
f (x | p) =nx"
# $ %
& ' px (1− p)n−xPDF:
Likelihood of prevalence (given data)
Data
Observation model
n = 3092
€
L(p | x) = nx" # $ % & ' px (1− p)n−xLIKELIHOOD:
€
f (x | p) =nx"
# $ %
& ' px (1− p)n−xPDF:
Data
Data
Observation model
€
f (xt | pt ) =ntxt" # $
% & ' pt
xt (1− pt )nt −xt
t∏
PDF:
Likelihood of prevalence trajectory (given data)
Data
Observation model
€
L(pt | xt ) =ntxt" # $
% & ' pt
xt (1− pt )nt −xt
t∏
LIKELIHOOD: €
f (xt | pt ) =ntxt" # $
% & ' pt
xt (1− pt )nt −xt
t∏
PDF:
Likelihood of parameters (given data)
Data
Observation model
Time series expectation
or distribution of latent variables
Process Model
S I
Parameters
some (possibly) fixed and others to be fitted
Likelihood of the model (given data)
Data
Observation model
Time series expectation
or distribution of latent variables
Process Model
S I
Parameters
some (possibly) fixed and others to be fitted
Process Model
S I
Why do we fit models to data in infectious
disease epidemiology?
Koopman’s Inference Robustness Assessment
Framework
Koopman et al. 2014. Transmission modeling to enhance surveillance system function.
Koopman’s Inference Robustness Assessment
Framework 1. Select the policy inference to be pursued
Koopman et al. 2014. Transmission modeling to enhance surveillance system function.
Koopman’s Inference Robustness Assessment
Framework
2. Construct & Analyze Simple
Models
1. Select the policy inference to be pursued
Koopman et al. 2014. Transmission modeling to enhance surveillance system function.
Koopman’s Inference Robustness Assessment
Framework
2. Construct & Analyze Simple
Models
3. Constrain Parameter Space with Data
1. Select the policy inference to be pursued
Koopman et al. 2014. Transmission modeling to enhance surveillance system function.
Koopman’s Inference Robustness Assessment
Framework
2. Construct & Analyze Simple
Models
3. Constrain Parameter Space with Data
4. Make inference across parameter space
1. Select the policy inference to be pursued
Koopman et al. 2014. Transmission modeling to enhance surveillance system function.
Koopman’s Inference Robustness Assessment
Framework
2. Construct & Analyze Simple
Models
3. Constrain Parameter Space with Data
4. Make inference across parameter space
Inference Same Across Parameter Space
Inference Differs Across Parameter Space
1. Select the policy inference to be pursued
Koopman et al. 2014. Transmission modeling to enhance surveillance system function.
Koopman’s Inference Robustness Assessment
Framework
2. Construct & Analyze Simple
Models
3. Constrain Parameter Space with Data
4. Make inference across parameter space
Inference Same Across Parameter Space
5. Relax Assumptions with More Realistic
Model
Inference Differs Across
Parameter Space
1. Select the policy inference to be pursued
Koopman et al. 2014. Transmission modeling to enhance surveillance system function.
Koopman’s Inference Robustness Assessment
Framework
2. Construct & Analyze Simple
Models
3. Constrain Parameter Space with Data
4. Make inference across parameter space
Inference Same Across Parameter Space
5. Relax Assumptions with More Realistic
Model
Inference Differs Across
Parameter Space
6. Find other types and sources of available data OR study cost benefit of new data collection to justify getting new data.
1. Select the policy inference to be pursued
Koopman et al. 2014. Transmission modeling to enhance surveillance system function.
Koopman’s Inference Robustness Assessment
Framework
4. Make inference across parameter space
Inference Same Across Parameter Space
5. Relax Assumptions with More Realistic
Model
Koopman et al. 2014. Transmission modeling to enhance surveillance system function.
Inference Robustness Assessment Loop
Koopman’s Inference Robustness Assessment
Framework
3. Constrain Parameter Space with Data
4. Make inference across parameter space
Inference Differs Across
Parameter Space
6. Find other types and sources of available data OR study cost benefit of new data collection to justify getting new data.
Koopman et al. 2014. Transmission modeling to enhance surveillance system function.
Inference Identifiability Assessment
Loop
• Assess inference robustness to realistic relaxation of simplifying model assumptions
Koopman’s Inference Robustness Assessment
Framework
Koopman et al. 2014. Slide courtesy of JS Koopman (with modification).
• Assess inference robustness to realistic relaxation of simplifying model assumptions
• Pursue complexity that matters by keeping models as simple as possible but not so simple that they lead to an incorrect inference
Koopman’s Inference Robustness Assessment
Framework
Koopman et al. 2014. Slide courtesy of JS Koopman (with modification).
• Assess inference robustness to realistic relaxation of simplifying model assumptions
• Pursue complexity that matters by keeping models as simple as possible but not so simple that they lead to an incorrect inference
Koopman’s Inference Robustness Assessment
Framework
Koopman et al. 2014. Slide courtesy of JS Koopman (with modification).
Validate the inference!
• Assess inference robustness to realistic relaxation of simplifying model assumptions
• Pursue complexity that matters by keeping models as simple as possible but not so simple that they lead to an incorrect inference
Koopman’s Inference Robustness Assessment
Framework
Koopman et al. 2014. Slide courtesy of JS Koopman (with modification).
Validate the inference! not the model or method you’re working with
This presentation is made available through a Creative Commons Attribution-Noncommercial license. Details of the license and permitted uses are available at
http://creativecommons.org/licenses/by-nc/3.0/
© 2013 – 2016 Juliet Pulliam & Steve Bellan, Clinic on the Meaningful Modeling of Epidemiological Data, ICI3D Program
Likelihood Fitting and dynamic models: Part I (Updated: 3 June 2016) Attribution: JRC Pulliam and SE Bellan, Clinic on the Meaningful Modeling of Epidemiological Data
Source URL: http://mmed2015.ici3d.org/lectures/fittingDynamicModels_PartI.pdf For further information please contact [email protected].
