Consequences of Heterogeneity

Public Health, Epidemiology, and Models

Introduction to Dynamics of Infectious Diseases

Foundations of Dynamic Modeling

(Hidden) Assumptions of Simple ODE’s

Breaking Assumptions!

Difference Equations and Discrete Time Intervals

Discrete Individuals and Finite Populations

Non-exponential Waiting Times

Heterogeneity tutorial

Epidemic Cards

Introduction to Thinking about Data

Introduction to Infectious Disease Data

Thinking about Data II: Confounding, bias, and

noise

Working with Data and Databases

Formulating Research Questions

Creating a Model World

Study Design and Analysis: Epi methods & RCTs

Introduction to Statistical Philosophy

Variability, Sampling Distributions, and

Simulation

HIV in Harare tutorial

Integration!

Introduction to Likelihood

Demographic Stochasticity and Ensemble Models

Fitting Dynamic Models I - III

Consequences of Heterogeneity

Public Health, Epidemiology, and Models

Introduction to Dynamics of Infectious Diseases

Foundations of Dynamic Modeling

(Hidden) Assumptions of Simple ODE’s

Breaking Assumptions!

Difference Equations and Discrete Time Intervals

Discrete Individuals and Finite Populations

Non-exponential Waiting Times

Heterogeneity tutorial

Epidemic Cards

Introduction to Thinking about Data

Introduction to Infectious Disease Data

Thinking about Data II: Confounding, bias, and

noise

Working with Data and Databases

Formulating Research Questions

Creating a Model World

Study Design and Analysis: Epi methods & RCTs

Introduction to Statistical Philosophy

Variability, Sampling Distributions, and

Simulation

HIV in Harare tutorial

Integration!

Introduction to Likelihood

Demographic Stochasticity and Ensemble Models

Fitting Dynamic Models I - III

Modeling for Policy

Model Assessment

Likelihood fitting and dynamic

models: Part 1

Steve Bellan, PhD, MPH Center for Computational Biology and Bioinformatics

The University of Texas at Austin

Clinic on the Meaningful Modeling of Epidemiological Data ICI3D Program and AIMS - South Arica June 3, 2016

Integra(on  Is  Hard  If true prevalence were 30%, then p(28 or more extreme) is

prob

abilit

y

2X

0 10 20 30 40 50 60 70 80 90 1000.00

0.05

0.10

number HIV+

2*pbinom(28, 100, 0.3, lower.tail = TRUE)p = 0.754

hypothetical prevalence: 30 %pr

obab

ility

0 10 20 30 40 50 60 70 80 90 1000.00

0.05

0.10

number HIV+

dbinom(28, 100, 0.3) = 0.0804

Do we want to sum or integrate the area under a curve?

Or just evaluate a function at one point?

P Values

Likelihoods

hypothetical prevalence

−log

(like

lihoo

d)

0.0 0.2 0.4 0.6 0.8 1.00

100

200

300

400

we usually minimize the −log(likelihood)

Let’s zoom in…

Building  Confidence  Intervals  Likelihood  Ra(o  Test  

Maximum Likelihood Estimate ! = !

!!

!! = !

! =28100 = 0.28!!!!

hypothetical prevalence

−log

(like

lihoo

d)

0.0 0.2 0.4 0.6 0.8 1.02

4

6

Building  Confidence  Intervals  Likelihood  Ra(o  Test  

! = !!!

!! = !

! =28100 = 0.28!!!!

Maximum Likelihood Estimate

Building  Confidence  Intervals  Likelihood  Ra(o  Test  

! = !!!

!! = !

! =28100 = 0.28!!

2 log L(alternative*hypothesis)L(null%hypothesis) ~!!"!!! !

!!

2log!(!!"#$%&!#'($)− 2 log !!"## ~!!"!!! !!−2!!"# + 2!!"## > !!"!!,!!.!"! = 3.84!!!!"## − !!"# > 1.92!

hypothetical prevalence

−log

(like

lihoo

d)

0.0 0.2 0.4 0.6 0.8 1.02

4

6

1.92

Building  Confidence  Intervals  Likelihood  Ra(o  Test  

hypothetical prevalence

−log

(like

lihoo

d)

0.0 0.2 0.4 0.6 0.8 1.02

4

6

95% CI includes HIV prevalences of 19.9% to 37.2%

0.199 0.372

0.0 0.2 0.4 0.6 0.8 1.0

hypothetical prevalence (null hypothesis)

p−va

lue

0.0

0.5

1.0

0.195 0.37895% CI includes 19.5% to 37.8%

0.0 0.2 0.4 0.6 0.8 1.0

potential prevalences (our models)

−log

(like

lihoo

d)

2

4

695% CI includes 19.9% to 37.2%

0.199 0.372

Comparing  Confidence  Intervals  

Process Model

S I

Process Model

S I

Parameters

Process Model

S I

Parameters

Where do parameters come

from?

A priori parameterization

¨  Use external data to determine values for the parameters in your model

CASCADE study

A priori parameterization

¨  Use external data to determine values for the parameters in your model ¤ eg, time from seroconversion to death

¨  Plug estimates into models to determine expected dynamics

A priori parameterization

¤  Long-term time series are not available ¤  Designing a new study ¤  Data are limited and your goal is to estimate a particular quantity that has not been directly measured ¤  Comparing model structures, especially when multiple long-term time series are not available for validation

Fitting models to data

¨  A priori parameterization ¤ Use external data to determine values for the

parameters in your model ¤ Rarely possible for all model parameters

Fitting models to data

¨  A priori parameterization ¤ Use external data to determine values for the

parameters in your model ¤ Rarely possible for all model parameters

¨  Trajectory matching

¨  Feature matching

Process Model

S I

Parameters

some (possibly) fixed and others to be fitted

Time series

Process Model

S I

Parameters

some (possibly) fixed and others to be fitted

Process Model

S I

Parameters

some (possibly) fixed and others to be fitted

Time series expectation

or distribution of latent variables

Process Model

S I

Parameters

some (possibly) fixed and others to be fitted

Time series expectation

or distribution of latent variables

Deterministic models

Process Model

S I

Parameters

some (possibly) fixed and others to be fitted

Time series expectation

or distribution of latent variables

Deterministic models

Stochastic models

Data

Data n = 3092

Data

Observation model

n = 3092

€

f (x | p) =nx"

# $ %

& ' px (1− p)n−xPDF:

Likelihood of prevalence (given data)

Data

Observation model

n = 3092

€

L(p | x) = nx" # $ % & ' px (1− p)n−xLIKELIHOOD:

€

f (x | p) =nx"

# $ %

& ' px (1− p)n−xPDF:

Data

Data

Observation model

€

f (xt | pt ) =ntxt" # $

% & ' pt

xt (1− pt )nt −xt

t∏

PDF:

Likelihood of prevalence trajectory (given data)

Data

Observation model

€

L(pt | xt ) =ntxt" # $

% & ' pt

xt (1− pt )nt −xt

t∏

LIKELIHOOD: €

f (xt | pt ) =ntxt" # $

% & ' pt

xt (1− pt )nt −xt

t∏

PDF:

Likelihood of parameters (given data)

Data

Observation model

Time series expectation

or distribution of latent variables

Process Model

S I

Parameters

some (possibly) fixed and others to be fitted

Likelihood of the model (given data)

Data

Observation model

Time series expectation

or distribution of latent variables

Process Model

S I

Parameters

some (possibly) fixed and others to be fitted

Process Model

S I

Why do we fit models to data in infectious

disease epidemiology?

Koopman’s Inference Robustness Assessment

Framework

Koopman et al. 2014. Transmission modeling to enhance surveillance system function.

Koopman’s Inference Robustness Assessment

Framework 1. Select the policy inference to be pursued

Koopman et al. 2014. Transmission modeling to enhance surveillance system function.

Koopman’s Inference Robustness Assessment

Framework

2. Construct & Analyze Simple

Models

1. Select the policy inference to be pursued

Koopman et al. 2014. Transmission modeling to enhance surveillance system function.

Koopman’s Inference Robustness Assessment

Framework

2. Construct & Analyze Simple

Models

3. Constrain Parameter Space with Data

1. Select the policy inference to be pursued

Koopman et al. 2014. Transmission modeling to enhance surveillance system function.

Koopman’s Inference Robustness Assessment

Framework

2. Construct & Analyze Simple

Models

3. Constrain Parameter Space with Data

4. Make inference across parameter space

1. Select the policy inference to be pursued

Koopman et al. 2014. Transmission modeling to enhance surveillance system function.

Koopman’s Inference Robustness Assessment

Framework

2. Construct & Analyze Simple

Models

3. Constrain Parameter Space with Data

4. Make inference across parameter space

Inference Same Across Parameter Space

Inference Differs Across Parameter Space

1. Select the policy inference to be pursued

Koopman et al. 2014. Transmission modeling to enhance surveillance system function.

Koopman’s Inference Robustness Assessment

Framework

2. Construct & Analyze Simple

Models

3. Constrain Parameter Space with Data

4. Make inference across parameter space

Inference Same Across Parameter Space

5. Relax Assumptions with More Realistic

Model

Inference Differs Across

Parameter Space

1. Select the policy inference to be pursued

Koopman et al. 2014. Transmission modeling to enhance surveillance system function.

Koopman’s Inference Robustness Assessment

Framework

2. Construct & Analyze Simple

Models

3. Constrain Parameter Space with Data

4. Make inference across parameter space

Inference Same Across Parameter Space

5. Relax Assumptions with More Realistic

Model

Inference Differs Across

Parameter Space

6. Find other types and sources of available data OR study cost benefit of new data collection to justify getting new data.

1. Select the policy inference to be pursued

Koopman et al. 2014. Transmission modeling to enhance surveillance system function.

Koopman’s Inference Robustness Assessment

Framework

4. Make inference across parameter space

Inference Same Across Parameter Space

5. Relax Assumptions with More Realistic

Model

Koopman et al. 2014. Transmission modeling to enhance surveillance system function.

Inference Robustness Assessment Loop

Koopman’s Inference Robustness Assessment

Framework

3. Constrain Parameter Space with Data

4. Make inference across parameter space

Inference Differs Across

Parameter Space

6. Find other types and sources of available data OR study cost benefit of new data collection to justify getting new data.

Koopman et al. 2014. Transmission modeling to enhance surveillance system function.

Inference Identifiability Assessment

Loop

•  Assess inference robustness to realistic relaxation of simplifying model assumptions

Koopman’s Inference Robustness Assessment

Framework

Koopman et al. 2014. Slide courtesy of JS Koopman (with modification).

•  Assess inference robustness to realistic relaxation of simplifying model assumptions

•  Pursue complexity that matters by keeping models as simple as possible but not so simple that they lead to an incorrect inference

Koopman’s Inference Robustness Assessment

Framework

Koopman et al. 2014. Slide courtesy of JS Koopman (with modification).

•  Assess inference robustness to realistic relaxation of simplifying model assumptions

•  Pursue complexity that matters by keeping models as simple as possible but not so simple that they lead to an incorrect inference

Koopman’s Inference Robustness Assessment

Framework

Koopman et al. 2014. Slide courtesy of JS Koopman (with modification).

Validate the inference!

•  Assess inference robustness to realistic relaxation of simplifying model assumptions

•  Pursue complexity that matters by keeping models as simple as possible but not so simple that they lead to an incorrect inference

Koopman’s Inference Robustness Assessment

Framework

Koopman et al. 2014. Slide courtesy of JS Koopman (with modification).

Validate the inference! not the model or method you’re working with

This presentation is made available through a Creative Commons Attribution-Noncommercial license. Details of the license and permitted uses are available at

http://creativecommons.org/licenses/by-nc/3.0/

© 2013 – 2016 Juliet Pulliam & Steve Bellan, Clinic on the Meaningful Modeling of Epidemiological Data, ICI3D Program

Likelihood Fitting and dynamic models: Part I (Updated: 3 June 2016) Attribution: JRC Pulliam and SE Bellan, Clinic on the Meaningful Modeling of Epidemiological Data

Source URL: http://mmed2015.ici3d.org/lectures/fittingDynamicModels_PartI.pdf For further information please contact faculty@ici3d.org.