JOHNS HOPKINS UNIVERSITY CFAR HIV METHODOLOGY SUMMIT

NOVEMBER 13, 2013

Background

•  RCT is the comfort zone in NIH study sections and policy discussions. •  Decision makers and leaders in HIV prevention were

trained on the RCT when there were intervention naïve populations and settings and when the resources for such research were considerable. •  Standard ITT approaches for RCTs are frequently

unable to assess causality, especially for interventions that are subjected to lack of compliance and/or to switching/contamination.

Current Challenges

•  Availability and implementation of many effective prevention strategies including treatment of HIV infected subjects •  Low HIV incidence rates in many populations •  Assessing causality in a multi-modal intervention

(e.g. combination prevention). •  Increasing interest in the role of statistical modeling •  in program evaluation •  “model-based” approaches (statistical) vs. “design-based”

approaches (RCT) •  the need of model-based approaches to be robust to

violations of their assumptions.

Why are we here today?

•  We believe with some innovation and ingenuity we can demonstrate causality with approaches that are appropriate, timely and efficient. •  JHU and the JHU CFAR are well positioned to lead in

crafting alternative designs to test real world problems. •  To develop an inter-disciplinary team of

methodologists and prevention scientists and set an agenda for solving big impact, statistical challenges.

Question 1

•  What are the potential explanations for why some RCTs successfully replicate results from an observational study (e.g., 052, circumcision) while others do not (e.g., HSV-2)? Confounding? Effect modification and differences in populations?

Example where causal methods explain discrepancy: Effect of postmenopausal hormone therapy on coronary heart disease; Nurses’ Health Study vs. Women’s Health Initiative.

Question 2

•  How can causal inference methods disentangle different mechanisms in HIV prevention studies?

•  Example: Does Hormonal contraception (e.g., Depo-Provera) impact susceptibility to HIV acquisition?

Observational Studies: Associations in observational studies possibly due to confounding by sexual risk behavior and/or other factors.

Potential problems with RCT: non-compliance; causal pathways other than biological mechanism of interest, e.g., receiving Depo-Provera may lead to changed risk behaviors and/or change chance of pregnancy (which may increase HIV risk).

Question 3

•  How can we effectively assess causality with structural interventions that involve community level change? •  For example, how do we assess the impact of

traditional elements of combination prevention, e.g., peer education, access to condoms, HIV/STI testing, male circumcision and earlier ART? •  How do we assess the impact of community based

activities, e.g., service demand creation and community mobilization?

Question 4

•  How can we evaluate very large, multi-country interventions such as PEPFAR? How can we overcome obstacles such as no randomization or use of historical controls?

Double-sampling designs to address dropout in PEPFAR evaluation

I dropout ∼ 39% in two years:I double-sampling design is essential to address dropout.

– select a subset of dropout patients– devote increased resources to find the subset

Patienttype

E i

(a) (c)

“Now”

0

iS

0 1

(d)

iT

.

.

1

0

0

“Now”

0

0 1

1

0

1

0

idR

0 1

1

1

1

“Now”

0

1 1

0

0

0

trueidR obs

i

L i

3

1 2

4

5

6

3

1 2

4

5

6

(b)

Patienttype

iC

Analyses of double-sampling give substantially different results

than simple adjustmentsI One-year mortality rate:

– around 2% (when using standard adjustments)– around 9 % (when using double-sampling methods)– b/c dropouts are much more likely to die than can be explained

by observed covariates

0100 200 3000

24

68

10

Days since enrollment

% M

orta

lity

method using double-sampling

method withoutusing double-sampling

An, Frangakis, Musick, Yiannoutsos, Biometrics, 2009;65:301306

Whom to double-sample ?

I resources need to be spread, in order to:– select patients with high likelihood to benefit if found– provide information to evaluate the program

I preliminary results suggest focus on patients with high risk profile

Risk for dropoutlow med high

Dropout timeshort 32 79 88med 37 102 78long 36 88 81

(a) original design

Risk for dropoutlow med high

short 127 86 26med 0 183 199long 0 0 0

(b) targetted design

An, Frangakis, Yiannoutsos, Statistics in Medicine, to appear

thanks to an R01 from NIAID