Assessing causality in a post RCT world: Challenges to HIV research
-
Upload
hopkinscfar -
Category
Education
-
view
241 -
download
0
description
Transcript of Assessing causality in a post RCT world: Challenges to HIV research
JOHNS HOPKINS UNIVERSITY CFAR HIV METHODOLOGY SUMMIT
NOVEMBER 13, 2013
Background
• RCT is the comfort zone in NIH study sections and policy discussions. • Decision makers and leaders in HIV prevention were
trained on the RCT when there were intervention naïve populations and settings and when the resources for such research were considerable. • Standard ITT approaches for RCTs are frequently
unable to assess causality, especially for interventions that are subjected to lack of compliance and/or to switching/contamination.
Current Challenges
• Availability and implementation of many effective prevention strategies including treatment of HIV infected subjects • Low HIV incidence rates in many populations • Assessing causality in a multi-modal intervention
(e.g. combination prevention). • Increasing interest in the role of statistical modeling • in program evaluation • “model-based” approaches (statistical) vs. “design-based”
approaches (RCT) • the need of model-based approaches to be robust to
violations of their assumptions.
Why are we here today?
• We believe with some innovation and ingenuity we can demonstrate causality with approaches that are appropriate, timely and efficient. • JHU and the JHU CFAR are well positioned to lead in
crafting alternative designs to test real world problems. • To develop an inter-disciplinary team of
methodologists and prevention scientists and set an agenda for solving big impact, statistical challenges.
Question 1
• What are the potential explanations for why some RCTs successfully replicate results from an observational study (e.g., 052, circumcision) while others do not (e.g., HSV-2)? Confounding? Effect modification and differences in populations?
Example where causal methods explain discrepancy: Effect of postmenopausal hormone therapy on coronary heart disease; Nurses’ Health Study vs. Women’s Health Initiative.
Question 2
• How can causal inference methods disentangle different mechanisms in HIV prevention studies?
• Example: Does Hormonal contraception (e.g., Depo-Provera) impact susceptibility to HIV acquisition?
Observational Studies: Associations in observational studies possibly due to confounding by sexual risk behavior and/or other factors.
Potential problems with RCT: non-compliance; causal pathways other than biological mechanism of interest, e.g., receiving Depo-Provera may lead to changed risk behaviors and/or change chance of pregnancy (which may increase HIV risk).
Question 3
• How can we effectively assess causality with structural interventions that involve community level change? • For example, how do we assess the impact of
traditional elements of combination prevention, e.g., peer education, access to condoms, HIV/STI testing, male circumcision and earlier ART? • How do we assess the impact of community based
activities, e.g., service demand creation and community mobilization?
Question 4
• How can we evaluate very large, multi-country interventions such as PEPFAR? How can we overcome obstacles such as no randomization or use of historical controls?
Double-sampling designs to address dropout in PEPFAR evaluation
I dropout ∼ 39% in two years:I double-sampling design is essential to address dropout.
– select a subset of dropout patients– devote increased resources to find the subset
Patienttype
E i
(a) (c)
“Now”
0
iS
0 1
(d)
iT
.
.
1
0
0
“Now”
0
0 1
1
0
1
0
idR
0 1
1
1
1
“Now”
0
1 1
0
0
0
trueidR obs
i
L i
3
1 2
4
5
6
3
1 2
4
5
6
(b)
Patienttype
iC
Analyses of double-sampling give substantially different results
than simple adjustmentsI One-year mortality rate:
– around 2% (when using standard adjustments)– around 9 % (when using double-sampling methods)– b/c dropouts are much more likely to die than can be explained
by observed covariates
0100 200 3000
24
68
10
Days since enrollment
% M
orta
lity
method using double-sampling
method withoutusing double-sampling
An, Frangakis, Musick, Yiannoutsos, Biometrics, 2009;65:301306
Whom to double-sample ?
I resources need to be spread, in order to:– select patients with high likelihood to benefit if found– provide information to evaluate the program
I preliminary results suggest focus on patients with high risk profile
Risk for dropoutlow med high
Dropout timeshort 32 79 88med 37 102 78long 36 88 81
(a) original design
Risk for dropoutlow med high
short 127 86 26med 0 183 199long 0 0 0
(b) targetted design
An, Frangakis, Yiannoutsos, Statistics in Medicine, to appear
thanks to an R01 from NIAID