Treatment Effect Heterogeneity & Dynamic Treatment Regime Development
description
Transcript of Treatment Effect Heterogeneity & Dynamic Treatment Regime Development
Treatment Effect Heterogeneity &
Dynamic Treatment Regime Development
S.A. Murphy
2
Dynamic treatment regimes (DTRs) are individually tailored treatments, with treatment type and dosage changing according to individual outcomes.
***utilize treatment effect heterogeneity to individualize treatment***
3
Example of a DTR
•Adaptive Drug Court Program for drug abusing offenders.
•Goal is to minimize recidivism and drug use.
•Marlowe et al. (2008, 2009, 2011)
4
non-responsiveAs-needed court hearings As-needed court hearings
low risk + standard counseling + ICM
non-complianthigh risk
non-responsiveBi-weekly court hearings Bi-weekly court hearings + standard counseling + ICM
non-compliant Court-determined disposition
Adaptive Drug Court Program
Treatment Effect Heterogeneity
• Focus on Theory: Used to deepen understanding of underlying causal, mechanistic structure
• Focus on Practice: Used to improve decision making in practice– For Whom, When, and in Which Context, might a
specific treatment be most useful?
– This is our focus today
Treatment Effect Heterogeneity &
DTR Development
• Take Advantage of Treatment Effect Heterogeneity in Design of Intervention Trial– Embedded tailoring variables
– Part of “treatment action”
• Take Advantage of Treatment Effect Heterogeneity in Design of the DTR.– Data analyses
Pelham ADHD Study
Begin low doseMed
8 weeks
Assess-Adequate response?
Continue, reassess monthly; randomize if deteriorate
Intensify Current Treatment
Randomassignment:
Augment with other Treatment
No
Begin low-intensity BMOD
8 weeks
Assess-Adequate response?
Continue, reassess monthly;randomize if deteriorate
Augment with other treatment
Randomassignment:
Intensify Current Treatment
Yes
No
Randomassignment:
Txt Effect Heterogeneity Embedded Tailoring Variable
• Embedded Tailoring Variables: (a) Teacher reported Impairment Scale, (b) Teacher reported individualized list of target behaviors
• Non-response is assessed at 8 weeks and every 4 weeks thereafter.
Txt Effect Heterogeneity Embedded DTRs
4 Embedded DTRs1) Start with BMOD; only if nonresponse
criterion reached, augment with MED
2) Start with BMOD; only if nonresponse criterion reached, intensify BMOD
3) Start with MED; only if nonresponse criterion reached, augment with BMOD
4) Start with MED; only if nonresponse criterion reached, intensify MED
Oslin Alcoholism Trial
Late Trigger forNonresponse
8 wks Response
TDM + NTX
CBI +MMRandom
assignment:
CBI +NTX+MM
Nonresponse
Early Trigger for Nonresponse
Randomassignment:
Randomassignment:
Randomassignment:
NTX
8 wks Response
Randomassignment:
CBI +NTX+MM
CBI+MM
TDM + NTX
NTX
Nonresponse
Txt Effect Heterogeneity Embedded Tailoring Variable &
Embedded DTR
• Embedded Tailoring Variable: heavy drinking days (HDD)
• First randomization is between treatment actions: move to stage 2 if 2 HDDs versus move to stage 2 if 5 HDDs
• 8 Embedded DTRs
12
A Data Analysis Method for Utilizing Treatment Effect Heterogeneity to Construct a “More Deeply Tailored” DTR: Q-Learning
Subject data from sequential, multiple assignment, randomized trials. At each stage subjects are randomized among alternative options.
Aj is a randomized action with known randomization probability. Binary actions with P[Aj=1]=P[Aj=-1]=.5
13
Dynamic Treatment Regime (DTR)
•The DTR is given by a sequence of decision rules, one per stage of treatment (here 2 stages)
DTR=
•Goal: Construct
for which the expected outcome is
maximal.
Ed1;d2 [Y ]
14
• Q-Learning (Watkins, 1989; Ernst et al., 2005; Murphy, 2005) (a popular method from computer science)—generalizes regression to multiple stages
• Q-Learning uses dynamic programming arguments combined with linear regression estimation of conditional means.
Q-Learning
15
There is a regression for each stage.
Simple Version of Q-Learning –
• Stage 2 regression: Regress Y on to obtain
• Stage 1 regression: Regress on to obtain
16
for subjects entering stage 2:
• is the predicted end of stage 2 response when the stage 2 treatment is equal to the “best” treatment.
• is the dependent variable in the stage 1 regression for patients moving to stage 2
17
A Simple Version of Q-Learning –
• Stage 2 regression, (using Y as dependent variable) yields
• Arg-max over a2 yields
18
A Simple Version of Q-Learning –
• Stage 1 regression, (using as dependent variable) yields
• Arg-max over a1 yields
Pelham ADHD Study
Begin low doseMed
8 weeks
Assess-Adequate response?
Continue, reassess monthly; randomize if deteriorate
Intensify Current Treatment
Randomassignment:
Augment with other Treatment
No
Begin low-intensity BMOD
8 weeks
Assess-Adequate response?
Continue, reassess monthly;randomize if deteriorate
Augment with other treatment
Randomassignment:
Intensify Current Treatment
Yes
No
Randomassignment:
2020
• (X1, A1, R1, X2, A2, Y)
– Y = end of year school performance
– R1=1 if early responder; =0 if early non-responder
– X2 includes the month of non-response, M2, and a measure of adherence in stage 1 (S2 )
– S2 =1 if adherent in stage 1; =0, if non-adherent
– X1 includes baseline school performance, Y0 , whether medicated in prior year (S1), ODD (O1)
– S1 =1 if medicated in prior year; =0, otherwise.
ADHD Example
2121
• Stage 2 regression for Y:
• Stage 1 outcome:
ADHD Example
(1;Y0;S1;O1;A1;M2;S2)®2+A2(¯21+A1¯22+S2¯23)
R1Y +(1¡ R1)Y
Y = (1;Y0;S1;O1;A1;M2;S2)®2+maxa2 ( ^21+A1
^22+S2 ^23)a2
2222
IF medication was not used in the prior year THEN begin with BMOD;
ELSE select either BMOD or MED.
IF the child is nonresponsive and was non-adherent, THEN augment present treatment;
ELSE IF the child is nonresponsive and was adherent, THEN select intensification of current treatment.
Dynamic Treatment Regime Proposal
23
•High dimensional data; investigators want to collect real time data
• Feature construction & Feature selection
•Many stages or infinite horizon
This seminar can be found at:
http://www.stat.lsa.umich.edu/~samurphy/
seminars/JSM_Txt_Heterogeneity2012.ppt
Future Challenges