Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making
description
Transcript of Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making
![Page 1: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/1.jpg)
Using Clinical Trial Data to Construct Policies for Guiding
Clinical Decision Making
S. Murphy & J. PineauAmerican Control Conference Special Session
June, 2009
![Page 2: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/2.jpg)
2
Outline
– Sequential Clinical Decision Making– Clinical Trials– Challenges
• Incomplete, primitive, mechanistic models• Measures of Confidence
– Illustration
Long Term Goal: Improving Clinical Decision Making Using Data
![Page 3: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/3.jpg)
3
![Page 4: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/4.jpg)
4
Critical Decisions
• Which treatments should be offered first?• How long should we wait for these
treatments to work?• How long should we wait before offering a
transition to a maintenance stage?• Which treatments should be offered next?• All of these questions relate to the
formulation of a policy.
![Page 5: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/5.jpg)
5
Examples of Clinical Trials
• Sequenced RBT: Goal is to improve neonatal outcomes
• STAR*D: Goal is to achieve depression remission.
![Page 6: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/6.jpg)
6
Jones’ Study for Drug-Addicted Pregnant Women
rRBT
2 wks Response
rRBT
tRBTRandom
assignment:
rRBT
Nonresponse
tRBT
Randomassignment:
Randomassignment:
Randomassignment:
aRBT
2 wks Response
Randomassignment:
eRBT
tRBT
tRBT
rRBT
Nonresponse
![Page 7: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/7.jpg)
7
Stage 1 Intermediate Stage 2Preference Treatment Outcome Preference Treatment
Bup Continue Remission on Present
Switch R Ven Treatment
Ser MIRT Switch R
+ Bup No NTPAugment R Remission
+ Bus +LI
Augment R +THY
STAR*D
![Page 8: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/8.jpg)
8
Challenges
• Incomplete Mechanistic Models– non-causal “associations” in data occur due to
the unknown causes of the observations
• Small, Expensive, Data Sets with High Noise to Signal Ratio– Measures of confidence are essential
![Page 9: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/9.jpg)
9
Unknown Unknown Causes Causes
Observations Action Observations Action RewardStage 1 Stage 2
Stage 2
Conceptual Structure in the Behavioral Sciences (clinical trial data)
![Page 10: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/10.jpg)
10
Maturity/
Unknown DecisionCauses to join "Adult"
Society
+ -
Binge Drinking Counseling on - Binge Drinking Sanctions FunctionalityYes Health Yes/No + counseling
Consequences Time 2 Yes/No Time 3 Yes/No
Unknown, Unobserved Causes (Incomplete Mechanistic Models)
![Page 11: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/11.jpg)
11
• The problem: Even when treatments are randomized, non-causal associations occur in the data.
• Solutions:– Recognize that parts of the transition probabilities
(“system dynamics”) can not be informed by domain expertise as these parts reflect non-causal associations
– Or use methods for constructing policies that “average” over the non-causal associations between action and cost or reward.
Unknown, Unobserved Causes (Incomplete Mechanistic Models)
![Page 12: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/12.jpg)
12
Measures of Confidence • We would like measures of confidence for
the following:– To assess if there is sufficient evidence that a
particular observation (e.g. output of a biological test) should be part of the policy.
– To assess if there is sufficient evidence that a subset of the actions lead to lower cost than the remaining actions.
(reward=-cost)
![Page 13: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/13.jpg)
13
Measures of Confidence • Traditional methods for constructing
measures of confidence require differentiability (if frequentist properties are desired).
• Optimal policies are constructed via non-differentiable operations (e.g. minimization/maximization).
![Page 14: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/14.jpg)
14
Stage 1 Intermediate Stage 2Preference Treatment Outcome Preference Treatment
Bup Continue Remission on Present
Switch R Ven Treatment
Ser MIRT Switch R
+ Bup No NTPAugment R Remission
+ Bus +LI
Augment R +THY
STAR*D
![Page 15: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/15.jpg)
15
STAR*D • Stage 1 Observation:
• QIDS: low score is desirable• Preference for type of Stage 1 treatment: Switch or Augment
• Stage 1Treatment Action: If Stage 1 preference is Switch then randomize switch to either Ser, Bup or Ven; if Stage 1 preference is Augment then randomize to augment with Bup or Bus.
• Stage 2 Observation:• QIDS: low score is desirable• Preference for type of Stage 2 treatment: Switch or Augment
• Stage 2 Treatment Action: If Stage 2 preference is Switch then randomize switch to either Mirt or Ntp: if Stage 2 preference is Augment then randomize to augment with Li or Thy
• Patients exit to follow-up if remission is achieved (QIDS ≤ 5).
![Page 16: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/16.jpg)
16
Construct the policy to minimize cost (or maximize reward)
•Cost: minimum of time to remission and 30 weeks.
•Construct policy so as to minimize average cost
![Page 17: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/17.jpg)
17
Algorithm
• Fitted Q-iteration with linear function approximation. One estimates the “state-action cost” function at stages 1,2 via a linear model.
•Use voting across bootstrap samples (approximate double bootstrap) to assess confidence that a particular action is best.
(cost=-value=-benefit-to-go)
![Page 18: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/18.jpg)
18
![Page 19: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/19.jpg)
19
Conclusion for Stage 1(level 2)
• If QIDS is >13 then both Ven and Bup are best treatment actions
• If QIDS is <9 then Ser is best treatment action.
• If QIDS is around 10-13 then no real winner(s).
![Page 20: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/20.jpg)
20
Discussion
If modern control methods are to be used with clinical trial data then these methods
•must accommodate the existence of unknown, unobserved variables influencing observations at multiple stages,
•should provide measures of confidence and
•must be combined with modern missing data methods.
![Page 21: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/21.jpg)
21
This seminar can be found at:http://www.stat.lsa.umich.edu/~samurphy/seminars/ACC06.09.ppt
Email me with questions or if you would like a copy!
![Page 22: Using Clinical Trial Data to Construct Policies for Guiding Clinical Decision Making](https://reader036.fdocuments.in/reader036/viewer/2022062521/568167b5550346895ddd03bb/html5/thumbnails/22.jpg)
22
The Problem
• Many patients dropout of the study.
Stage 1 Stage 2
Remit 383 36
Move to next stage
456 260
Dropout 362 160
Sum 1201 456