Lecture 1: Decision analysis UCSF DCEA 2004 Objectives To understand what decision analysis is and...
35
Lecture 1: Decision Lecture 1: Decision analysis analysis UCSF DCEA 2004 UCSF DCEA 2004 Objectives Objectives To understand what decision To understand what decision analysis is and when it might be analysis is and when it might be used used To understand the sequence of steps To understand the sequence of steps in doing a decision analysis in doing a decision analysis
-
Upload
angela-bailey -
Category
Documents
-
view
213 -
download
0
Transcript of Lecture 1: Decision analysis UCSF DCEA 2004 Objectives To understand what decision analysis is and...
Lecture 1: Decision analysisLecture 1: Decision analysisUCSF DCEA 2004UCSF DCEA 2004
ObjectivesObjectives
To understand what decision analysis is and when it To understand what decision analysis is and when it might be usedmight be used
To understand the sequence of steps in doing a To understand the sequence of steps in doing a decision analysisdecision analysis
Decision analysisDecision analysis = = explicit, quantitative method to explicit, quantitative method to
make (or think about) decisions in the face of uncertainty.make (or think about) decisions in the face of uncertainty.
• Portray Portray optionsoptions and their and their consequencesconsequences
• Quantify Quantify uncertaintyuncertainty using using probabilitiesprobabilities
• Quantify theQuantify the desirability of outcomes desirability of outcomes using using utilitiesutilities
• Calculate the Calculate the expected utilityexpected utility of each option of each option (alternative course of action) (alternative course of action)
• Choose the option thatChoose the option that on average on average leads to leads to most desirable outcomesmost desirable outcomes
Indications For Decision AnalysisIndications For Decision Analysis
1. Developing policies, treatment guidelines, etc.1. Developing policies, treatment guidelines, etc.2. At the bedside (i.e. helping patients make decisions)2. At the bedside (i.e. helping patients make decisions)3. Focus discussion and identify important research needs3. Focus discussion and identify important research needs4. In your life outside of medicine4. In your life outside of medicine5. As teaching tool to discourage dogmatism and to 5. As teaching tool to discourage dogmatism and to
demonstrate rigorously the need to involve patients in demonstrate rigorously the need to involve patients in decisionsdecisions
Uncertainty about outcomes of alternative courses of action.Uncertainty about outcomes of alternative courses of action.
Overview of DA StepsOverview of DA Steps
1. 1. Formulate an explicit questionFormulate an explicit question
2. 2. Make a decision tree. Make a decision tree. (diamonds = decision nodes, circles = chance nodes) (diamonds = decision nodes, circles = chance nodes)
a) a) Alternative actions = branches of the decision node.Alternative actions = branches of the decision node.
b) b) Possible outcomes of each = branches of chance nodes.Possible outcomes of each = branches of chance nodes.
3. 3. Estimate probabilities of outcomes at each chance node.Estimate probabilities of outcomes at each chance node.
4. 4. Estimate utilities = numerical preference for outcomes.Estimate utilities = numerical preference for outcomes.
5. 5. Compute the expected utility of each possible actionCompute the expected utility of each possible action
6. 6. Perform sensitivity analysisPerform sensitivity analysis
Motivating CaseMotivating Case::
Ms. Brooks is a 50 year old woman with an incidental Ms. Brooks is a 50 year old woman with an incidental cerebral aneurysm. She presented with new vertigo 3 cerebral aneurysm. She presented with new vertigo 3 weeks ago and her primary MD ordered a head MRI. Her weeks ago and her primary MD ordered a head MRI. Her vertigo has subsequently resolved and has been attributed vertigo has subsequently resolved and has been attributed to labyrinthitis.to labyrinthitis.
Her MRI suggested a left posterior communicating artery Her MRI suggested a left posterior communicating artery aneurysm, and a catheter angiogram confirmed a 6 mm aneurysm, and a catheter angiogram confirmed a 6 mm berry aneurysm.berry aneurysm.
Case Presentation (cont’d)Case Presentation (cont’d)
Past medical history is remarkable only for 35 pack-years of Past medical history is remarkable only for 35 pack-years of cigarette smoking. cigarette smoking. Exam is normal. Exam is normal. Ms. Brooks: “I don’t want to die before my time.” Ms. Brooks: “I don’t want to die before my time.”
Question is: Do we recommend surgical clipping of the Question is: Do we recommend surgical clipping of the aneurysm or no treatment? aneurysm or no treatment?
Alternative ways of dealing with uncertaintyAlternative ways of dealing with uncertainty
• Dogmatism.Dogmatism. All aneurysms should be surgically clipped.All aneurysms should be surgically clipped.
• Policy.Policy. At UCSF we clip all aneurysms.At UCSF we clip all aneurysms.
• Experience.Experience. I’ve referred a number of aneurysm patients for surgery and they have I’ve referred a number of aneurysm patients for surgery and they have done well.done well.
• Whim.Whim. Let’s clip this one.Let’s clip this one.
• Nihilism.Nihilism. It really doesn't matter.It really doesn't matter.
• Defer to experts.Defer to experts. Vascular neurosurgeons say clipVascular neurosurgeons say clip..
• Defer to patients.Defer to patients. Would you rather have surgery or live with your aneurysm Would you rather have surgery or live with your aneurysm untreated?untreated?
OR
Decision Analysis
1. FORMULATE AN EXPLICIT QUESTION1. FORMULATE AN EXPLICIT QUESTION
- Formulate explicit, answerable question. - Formulate explicit, answerable question. - May require modification as analysis progresses. - May require modification as analysis progresses. - The simpler the question, without losing important - The simpler the question, without losing important
detail, the easier and better the decision analysis.detail, the easier and better the decision analysis.
In the aneurysm example, our interest is in determining what’s In the aneurysm example, our interest is in determining what’s best for Ms. Brooks so we'll take her perspective. We will begin best for Ms. Brooks so we'll take her perspective. We will begin with the following question:with the following question:
Which treatment strategy, surgical clipping or no Which treatment strategy, surgical clipping or no treatment, is better for Ms. Brooks considering her primary treatment, is better for Ms. Brooks considering her primary concern about living a normal life span?concern about living a normal life span?
2. MAKE A DECISION TREE2. MAKE A DECISION TREE
Creating a decision tree = Creating a decision tree = structuring the problemstructuring the problem
Provide a reasonably complete depiction of the problem.Provide a reasonably complete depiction of the problem.
Best is one Best is one decision nodedecision node (on the left, at the beginning of (on the left, at the beginning of the tree). the tree).
Branches of each Branches of each chance nodechance node -- -- exhaustiveexhaustive and and mutually mutually exclusiveexclusive. .
Proceed incrementally. Begin simple. Proceed incrementally. Begin simple.
Decision Trees: Simple to …Decision Trees: Simple to …
Ms. Brooks
No treatment
Surgery
Normal survival
Early Death
Surgery:yes or no?
Normal survival
Early Death
… … to Less Simple…to Less Simple…
Ms. Brooks
No treatment
Surgery
Surgery:yes or no?
AneurysmRupture?
No Normal survival
Yes Early Death
SurgicalDeath?
No
Yes
AneurysmRupture?
No Normal survival
Yes Early Death
Early Death
……to Complexto Complex
Ms. Brooks
No treatment
Surgery
Surgery:yes or no?
AneurysmRupture?
No Normal survival
Yes
Early Death
SurgicalDeath?
No
Yes Early Death
Death?
No
Yes
Normal survival
AneurysmRupture?
No Normal survival
Yes
Early Death
Death?
No
Yes
Normal survival
No aneurysm rupture
No surgeryDie
Aneurysm rupture Survive
No aneurysm rupture
Survive surgeryDie
Aneurysm ruptureClipping Survive
Surgical death
Ms. Brooks
Normal survival
Immediate death
Normal survival
Normal survival
Normal survival
Early death
Early death
Figure 1Figure 1
3. ESTIMATE PROBABILITIES3. ESTIMATE PROBABILITIES
From the most reliable results applicable to the patient or From the most reliable results applicable to the patient or scenario of interest.scenario of interest.
Standard hierarchies of data quality Standard hierarchies of data quality
Definitive trials/meta-analysis of trials > smaller trials Definitive trials/meta-analysis of trials > smaller trials > large cohort studies > small cohort studies > opinion> large cohort studies > small cohort studies > opinion
3. Fill in the probabilities: 3. Fill in the probabilities: No treatment nodeNo treatment node
•Prob rupture =exp life span x rupture/yr-Expected life span:
From US mortality figures: 35 years
-Probability of untreated aneurysm rupture. Cohort study
-0.05%/yr for <10 mm
-Lifetime prob rupture = 0.05%/y x 35 y = 1.75%
•Case fatality of ruptureMeta-analysis: 45%
3. Fill in the probabilities3. Fill in the probabilities
Ms. Brooks
No treatment
Surgery
Surgery:yes or no?
AneurysmRupture?
Nop=0.9825 Normal survival
Yesp=0.0175
Early Death
SurgicalDeath?
Nop=0.977
Yesp=0.023 Early Death=0
Death?
Nop=.55
Yesp=.45
Normal survival
AneurysmRupture?
Nop=1.0 Normal survival=1
Yesp=0
Early Death=0
Death?
Nop=.55
Yesp=.45
Normal survival=1
3. Fill in the probabilities: 3. Fill in the probabilities: Surgery nodeSurgery node
•Probability of treated aneurysm rupture.
-No data: probably very small ~ 0 (Opinion)
•Surgical mortality. Options:-Meta-analysis of case series: 2.6%-Clinical databases: 2.3%-The numbers at UCSF: 2.3%
3. Fill in the probabilities3. Fill in the probabilities
Ms. Brooks
No treatment
Surgery
Surgery:yes or no?
AneurysmRupture?
Nop=0.9825 Normal survival=1
Yesp=0.0175
Early Death=0
SurgicalDeath?
Nop=0.977
Yesp=0.023 Early Death
Death?
Nop=.55
Yesp=.45
Normal survival=1
AneurysmRupture?
Nop=1.0 Normal survival
Yesp=0
Early Death
Death?
Nop=.55
Yesp=.45
Normal survival
4. Estimate utilities4. Estimate utilities
• Valuation of an outcome (more restrictive use in the next Valuation of an outcome (more restrictive use in the next lecture).lecture).
• Best = 1Best = 1
• Worst = 0Worst = 0
• In this case, she wants to avoid early death:In this case, she wants to avoid early death:
– Normal survival = 1Normal survival = 1
– Early death = 0Early death = 0
4. Fill in the utilities4. Fill in the utilities
Ms. Brooks
No treatment
Surgery
Surgery:yes or no?
AneurysmRupture?
Nop=0.9825 Normal survival=1
Yesp=0.0175
Early Death=0
SurgicalDeath?
Nop=0.977
Yesp=0.023 Early Death=0
Death?
Nop=.55
Yesp=.45
Normal survival=1
AneurysmRupture?
Nop=1.0 Normal survival=1
Yesp=0
Early Death=0
Death?
Nop=.55
Yesp=.45
Normal survival=1
5. COMPUTE THE EXPECTED UTILITY OF 5. COMPUTE THE EXPECTED UTILITY OF EACH BRANCHEACH BRANCH
Called "folding back" the tree. Called "folding back" the tree.
Expected utility of action = each possible outcome weighted Expected utility of action = each possible outcome weighted by its probability. by its probability.
Simple arithmetic calculationsSimple arithmetic calculations
5. Compute expected utility of each branch5. Compute expected utility of each branch
Ms. Brooks
No treatment
Surgery
Surgery:yes or no?
AneurysmRupture?
Nop=0.9825 Normal survival=1
Yesp=0.0175
Early Death=0
SurgicalDeath?
Nop=0.977
Yesp=0.023 Early Death=0
Death?
Nop=.55
Yesp=.45
Normal survival=1
AneurysmRupture?
Nop=1.0 Normal survival=1
Yesp=0
Early Death=0
Death?
Nop=.55
Yesp=.45
Normal survival=1
=0
=0
=.55
=.55
5. Compute expected utility of each branch5. Compute expected utility of each branch
.865 vs .977
Ms. Brooks
No treatment
Surgery
Surgery:yes or no?
AneurysmRupture?
Nop=0.9825 Normal survival=1
Yesp=0.0175
Early Death=0
SurgicalDeath?
Nop=0.977
Yesp=0.023 Early Death=0
Death?
Nop=.55
Yesp=.45
Normal survival=1
AneurysmRupture?
Nop=1.0 Normal survival=1
Yesp=0
Early Death=0
Death?
Nop=.55
Yesp=.45
Normal survival=1
=1.0
=.55
=.55
=.9825
=0
=.9921
=.977
Diff = -0.0151
6. Perform sensitivity analysis6. Perform sensitivity analysis
• How certain are we of our recommendation?How certain are we of our recommendation?
• Change the input parameters to see how they affect the final Change the input parameters to see how they affect the final result.result.
– What if her life expectancy were shorter?What if her life expectancy were shorter?
– What if the rupture rate of untreated aneurysms were What if the rupture rate of untreated aneurysms were lower?lower?
– How good a neurosurgeon is required for a toss up?How good a neurosurgeon is required for a toss up?
Point at which the two lines cross = Point at which the two lines cross = treatment thresholdtreatment threshold.
Figure 4Figure 4 Sensitivity Analysis
0.965
0.97
0.975
0.98
0.985
0.99
0.995
1
1.005
0 0.005 0.01 0.015 0.02 0.025 0.03
Surgical Mortality
Exp
cete
d U
tility
No Treatment
Surgery
Surgical mortality = 0.008
Base Case
STEP BACK AND REVIEW THE ANALYSISSTEP BACK AND REVIEW THE ANALYSIS
As each iteration is completed, step back … As each iteration is completed, step back … Have we answered the question? Have we answered the question? Did we ask the right question? Did we ask the right question? Are there other details that might be important? Are there other details that might be important?
Consider adding complexity to improve accuracyConsider adding complexity to improve accuracy.
Ms. BrooksMs. Brooks
• ““Thanks… But I meant I wanted to live the most years Thanks… But I meant I wanted to live the most years possible. Dying at age 80 isn’t as bad as dying possible. Dying at age 80 isn’t as bad as dying tomorrow…”tomorrow…”
We recommend NO surgery.
Improve the AnalysisImprove the Analysis
Add layers of complexity to produce a more Add layers of complexity to produce a more realistic analysis.realistic analysis.
Solution: Another OutcomeSolution: Another Outcome
Ms. Brooks
No treatment
Surgery
Surgery:yes or no?
AneurysmRupture?
No Normal survival
Yes
Early Death
SurgicalDeath?
No
Yes Immediate Death
Death?
No
Yes
Normal survival
AneurysmRupture?
No Normal survival
Yes
Early Death
Death?
No
Yes
Normal survival
Three outcomesDetermine utility as a portion of expected life span
-Normal survival 1.0-Early death 0.5-Immediate death 0
No aneurysm rupture0.9825
No surgeryDie
Aneurysm rupture 0.45
0.0175 Survive0.55
No aneurysm rupture1
Survive surgery0.977 Die
Aneurysm rupture 0.45
Clipping 0 Survive0.55
Key InputsRupture risk/yr 0.0005 Surgical deathExpected life span 35 0.023
RR rupture w/ surgery 0Surgical mortality 0.023
Ms. Brooks
Normal survival
Immediate death
Normal survival
Normal survival
Normal survival
Early death
Early death
Figure 2Figure 2
"Utility"No aneurysm rupture0.9825
No surgery0.996 Die
Aneurysm rupture 0.45
0.0175 Survive0.55
No aneurysm ruptureDifference 1
-0.019 Survive surgery0.977 Die
Aneurysm rupture 0.45
Clipping 0 Survive0.977 0.55
Key InputsRupture risk/yr 0.0005 Surgical deathExpected life span 35 0.023
RR rupture w/ surgery 0Surgical mortality 0.023
Normal survival 1.0
Normal survival
Normal survival
Early death
Early death
1.0
0.5
1.0
0.0
Ms. Brooks
0.5
1.0Normal survival
Immediate death
Figure 3aFigure 3a
123456789
1011121314151617181920
B C D E F G H I J K
"Utility"No aneurysm rupture0.9825
No surgery =F3*K2+F7*(H6*K5+H8*K7) Die
Aneurysm rupture 0.450.0175 Survive
0.55
No aneurysm ruptureDifference 1 =D17-D5 Survive surgery
0.998 DieAneurysm rupture 0.45
Clipping 0 Survive =E14*(F12*K11+F16*(H15*K14+H17*K16))+E20*K19 0.55
Surgical death0.002
0.0
Ms. Brooks
0.5
1.0Normal survival
Immediate death
Normal survival 1.0
Normal survival
Normal survival
Early death
Early death
1.0
0.5
1.0
Figure 3bFigure 3b
Ms. BrooksMs. Brooks
• ““Wait a minute… Nobody said anything about being Wait a minute… Nobody said anything about being disabled. If I lived with a disability because of surgery, disabled. If I lived with a disability because of surgery, that would stink. Did you factor that in?”that would stink. Did you factor that in?”
"Utility"No aneurysm rupture0.9825
No surgery0.996 Die
Aneurysm rupture 0.45
0.0175 Survive0.55
No aneurysm ruptureDifference 1
-0.082 Survive surgery0.902 Die
Aneurysm rupture 0.45
Clipping 0 Survive0.915 0.55
Key Inputs Surgery-induced disabilityRupture risk/yr 0.0005 0.075
Expected life span 35RR rupture w/ surgery 0 Surgical deathSurgical mortality 0.023 0.023
Surg morb (disability) 0.075
0.0
Ms. Brooks
0.5
1.0Normal survival
Disability, shorter survival
0.17
Immediate death
Normal survival 1.0
Normal survival
Normal survival
Early death
Early death
1.0
0.5
1.0
Figure 5Figure 5
SummarySummary
• Explicit question.Explicit question.• Decision tree.Decision tree.• Probabilities of each outcome.Probabilities of each outcome.• Utilities for each outcome.Utilities for each outcome.• Expected utility of each course of action.Expected utility of each course of action.• Sensitivity analysis.Sensitivity analysis.
Next time:Next time: Utilities, QALYs, discounting Utilities, QALYs, discounting
