Net Clinical Benefit: The Art And Science Of Jointly Estimating Benefits And Risks Of Medical

TreatmentAdrian Towse

Office of Health Economicsatowse@ohe.org

www.ohe.org

Benefit-risk Assessment

Benefits Risks

(Harms)

Agenda• What is the issue?

– Licensing and HTA• How can we improve the information available to

decision makers?– Incremental Net Health Benefit (IHNB) – QALYs– Incremental Net Health Benefit (IHNB) – RVALYs– Stated Preferences for Benefit versus Risk

• Better decision making– Multi Criteria Decision Analysis (MCDA)

• Way forward

What is the issue?

• Need to understand benefits and harms (risks) and to understand “willingness to trade”

• A critical task for regulators (licensing bodies) –Vioxx, Lotronex, …..

• But also key for HTA bodies and P&R bodies• n.b. Although similar tools are used in

economics and decision analysis, it’s NOT about COSTS

Pharmacoeconomics & Pharmacoepidemiology:Curious bedfellows or a match made in heaven?

Briggs and Levy, 2006

• How to trade off benefits and risks (harms)– Example of DVT (Lynd and O’Brien)

• Incremental benefit – risk with “benefit - risk acceptability threshold”– Need a “willingness to trade” a benefit and a risk

• Using a common currency:– Willingness to pay– QALYs

Summary of events from a clinical trial of low-dose unfractionated heparin vs. enoxaparin for the

prevention of DVT following major traumaNumbers experiencing an eventa

Event Unfractionatedheparin

Enoxaparin Differenceb

Benefits

Distal DVT 40 32 −8

Proximal DVT 20 8 −12

Risks

Major bleeds 1 5 +4

AFocus on numbers (rather than probabilities) ignores the slight imbalance between arms of the trial with 136 patients randomised to low-dose unfractionated heparin and 129 randomised to enoxaparin in the original trial. bEnoxaparin minus heparin: negative numbers are events avoided, positive numbers indicate excess events.

Proximal DVT vs Major Bleed(µ = threshold ratio) shown on as a joint distribution of risk

and benefit on a cost-effectiveness plane

IRBR= 0.031/0.088

= 0.35

Source: Figure 1 from Lynd & O’Brien, 2004

Pharmacoeconomics & Pharmacoepidemiology:Curious bedfellows or a match made in heaven?

Briggs and Levy, 2006

• How to trade off benefits and risks (harms)– Example of DVT (Lynd and O’Brien)

• Incremental benefit – risk with “benefit - risk acceptability threshold”– Need a “willingness to trade” a benefit and a risk

• Using a common currency:– Willingness to pay– QALYs

Challenges and Opportunities for Improving Benefit-Risk Assessmentof Pharmaceuticals from an Economic Perspective

Cross and Garrison, Office of Health Economics 2008Available on the OHE website for download

www.ohe.org

Agenda• What is the issue?

– Licensing and HTA• How can we improve the information

available to decision makers?– Incremental Net Health Benefit (IHNB) – QALYs– Incremental Net Health Benefit (IHNB) – RVALYs– Stated Preferences for Benefit versus Risk

• Better decision making– Multi Criteria Decision Analysis (MCDA)

• Way forward

Incremental Net Health Benefit (INHB) of New Drug (2) vs. Current Therapy (1)

INHB= (E2 – E1) – (R2-R1)where– Effectiveness E is measured in QALYs– Risk R is measured in QALYs

(E2 – E1) > (R2-R1) Favorable benefit-risk balance

Estimating aggregate INHB

Risk-Benefit Analysis of Rofecoxib versus Naproxen using Discrete Event Simulation:

Disutility and Duration of Adverse EventsDisutility Duration (days)

Dyspepsia (resolved)severe -0.13 12moderate

Dyspepsia (unresolved)-0.09-0.13

32Duration

GI Hemorrhage with surgery -0.51 4

severe dyspepsia -0.13 35MI non-fatal

acute event 0 4post-MI -0.12 duration

Death (all causes) 0 duration

Risk-Benefit Analysis of Rofecoxib versus Naproxen using Discrete Event Simulation:Results: number of events per 10,000 patients

Rofecoxib NaproxenDyspepsia 415 756GI Bleed 117 236GI Perforation 7 18

Fatal MI 12 5Non-Fatal MI 30 8Death (all causes) 1 2

No Adverse Event 9418 8975

Results: assuming equal efficacy

96% chance INB ≥0

Agenda• What is the issue?

– Licensing and HTA• How can we improve the information

available to decision makers?– Incremental Net Health Benefit (IHNB) – QALYs– Incremental Net Health Benefit (IHNB) – RVALYs– Stated Preferences for Benefit versus Risk

• Better decision making– Multi Criteria Decision Analysis (MCDA)

• Way forward

Lotronex® - alosetron for IBS

• Originally indicated in women with diarrhea predominant IBS.

• Demonstrated benefits• Potential harms• Withdrawal over concern for harm, then

reintroduced with restricted indication + RMP– Severe, diarrhea predominant irritable

bowel syndrome.

Preference weights for outcomes (RTI Health Solutions)

Stated Preference Weights (Scaled)

Frequency of Abdominal Pain

1-2 Days a Week3-5 Days a Week6-7 Days a Week

-0.0151-0.0378-0.0529

Frequency of Urgency 1-2 Days a Week3-5 Days a Week 6-7 Days a Week

-0.0198-0.0753-0.0817

Frequency of Diarrhea 1-2 Times a Day3-4 Times a Day>4 Times a Day

-0.0269-0.0827-0.1042

Frequency of Constipation

1-2 Days a Week3-5 Days a Week6-7 Days a Week

-0.0143-0.0359-0.0502

Adverse Events Moderate ColitisSevere ColitisImpacted BowelPerforated Bowel

-0.0177-0.1258-0.0987-0.3072

Distribution of INB, PSA results99% chance of +ve INB

-0.0060 0.0006 0.0072 0.0138 0.0204 0.0270 0.0336 0.0402 0.0468 0.0534 0.0600

incerementan net benefit

0

10

20

30

perc

ent

Incremental Net Benefit

30.4 RVALYs gained per 1,000 patients treated with alosetron relative to placebo over 52 weeks

Agenda• What is the issue?

– Licensing and HTA• How can we improve the information

available to decision makers?– Incremental Net Health Benefit (IHNB) – QALYs– Incremental Net Health Benefit (IHNB) – RVALYs– Stated Preferences for Benefit versus Risk

• Better decision making– Multi Criteria Decision Analysis (MCDA)

• Way forward

Stated-Preference Methods• “When asking the public to assist in determining

health priorities, we should use techniques that allow people to reveal their true preferences. If not, why bother asking them at all?” Amiram Gafni (Social Science and Medicine, 1995)

• Subjects state preference for series of hypothetical alternatives

• Alternatives consist of combinations of attributes• Preferences among alternatives depend on the

relative importance of attributes• Pattern of choices identifies trade-off relationships• Statistical model predicts choice probabilities

page 23

Choice-Format Stated-Preference or Conjoint Question

Feature Treatment A Treatment B

PAIN

STIFFNESS

STOMACH PROBLEMS

Occasional mild symptoms.

Treat with over-the-counter medicines

Frequent moderate symptoms.

Treat with a prescription medicine

RISK OF BLEEDING ULCER

1 patient out of 100 (1%) will have a bleeding ulcer

5 patients out of 100 (5%) will have a bleeding ulcer

RISK OF HEART ATTACK or STROKE

5 patients out of 100 (5%) will have a stroke

15 patients out of 100 (15%) will have a heart attack

No Stiffness Extreme StiffnessNo Stiffness Extreme Stiffness

No Pain Extreme Pain No Pain Extreme Pain

Effic

acy

Mild

-M

oder

ate

Side

Effe

cts

Serio

us

Side

-Effe

ct

Ris

ks

page 24

Example Results: MS 10-Year PML Risk

Slow progressionbenefit

PMLLiver failureLeukemia

14%

12%

10%

8%

6%

4%

2%

0.0%Clinically relevant

benefitLargest tested

benefit

Observed Risk

Max

imum

acc

epta

ble

risk

Manuscript forthcoming in Multiple Sclerosis

page 25

Example Results: HRT Annual Heart-Attack Risk

0.050

Severe tomild symptoms

Max

imum

acc

epta

ble

risk Absolute Risk

Relative Risk

Severe tono symptoms

WHI Risk

0.045

0.040

0.035

0.030

0.025

0.020

0.015

0.010

0.005

0.000Severe to

moderate symptoms

Johnson, et al. (2007) Journal of Women’s Health.

Agenda• What is the issue?

– Licensing and HTA• How can we improve the information available to

decision makers?– Incremental Net Health Benefit (IHNB) – QALYs– Incremental Net Health Benefit (IHNB) – RVALYs– Stated Preferences for Benefit versus Risk

• Better decision making– Multi Criteria Decision Analysis (MCDA)

• Way forward

Tasks for experts and modelsExperts are good at….• Identifying the elements

contributing to a decision• Making judgements

about the individual elements

• Exploring the results of a model that combines the elements

• Developing new insights• Forming preferences

Models are good at…• Providing Structure for

thinking clearly• Separating facts from

value judgements• Combining the elements

to form an overall results• Showing consequences

of imprecision in judgements and differences of opinion

Source: Phillips

Multi-criteria decision analysis• A methodology for appraising options on

individual, often conflicting criteria, and combining them into one overall appraisal

• Incorporates judgements about the impact of data, turning performance into value added

• Allows for differential importance of decision criteria

• Based on sound theory: assumes decision makers want choices to be consistent

Source: Phillips

Putting the ingredients togetherMCDA model Group judgement

= Smart decisions!

Source: Phillips

Agenda• What is the issue?

– Licensing and HTA• How can we improve the information available to

decision makers?– Incremental Net Health Benefit (IHNB) – QALYs– Incremental Net Health Benefit (IHNB) – RVALYs– Stated Preferences for Benefit versus Risk

• Better decision making– Multi Criteria Decision Analysis (MCDA)

• Way forward

Way forward• These are decision support tools

– The techniques have to be understood and acceptedby decision makers

– Where risks and benefits are combined the weightings need to be explained and the results of alternative assumptions presented

• These tools offer decision makers information and the opportunity to be more transparent and consistent in decision making

• Can they “work”? Need for more studies and pilots?

References• Aronson J Balancing benefits and harms in health care. BMJ 2004;329;30• Briggs and Levy. Pharmacoeconomics & Pharmacoepidemiology: Curious

bedfellows or a match made in heaven? PharmacoEconomics 2006; 24 (11): 1079-86

• Cross and Garrison. Challenges and Opportunities for Improving Benefit-Risk Assessment of Pharmaceuticals from an Economic Perspective Office of Health Economics 2008

• Garrison LP, Towse A, Bresnahan, B. Assessing a structured, quantitative health outcomes approach to drug risk-benefit analysis. Health Affairs, 2007. 26(3):684-95.

• Dodgson, J., Spackman, M., Pearman, A., & Phillips, L. Multi-Criteria Analysis: A Manual. London: Department of the Environment, Transport and the Regions 2000.

• Keeney, R. L., & Raiffa, H. Decisions With Multiple Objectives: Preferences and Value Tradeoffs. New York: John Wiley 1976

• Lynd LD, O'Brien BJ. Advances in risk-benefit evaluation using probabilistic simulation methods: an application to the phrophylaxis of deep vein thrombosis. Journal of Clinical Epidemiology 2004; 57:795-803.

• Lynd LD, Najafzadeh M, Colley L, Willan AR, Sculpher MJ. Quantitative harm-benefit analysis of alosetron in irritable bowel syndrome: a patient level meta-cohort analysis. Poster

• Lynd LD. Using Incremental Net Benefit for Quantitative Benefit-Risk Analysis – Case studies of Vioxx® and Lotronex®. Presented at the Benefit-Risk Assessments for Drugs Workshop. Office of Health Economics. 24 October 2007

• Walker, S., Phillips, L., & Cone, M. Benefit-Risk Assessment Model for Medicines: Developing a Structured Approach to Decision Making. Epsom: Centre for Medicines Research International, Institute for Regulatory Science.. 2006