Dangerous Omissions – the Cost of Ignoring Decision Uncertainty

Dangerous Omissions – the Cost of Ignoring Decision Uncertainty

Mark Sculpher

Susan Griffin

Karl Claxton

Steve Palmer

Centre for Health Economics, University of York,

Background

• Increasing demands to assess new drugs closer to launch• Inevitable uncertainty in evidence• Hence a series of linked questions

– Should a technology be adopted?– How uncertain is this decision?– Is more evidence needed?

• But decisions may be fragmented• Need a methods framework to handle these questions

What are the decisions?

• Should a technology be adopted given existing information?– Which clinical strategies are worthwhile? – For which patient groups?

• Is current evidence sufficient to support use in NHS?– Do we need more evidence?– What type of evidence is required?– What additional research should be conducted to provide this

evidence?

£20,000

2 QALYs =

= 2 – £20,000

£20,000

Is it cost-effective?Is it worthwhile?

Is the ICER less than the cost-effectiveness threshold?

If the cost-effectiveness threshold is £20,000 per QALY, B is cost-effective

Is net benefit positive? Net health benefit = QALYs gained – QALYs lost

Net money benefit = £ value of QALYs gained – additional costs

= 2 x £20,000 – £20,000

Additional cost

QALYs gained ICER = = £10,000 per QALY

= 2 – 1 = 1 QALY

= £20,000 = 1 QALY

= 2 – £20,000

£20,000

Should a technology be adopted?

Treatment A

QALY Cost

Treatment B

QALY Cost

2 £30,000

3 £20,0004 £40,000

1 £10,000

0 £ 5,0002 £15,000

1 £10,000

3 £30,000

Additional cost

QALYs gained ICER =

£20,000

2 QALYs = = £10,000 per QALY

Is the ICER less than the cost-effectiveness threshold?

£10,000 per QALY < £20,000 per QALY, B is cost-effective

Is net benefit positive?

Net health benefit = QALYs gained – QALYs lost

= 2 – 1 = 1 QALY

Net money benefit = £ value of QALYs gained – additional costs

= 2 x £20,000 – £20,000 = £20,000 = 1 QALY

What are the decisions?

• Should a technology be adopted given existing information?– Which clinical strategies are cost-effective? – For which patient groups?

• Is current evidence sufficient to support use in NHS?– Do we need more evidence?– What type of evidence is required?– What additional research should be conducted to provide this

evidence?

How uncertain is a decision?

What’s the best we can do now? But we are not always right

Choose B and expect 13 QALYs Chance that B is the best = 3/5 = 0.6

Chance that A is the best = 2/5 = 0.4

Chance that C is the best = 0/5 = 0

So if we adopt B the probability of error = 0.4

How things could turn out

Net Health Benefit

Best choiceTreatment A Treatment B Treatment C

Possibility 1 9 12 8 B

Possibility 2 12 10 9 A


Possibility 4 11 10 10 A


Average 12 13 10

How uncertain is the decision?

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

£0 £10,000 £20,000 £30,000 £40,000 £50,000 £60,000

Cost-effectiveness threshold

Pro

ba

bili

ty c

ost

-effe

ctiv

e

B

A

C

Choose A Choose B

ICER = £25,000 per QALY

Why does uncertainty matter?

What’s the best we can do now? Could we do better?

Choose B and expect 13 QALYs If we knew we get 13.6 QALYs

Maximum benefit of more evidence is 0.6 QALYs

But is it worth it?

How things could turn out

Net Health Benefit Best we could do if we knew

What we could loseTreatment A Treatment B Best choice

Possibility 1 9 12 B 12 0

Possibility 2 12 10 A 12 2


Possibility 4 11 10 A 11 1


Average 12 13 13.6 0.6

Do we need more evidence?

£0

£5,000,000

£10,000,000

£15,000,000

£20,000,000

£25,000,000

£0 £10,000 £20,000 £30,000 £40,000 £50,000 £60,000


Ma

xiu

m b

en

efit

of

evi

de

nce

.

Choose A Choose B

Cost of research

Cost of research

£0

£5,000,000

£10,000,000

£15,000,000

£20,000,000

£25,000,000

£30,000,000

£35,000,000

£0 £5,000 £10,000 £15,000 £20,000 £25,000 £30,000 £35,000 £40,000 £45,000 £50,000

Threshold for cost-effectiveness

Po

pu

latio

n E

VP

I

WT

Ritalin

Zanamivir

Riluzole

Orlistat

Do we need more evidence?

Decisions in a joined up world?

• Adopt technologies if we expect them to be cost effective based on existing evidence

• But only if we simultaneously address question: Is the evidence sufficient?

• Demand or commission further research to inform this choice in the future

A fragmented world

• Separation of adoption and research decisions– Adoption decisions without accountability for impact on

future research– Research decisions without accountability for relevance to

adoption decisions • Dangers

– Adoption decisions undermine evidence base for practice• Incentives and ethics

– Commissioned research does not inform decisions• Adoption becomes the only policy instrument

£0

£10

£20

£30

£40

£50

£60

£70

£80

£0 £5,000 £10,000 £15,000 £20,000 £25,000 £30,000 £35,000 £40,000 £45,000 £50,000


Op

po

rtu

nity

loss

Account for the cost of uncertainty

What we loose if we accept technology

What we loose if wereject a technology

£0

£10

£20

£30

£40

£50

£60

£70

£80

£0 £5,000 £10,000 £15,000 £20,000 £25,000 £30,000 £35,000 £40,000 £45,000 £50,000


Op

po

rtu

nity

loss

Clear signals and incentives

Provide more evidence!

£0

£10

£20

£30

£40

£50

£60

£70

£80

£0 £5,000 £10,000 £15,000 £20,000 £25,000 £30,000 £35,000 £40,000 £45,000 £50,000


Op

po

rtu

nity

loss

Clear signals and incentives

Reduce price

Why say no (or only in research)?

• Clear signals – No because it is not a cost-effective use of resources– No because there is currently insufficient evidence to justify

NHS use– Spell out the key evidence needed (not the research)

• Clear incentives– If and when additional evidence is made available then

considered for early review– Incentives to sponsors (evidence and price)– Incentives for others stakeholders to lobby for publicly

funded research– Clear signals to research commissioners

Conclusions

• All decisions about new technologies involve uncertainty• Is uncertainty being used in decision making?• Need to address adoption decision and need for further

research simultaneously• But adoption decision may be only policy lever

Dangerous Omissions – the Cost of Ignoring Decision Uncertainty

Documents

Transcript of Dangerous Omissions – the Cost of Ignoring Decision Uncertainty