Setting the Scene

Dr Andrew Walker

Contents

• SMC Process (very briefly)

• Analysis of decisions to date, emphasising economics aspects

• Quality of economics evidence

• Issues for economic models

• Issues for sensitivity analysis

SMC process

• New medicines and major new indications• Onus on manufacturer to submit evidence• Forms and guidance on website• Sections on

– Clinical trials– Safety– Effectiveness– Economics evidence– Budget impact

Upon receipt …

• Assigned to meeting and reviewer• Checklist for economics• Access to clinical experts, statistician, etc.• With pharmacist, joint recommendation• Goes to New Drugs Committee …• … their draft rec. to manufacturer …• SMC sees NDC view, manufacturer view,

patient group view

NICE (HTA (medicines (Scotland)))?

• Selection of topics

• Timescales

• Resources

• Nature of guidance

• Processes (e.g. consultation)

• Tasks undertaken

• Possibly: burden of proof

Submissions

24% no economic evaluation

1% cost-consequence analysis

27% cost-effectiveness analysis

13% cost impact

4% cost-minimisation analysis

30% cost-utility analysis

How many trees died?

• Recommended limit is 50 but exactly half the submissions received exceed that

• Average submission is 56 pages long (including economics appendices)

• Average econ section is 15 pages

• Econ is typically 24% of total

• Largest submission 236 pages

Guidance issued

• Just over 100 pieces of guidance issued

• www.scottishmedicines.org/medicines/default.asp

• Acceptance rate about 70%

• Falling very slightly over time

• Goes to NHS boards and Area D&T Cmts

• Sticking to timescales

Medicine D - accept 11/02

0

50000

100000

150000

200000

250000

300000

350000

Medicine C - not rec 7/02, accepted 5/03

0100002000030000400005000060000700008000090000

100000

Medicine A - SMC decision 5/03

0

1000

2000

3000

4000

5000

6000

7000

10/02

11/02

12/02 1/

032/

033/

034/

035/

036/

037/

038/

039/

0310

/0311

/0312

/03 1/04

2/04

Medicine B - SMC decision 8/03

02000400060008000

1000012000140001600018000

NICE overlap

• 12 products reviewed by both groups, every decision has been the same

• Accepted: peg interferon for hep C, imatinib for CML, rituximab for NHL

• Not acepted: anakinra for RA’ pimecrolimus for eczema (still provisional?)

• Typically SMC announcement ten months earlier• A quarter of the duration and a tenth of the cost!

PBAC overlap

PBAC accept

PBAC reject

SMC accept 20 4

SMC reject 2 5

What helps or hinders?

Medicines do slightly better if:

• Mainly for use in for secondary care

• Potential to save life / improve compliance

• 57th in class

• New version of a medicine (e.g. pegylated)

• Manufacturer is “frequent submitter”

But NONE of these are deliberate

SUBMITTED budget impact

Not rec Restrict General Accept

No clear estimate 5 10 1 69%

Savings/zero 4 4 6 71%

< £100k 3 8 2 77%

£100k-£1m 13 19 3 63%

>£1m 3 7 3 77%

Average (excl. “No clear estimate”)

£548k £821k. £532k

SUBMITTED £/QALY or LY

Not rec Restrict General Accept

Dominant (savings and QALY gain

2 2 0 50%

<£5k/QALY 2 4 1 71%

£5k-£10k/QALY 0 8 0 100%

£10k-£20k/QALY 3 6 2 73%

>£20k/QALY 1 2 0 67%

Average (excl. dominant)

£12k £10k

Role of economics at SMC

Econ view supported 37% of submissions

– When econ view is supportive, 92% were accepted

– When econ view is not supportive, 56% were accepted

When SMC guidance is “not recommended” the econ view was a factor in 90% of cases

So what does matter?

1. Quality of case submitted and whether it is fit for purpose

2. … er …

3. That’s it really.

The SMC scepticism index

• Almost every submission has some flaws• Not every flaw damns the submission• Flaws act cumulatively and probably interact as

well to produce scepticism• At some point the evidence submitted is

perceived to be so flawed that it can no longer be taken as offering convincing support

• “The economic case has not been demonstrated”

How do you define quality?

• Guidance on our website, FAQs, checklist

• Keeps fairly close to NICE guidance

• Today is about starting to think how we take that forward in two key areas:– Economic models– Sensitivity analysis

• So what’s the problem?

3 roles for models

Economic models perform three roles:

1. Manufacturer constructs model to estimate costs and benefits beyond RCT follow-up

2. Partial or complete departure from RCT, model estimates all costs and benefits

3. A few key clinical results are used in a pre-existing model

The Norwegian Blue problem

• Basic design:– Choice of comparator for economic evaluation– RCT data that convinces SMC clinicians– Generalisable measure of health benefit

• Can clever stats ever conceal a big design flaw?

• Stirling or Kilmarnock?

Thumbnail of submission

• “Markov model as basis for CUA”, time horizon for model, cycle length– Justified?

• Diagram, data inputs – table and refs.– Relevance to Scotland 2004? – Is the model valid? (section missing???)

• Key results and ICER– Can we really see what’s going on?

Key issues for modelling

• Data inputs (but not for today)

• How can we make the predictions of the model as transparent as possible?

• How do we know if the model is valid or not?

• What is it reasonable to expect a manufacturer to have available at launch?

Transparency

Time Mild Moderate Severe

1 100 0 0

2 90 8 2

3 75 18 7

4 58 30 12

5 40 40 20

£0

£5,000

£10,000

£15,000

£20,000

£25,000

Tam Ari Tam Ari Tam Ari Tam Ari Tam Ari

1 5 10 20 25

Years

Discounted follow-up costs

Discounted AE costs

Discounted recurrence & death cost

Discounted drug costs

What is validity?

• Mount Hood challenge as the ideal

• Reality: model constructed by one economist, with more-or-less plausible data (including RCTs with one year follow-up), predicting “safe” cost per QALY over 25 years

• The ideal is not feasible …

• … but what is?

Calling John Hutton …

• John will address this issue in a minute, but we have had submissions which:

• Ignored trial data, just used focus group to estimate likely effects

• RCT data collection was poor, so started again with a model

• Took one key result and applied it in a pre-existing model

Sensitivity analysis

• About a third of econ submissions now have PSA, visual aspect is very helpful

• Issues:• If it is possible to “spin” a PSA, how can

we critique it to spot this?• How do we interpret the results?• Should PSA be a requirement or does it

have a more limited role? Or are there other options?

Calling Andy and Adrian …

What evidence should the manufacturer submit to support:

• choice of range for a variable?

• form of distribution?

• number of simulations?

• choice of ICER for “threshold”?

• Interpretation of CEAC, NHB, etc.?

-6000

-4000

-2000

0

2000

4000

6000

-0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8

Δ QALE (years)

Δ C

osts

(£)

0

10

20

30

40

50

60

70

80

£0 £5,000 £10,000 £15,000 £20,000 £25,000 £30,000 £35,000 £40,000 £45,000 £50,000

Willingness to pay

Pro

babi

lity

that

Det

erm

ir c

ombi

natio

n th

erap

y w

ill b

e co

st-e

ffec

tive

(%)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

-£15,000 -£10,000 -£5,000 £0 £5,000 £10,000 £15,000

Net health benefit provided by pioglitazone

Summary

• Process and guidance

• What matters (and what doesn’t)

• Quality is key, not trying to deal with all aspects of that today

• Focus is on:– Models: transparency and validity– PSA: role and/or niche