Economic evaluation in health careEconomic evaluation in health care - - IIII

Zahidul Quayyum

Health Economic Research Unit

Economic Evaluation II- OutlineEconomic Evaluation II- Outline

• Costing in Economic Evaluation– Identification of Costs, Measurement of

Costs– Valuation of Costs, Main Types of Costs

• Challenges in Costing• Economic evaluation in health care- Critical

Appraisal• Valuing Health Care for CBA• Limitations of Economic Evaluation

Costing in economic evaluationCosting in economic evaluation

• All resources used in health programmes /interventions have opportunity costs, and therefore we want to identify, measure and value resource used in health care/services, programmes and intervetions

• Items to be identified for inclusion on the cost side of an economic evaluation are any resources which have an opportunity cost as a result of being used in the health care programs under consideration

Identification of costsIdentification of costsDirect costs:• Health care resources: staffing, consumables,

overheads, capital• Other related services: community services; ambulance

services; voluntary services

• Costs incurred by patients and their families– inputs to treatments– out-of-pocket expenses

Identification of costsIdentification of costs

Indirect Costs:• Time lost from work (production effects)• Production effects arise from working in the labour

force and housework• Indirect costs arise because health care may result in

lost production• Value of indirect cost is the value of the lost

production

How far and wide should we go?How far and wide should we go?

• Consider the viewpoint or perspective of the study– Societal, NHS/health care sector, patient or employer

• Identify common costs– can include only those costs that differ between the

programmes

• Time frame

• Magnitude or quantitative importance of costs

Measurement of costsMeasurement of costs

• Measuring resource use in naturally occurring units

• Health care resources Measured how?– staffing time (hours, minutes)– consumables units/amounts consumed– overheads units/amounts consumed – capital units/amounts consumed

• How? Medical records and patient surveys

• Important to report quantity of resource use – Variations between countries/regions

Valuation of costsValuation of costs

• Many elements are straight forward– For example staff costs - wage rates (grades) +

national insurance and superannuation

• Others are more difficult– e.g. overheads, capital

• Calculating costs (multiply quantity of resource use by unit cost)

Main types of costMain types of costTotal cost= sum of all the costs of producing a

particular quantity of output

Average cost = total cost /quantity e.g. cost per patient,cost per diem

Marginal cost = the extra cost of producing one extraunit of output

Fixed costs = cost which do not vary with the quantity ofoutput in the short run e.g. rent,equipment, lease payments, some wagesand salaries

Variable costs = costs which vary with the level of output,e.g. food, medical/surgical supplies

Measure and Valuation of Costs: Measure and Valuation of Costs: Cost AnalysisCost Analysis

• Perspectives is important, Top down & bottom up or ingredients approach

• Direct and Indirect Costs

• Recurrent and Capital Costs

• Overhead and Shared Costs

• Apportionment and allocation, period of costing and adjustments for inflation

Measure and Valuation of Costs: Cost AnalysisMeasure and Valuation of Costs: Cost Analysis

• Perspectives needs to be defined: Cost included depend on the point of view:

– Health service or institution (direct costs)– Government budget (direct and some indirect

costs)– Community (direct and indirect costs)

• Economic Costing vs. Financial Costing– Full Economic Costing (Opportunity Costs) vs

Accounting/Financial Costing- We need to show the difference, societal perspective

Measure and Valuation of Costs: Cost AnalysisMeasure and Valuation of Costs: Cost Analysis

• Top down approach – Macro level costing– Identify activities, identify the cost centres, e.g.

maternity ward, the related department, the overheads– Major resources used in cost centres– Measure (Quantify) and value them – Allocate and estimate costs- directly through

accounting or determining allocation factor• Bottom Up Approach – Micro level costing

– Identify Activities, Identify all resources and inputs, assured measure of allocation, Measure (Quantify), using appropriate allocation factor

– Value them,• Top Down or Bottom up -depend on what you want/can

afford to do, and how useful

UK costing dataUK costing data

• Curtis (2009) Unit costs of health and social care, PSSRU, Canterbury.

http://www.pssru.ac.uk/pdf/uc/uc2009/uc2009.pdf• ISD Scotland (2009) Scottish health service costs.

Year ended 31st March, 2009, ISD Scotland, Edinburgh.

http://www.isdscotland.org• NHS reference costs – 2008-2009http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuidance/DH_111591

Challenges, refinements and pitfallsChallenges, refinements and pitfalls

1. Average versus marginal costs 2. Double counting3. Counting costs in a base year4. Discounting5. Dealing with overheads and capital items6. Value of time

1. 1. Average versus marginal costsAverage versus marginal costs

• What is the cost of delivery at a Scottish maternity unit? (Drummond and Mooney,1983)

• Average cost = total cost of the maternity unit/number of deliveries = £540

• Marginal cost = if we wanted to increase the number of deliveries in a Scottish specialist maternity units,what would be the extra health service unit cost per delivery = £510

2. 2. Double CountingDouble Counting

Double counting including the same cost more than

once:• e.g. including time costs and fees and adding them

together would be double counting, often GP fee includes time spent on activities

• e.g. detailed costing of staff in an operating theatre then adding hourly cost of theatre time which already includes staffing costs as well as amounts for supplies, drugs, equipment etc.

3. 3. Counting costs in a base yearCounting costs in a base year

• Remove the effect of inflation

• Transform costs into a single base year

• Inflation indices – hospital and community health services

Assuming 5% inflation:£100 this year £100 in a year’s time£100 this year = £105 in a year’s time

4. 4. Discounting Discounting

• Question: “Should costs (and benefits) occurring at different points in time, be given equal weighting?

• If I offered to give you £1000 today OR £1000 in 5 years which would you choose?

• Would you rather pay me £1000 today OR £1000 in 5 years?

DiscountingDiscounting• Generally individuals prefer to receive a benefit

today and to incur a cost later

• Economists call this the notion of time preference• This is important for economic evaluation as cost and

benefits often arise at different points in time (e.g. prevention versus cure)

• Discounting is a process by which costs and benefits which occur at different times are made comparable by expressing future costs and benefits in terms of their present value

5. 5. OverheadsOverheads

• Resources shared by more than one programme, department e.g. heat, light, laundry, cleaning, administration

• How much should be allocated to the intervention or programme being evaluated?

• Consider which costs would change if a services/programme were introduced

• Number of methods– direct allocation method eg based on floor space

and patient throughput

5. 5. Capital itemsCapital items

• Examples - land, buildings and equipment

• The cost of capital occurs at a single point in time

• Although there is one initial outlay, the opportunity cost is spread over time

• Calculate the equivalent annual cost (EAC)

6. 6. Value of timeValue of time

• Depends on the alternative use of time

• Paid employment human capital approach (average wage rate)

• Unpaid labour? – Dept Transport (1989) suggest to value it at 54% of

average wage rate

• Leisure time?– Dept of Transport (1989) suggest to value it at 43% of

average wage rate

Economic evaluation in health care- Economic evaluation in health care- Critical AppraisalCritical Appraisal

• Drummond “10 points” check list *• Break• Exercise: Appraise and economic

evaluation• Discussion and considerations about other

possible types of studies

* Drummond M.F., Sculpher M.J., Torrance G.W., O’Brien B.J., Stoddart G.L.: Methods for Economic Evaluation of Health Care Programmes. 3rd Ed. Oxford University Press. 2005

Introduction

• Why critically appraise? – trust the results of a study?– can apply the study results

elsewhere?

Drummond “10 points” check listDrummond “10 points” check list

1. Well-defined question?2. Comprehensive description of alternatives?3. Effectiveness established?4. All relevant cost and consequences?5. Appropriate measurement?6. Credible valuation?7. Differential timing?8. Incremental costs and consequences?9. Allowance made for uncertainty?10.Appropriate interpretation of results?

1. Was a well-defined question posed in an answerable form?

• Examples of NOT good EE questions:– Is a chronic home care programme

worth it?– Will a community hypertension

screening programme do any good?– How much does it cost to run our

intensive care unit?– What are the costs and outcomes of

adolescent counselling by social workers?

1. Was a well-defined question posed in an answerable form?

1.1 Did the study examine both cost and effects of the service(s) or programme(s)?

1.2 Did the study involve a comparison of alternatives?

1.3 Was a viewpoint of the analysis stated and was the study placed in any particular decision making context?

1.3 Was a viewpoint of the analysis stated and was the study placed in any particular decision making context?

• Viewpoints:– Societal; Patient; Single provider (health

service, hospital, clinic, etc.); Insurer (third party payer); healthcare system; decision-maker (e.g. the Government).

• Settings:– community, primary care, secondary care,

tertiary care, institution, other.– Specific country where the EE was conducted

2. Was a comprehensive description of the competing alternatives given?

• that is, can you tell who did what to whom, where, and how often?

Example: Ingeborg et al. BMJ, Apr 2003;326:911

2. Was a comprehensive description of the competing alternatives given?

2.1 Were any relevant alternatives omitted?

2.2 Was (should) a do-nothing alternative (be) considered?

3. Was the effectiveness of the programmes or services established?

3.1 Was this done through a randomised, controlled clinical trial? If so, did the trial protocol reflect what would happen in regular practice?

3.2 Were effectiveness data collected and summarised through a systematic overview of clinical studies? If so, were the search strategy and rules for inclusion and exclusion criteria outlined?

3.3 Were observational data or assumptions used to establish effectiveness? If so, what are the potential biases in results?

4. Were all the important and relevant costs and consequences for each alternative identified?

4.1 Was the range wide enough for the research question at hand?

4.2 Did it cover all relevant viewpoints? (Societal, community, patients, third-party payers. Other potentially relevant viewpoints?)

4.3 Were capital costs, as well as operating costs, included?

5. Were costs and consequences measured accurately in appropriate physical units?

for example, hours of nursing time, number of physician visits, lost work-days, gained life years

5.1 Were the sources of resource utilisation described and justified?

5.2 Were any of the identified items omitted from measurement? If so, does this means that they carried no weight in the subsequent analysis?

5.3 Were there any special circumstances (for example, joint use of resources) that made measurement difficult? were these circumstances handled appropriately?

Example: Ingeborg et al. BMJ, Apr 2003;326:911

Example: Ingeborg et al. BMJ, Apr 2003;326:911

6. Were costs and consequences valued credibly?

6.1 Were the sources of all values clearly identified? (Possible sources include market values, patient or client preferences and views, policy-makers’ views, and health professionals judgements).

6.2 Were market values employed for changes involving resources gained or depleted?

6. Were costs and consequences valued credibly?

6.3 Where market values were absent (for example, volunteer labour), or market values did not reflect actual values (such as a clinic space donated at a reduced rate), were adjustments made to approximate market values?

6.4 Was the valuation of consequences appropriate for the question posed (that is, has the appropriate type or types of analysis –CEA, CUA, CBA- been selected)?

7. Were costs and consequences adjusted for differential timing?

7.1 Were cost and consequences that occur in the future ‘discounted’ to their present values?

‘Discounting is a process by which costs and benefits which occur at different times are made comparable by expressing future costs and benefits in terms of their present value’

Need to allow for inflation adjustments also: so, express all cost in prices of a chosen year, then, discount to get everything to its Present Value.

7.2 Was any justification given for the discount rate used?

(i.e. Discount rate use by Government for their investments. NICE nowadays ask 3.5% for costs and utilities/benefits)

8. Was an incremental analysis of costs and consequences of alternatives performed?

8.1 Were the additional (incremental) costs generated by one alternative over another compare to the additional effect, benefits, or utilities generated?

Intervention A

(e.g. current practice)

Intervention B

(e.g. new medication)

Total costs A Total effects A

Total costs B Total effects B

Difference in costs: Costs B - Costs A

Difference in effects: Effects B - Effects A

Costs A Effects A

Costs B Effects B

Costs B - Costs A Effects B - Effects AICER:

Intervention A

(e.g. current practice)

Intervention B

(e.g. new medication)

Total costs A Total effects A

Total costs B Total effects B

Difference in costs: Costs B - Costs A

Difference in effects: Effects B - Effects A

Costs A Effects A

Costs B Effects B

Costs B - Costs A Effects B - Effects AICER:

8. Was an incremental analysis of costs and consequences of alternatives performed?

A

B

Effectiveness

Cost

EA EB

CA

CB Costs B Effects B

Costs A Effects A

Costs B - Costs A Effects B - Effects

A

Incremental Cost Effectiveness

Ratio:

Cost Effectiveness

Ratios:

9. Was allowance made for uncertainty in the estimates of costs and consequences?

• Any Sensitivity Analysis?

– One way; two-way; threshold; scenarios; probabilistic SA ?

• Parameter uncertainty?

• Structural uncertainty?

9. Was allowance made for uncertainty in the estimates of costs and consequences?

9.1 If patient level data on costs or consequences were available, were appropriate statistical analyses performed?

9.2 If a sensitivity analysis was employed, was a justification provided for the ranges or distributions of values (for key study parameters), and the form of the sensitivity analysis used?

9.3 Were the conclusions of the study sensitive to the uncertainty in the results, as quantified by the statistical and/or sensitivity analysis?

10 Did the presentation and discussion of study results include all issues of concern to users?

10.1 Were the conclusions of the analysis based on some overall index or ratio of costs to consequences (for example, cost-effectiveness ratio)? If so, was the index interpreted intelligently or in a mechanic fashion?

10.2 Were the results compared with those of others who have investigated the same question? If so, were the allowances made for potential differences in study methodology?

10 Did the presentation and discussion of study results include all issues of concern to users?

10.3 Did the study discuss the generalizability of the results to other settings and patients/client groups?

10.4 Did the study allude to, or take account of, other important factors in the choice or decision under consideration (for example, distribution of cost and consequences, or relevant ethical issues)?

10.5 Did the study discuss issues of implementation, such as the feasibility of adopting the ‘preferred’ programme given existing financial or other constraints, and whether any freed resources could be redeployed to other worthwhile programmes?

• Break 15• Exercise – Critical Appraisal of Economic

Evaluation: 15

References• Drummond M, O’Brien B, Stoddart G, Torrance G. Methods of

Economic Evaluation of Health Care Programmes. 2nd Ed. Oxford University Press. 1997

• Ingeborg et al. Cost effectiveness of physiotherapy, manual therapy, and general practitioner care for neck pain: economic evaluation alongside a randomised controlled trial. BMJ, Apr 2003;326:911

• Mowatt, G., Vale, L., Brazzelli, M., Hernández, R., Murray, A., Scott, N., et al. Systematic review of the effectiveness and cost-effectiveness, and economic evaluation, of myocardial perfusion scinitigraphy for the diagnosis and management of angina and myocardial infarction. Health Technology Assessment 2004 Vol.8 No.30.

• NHS Economic Evaluation Database (NHS EED). National Health Service Centre for Reviews and Dissemination (NHS CRD). University of York. UK. (http://nhscrd.york.ac.uk/nhsdhp.htm)

Outline- Valuing Health Benefits in Outline- Valuing Health Benefits in Monetary TermsMonetary Terms

• Introduction to willingness to pay (WTP)• Introduction to the literature• Methods to value health outcomes in

monetary terms• Advantages/disadvantages• Conclusions

Valuing health outcomesValuing health outcomes

• Measurement of outcomes is central to economics – resources deployed to where they will be of greatest benefit

• How do we define and measure a health benefit?

• How do we quantify benefits with precision?• How do we ensure comparability?

Valuing health outcomes using WTPValuing health outcomes using WTP

• Approach to value non-market goods • WTP is a technique which can potentially be used to

place monetary values on any aspect of health or health care- including the value of human life

• Use survey methods to present respondents with hypothetical scenarios about the programmes/ intervention

• Respondents required to imagine that the market exists• Reveal the maximum they would be willing to pay (WTP)• Attempt to replace missing market

WTP MethodWTP Method

• WTP is a method developed to provide monetary valuation of benefits or outcomes and can be used in cost-benefit-analysis (CBA).

• Respondents can be asked for:– Use value (value to current patients)– Option value (availability should care be needed in

future)– Existence value (Value of any concern that the

programme will be available for others)

How do we ask WTP questions to ensure:How do we ask WTP questions to ensure:

• Acceptability• Plausibility• Consistency• External and internal validity• Avoidance of bias (hypothetical,…...)• Avoidance of embedding (part-whole bias)

and ‘warm glow’ yea saying etc……

Methods of obtaining WTP valuations Methods of obtaining WTP valuations

There are two broad approaches to eliciting WTP values. One method captures values with direct questions (often called an open-ended

format). An alternative method obtains values using binary choices (often called a closed-

ended format).

Techniques for eliciting WTPTechniques for eliciting WTP

• Open ended• Payment card• Take it or leave it (TIOLI)• Dichotomous choice (single, double, triple)• Bidding game • Structured haggling (exclusively used in

developing countries)

Open ended exampleOpen ended example

We are interested in the value you place on X

One way of doing this is to ask how much you would theoretically be WTP for X

What is the maximum amount of money you would be WTP for X? ……………

Payment card/scalePayment card/scale

Put a * next to the amounts that you are sure you would pay

05

1015

Put an X next to all the amountsThat you are sure you would not pay

2025

3035

Put a circle around the maximum amount you would be prepared to pay

50

100

TIOLI/DC exampleTIOLI/DC example

Would you pay £10 for X? Y/N

DC: There would be one or more follow up questions

Bidding game exampleBidding game example

WTP 5000?

Yes NoWTP 5500? WTP 4500?

Yes No Yes NoWTP 6000? Stop Stop WTP 4000?

Yes No Yes NoStop Stop

WTP 6500? WTP 3500?

Yes No Yes No Stop Stop

WTP 7000? WTP 3000?

Yes No Yes NoHow much are you WTP? Stop Stop How much are you WTP?

Advantages/disadvantages of using Advantages/disadvantages of using WTPWTP

• In tune with economic theory (welfare economics)

• WTP feeds consumer (patient or potential patient) preferences into decisions about health care (decision making is more democratic?)

• Value more than ‘health’ and ‘time’

• Acceptability, Validity and reliability of method

• Biases

• WTP and distributional issues- especially if WTP results are heavily influenced by ability

to pay

• If WTP is correlated with ability to pay, preferences of rich people are given greater weight

• However, even if WTP values rise with income, it is not necessarily a problem

• Good practice in WTP studies is to conduct diagnostic tests to see whether preferences depend on income

• If there are systematic differences, one option may be to use distributional weights

Advantages/disadvantages of using WTPAdvantages/disadvantages of using WTP

• Weighting may be applied on grounds other than income- for example, location, age, gender, patient status

(current patient, at risk, not at risk)

• Many potential methodological problems of WTP, some fairly easily avoided, others not

• Protest votes can be a problem but the most obvious potential protest vote – respondents believe they should not have to pay for health or health care – is fairly easily avoided

Advantages/disadvantages of using WTPAdvantages/disadvantages of using WTP

• Respondents are not (usually) asked what they are WTP in personal, out of pocket payments

• The alternatives are voluntary donations but particularly (the respondent’s share of) earmarked taxation

• WTP studies face a choice of trying to gain information through either open ended payment responses or the provision of cost information or payment scales

Advantages/disadvantages of using WTPAdvantages/disadvantages of using WTP

• Failure to give respondents any payment structure invites large non response rates or non credible responses

• However, supply of cost information or use of payment scales can lead to anchoring effects; in particular, WTP responses tend to cluster round the cost figure(s) or the midpoint of payment scale

Advantages/disadvantages of using WTPAdvantages/disadvantages of using WTP

Concluding commentsConcluding comments

• Future for monetary valuation• Experimental method

- different elicitation techniques- issues of validity

• Not necessarily a rival for other methods (Clinical measures, QALYs, DALYs)

Limitations of Economic EvaluationLimitations of Economic Evaluation

• Economic Evaluation techniques, needs to be carefully conducted and cautiously interpreted for a valuable aid to decision making about allocation of health care resources

• Placing monetary values on health and health care, which is necessary if a full blown economic evaluation is to be conducted, is especially problematic

- in methodological terms- more importantly, as to the assumptions made about

the nature of economic behaviour towards health and health care

Limitations of Economic EvaluationLimitations of Economic Evaluation

• Increasingly, evaluation results are treated as accurate, reliable measures of people’s preferences and the process of evaluation has become much more mechanical

• Actual use of Economic Evaluation is quite limited in relation to potentials

• Not possible to undertake economic evaluation for all decisions due to time limitations

• Very little known about decisions-makers attitudes to economic evaluation- future research