Resource useResource use

San SebastiánSan Sebastián14 May 200814 May 2008

Pablo Alonso, Victor Montori, Andy Oxman, Holger Schünemann

A balance sheet is a simple but powerful A balance sheet is a simple but powerful way to present the advantages and way to present the advantages and

disadvantages of the management options disadvantages of the management options under considerationunder consideration

What about the costs of What about the costs of compression stockings?compression stockings?

1.1. All the stockings in the 9 trials included in this review were All the stockings in the 9 trials included in this review were below-knee compression stockings.  In four trials the below-knee compression stockings.  In four trials the compression strength was 20-30 mm Hg at the ankle. It compression strength was 20-30 mm Hg at the ankle. It was 10-20 mm Hg in the other four trials. Stockings come was 10-20 mm Hg in the other four trials. Stockings come in different sizes. If a stocking is too tight around the knee in different sizes. If a stocking is too tight around the knee it can prevent essential venous return causing the blood to it can prevent essential venous return causing the blood to pool around the knee. Compression stockings should be pool around the knee. Compression stockings should be fitted properly. A stocking that is too tight could cut into fitted properly. A stocking that is too tight could cut into the skin on a long flight and potentially cause ulceration the skin on a long flight and potentially cause ulceration and increased risk of DVT. Some stockings can be slightly and increased risk of DVT. Some stockings can be slightly thicker than normal leg covering and can be potentially thicker than normal leg covering and can be potentially restrictive with tight foot wear. It is a good idea to wear restrictive with tight foot wear. It is a good idea to wear stockings around the house prior to travel to ensure a stockings around the house prior to travel to ensure a good, comfortable fitting. Stockings were put on 2 to 3 good, comfortable fitting. Stockings were put on 2 to 3 hours before the flight in most of the trials. The availability hours before the flight in most of the trials. The availability and cost of stockings can vary.and cost of stockings can vary.

What about the costs of What about the costs of compression stockings?compression stockings?

1.1. All the stockings in the 9 trials included in this review were All the stockings in the 9 trials included in this review were below-knee compression stockings.  In four trials the below-knee compression stockings.  In four trials the compression strength was 20-30 mm Hg at the ankle. It compression strength was 20-30 mm Hg at the ankle. It was 10-20 mm Hg in the other four trials. Stockings come was 10-20 mm Hg in the other four trials. Stockings come in different sizes. If a stocking is too tight around the knee in different sizes. If a stocking is too tight around the knee it can prevent essential venous return causing the blood to it can prevent essential venous return causing the blood to pool around the knee. Compression stockings should be pool around the knee. Compression stockings should be fitted properly. A stocking that is too tight could cut into fitted properly. A stocking that is too tight could cut into the skin on a long flight and potentially cause ulceration the skin on a long flight and potentially cause ulceration and increased risk of DVT. Some stockings can be slightly and increased risk of DVT. Some stockings can be slightly thicker than normal leg covering and can be potentially thicker than normal leg covering and can be potentially restrictive with tight foot wear. It is a good idea to wear restrictive with tight foot wear. It is a good idea to wear stockings around the house prior to travel to ensure a stockings around the house prior to travel to ensure a good, comfortable fitting. Stockings were put on 2 to 3 good, comfortable fitting. Stockings were put on 2 to 3 hours before the flight in most of the trials. hours before the flight in most of the trials. The The availability and cost of stockings can vary.availability and cost of stockings can vary.

What about the costs of What about the costs of compression stockings?compression stockings?

1.1. All the stockings in the 9 trials included in this review were All the stockings in the 9 trials included in this review were below-knee compression stockings.  In four trials the below-knee compression stockings.  In four trials the compression strength was 20-30 mm Hg at the ankle. It compression strength was 20-30 mm Hg at the ankle. It was 10-20 mm Hg in the other four trials. Stockings come was 10-20 mm Hg in the other four trials. Stockings come in different sizes. If a stocking is too tight around the knee in different sizes. If a stocking is too tight around the knee it can prevent essential venous return causing the blood to it can prevent essential venous return causing the blood to pool around the knee. pool around the knee. Compression stockings should Compression stockings should be fitted properly.be fitted properly. A stocking that is too tight could cut A stocking that is too tight could cut into the skin on a long flight and potentially cause into the skin on a long flight and potentially cause ulceration and increased risk of DVT. Some stockings can ulceration and increased risk of DVT. Some stockings can be slightly thicker than normal leg covering and can be be slightly thicker than normal leg covering and can be potentially restrictive with tight foot wear. It is a good idea potentially restrictive with tight foot wear. It is a good idea to wear stockings around the house prior to travel to to wear stockings around the house prior to travel to ensure a good, comfortable fitting. Stockings were put on ensure a good, comfortable fitting. Stockings were put on 2 to 3 hours before the flight in most of the trials. The 2 to 3 hours before the flight in most of the trials. The availability and cost of stockings can vary.availability and cost of stockings can vary.

BBalance sheets should inform alance sheets should inform judgments about whether the net judgments about whether the net benefits are worth the incremental benefits are worth the incremental

costscosts

Economic models versus Economic models versus balance sheetsbalance sheets

Given the complexity of economic models Given the complexity of economic models and limited resources for constructing these, and limited resources for constructing these, it is not feasible to undertake full economic it is not feasible to undertake full economic models for all recommendations. models for all recommendations.

Moreover, even when economic models are Moreover, even when economic models are used, a balance sheet is still useful for used, a balance sheet is still useful for ensuring decision-makers’ understanding and ensuring decision-makers’ understanding and appraisal of the key estimates used in an appraisal of the key estimates used in an economic model. economic model.

They can also help decision makers to focus They can also help decision makers to focus on the quality of the evidence used in an on the quality of the evidence used in an economic model.economic model.

Important economic consequences Important economic consequences should be included in SoF tables along should be included in SoF tables along

with other important outcomeswith other important outcomes (whether or not an economic model is (whether or not an economic model is

used)used)4 steps4 steps Identify resource consequencesIdentify resource consequences

– Guideline developers must identify resource Guideline developers must identify resource consequences that are potentially important to consequences that are potentially important to decision makers.decision makers.

Find evidenceFind evidence– They must then find evidence of the differences in They must then find evidence of the differences in

resource use for the options being compared. resource use for the options being compared. Appraise the evidenceAppraise the evidence

– They then need to appraise the quality of that They then need to appraise the quality of that evidence.evidence.

Value the resourcesValue the resources– They may then need to value the resources used in They may then need to value the resources used in

monetary terms for the specific setting for which monetary terms for the specific setting for which recommendations are being made.recommendations are being made.

The perspective should be The perspective should be explicitexplicit

The audience may be narrow The audience may be narrow – a single hospital pharmacya single hospital pharmacy– an individual hospitalan individual hospital– a health maintenance organizationa health maintenance organization

Alternatively it could be broadAlternatively it could be broad– a health regiona health region– a countrya country– an international audiencean international audience

A broad perspective is A broad perspective is desirabledesirable

In a publicly funded health system the In a publicly funded health system the patient perspective would ignore most of the patient perspective would ignore most of the costs generatedcosts generated

A pharmacy perspective would ignore down-A pharmacy perspective would ignore down-stream cost savings resulting for adverse stream cost savings resulting for adverse events (e.g. stroke or myocardial infarction) events (e.g. stroke or myocardial infarction) prevented by a drugprevented by a drug

A hospital perspective would ignore out-A hospital perspective would ignore out-patient costs either incurred, or prevented. patient costs either incurred, or prevented.

In the private sector, where disenrollment In the private sector, where disenrollment and loss of insurance can shift the burden of and loss of insurance can shift the burden of costs from one system to another, estimates costs from one system to another, estimates of resource use should include the down-of resource use should include the down-stream costs of all treated patients, not just stream costs of all treated patients, not just those who remain in a particular health plan.those who remain in a particular health plan.

Advantages of a broad Advantages of a broad perspectiveperspective

A comprehensive display of the resource use A comprehensive display of the resource use associated with alternative management associated with alternative management strategies allows an individual or group – a strategies allows an individual or group – a patient, a hospital or a pharmacy – to patient, a hospital or a pharmacy – to examine the relative merits of the examine the relative merits of the alternatives from their particular perspective.alternatives from their particular perspective.

Indirect costs or savings (e.g. lost wages). Indirect costs or savings (e.g. lost wages). – difficult to estimatedifficult to estimate– controversial because they assume that lost controversial because they assume that lost

productivity will not be replaced by an individual productivity will not be replaced by an individual who otherwise would be unemployed or who otherwise would be unemployed or underemployed, and implicitly place lower value on underemployed, and implicitly place lower value on individuals not working (e.g. the retired)individuals not working (e.g. the retired)

Identifying resource Identifying resource consequencesconsequences

1) Consider whether overall resource use 1) Consider whether overall resource use (costs) is important relative to health (costs) is important relative to health outcomes.outcomes.

2) If it is, consider specific types of 2) If it is, consider specific types of resources and consider their relative resources and consider their relative importance; i.e. which should be importance; i.e. which should be included in a summary of findings included in a summary of findings table and which are critical to a table and which are critical to a decision.decision.

Potentially important resource consequencesPotentially important resource consequences

Identifying resource Identifying resource consequencesconsequences

Included resource consequences should be Included resource consequences should be classified as critical or important classified as critical or important

It may be appropriate to aggregate different It may be appropriate to aggregate different types of resource usetypes of resource use– Sometimes necessary due to inadequate reportingSometimes necessary due to inadequate reporting

Consider whether resource consequences are Consider whether resource consequences are potentially important prior to summarizing potentially important prior to summarizing the available evidencethe available evidence– Reconsider those judgments after the available Reconsider those judgments after the available

evidence is summarizedevidence is summarized

Finding economic evidenceFinding economic evidence

Systematic reviewsSystematic reviews Randomized trials Randomized trials

– In or separate from clinical studies In or separate from clinical studies Observational studiesObservational studies National or local databasesNational or local databases

– drug use from prescription databasesdrug use from prescription databases– hospitalizations from hospital databaseshospitalizations from hospital databases

Finding economic evidenceFinding economic evidence

Finding economic evidenceFinding economic evidence

Often guideline developers will not find Often guideline developers will not find sufficient evidence for resource consequences sufficient evidence for resource consequences that they consider importantthat they consider important

For example, WHO guideline on the For example, WHO guideline on the prevention prevention of postpartum haemorrhage of postpartum haemorrhage – Should oral misoprostol be used for all women by Should oral misoprostol be used for all women by

skilled providers to prevent postpartum skilled providers to prevent postpartum haemorrhage instead of IM oxytocin?haemorrhage instead of IM oxytocin?

– Hospitalization, personnel and drugs were Hospitalization, personnel and drugs were considered important resourcesconsidered important resources

– No evidence was found for hospitalization or No evidence was found for hospitalization or personnel personnel

If possible, resource use should If possible, resource use should be presented in natural unitsbe presented in natural units

When only total costs are reported, resource When only total costs are reported, resource use cannot be separated from unit costsuse cannot be separated from unit costs

Without information about resource use it is Without information about resource use it is difficult to make judgments about the difficult to make judgments about the applicability of the evidence to different applicability of the evidence to different settingssettings

Unit prices can be misleading, for example if Unit prices can be misleading, for example if mean costs are used rather than marginal mean costs are used rather than marginal costs (ref), can be difficult to estimate, and costs (ref), can be difficult to estimate, and may vary across settingsmay vary across settings

When global recommendations are made When global recommendations are made different context specific patterns of resource different context specific patterns of resource use can be considered, if relevantuse can be considered, if relevant

Attaching monetary values Attaching monetary values to resource useto resource use

When a recommendation is made in a specific When a recommendation is made in a specific context, attaching appropriate monetary context, attaching appropriate monetary values to resource use can have several values to resource use can have several advantages.advantages.

Monetary values may be easily understood Monetary values may be easily understood by decision makersby decision makers

Attaching monetary values makes it possible Attaching monetary values makes it possible to aggregate different types of resource useto aggregate different types of resource use

The use of appropriate monetary values can The use of appropriate monetary values can help to ensure that resource use is valued help to ensure that resource use is valued consistently and appropriately by decision consistently and appropriately by decision makersmakers

Attaching monetary values to Attaching monetary values to resource useresource use

Use data specific to the context of a Use data specific to the context of a recommendation, providing transparent unit recommendation, providing transparent unit costscosts

Costs should, if possible, take account of Costs should, if possible, take account of severityseverity– For example, to estimate costs for MI of various For example, to estimate costs for MI of various

degrees of severity degrees of severity – Appropriate for nursing and physician time based Appropriate for nursing and physician time based

on severityon severity– Time spent providing care multiplied by an hourly Time spent providing care multiplied by an hourly

wage to arrive at the cost of labor timewage to arrive at the cost of labor time For resources from different health contexts For resources from different health contexts

and timeframes, purchasing power parity and timeframes, purchasing power parity (PPP) exchange rates and inflation factors (PPP) exchange rates and inflation factors should be used should be used

Probably best NOT to discount in SoF tables Probably best NOT to discount in SoF tables

Appraising the quality of Appraising the quality of evidence for resource useevidence for resource use

There are more than 20 checklists and There are more than 20 checklists and instruments for assessing the quality of instruments for assessing the quality of economic analyses in the healthcare economic analyses in the healthcare literature literature

These instruments are mainly related to These instruments are mainly related to the development and reporting of the development and reporting of economic models. economic models.

They are not constructed to assess the They are not constructed to assess the quality of evidence as defined by GRADE quality of evidence as defined by GRADE

Appraising the quality of Appraising the quality of evidence for resource useevidence for resource use

The quality of evidence should be The quality of evidence should be appraised explicitly for each important appraised explicitly for each important resource consequence using the same resource consequence using the same criteria as for health outcomes criteria as for health outcomes

As with health outcomes, in As with health outcomes, in determining the overall quality of determining the overall quality of evidence across outcomes only critical evidence across outcomes only critical resources consequences should be resources consequences should be taken into accounttaken into account

Study limitations (risk of Study limitations (risk of bias)bias)

There may be additional risks of bias There may be additional risks of bias for resource consequences related to for resource consequences related to

The measurement of resource useThe measurement of resource use– Use of unreliable cost data Use of unreliable cost data – Use of cost data from non-comparable Use of cost data from non-comparable

settings should be avoidedsettings should be avoided Follow-upFollow-up Sampling of patients to collect Sampling of patients to collect

resource dataresource data Validation of self-reported dataValidation of self-reported data

InconsistencyInconsistency Judgements about the consistency of estimates of Judgements about the consistency of estimates of

differences in resource use are difficult due to differences in resource use are difficult due to – Poor reporting Poor reporting – Variation can be expected if there are different patterns Variation can be expected if there are different patterns

of resource use in the settings where studies were doneof resource use in the settings where studies were done Generally, directness of the evidence (from the same Generally, directness of the evidence (from the same

or a similar setting) is more likely to be important for or a similar setting) is more likely to be important for resource consequences than for health outcomesresource consequences than for health outcomes

As a consequence of wide variation in patterns of As a consequence of wide variation in patterns of resource use, guideline developers will frequently resource use, guideline developers will frequently choose to focus on the evidence that is most direct, choose to focus on the evidence that is most direct, rather than on an average across different settingsrather than on an average across different settings

When data are available from more than one study When data are available from more than one study from a similar setting, judgements about the from a similar setting, judgements about the consistency of estimates should be based on resource consistency of estimates should be based on resource use rather than on costsuse rather than on costs

IndirectnessIndirectness

Another type of indirectness that is common Another type of indirectness that is common for evidence of resource use is the length of for evidence of resource use is the length of follow-up, since decision makers are often follow-up, since decision makers are often interested in resource use beyond the length interested in resource use beyond the length of follow-up for clinical studiesof follow-up for clinical studies– In particular for chronic diseasesIn particular for chronic diseases

Indirectness of evidence of resource use may Indirectness of evidence of resource use may also result from differences in providersalso result from differences in providers– For example, teaching and research-based For example, teaching and research-based

hospitals have higher costs relative to nonteaching hospitals have higher costs relative to nonteaching hospitalshospitals

Evidence from older studies may be indirect Evidence from older studies may be indirect due to changes in the use of technologiesdue to changes in the use of technologies– For example, decreasing prices for generic drugs For example, decreasing prices for generic drugs

could change prescribing patternscould change prescribing patterns

ImprecisionImprecision

Large variability in healthcare resourceLarge variability in healthcare resource– Larger sample sizes may be required for Larger sample sizes may be required for

resource use data than for health resource use data than for health outcomes outcomes

Clinical trials are often underpowered Clinical trials are often underpowered to detect differences in resource use to detect differences in resource use

Moreover, a number of studies do not Moreover, a number of studies do not report confidence intervals or p-values report confidence intervals or p-values for economic estimatesfor economic estimates

Publication biasPublication bias

Evidence of publication bias for Evidence of publication bias for economic analyses as well as for economic analyses as well as for clinical studies clinical studies

For example, studies of depression For example, studies of depression were more likely to report an economic were more likely to report an economic evaluation if they had larger clinical evaluation if they had larger clinical effectseffects

Using economic models to Using economic models to inform recommendationsinform recommendations

Economic models have advantages Economic models have advantages over balance sheetsover balance sheets– Particularly allowing for more explicit and Particularly allowing for more explicit and

complex sensitivity analysescomplex sensitivity analyses Economic models should be Economic models should be

considered whenconsidered when– Decisions are complex and involve Decisions are complex and involve

multiple important outcomes and resource multiple important outcomes and resource consequences over long periods of time or consequences over long periods of time or

– Large capital investments, such as Large capital investments, such as building new facilities or purchasing new, building new facilities or purchasing new, expensive equipmentexpensive equipment

Using economic models to Using economic models to inform recommendationsinform recommendations

Evidence profiles should be Evidence profiles should be constructed to inform constructed to inform recommendations whether or not an recommendations whether or not an economic model is usedeconomic model is used

The results of economic models should The results of economic models should not be incorporated in evidence not be incorporated in evidence profiles because profiles because – They represent an attempt to analyse They represent an attempt to analyse

health and resource information already health and resource information already summarised in the evidence profile and summarised in the evidence profile and

– They are a mixture of evidence of varying They are a mixture of evidence of varying quality and assumptionsquality and assumptions

The quality of the evidence used in The quality of the evidence used in economic models is rarely explicitly economic models is rarely explicitly

assessedassessed

Sensitivity analyses are used to test Sensitivity analyses are used to test assumptions, but rarely directly take assumptions, but rarely directly take into account the quality of the into account the quality of the evidence evidence

Tables that present the data used to Tables that present the data used to build an economic model build an economic model rarely rarely include assessments of the quality of include assessments of the quality of evidence for each estimate that is evidence for each estimate that is usedused

Cost per quality-adjusted life Cost per quality-adjusted life year (QALY) thresholdsyear (QALY) thresholds

Cost per QALY allows for comparisons across Cost per QALY allows for comparisons across different interventions and conditionsdifferent interventions and conditions

However, it is uncertain how helpful these However, it is uncertain how helpful these estimates are to decision makers or how they estimates are to decision makers or how they are used are used

On the other hand, balance sheets are On the other hand, balance sheets are generally not helpful for comparisons across generally not helpful for comparisons across different interventions in a health systemdifferent interventions in a health system

Thus, if costs per QALY are used, they should Thus, if costs per QALY are used, they should be used cautiously, and decision makers be used cautiously, and decision makers should be presented with evidence profiles should be presented with evidence profiles together with the results of economic modelstogether with the results of economic models

Presenting the results of economic models together with SoF Presenting the results of economic models together with SoF tablestables

When should resource use be When should resource use be presented?presented?

A panel may legitimately choose to A panel may legitimately choose to leave considerations of resource use leave considerations of resource use aside, and offer a recommendation aside, and offer a recommendation solely on the basis of other advantages solely on the basis of other advantages and disadvantages of the alternatives and disadvantages of the alternatives being considered. being considered.

Resource allocation must then be Resource allocation must then be considered at the level of the ultimate considered at the level of the ultimate decision-makerdecision-maker

When should resource use be When should resource use be presented?presented?

If a panel considers resource use it should, If a panel considers resource use it should, prior to bringing cost into the equation, first prior to bringing cost into the equation, first decide on the quality of evidence regarding decide on the quality of evidence regarding other outcomes, and weigh up the other outcomes, and weigh up the advantages and disadvantages. advantages and disadvantages.

Resource implications may be irrelevant ifResource implications may be irrelevant if– evidence of net health benefits is lackingevidence of net health benefits is lacking– advantages of an intervention far outweigh advantages of an intervention far outweigh

disadvantagesdisadvantages Resource use usually becomes important Resource use usually becomes important

when advantages and disadvantages are when advantages and disadvantages are closely balanced.closely balanced.

Even when – as in this case – Even when – as in this case – cost/effect estimates are credible, cost/effect estimates are credible, they do not provide clear answers they do not provide clear answers

regarding appropriate actionregarding appropriate action Most people would consider the cost per Most people would consider the cost per

episode of eclampsia prevented for severe episode of eclampsia prevented for severe pre-eclampsia well worth it.pre-eclampsia well worth it.

For pre-eclampsia that is not severe, and For pre-eclampsia that is not severe, and particularly for low-income countries, the particularly for low-income countries, the decision may be more difficult. decision may be more difficult.

Ultimately, decision-makers would need to Ultimately, decision-makers would need to weigh the relative value of preventing pre-weigh the relative value of preventing pre-eclampsia against the benefits the health eclampsia against the benefits the health system, or society, would forego in allocating system, or society, would forego in allocating resources to magnesium sulphate resources to magnesium sulphate administration.administration.

Strength of recommendationStrength of recommendationThe degree of confidence that the The degree of confidence that the

desirable effects of adherence to a desirable effects of adherence to a recommendation outweigh the recommendation outweigh the undesirable effects. undesirable effects.

Desirable Desirable effectseffects•health benefitshealth benefits•less burdenless burden•savingssavings

Undesirable Undesirable effectseffects•harmsharms•more burdenmore burden•costscosts

Priority setting:Priority setting:From a health system From a health system

or public health or public health perspectiveperspective

A A strongstrong recommendation recommendation may not be may not be importantimportant

Possible criteria for making a Possible criteria for making a recommendation or a decisionrecommendation or a decision

Treatment effect (benefit)Treatment effect (benefit) Adverse effectsAdverse effects Cost (affordability)Cost (affordability) Cost-effectivenessCost-effectiveness EquityEquity Seriousness of the problemSeriousness of the problem Administrative restrictionsAdministrative restrictions

From the perspective of From the perspective of individual patients and individual patients and

cliniciansclinicians Most patients would want and should Most patients would want and should

receive receive strongly recommendedstrongly recommended courses of actioncourses of action

The majority of people would want The majority of people would want weakly recommendedweakly recommended courses of courses of action, but many would not and clinicians action, but many would not and clinicians should be prepared to help patients make should be prepared to help patients make a decision that is consistent with their a decision that is consistent with their own valuesown values

The issue of priority setting (importance) The issue of priority setting (importance) is usually not important.is usually not important.

Should patients with partial epilepsy be Should patients with partial epilepsy be treated with lamotrigine instead to treated with lamotrigine instead to

carbamazepine?carbamazepine?