Health and Cost Data

Inputs

Advanced Training in Clinical Research DCEA Lecture 4

February 12, 2009

UCSF Department of Epidemiology and Biostatistics

James G. Kahn, MD, MPHjgkahn@ucsf.edu

Brian Harris, BA, MPP Candidatebrianharris1113@hotmail.com

Lectures

1 - Introduction to Decision Analysis

2 - Consideration of Utility and QALYs

3 - CEA Overview/Developing an Analysis/Costs

Now: Data Inputs

• What type of data do we use to determine– The health inputs that are changing?– The costs that are changing?

• Where do we find these data?

Today’s Objectives:Data Type, Quality & Source

• To Understand the General Issues in Gathering and Presenting Health and Cost Data Inputs

• To Understand Data Sources and Synthesis Methods for Health and Cost Inputs

• To Understand Common Criticisms Surrounding Health and Cost Data Inputs

Lecture Structure

A. Data Type, Quality, & Source1. General Approach2. Hierarchies of Types of Studies3. Health Inputs4. Cost Inputs5. Presentation of Inputs6. Review/References

B. Data Searches7. Resources available at UCSF8. Using PubMed9. Other Sources of Data … the big G

1: General Approach

A. Research Question & Conceptual Model

B. Model Inputs:1. Measures of Health States & Preferences2. Measures of Resource Use

C. Tradeoffs - Refine Model Given Data Availability

D. Best Estimates & Plausible Ranges - Base Case & Range

1-C: Tradeoffs--Refine the Model, Given Data

Availability1. CEA models usually have many inputs

2. Resources are limited a. Your timeb. Your budgetc. Time for decision to be made

3. Therefore, model can be modified to utilize reliable data currently available. Iterative process highlights critical vs. marginal components.

1-D: Best Estimates and Plausible Ranges

1. Best Estimate = Base Case = Baselinea. The most likely value for the input: the

value in the center of the best available data.

b. You can intentionally err in one direction to prove the strength of your result.

2. Plausible Range: similar to 95%, not 99.99%, C.I.

a. When using a single empirical data base, calculate a formal confidence interval (typically + or – 95%).

b. With multiple sources, informal use of best range.

1-Example: Model – To Clip Or Not To Clip

QALYsdisc Costs

No aneurysm rupture0.9825

No surgery21.37 Die

Aneurysm rupture 0.45

0.0175 Survive0.55

No aneurysm ruptureDifference 1

Ä QALYs -1.63 Survive surg.0.902 Die

Aneurysm rupture 0.45

Clipping 0 Survive19.74 0.55

Key Inputs Surgery-induced disabilityRupture risk/yr 0.0005 0.075

Expected life span 35RR rupture w/ surgery 0 Surgical deathSurgical mortality 0.023 0.023Surg morb (disability) 0.075

0.0

Disability, shorter survival

SAH if reach hosp

SAH

$/yr, exp'd life

Surg

none

SAH if reach hosp

SAH

Surg

13.4

Ms. Brooks

13.3

21.4Normal survival,

worry

21.5

4.8

Immediate death

Normal survival,worry

21.4

Normal survival

Normal survival

Early death,worry

Early death

21.5

2: Hierarchies of Types of

StudiesA. Hierarchy used by this Decision Cost-

Effectiveness Analysis CourseB. Hierarchy recommended by the U. S.

Preventive Services Task ForceC. Hierarchy used by Drummond, et al. in

“Methods for the Economic Evaluation of Health Care Programmes”

D. Melded Hierarchy

2-A: DCEA Hierarchy

1. Definitive trials (large randomized controlled trials)

2. Meta-analysis of trials3. Systematic Review4. Smaller Trials5. Large Cohort Studies6. Small Cohort Studies7. Expert Opinion

2-A-1: Definitive Trials

a. Large RCTs (Randomized Controlled Trials)

b. Participants are assigned randomly to treatment group or control group

c. Confounding variables therefore randomly distributed across groups

d. Problems: i. Does study population represent your modeled

group? ii. Clinical trial conditions > real world conditions

2-A-2: Meta Analysis

a. Combines several studies, diluting errors of a single study and increasing statistical power.

b. Problems:i. “File Drawer Syndrome”ii. Unresolvable heterogeneityiii. LeLorier suggests large RCT > Meta iv. Meta-analysis often requires $100,000

c. The Cochrane Library has established useful protocols for Meta Analysis methodology.

2-A-3: Systematic Review

a. Excellent method: Far easier than Meta-Analysis, often with similar results.

b. Careful, critical, structured compendium of studies, summarizing methods & findings.

c. Describe: subjects, study design, outcome measurement, and outcomes.

d. Baseline & range selected by informal weighting of study size and quality.

2-A-3: Systematic Review (CONT.)

e. Most common method of obtaining inputs, often used for several inputs in a CEA.

f. When no definitive study is available (usually the case), this is far better than relying on a single non-definitive report.

g. At least 95% less work than a formal Meta-Analysis.

2-A-4: Smaller Trials

a. Unbiased if well-done, but imprecise due to small size.

b. Point estimate may be wrong, C.I. large.

c. Often used, perhaps because of promising clinical results . . . . Reader beware: recall “File Drawer Syndrome”.

2-A-5: Large Cohort Studies

a. Cohort = persons already exposed to risk factor, treatment, etc., vs. unexposed or untreated control group.

b. No random assignment, therefore selection bias and confounding variables.

c. In untreated populations (when no treatment was available), can be used to estimate outcomes.

2-A-6: Small Cohort Studies

a. Same problems as large cohort studies.

b. Additionally, imprecise due to small size.

2-A-7: Expert Opinion

a. Virtues:i. Absent data, it’s all you’ve got. You need to use something; expert opinion is common for 1-2 inputs in a CEA.ii. Experts do know a lot.iii. Experts are influential & can help you.

b. Problems: they’re just opinions; small sample creates bias or random error.

2-B: U. S. Preventive Services Task Force

Hierarchy1. Randomized Controlled Trials.

2. Cohort Studies.

3. Case-Control Studies.

2-B-3: Case-Control Studies

a. Study and control groups selected on the basis of having or not having the disease.

b. Therefore, retrospective study.c. Problems:

i. Confounding variables hard to identify.ii. Observer bias: health outcome known.iii. Recall bias: imperfect recall.iv. Selection problems similar to cohort

studies.

2-C: Drummond, et al. Hierarchy

1. Randomized Controlled Trials.2. Cohort Studies.3. Case-Control Studies.4. Case Series.

a. No control group.b. E.g.—Heart Transplant in UK: Untreated group considered unethical.c. Common with new procedures.

2-C: Drummond, et al. (cont.)

5. RCTs may lack precision important to economic analysis but not important to clinical outcomes.

6. Precise clinical setting creates unbiased and precise data: high signal to noise (high internal validity).

7. Clinical precision unlikely to be replicated in real world, so results might not be generalizable (low external validity).

2-D: Melded Hierarchy

1. Definitive trials (large RCTs)2. Meta-analysis of trials3. Systematic Review4. Smaller Trials5. Large Cohort Studies6. Small Cohort Studies7. Case-Control Studies8. Case Series9. Expert Opinion

3: Health Inputs

A. Overview

B. Steps

C. How to Find Inputs

3-A: Health Inputs Overview

1. Health State Outcomesa. Relevant Outcome Statesb. Probability Estimates

2. Health Preferences Weightsa. Preference Weights for Outcomesb. Utilities, QALYs

3. Population Characteristicsa. Relevant Populationb. Disease Prevalence in Population

3-B: Health Inputs - Steps

1. List Potential Health Outcome States and Relevant Population

2. Find Data on States & Probabilities - Start With Comprehensive Literature Search

3. Find Data on Utilities

4. Find Data on Population Characteristics

3-B-1 & 2: Health StatesKey Questions:

• What Are the Relevant Health States Over Time for the Disease Under Study?

• When Do These States Occur, and How Long Do They Last?

• What Are the Likely Side Effects or Other Unintended Consequences for Each Group? What Are Key Outcomes of Interest for Relevant Stakeholders, e.g. Patients, Clinicians, Payers, Employers, Policy Makers, Society as a Whole?

• For Which Health States, Are There Credible Estimates?

• Are These Estimates Appropriate for Your RQ (Research Question) and for Your Population?

3-B-1 & 2: Example – Aneurysm Analysis

For the aneurysm analysis, health outcomes were estimated from multiple sources:

• Aneurysm rupture rates large cohort study• SAH case fatality meta-analysis• SAH disability medium cohort study, • meta-analysis• RR mortality with disability small cohort study• Surgical mortality, disability meta-analysis• RR rupture (= 0) expert opinion

(informal)

3-B-3: Preference Weights - Utilities

a. Disease-specific Utilitiesb. Generic Utilitiesc. Key Questions:

i. Do Credible Estimates Exist for Your RQ?

ii. Are the Data Appropriate for Your RQ and Your Model?

iii. Whose Perspective Are You Taking? iv. Disease Specific Ratings v. Community Ratings vs. Patient

Ratings

3-B-4: Population Characteristics

a. Prevalence of the Disease

b. Key Questions:i. What is the Relevant Disease Prevalence? ii. National, Representative Samples iii. Disease Surveillance Databases iv. Integrated Delivery System Databases v. Claims Databases

c. What Competing Risks Exist (Unrelated to RQ)?

d. Do Credible Estimates Exist?

e. Are These Estimates Appropriate for Your RQ?

3-B-4-Example: Population Characteristics - Aneurysm

Analysis

• Prevalence of disease - not needed, not a study about screening or an estimate of total societal costs

• All-cause mortality - very important because of the low risk of aneurysm rupture and hence the high risk of dying before rupture occurs

• Estimated by age and sex from a data base maintained at CDC, available on the internet.

3-C: Health—How to Find Inputs

1. Comprehensive Literature Review

2. Primary Data Collection

3. Commonly Used Health Data Sources

4. Current Recommendations

3-C-1: Comprehensive Literature Review

1. Employ hierarchies presented in Part 2.

2. Always consider relevance to your model.

3. Revise your model when the cost of the data your model requires is greater than the benefit from not revising the model.

3-C-3: Commonly Used Health Data Sources

a. Clinical Trialsb. CMS/VA Databasesc. Disease Registriesd. Quality of Well-Being Index (QWB),

Health Utilities Index (HUI) - www.healthutilities.com/overview.htm

e. Disability/Distress Index, EuroQol Instrument

3-C-4: Current Recommendations*:

US Panel on Cost-effectiveness in Health and Medicine

a. Provide a Reference Case Analysisb. Select Data Inputs From Highest Quality Sources

That Are Relevant to the RQ and the Population

c. Expert Opinion Is Relevant Only When No Other Adequate Data Exists

d. Use QALYs to Incorporate Morbidity and Mortality Into a Single Measure

e. Use Community Preferences for Health Statesf. Perform a Sensitivity Analysis on Inputs

* See Gold MR et al. for Complete Set of Recommendations

4: Cost Inputs

A. Direct Costs

B. Fixed vs. Variable Costs

C. Time Costs

D. Current Recommendations

4-A: Direct Costs

1. Published Estimates

2. Resources Used x Cost per unit Used

3. Cost Data Bases

4-A-1: Published Estimates

a. If available, these are the easiest

b. Should be well-done

c. Health care scenarios should match

d. Often, costs must be updated

4-A-1-b: Should be well done

i. Sources of data used should be documented.

ii. The data used should be appropriate (e.g.—not unadjusted, unnegotiated billed charges)

iii. Algorithms used to process that data should be transparent and credible

4-A-1-c: Health-Care Scenarios Should Match

i. Diagnosis & severity/complications

ii. Age mix

iii. Care practices

iv. Care setting

4-A-1-d: Costs must be Updated

i. Adjust using the medical component of the Consumer Price Index (CPI), maintained by the Bureau of Labor Statistics (BLS)

ii. Substitute particular unit costs with updated values for the same services

iii. Whichever method is used, all costs must be denominated in a single currency adjusted to a single year

4-A-2: Resources Used x Cost per

Unit Used

a. Resources

b. Costs

4-A-2-a: Resources

i. Theoretical model

ii. Empirical evidence—typically better, but more difficult

iii. Micro vs. Macro data

4-A-2-b: Unit Costs

i. Reimbursements

ii. Billed Charges

iii. Cost Accounting Systems

iv. Price References

4-A-2-b-i: Reimbursements

a. Acceptable, especially if based on negotiated rates

b. Medicare reimbursement for inpatient care is based on prices established for DRGs (Diagnostic Related Groups)

c. Excellent approach: RBRVS (resource-based relative value scale) used by Medicare for outpatient services

d. When deductibles and copayments are charged, these should be included when calculating cost for the societal perspective

e. In some databases, “allowed charges” summarize total reimbursement

4-A-2-b-ii: Billed Charges

a. Can be used

b. Must be adjusted with hospital department- specific cost-to-charge ratios

c. Even then, imperfect: a single cost-charge ratio is used for all services in a department

4-A-2-b-iii: Cost-Accounting Systems

a. Costs calculated from time and motion studies

b. Most common system is TSI (Transition System, Inc.)

4-A-2-b-iv: Price References

a. Drugs: “Pharmacy Red Book” of average wholesale prices

b. There are varied publicly available estimates for health worker hourly wages, diagnostic and laboratory equipment, and even for most supplies

4-A-3: Cost Data Bases

a. Record both resources and costs for specific diseases

b. Examples:

i. California Office of Statewide Health Planning & Development (OSHPD) hospital discharge data base

ii. Medical Expenditure Panel Survey

4-A-3-b-i: California OSHPD Hospital

Discharge Data Base

a. Can be used to estimate one inpatient stay for one condition

b. Can also be used to examine multiple inpatient stays over the course of one or more years

4-A-3-b-ii: Medical Expenditure Panel Survey

a. Fielded by the Agency for Health Care Research and Quality

b. Contains longitudinal data for all costs (inpatient, outpatient, etc.) over time

c. This and other longitudinal data sets are well-documented, but complicated and time-consuming to navigate (the first time).

4-B: Fixed vs. Variable Costs

1. Costing methods above implicitly combine fixed and variable costs

2. Sometimes it is most accurate to distinguish between these costs

3. Definitions & Guidelines4. A note on scale5. Fixed cost measurement methods

4-B-3: Definitions & Guidelines

a. Fixed costs do not vary with each unit of service provided; variable costs do.

b. If an intervention is unlikely to require new fixed costs, they can be omitted.

c. In the long run, there are no fixed costs.

4-B-4: A Note on Scale

Including fixed costs in your calculations will increase the average cost per patient when the patient population is held constant.

However, as your patient population increases, the fixed cost per patient will drop.

Therefore, it is sometimes relatively economical to expand the scope of your program (e.g.—include individuals at lower risk). This could affect cost-effectiveness in important ways.

4-B-5: Fixed-Cost Measurement Methods

a. Sometimes, data is available in the literature (e.g., for common diagnostic equipment).

b. Otherwise, you must do an analysis using an institutional cost accounting system.

c. Several methods (single allocation, step-down, multiple step-down, etc.) produce different results.

d. If you must do this, consult or consider hiring an economist or an accountant.

4-C: Time Costs

1. Opportunity cost (to the patient & possibly caregivers) of receiving an intervention

2. Opportunity cost = value of activities foregone = value of lost work & household productivity

3. What time is properly counted?

4. How is time counted & valued?

4-C-3: What Time is Counted

a. Time required for the intervention is counted

b. Time lost due to the illness (care, disability, early death) is not counted: the valuation of this time is captured in the utility assessment, and should not be double-counted

c. Note that “intervention” vs. “illness care” varies across research questions: HIV is “illness” for an HIV prevention program, but part of “intervention” in a CEA on antiretroviral therapy

4-C-4: How is Time Counted & Valued?

a. Time Cost = Time Lost x Cost per Time Unit

b. Time lost is synthesized from estimates of the time required for an intervention

c. Assessment of informal (friends & family) care might require patient interviews

d. Cost per time unit is based on age- and gender-adjusted values from (frequently updated) published tables

e. Published time cost estimates are available for some illnesses

4-Example: Aneurysm Analysis

Cost inputCost input Value (range)Value (range) SourceSource

ClippingClipping $25,150 (18,000-35,000)$25,150 (18,000-35,000) Cohort study – Cohort study – cost accounting cost accounting systemsystem

Moderate/severe Moderate/severe disabilitydisability $20,000/yr (13,000-30,000)$20,000/yr (13,000-30,000) Published Published

estimateestimateSAH hospitalizationSAH hospitalization $47,000 ($33,000-$67,000)$47,000 ($33,000-$67,000) Cohort study – Cohort study –

cost accounting cost accounting systemsystem

Discount rateDiscount rate 3% (0-5)3% (0-5) CEA guidelinesCEA guidelines

Time Costs Not Included - Could be based on the time for surgery and recovery. Assuming one month of lost time, at 10 hours/day and $10/hour = $3,000.

4-D: Current Recommendations*:

US Panel on Cost-effectiveness in Health and Medicine1. Use Societal Perspective for Calculating Resource

Units and Costs

2. Include All “Relevant” Direct Costs

3. Exclude Time (“Productivity”) Costs associated with illness, but include those associated with the intervention

4. Include Variable Costs, but Exclude Fixed Costs

5. Use Incremental Costs Rather Than Average Costs

* See Gold MR et al. for Complete Set of Recommendations

QALYsdisc

Costdisc

No aneurysm rupture0.9825

No surgery21.37 Die$534 Aneurysm rupture 0.45

0.0175 Survive0.55

No aneurysm ruptureDifferences 1

Ä QALYs -1.63 Survive surg.Ä $ $39,132 0.902 Die

Aneurysm rupture 0.45Clipping 0 Survive

19.74 0.55Key Inputs $39,666 Surgery-induced disabilityRupture risk/yr 0.0005 0.075Expected life span 35RR rupture w/ surgery 0 Surgical deathSurgical mortality 0.023 0.023

Surg morb (disability) 0.075Cost of aneurysm hosp 47,000$

21.5

4.8

Immediate death

Normal survival,worry

21.4

Normal survival

Normal survival

Early death,worry

Early death

21.5

13.4

Ms. Brooks

13.3

21.4Normal survival,

worry

-$

23,941$

35,912$

25,150$

Dominant Strategy

$ / QALY49,091$

61,062$

218,700$

25,150$ 0.0

Disability, shorter survival

5-Example: Final Product - Aneurysm Analysis

5-A: Presentation/Transparency*

Input Base Case Value

Range Data Source

Health

Starting State

0.47 0.21 - 0.59CDC 98

Probability 0.9% 0.1 – 2.4% Smith et al.

Costs$1,500 $1.2 – 1.6K

AWP (Red book) 1999

Utilities 0.80 0.75 - 0.85 Jones et al.

Discount Rate

3% 0 – 5%Gold et al.

* Societal Perspective, Funding R01-123456 • Clearly Stated Reference / Base Case• Clearly Stated Data Source, Perspective, and Potential Conflicts of Interest• Sensitivity Analysis Across Range of Values• Clearly Stated Discount Rate

6-A: Common Criticisms

1. Input Estimation Not Credible

2. Analysis Not Valid

3. Structure Not Transparent a. Perspectiveb. Modeling Assumptionsc. Discount Rates

4. Bias a. Funding Sources

6-B: Quality of Literature*1. Increasing Number Over Time – 228 Articles

Reviewed2. 89% Listed Modeling Assumptions3. 83% Described the Comparator Intervention4. 89% Reported Sensitivity Analyses5. 52% Stated Perspective6. 54% Provided Model Diagram7. 64% Listed Discount Rate8. 66% Reported Funding Source9. Methods for Reporting Health and Cost Inputs

Varied Widely

* Neumann PJ et al. The quality of reporting in published cost-utility analyses, 1976-1997. Ann of Intern Med. 2000; 132: 964 – 972.

6-C: Additional References

1. US Preventative Services Task Force, Guide to Clinical Preventative Services, 2nd Edition, 1996

2. Drummond MF et al. Methods for the economic evaluation of health care programmes. NY: Oxford University Press, 1997.

3. LeLorier J et al. Discrepancies between meta-analyses and subsequent large randomized, controlled trials. NEJM 1997; 337: 536 - 542.

4. Light RJ, Pillemer DB. Summing up: the science of reviewing research. Cambridge, MA: Harvard University Press, 1984.

5. Gold MR et al. Cost-effectiveness in Health and Medicine. NY: Oxford University Press, 1996.

6. Neumann PJ et al. The quality of reporting in published cost-utility analyses, 1976-1997. Ann of Intern Med. 2000; 132: 964 – 972.

7: Resources Available at UCSF

A. Galen Digital Library Collections and Resources

B. Galen Clinical Resources Handout

C. Galen Databases

7-B: Galen Clinical Resources

• LIBRARY RESOURCES AND SERVICES– Library Website: GALEN, the Digital Library of UCSF:

www.library.ucsf.edu– Key Clinical Resources:

www.library.ucsf.edu/collres/reflinks/clinical/• PubMed@UCSF: www.library.ucsf.edu/db/pubmed/• MD Consult (textbooks, practice guidelines, and journal

articles):www.library.ucsf.edu/db/

mdconsult.html• AccessMedicine (Harrison’s, Current Diagnosis and

Treatment series): www.library.ucsf.edu/db/accessmedicine.html

• Stat!Ref (textbooks, medical references): www.library.ucsf.edu/db/statref.html

• UpToDate (concise topic reviews): www.library.ucsf.edu/db/uptodate.html

7-B: Galen Clinical Resources (cont.)

• LIBRARY RESOURCES AND SERVICES– Online Journals:

www.library.ucsf.edu/collres/journals/– Contact Us for Reference Assistance:

www.library.ucsf.edu/contact/– Informatics Classes:

www.library.ucsf.edu/edtech/class/

7-B: Galen Clinical Resources (cont.)

• REMOTE ACCESS TO LIBRARY RESOURCES– Information about Remote Access:

its.ucsf.edu/information/network/

• GALEN ACCOUNTS– Required for wireless access to the UCSF

network, printing from Library computers, and use of Library computer labs: www.library.ucsf.edu/info/accounts.html

7-B: Galen Clinical Resources (cont.)

• AFFILIATED HOSPITAL LIBRARIES– San Francisco General Hospital Medical

Center Library: sfghdean.ucsf.edu/barnett/

– UCSF Medical Center at Mount Zion Library: mountzion.ucsfmedicalcenter.org/library/

– Veterans Affairs Medical Center Library: www.sf.med.va.gov/medlib/

7-C: Galen Databases

www.library.ucsf.edu/db/

Over 120 Databases

All have links

All have access descriptions (UCSF, UC, Public, etc.)

8: Using PubMed

A. www.pubmed.gov (or, UCSF link)

B. Search and Sort options

8-B: Search and Sort Options

• Search: topic(s), author(s), journal, year(s), [more]

• Display: summary, brief, abstract, citation, [more]

• Sort by: pub date, first/last author, journal• UC eLinks: full text (when available), UCSF

catalog, MELVYL, InterLibrary Loan, Citation, [more]

• Related Articles• Links: cited articles, cited in PMC, books, link

out (links include full text)

9: Other Information Sources

• Currency Conversion: www.cia.gov/cia/publications/factbook/

• USA Price Series: http://www.bls.gov/

http://research.stlouisfed.org/fred2/data/GDPCTPI.txt

• Acronym Finders: http://www.acronymfinder.com/

http://www.acronymattic.com/

9: Other Information Sources--Google

• Stunningly rich & efficient• Search with several words, and the right

words – by topic & agency• Refine search as see better / other

words / agencies

• If the above fails:Look for external hyperlinks at good sitesUltimate fall-back – email or call a person

at a relevant agency