Comparative Effectiveness Research, Personalized

Medicine, and Health Reform

Harold C. Sox, M.D., MACP

Co-chair, the IOM committee for Initial Priority Setting for CER

Editor EmeritusAnnals of Internal Medicine

Personalized Medicine

• The United States Congress defines personalized medicine as "the application of genomic and molecular data to better target the delivery of health care, facilitate the discovery and clinical testing of new products, and help determine a person's predisposition to a particular disease or condition."

Personalized Medicine:

The Health Policy Context

“Seriously, we basically have to solve the health cost problem, or nothing else matters.”

Paul Krugman

NY Times blog on restoring a healthy US economy, September 28, 2009

Cutting Costs: the Senate Finance Bill• Reduce market-basket updates of Medicare

payments to providers.• Reduce subsidies to pre-paid Medicare• Link Medicare payments to quality of care• Reduce Part D subsidies for the wealthy• Independent commission to advise Congress on

Medicare rates.• Reduce Medicare DSH payments.• Initiate Accountable Care Organizations (like a

medical home)

Cutting Costs: Senate Finance

• Create an Innovation Center in CMS– Test strategies for patient-centered care,

reduced costs, and better quality.

• Reduce payment for preventable hospitalizations.

• Increase Part D drug cost rebates

http://www.kff.org/healthreform/sidebyside.cfm

Will current legislation control costs?

• A member of the group, Elizabeth A. McGlynn, associate director of RAND Health, said that her firm’s research showed that the legislation would do more to provide benefits for the uninsured than to change the overall upward trajectory in spending.

• “We are not really seeing a lot of evidence that the trajectory would change very much,” Ms. McGlynn said.

Personalized Medicine

• The United States Congress defines personalized medicine as "the application of genomic and molecular data to better target the delivery of health care, facilitate the discovery and clinical testing of new products, and help determine a person's predisposition to a particular disease or condition."

Comparative Effectiveness Research (CER):

What is it?

Why all the interest?

What drives the costs of health care?

• The availability of expensive technology

• Technological innovation

• High prices

• Uncertainty about effectiveness

• Profit-taking

• Imperfect markets

• Patients need; doctors decide; someone else pays.

What drives the costs of health care?

• The availability of expensive technology

• Technological innovation

• High prices

• Uncertainty about effectiveness

• Profit-taking

• Imperfect markets

• Patients need; doctors decide; someone else pays.

$ 3,922$ 4,439$ 4,940$ 5,444$ 6,304

Per-capitaMedicare Spending1996 2000

Per-capita spending across Per-capita spending across intensity quintiles intensity quintiles

Ratio: High to Low: 1.61 1.58

$ 5,229$ 5.692$ 6,069$ 6,614$ 8,283

What expenditures drive small area variations?

Wennberg. Health Affairs. February 13, 2002Wennberg. Health Affairs. February 13, 2002

A rationale for better evidence

• When the evidence is good, service rates don’t vary across low and high utilization regions.– That should be reassuring.

• When evidence is lacking, rates are higher in regions with high utilization.

• Perhaps—just perhaps—better evidence will reduce unwanted variation in health care practices.

CER in the American Recovery and Reinvestment Act of 2009

• $1.1B for CER research– $400M to NIH– $300M to AHRQ– $400M to the Secretary, DHHS

• Mandated IOM study to establish initial priorities for conditions to study with CER funding.– Due date: June 30, 2009

The IOM Committee’s working definition of CER

The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition, or to improve the delivery of care.

The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.

Source: iom.edu/cerprioritiesSource: iom.edu/cerpriorities

What’s unique about CER?It includes all of the following

• Direct, head-to-head comparisons.• Broad range of topics.

– tests, treatments, strategies for prevention, care delivery and monitoring

• A broad range of beneficiaries: – patients, clinicians, purchasers, and policy

makers.

• Study populations representative of clinical practice

• Focus on patient-centered decision-making– tailor the test or treatment to the specific

characteristics of the patient.

“Patient-centered”

• Suppose a RCT shows that A>B, but many patients got better on B.– Lacking any additional knowledge, you should

prefer A.

• Is it possible that some patients would have done better on B than A?– Can we identify them in advance?

• Demographic predictors• Clinical predictors

The Promise of CER

Information to help doctors and patients make better decisions

2,606 recommended CER topics received from 1758 respondents to web-based questionnaire

IOM Committee’s Voting Process

Round1 Voting = 1,268 nominated topics 200 topics

Round 2 Voting = 145 rank-ordered topics

Committee discusses each topicRound 3 Voting on 155 nominated topics

Round 3 Results = Final 100 priority topics

Figure 5.1 Distribution of the recommended research priorities by primary and secondary research areas

The IOM: the CER program should also:

• Do priority-setting on an ongoing basis.• Have a broadly representative oversight

committee• Engage public participation at all levels of CER• Support large-scale, clinical and administrative

data networks• Do research on dissemination of CER findings• Support research and innovation in the methods

of CER• Expand and support the CER workforce

CER: Senate Finance

• Support comparative effectiveness research by establishing a public-private Center for Comparative Effectiveness Research to conduct, support, and synthesize research on outcomes, effectiveness, and appropriateness of health care services and procedures. – An independent CER Commission will oversee the

activities of the Center. – [E&C Committee amendment: Prohibit use of

comparative effectiveness research findings to deny or ration care or to make coverage decisions in Medicare.]

http://www.kff.org/healthreform/sidebyside.cfm

CER is coming. Everyone has an interest in seeing it succeed

What can you do to help?

Helping CER to succeed

• Learn what CER can do (and what it can’t or won’t do).

• Speak up. Share your knowledge with others.

How could CER improve decision making about

personalized medicine?

Measuring the value of genetic tests

• Genetic markers are tests

• What’s the best way to measure the value of tests?– Diagnostic: predicting current disease status – Prognostic: predicting future outcomes

What do tests do?• Disease detection

– Diagnostic tests– “What is the present state of this patient?”– “What is the probability that this patient has this

disease?”– How to measure: do a cross-sectional study

• Disease prediction– Prognostic tests– “What is the probability that this patient will develop

this disease in the future?”– How to measure: Do a cohort study.

Tests aren’t perfect• They miss disease, and they give false

alarms. • Therefore, we have to interpret them in terms

of probability, not certainty.• The question to ask:

– Diagnostic tests: “how much will the test change the probability that the patient has a disease?”

– Prognostic tests: “how much will the test change the probability that the patient will develop a disease?”

Evaluating diagnostic tests

• Measures of test performance – Sensitivity and specificity

• Sensitivity:

– % of diseased patients with + test

• Specificity:– % of non-diseased patients with - test

Types of test results

Disease present

Disease absent

Test pos

True-positive

False-positive

Test neg

False-negative

True-negative

ND+ ND-

Types of test results

Disease present

Disease absent

Test pos

TP FP

Test neg

FN TN

ND+ ND-

Sensitivity= TP/ND+

Specificity = TN/ND-

Evaluating diagnostic tests

• Sensitivity and specificity do not necessarily imply health effects– Need to measure consequences of test

results

• PET scanning in cancer: a political challenge for Medicare a method for using test performance

measures to estimate health effects

Evaluate studies of test performance

Test sensitivity and specificity

Calculate post-test probability

Does test result change probability enough to change

management?

Evaluate studies of test performance

Test sensitivity and specificity

Calculate post-test probability

Does test result change probability enough to change

management?

YesNoDon’t dotest

Do test

How much does the probability of disease change after a test

result?

• Bayes Theorem:

Post-test odds = pre-test odds x LR

• LR+= sens / (1-spec)

• LR- = (1-sens) / spec

Example: PET scanning to detect scar recurrence of colon cancer

• Is an firm area near the original incision – scar tissue?– a local recurrence of cancer?

• The choice:– Do a biopsy now– Do a PET scan and biopsy if it’s positive.

The effect of PET on management

• Does a negative PET scan lower the probability of recurrence enough to alter the decision to biopsy the mass?– Pre-test probability of recurrence = 0.69– Sensitivity of PET = 0.96– Specificity of PET = 0.98

• Use Bayes’ theorem to calculate post-test probability of recurrence

Post-test odds = pre-test odds x Likelihood Ratio

Post-test probability of recurrent CRC after PET scan of peri-operative scar

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

pre-test probability of recurrence

post-te

st p

robab

ility

of re

curr

ence

p[CRC if PET+]

p[CRC if PET-]

Prognostic tests

• What is the probability that this person will develop diabetes in 10 years?– Age, BP, blood sugar, body weight, TG level,

family history of diabetes, body mass index.

• How much will the probability change if the patient has genetic polymorphisms that predict future diabetes?

Joint Effects of Common Genetic Variants on the Risk for Type 2

Diabetes in U.S. Men and Women of European Ancestry

Cornelis et al. Ann Intern Med. 2009; 150: 541 - 550.

• Genome-wide association studies have identified genetic polymorphisms associated with diabetes mellitus (DM).– Individual variants are weakly associated

• Study questions:– With more polymorphisms, does the risk of

DM increase?– How much does genetic information improve

the prediction of DM compared with clinical information alone?

• Use 2 large cohorts (NHS [1976] and HPFS [1980]) followed through 2002.– Blood collected 23 and 13 years after start.

• Case control design:– Cases: 1297 men and 1612 women who developed DM– Controls: 1338 men and 2163 women without diabetes.

• Tested for 17 SNPs from 13 genetic loci.– Calculated genetic risk score (GRS)

• Tested association of SNP score with development of DM, adjusting for:– Body mass index, exercise, family history of diabetes,

diet

Analysis• Tested whether SNP score predicts the

development of DM, adjusting for:– Predictors of DM: BMI, exercise, FHx, diet

• Calculated area under ROC curve (a measure of discrimination)– Clinical factors only– Clinical factors + GRS

• Area under ROC = probability that someone who gets DM has a higher GRS than someone who does not get DM.

Cornelis, M. C. et. al. Ann Intern Med 2009;150:541-550

Association of reported loci and risk for type 2 diabetes in pooled analysis of men and women

Cornelis, M. C. et. al. Ann Intern Med 2009;150:541-550

Genetic risk score and risk for type 2 diabetes

Cornelis, M. C. et. al. Ann Intern Med 2009;150:541-550

Receiver-operating characteristic curves for type 2 diabetes

Study conclusions• The GRS significantly improved case–control

discrimination beyond that afforded by conventional risk factors, but the magnitude of this improvement was marginal: – Addition of the GRS increased the AUC by only 1%.

• Caveat: given the design of our study, we could not precisely estimate the predictive power of the GRS and were limited to discriminatory analysis.

• Comment: they did not do a net reclassification analysis.– Would show directly how many subjects change risk

category due to genetic information.

Conclusions

• The goal of CER: help doctors and patients make better decisions.

• CER can help measure the extra value of a test– Diagnostic tests: difference in probability of

disease.– Prognostic tests: difference in discrimination

or the probability of getting a disease.

• Better evidence about tests could reduce the cost of health care.

Questions for the future

• Will Congress enact a national CER program?• Will a CER Program promote research to

improve decision making?• Will doctors and patients use the results of

CER?• Will better evidence narrow differences in

utilization rates in high and low geographic areas lower health care costs.

• For which diseases will genetic testing improve prediction of disease susceptibility?