Using Decision Analysis Modeling to Translate Epidemiologic Research into Health Policy

Elena Losina, PhDHIV Epidemiology and Outcomes Research Program

Massachusetts General HospitalBoston, MA

Supported by NIAID R37 AI42006, R01 AI058736

Outline•

Rationale for using decision analysis modeling in HIV•

Overview of CEPAC model•

Model validation and calibration•

Examples of analyses using CEPAC model–

Life expectancy projections–

Cost-effectiveness of specific treatment strategies–

Informing design of clinical trials•

Challenges–

Need to make decisions in the absence of data•

Policy decisions in developing countries

–

Finding the ‘right’

data–

Prospective collaboration vs. retrospective secondary data analysis

Question

• How can we use epidemiologic, clinical, and economic data to understand current and project future impact and value of strategies for HIV treatment?

What should

be

the standard of care?

Abidjan

Cape Town

Chennai

Vail

Ideal Means of Evaluation

• Evaluations of strategies related to timing and sequencing of antiretroviral agents would require large-scale randomized trials with HIV-related mortality as the primary outcome

Barriers to Design of Such Trials• Feasibility:–

Size

– Recruitment

– Cost

– Time horizon

– Concern about results being outdated

•

INITIO study of initial HIV therapy was designed, enrolled, and conducted over six years. By the time results were presented in 2005, none of the original regimens were considered standard because of new evidence regarding toxicity or potency.

Role of Decision Sciences

• Synthesize data from multiple sources

• Help decision-makers understand the likely impact of different strategies and set priorities for further data acquisition

• Offer a practical framework for managing uncertainty via sensitivity and “what-if”

analyses• Permit analyses that extrapolate beyond the limitations of time horizons, geographic settings, and target populations

Goal to Convince You …

1.

Decision science and cost-effectiveness analysis are critical for informing health policy

2.

Data collected within epidemiologic studies can provide tremendous insight into policy

•

Inputs into probabilistic decision analysis model•

Means of calibration and validation of such models

HIV Clinical Policy Questions•

U.S.–

When should we start ART? Will RCT provide the answers? When?–

How do we identify patients earlier? Is routine HIV testing cost-

effective?

–

What are lifetime costs of HIV-infected patients?–

Should we be doing genotypic resistance testing on all naïve patients before starting ART? At what prevalence of resistance?

–

What is the per-person and population level impact of late initiation and premature discontinuation?

•

International –

What will be the impact on life expectancy, cost and cost-

effectiveness of ART in South Africa? Côte d’Ivoire? India?

–

Are second-line regimens cost-effective?–

When to start?

Cost Analysis

•

HIV Cost and Services Utilization Study (HCSUS) US$18,300/patient/year 1998–

Bozette, N Engl J Med 2001•

Estimated HIV care cost is $618,900 per person/2004 US$ –

Schackman, Med Care 2006, CEPAC, based on HIVRN data•

$9.2 Billion for HIV/AIDS care in low-middle income countries –

Schwartlander, Science 2001

Cost-effectiveness Analysis

• Two different outcome measures– Cost ($)– Effectiveness: Years of life saved (YLS)

QALYs• Cost-effectiveness ratio:

Additional Cost:Additional QALYs

• The value of resources spent

Who Cares?

• All about limited resources

• Need and capability exceed resources–

Individual, state, national, global

• Challenges–

Medicaid

– HDAP

– South Africa Ministry of Health

– PEPFAR

Cost-effectiveness of Preventing AIDS Complications (CEPAC) Model

•

Simulation state transition model of HIV•

Compares clinical outcomes (survival), cost- effectiveness

•

CD4 (true, observed), HIV RNA (true, observed), adherence, resistance, HIV testing

•

Antiretroviral strategies•

Prophylaxis for multiple opportunistic infections

Supported by NIAID, NIMH, NIAAA, CDC

Overview of CEPAC Capabilities

Screening/Intake Screening/Intake ModuleModule

New HIV screening New HIV screening programprogram

Detection via background HIV

Screening

CEPAC-treatment module

Undiagnosed HIV-infected patient

Detection via development of

an OI

The CEPAC Treatment Module

• CD4 cell count (current and nadir)• Resistance assays• Antiretroviral therapy (success, failure, prior regimen failure)• HIV RNA (current and setpoint)• Opportunistic infections (current and prior events) and prophylaxis• Treatment-related toxicity

Acute Clinical

Event

Death

Chronic HIVInfection

Primary HIVInfection

Methods

•

1 million simulations run - Monte Carlo

– Each “patient”

drawn randomly from initial

distribution of age, sex, CD4, RNA–

Track clinical events (e.g. PCP or TB cases)

– Life expectancy, quality-adjusted life expectancy

– Lifetime direct medical costs, total costs

•

Payer or societal perspective ($/QALY)

0

50

100

150

200

250

300

350

400

Time (years)

CD

4 co

unt (

cells

/ul)

Vira

l loa

d (c

opie

s/m

l)

CD4HVL >100,000

30-001-100,000

10,001-30,000

3,001-10,000

501-3,000

21-500

1st line ART2nd-line ART

3rd/4th line ART

10 QALYs: 8.47Total costs: $203,380

5

A Patient “Trace”: US

Bacterial pneumonia

CMV

Death

Data Domains1)

Natural history

2)

Treatment efficacy/effectiveness

3)

Cost

4)

Quality of Life

Where do the data come from?

Natural History: Data Sources•

US:–

MACS–

WIHS–

HIVRN•

Other developed countries–

France•

Developing countries–

India–

Côte d’Ivoire–

South Africa–

OECS

Natural History: Data Elements•

Incidence of specific OIs by CD4 cell count strata

•

Mortality –

by CD4 cell strata–

Acute–

Attribution of history of OI to mortality–

Non-HIV related•

Cost –

Chronic care, by CD4 cell count strata–

Cost of acute care–

Last month of life

Treatment Efficacy/Effectiveness

•

Two-dimensional–

HIV RNA–

CD4•

Time frame–

Early efficacy–

Late failures•

Collaboration with ALLRT (ACTG)•

Independent ART effect•

RCT vs. Actual clinical care

Treatment Efficacy: Time Frame

• ‘Early failure’–

Estimated directly from reported RCTs

– HIV RNA suppression

– CD4 change

• ‘Late failure’–

Assumes exponential rates

– Estimated from two time points beyond ‘initial period’

Dealing with Uncertainty

• Sensitivity analysis–

One and two-way sensitivity analyses

– Wide range of plausible values

– Thresholds for policy conclusions

Model Validation and Calibration

• Validation–

Internal

– External

• Calibration–

Flexibility of the model

– Ability to ‘match’

outcomes from cohorts and

RCTs

Model Validation and Calibration

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35

bcim

bcis

fngm

fngs

tb

mlr

iso

toxo

mac

othm

othsOther SevOther Mild

MACToxo

IsosporaMalaria

TBFungal SevFungal Mild

Bacterial SevBacterial Mild

Prevalence

Data

Model

Internal Validity: ANRS 059, Abidjan

Select Analyses Using the CEPAC US Model

0%10%20%30%40%50%60%70%80%90%

100%

0 5 10 15 20 25 30 35 40

Years from treatment start

% A

live

Untreated HIV/AIDS1989 - PCP1993 - PCP/MAC1996 - PCP/MAC + ART 11998 - PCP/MAC + ART 22000 - PCP/MAC + ART 32003 - PCP/MAC + ART 4

AIDS Survival by Era

Walensky, et al. JID 2007

Life Expectancy by CD4 at ART Initiation and Number of Regimens

0

5

10

15

20

25

30

Very Late

16 14 11

2118

13

2420

14

2621

15

Late

2823

15

Early

Life

Exp

ecta

ncy

(yea

rs)

3 2 1 5 4

# of Regimens

00.10.20.30.40.50.60.70.80.9

1

0 2 6 8 10 12 14 18 24 30 36 42 48 54 60 66 70

Life Expectancy for HIV- and HIV+

Populations in the US

Time (Years)

Surv

ival

Pro

babi

lity

00.10.20.30.40.50.60.70.80.9

1

0 2 6 8 10 12 14 18 24 30 36 42 48 54 60 66 70

Life Expectancy for HIV- and HIV+

Populations in the US

Time (Years)

Surv

ival

Pro

babi

lity

44

HIV negative

00.10.20.30.40.50.60.70.80.9

1

0 2 6 8 10 12 14 18 24 30 36 42 48 54 60 66 70

Life Expectancy for HIV- and HIV+

Populations in the US

Time (Years)

Surv

ival

Pro

babi

lity

44

HIV negative

28

Guideline-Concordant Care

44 –

28 = 16 YLL due to HIV

Years of Life Lost

00.10.20.30.40.50.60.70.80.9

1

0 2 6 8 10 12 14 18 24 30 36 42 48 54 60 66 70

Life Expectancy for HIV- and HIV+

Populations in the US

Time (Years)

Surv

ival

Pro

babi

lity

44

HIV negative

28

Guideline-Concordant Care

44 –

28 = 16 YLL due to HIV

Years of Life Lost

23Actual ART Use

44 –

23 = 21 YLL actual21 –

16 = 5 YLL due tosuboptimal care

Should We Genotype Test Naïve Patients?

CE

RA

tio($

/QA

LY)

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

0 5 10 15 20 25 30 35

% Resistant

Sax, et al. CID 2005

Expanded Testing for HIV: US

StrategyDiscounted

Lifetime Costs

Discounted Life

ExpectancyC-E ratio

(US$/YLS)

Background alone

Background & one-time EIA

Background & EIA every 5 years

Background & EIA every 3 years

Background & annual EIA

32,700

33,800

37,300

38,900

41,700

254

255

256

257

257

---

36,000

50,000

63,000

100,000

Paltiel et al. NEJM 2005

Cost-effectiveness Ratios for Screening Programs

C-E ratioScreening Program

($/QALY)*

Reference

HIV screening inpatients

$38,600 Walensky, Am J Med 2005

Breast cancer screeningAnnual mammogram, women 50–69 y/o

$57,500 Salzmann Ann Intern Med 1997

Colon cancer FOBT + SIG q5y, adults 50–85 y/o $57,700 Frazier JAMA 2000

HIV screening every 3 years High risk patients

$63,000 Paltiel, N Engl

J Med 2005

Diabetes Mellitus, Type 2 Fasting plasma glucose, adults >25 y/o $70,000 CDC C-E Study Grp. JAMA 1998

*all costs adjusted to 2001 US dollars

CEPAC-International: Informing Health Policy Decisions in

Developing Countries

Cost-Effectiveness of ART in Côte d’Ivoire: Single ART Regimen Available

Strategy Discounted Lifetime Costs

Discounted

Life

Expectancy

C-E ratio (US$/YLS)

No treatment 780 31.4 ---

T/S prophylaxis onlyT/S and ART* (start n=2, stop n=1)

8101,230

32.841.4

240590

T/S and ART* (start n=1, stop n=1) 1,720 50.7 620T/S and ART* (start n=1, stop n=3)T/S and ART (CD4-guided criteria)

2,1703,420

56.869.6

8901,180

*ART start/stop decisions based on clinical criteria (no CD4 available)

Goldie et al. NEJM 2006

One-way Sensitivity Analyses

$650 $1,050 $1,450 $1,850 $2,250 $2,650 $3,050 $3,450

OD treatment cost

ART efficacy

CD4 test cost

ART cost

Routine care cost

Cost-Effectiveness ($/Years of life saved)

(Cost x0.5 – 3)

(Cost x0.5 – 4)

(Cost x0.25 – 4)

(↓

20%, ↑

20%)

Base caseGDP

Côte d’Ivoire$708

3 x GDPCôte d’Ivoire

$2,124

$650 $1,050 $1,450 $1,850 $2,250 $2,650 $3,050 $3,450

(Cost x0.5 –

4)

$1,180Côte d’Ivoire

$708

3 x GDPCôte d’Ivoire

$2,124

(Cost x0.5 –

4)

(Cost x0.5 –

2)

(Cost x0.5 –

3)

ART Strategies in India

Strategy for ART Initiation

Mean Lifetime Mean

Survival (Months)

IncrementalC-E Ratio ($/YLS)

No treatment 680 35.2 ---

Costs (2000US$)

One line of ARTNNRTI, start ART < 200/μl 1,600 59.4 450NNRTI, start ART < 350/μl 1,950 66.7 580

NNRTI, start ART < 350/μl or severe OI 2,100 69.1 800Two lines of ART

NNRTI first, ART <350/μl or severe OI 5,470 93.0 1,680PI first, ART <350/μl or severe OI 6,510 96.2 3,920

Freedberg et al. CROI 2007

Informing the Design of Clinical Trials

New Motivation to Address “When to Start”

ART

• Increased stability in ART regimens

• ‘ART effect’

may be greater at higher CD4

• SMART study suggests that AIDS and non-AIDS related events were decreased on ART• Another reason to consider earlier therapy

To ‘Trial’ or Not to ‘Trial’

•

Will results will be obsolete by the time they are available?•

Long enrollment period: are there enough patients to study?•

Is it too expensive given the information to be learned?•

Where should it be done: developed vs. developing countries?

•

Is there enough data to inform the design of a trial?• US

• Developing countries

•

ACTG-CEPAC collaboration•

ACTG 5245/HPTN 052 --

A CEPAC-International Simulation

Question

• Can a simulation model help inform the question re earlier therapy, or the design of the RCT?

Selecting Trial Primary Endpoints: ACTG/CEPAC Collaboration

• Mortality

• Mortality or TB

• Mortality or AIDS-defining event

• Mortality or AIDS

• Mortality or any opportunistic disease

Reaching Mortality EndpointsYear Hazard Ratio

South AfricaHazard RatioCôte d’Ivoire

1 0.24 0.282 0.22 0.223 0.60 0.574 1.07 0.825 0.76 0.856 0.75 0.877 0.70 0.838 0.75 0.869 0.84 0.9410 0.90 0.93

South Africa

0.000.050.100.150.200.25

0 5 10Year

Ann

ual H

azar

d

Côte d’Ivoire

0.000.050.100.150.200.25

0 5 10Year

Ann

ual H

azar

d

Deferred

Immediate

Reaching Mortality or OI Endpoints

Year Hazard RatioSouth Africa

Hazard RatioCôte d’Ivoire

1 0.57 0.532 0.38 0.383 0.73 0.734 1.02 0.925 0.90 0.906 0.86 0.867 0.76 0.828 0.72 0.789 0.75 0.8310 0.81 0.82

South Africa

0.000.050.100.150.200.25

0 5 10Year

Ann

ual H

azar

d

Côte d’Ivoire

0.000.050.100.150.200.25

0 5 10Year

Ann

ual H

azar

d

Deferred

Immediate

Reaching Mortality or OI Endpoints, Omitting TB/Severe Bacterial Events:

South Africa

0.000.050.100.150.200.25

0 5 10Year

Ann

ual H

azar

d

DeferredImmediateDeferred, no TB or Bac SevImmediate, no TB or Bac Sev

Year Hazard RatioBase case

Hazard Ratio (-) TB and Bac

Sev

1 0.57 0.422 0.38 0.363 0.73 0.754 1.02 1.005 0.90 0.846 0.86 0.807 0.76 0.758 0.72 0.749 0.75 0.82

10 0.81 0.88

Main Findings

•

Hazard ratios change over time•

Hazard ratios for immediate therapy are further from 1.0 early and closer to 1.0 later

•

Time dependence of hazard ratios is partially due to:

–

The independent “ART effect”–

Differential incidence of ‘outcomes’

high CD4 cell strata: 200-350, 350-500, >500

•

Including non-CD4-dependent, “non-HIV” events

increases the needed sample size (not shown)

Lessons Learned• Optimal timing of when to start ART is of fundamental importance

• The answer is critical to individual survival, costs of care and global health policy

• The trial feasibility is a function of –

Definition of primary endpoint

– OI incidence in different regions of the world

– Diagnostic criteria

– Prevalence and incidence of HIV

Challenges

• Data are incomplete in many areas

• Models are approximations of reality

• Tension between model complexity and data availability

How to Overcome Challenges

• Prospective collaborations vs. retrospective secondary data analysis

• Participation in the design of study instruments

• Collaborative analyses–

ALLRT

– IeDEA

Conclusions

• Critical questions at the interface of HIV clinical care, epidemiology and policy in every country in the world

• Decision analysis is a method to integrate data from many sources to help inform optimal testing and treatment strategies for HIV

CEPAC Investigators: USMGH/BWHWendy Aaronson, MPHNomita Divi, MScMariam FofanaKenneth Freedberg, MD, MScHeather Hsu Elena Losina, PhDZhigang Lu, MDPaul Sax, MDCallie ScottStacie Waldman, MPHRochelle Walensky, MD, MPHBingxia

Wang, PhD

Harvard SPHSue Goldie, MD, MPHAlethea McCormack, PhDGeorge Seage, PhDMilton Weinstein, PhD

CornellBruce Schackman, PhD

YaleA. David Paltiel, PhD

Supported by NIAID, NIMH, CDC

Lindsey WolfHong Zhang, SM

IndiaN. Kumarasamy, MBBS, PhDTim Flanigan, MDKenneth Mayer, MDAnitha Cecelia, MS

Organization of Eastern Caribbean StatesKathleen Allen-Ferdinand, MDPaul Ricketts, MDHazel Williams-Roberts, MD

CEPAC Investigators: International

Côte d’IvoireXavier Anglaret, MD, PhDEugene Messou, MDCatherine Seyler, MD, PhDSiaka

Touré, MD, MPH South AfricaGlenda Gray, MDNeil Martinson, MBBCh, MPHJames McIntyre, MBChB, MRCOGLerato Mohapi, MBBCHRobin Wood, MDLille, FranceYazdan Yazdanpanah, MD, PhD

Supported by NIAID, NIMH, CDC