Challenges and Opportunities to evaluate Combination Prevention

Marie Laga Institute of Tropical Medicine

AntwerpBelgium

Challenges to evaluate combination Prevention

1. Measuring HIV incidence

2. The complexity of Prevention programs with long causal pathways

3. The scarcity of monitoring data and process evaluation of current Prevention programs

4. Interpreting National HIV trends and attributing changes to Prevention programs

5. The evaluation methods to establish program effectiveness

Measuring change in HIV incidence

• Cohort studies• Lab methods ( BED, STARHS,..)• Proxy’s: prevalence in 15-24y • Estimating incidence from successive

prevalence survey’s• Modelling: EPP, Asian Epidemic model, and

more recent methods

None ideal, most progress with modellingUrgent need to develop HIV incidence assay

The complexity of the combination prevention programs

need for Program impact PathwayHow is the program expected to reduce HIV incidence?

• Spelling out the different steps and make the connections explicit […if..then…]

• Attention to include underlying contextual factors resulting in long and complex causal chains

• Balance between simplifying reality and not being simplistic

• Need for guidelines on methodology and more practical examples

University of Wisconsin-Extension, Program Development and Evaluation

Statewide Tobacco Control: Smoke-free environmentsOUTCOMES

Mgrs of public areas/events

Coalition

Time

Money

Partnersincluding youth

Research and best practices

Organize and implement Smoke-free campaign

Organize and implement strategy for treating tobacco addiction

Public

Youth

Organize and implement strategy to prevent youth tobacco use

Influential others

Demonstrations of support

Reduction in tobacco use and exposure

Change in knowledge, attitude, motivations

Increased knowledge of availability of cessation resources

Increased use of cessation resources

Increased # of quit attempts

SF policies implemented, enforced

Elected officials

Tobacco users• Adults• Youth

Worksite contacts

Parents, schools, etc.

Changes in attitudes and motivations

Increased # of prevention programs, policies adopted, enforcement

Changes in awareness, knowledge and attitudes about SF

Residential owners, mgrs

Increased commitment, support, demand for SF environments

Policy makersIncreased commitment to eliminate access Retailers

Change in behaviors

Poor monitoring of the current prevention response

Is the program implemented according to plan? Coverage? Reach? Quality?

• Data not collected• Data not made available or not used• Coverage estimates: definition of numerator and

denominator (seize estimates of hard to reach population; Human rights issues )

• Definition of minimum quality standards• Missing baselines

7

Understanding National TrendsAdjusted HIV prevalence* among ANC women, by year,

1996-2006-CAMBODIA

1.9

0.9

2.22.42.9

2.5

1.8

2.3

2.72.8

2.62.4

1.81.5

1.3

2.0

2.2

2.8

0.0

0.5

1.0

1.5

2.0

2.5

3.0

1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Per

cen

t

Adjusted, weighted data Adjusted, weighted data smoothed with EPP

Source: Analysis and Advocacy (A2) Thailand Team, Pattaya presentation to ASAP meeting, January 2006.

Understanding National TrendsEstimated number of new HIV infections in Thailand 1988-2008

HIV surveillance is critical data source to evaluate the 100% condom program

0

5

10

15

20

25

30

19891991

19931995

19971999

20012003

Pe

rce

nt

HIV

po

sit

ive

ANC Male STD Direct sex worker Indirect sex worker Military conscripts

Source: Bureau of Epidemiology, Ministry of Public Health Thailand

Evidence of behavioral change in Thailand

Source: Relationships of HIV and STD declines in Thailand to behavioral change: A synthesis of existing studies, UNAIDS 1998

Understanding National trendsGeneralised epidemics

Source: UNAIDS Report on the global AIDS epidemic, 2008

Understanding National Trends: Impact of Prevention

The example of Zimbabwe

1980 1985 1990 1995 2000 2005 20100

2

4

6

8

10

Year

HIV

inci

de

nce

(p

er

10

0p

yar)

Natural decline in incidence ~ 1990

Accelerated decline in incidence, due to behaviour change: ~ 2000

Source: Hallett, Gregson, Gonese, et al., Epidemics, 2009

Evaluation of Public Health programs

Type of evidence

Type of statement Compared to what

Adequacy The expected change occurred (but no causality)

-No control group -Predefined criteria, or absolute or incremental value

Plausibility Program seemed to have effect over & above external influences based on a step by step ruling out of other confounding factors

-A non-random control group (historical, external, internal, simulated)

Probability -The program has an effect -(P<x% that the difference between program & non-program were due to confoundig / bias)

-Randomised control group or cluster

Ref: Habicht et al

• Teacher-led, peer-assisted in- school sexual and reproductive health education

• Youth-friendly sexual and reproductive health services

• Community-based condom promotors and distributors

• Community-wide activities, to create a supportive environment

The evaluation results• no impact on HIV and STI rates

after 3 and 8 years

• In and out of school youth• Parents and community

stakeholders improving their knowledge and changing their attitudes to adolescent reproductive health issues as well as improving their communications skills

• Clinic staff to improve accessibility and acceptability of rural clinics to young people

The evaluation results• no impact on HIV or HSV-2

prevalence• a significant reduction in the

number of reported current or past pregnancies

Tanzania Zimbabwe

Ross D et al, 2007; Cowan et al,2008

So far, ALL community-RCT of behavioural interventions with HIV as endpoints have shown flat results

• The intervention too weak, low adherence?

• The control group not enough different?• The outcome HIV too distal?……

The evidence dilemma

• Good quality RCT require “ tightly defined interventions” preferebly with a short impact pathway

• Combination prevention including “Social movements” advocacy, education, social mobilisation , are impractical to evaluate with RCT because less-well-defined and longer more complex impact pathway

Considerations about RCT in HIV prevention

• RCT essential for biomedical prevention tool evaluation such as microbicides

• RCT usefull for evaluation of well defined program components using intermediate outcomes in the impact pathway

• Balance cost of program implementation and learning while doing through “Convergence of evidence” versus cost of trying to prove the “unprovable”…?

Showing program impact by Convergence of Evidence

Triangulation HIV trends-Behaviour/STI-Program data

• ANC surveillance• DHS or other population based HIV prevalence

data• IBBS• Special studies: context and determinants

“documenting Social tranfsormations” • Attention to Baseline data

Criteria for effectiveness: “Plausible attribution’

• Time , place , person• Context and program factors

• Causality considerations: Bradford Hill criteria

• Modelling to simulate control groups and predict impact

Development of the 100% condom program and key strategic information in Thailand

1984 1986 1988 1990 1992 1994 1996 1998 1999 2000

The first case detected

Effective Drug for STI

Pilot project: the 100% condom from 1 to 3 to 7 provinces

All provinces implemented the 100% condom program since 1992 - Structural/policy intervention -STI and outreach -Condom promotion -Mass media

Routine STD case report and mapping, enumeration of sex establishments and sex workers Routine case report

Small scale HIV sentinel surveillance in selected provinceS

National HIV sentinel surveillance Surveillance

BSS in BKK 1993-1996

Provincial BSS since 1995 and special surveys Behavioral and outcome monitoring

The national survey of partner relations, 1990

The national survey on effectiveness of AIDS media and behavior, 1993

The national evaluation of the 100% condom promotion program, 1997

National cross-sectional surveys

Young men cohort study in the north 1991-1993

FSW cohort studies in the north 1989-1994 in the selected provinces in the north

IDU cohort study1989-1991 in BKK

IDU cohort study 1995-1996 in BKK and Chiang Mai

Cohort studies to monitor incidence

IMPACT OF SEX WORKER PROGRAMS in Benin

MODELLING OF HIV PREVALENCE in SW and General population [INTERVENTIONS OF PROJETS SIDA-1, SIDA-2, AND SIDA-3.]

IMPACT OF SEX WORKER PROGRAMS in Benin

MODELLING OF HIV PREVALENCE in SW and General population [INTERVENTIONS OF PROJETS SIDA-1, SIDA-2, AND SIDA-3.]

HIV among FSWsAssuming migration of FSWs

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1980 1985 1990 1995 2000 2005 2010 2015

Time (year)

HIV

pre

va

len

ce

Observed

No intervention

With Intervention

1st condom use increase

2nd condom use increase

Migration of FSWs

A

HIV among female general populationAssuming migration of FSWs

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

1980 1985 1990 1995 2000 2005 2010 2015

Time (year)

HIV

pre

va

len

ce

Observed

No intervention

With Intervention

B

C

Provided by M. Alary ICASA Report on West Africa, 2009

Combination prevention needs Combination evaluation

The use of a diverse range of data (mixed methods of quantitative and qualitative) and information sources , will likely provide more illuminating, relevant and sensitive evidence of effects than a single “definitive” study

Some conclusions

• Need for better Incidence measurement tools• Need for Program Impact Pathways• Back to the basics: collect and use program

Monitoring data in HIV Prevention evaluation• “Convergence of data sources” and modelling

gets us a long way in Prevention evaluation • The role of C-RCT in documenting efficacy has

been limited by complex nature of combination prevention programs

Some conclusions

• Need to better document Combination Prevention successes to respond the current crisis of confidence

• The glass is half full, but can be filled more