1

Monitoring and Evaluation: HIV/AIDS Programs

2

Learning Objectives

At the end of this session, participants will be able to:• Identify M&E implications of the global HIV/AIDS

program context• Identify M&E and information systems implications

of the HIV/AIDS program environment• Apply basic M&E concepts to an HIV/AIDS

program component• Explain the implications of broadened HIV/AIDS

programs, particularly treatment scale up, for monitoring program impact

3

Context

4

Global Summary of the HIV/AIDS

Epidemic December 2004

Number of people living with HIV/AIDS Total 39.4 million (35.9 - 44.3 million) Adults 37.2 million (33.8 - 41.7 million)

Children under 15 years 2.2 million (2.0 - 2.6 million)

People newly infected with HIV in 2003 Total 4.9 million (4.3 - 6.4 million)

Adults 4.3 million (3.7 – 5.7 million)

Children under 15 years 640 000 (570 000 – 750 000)

AIDS deaths in 2003 Total 3.1 million (2.8 – 3.5 million)

Adults 2.6 million (2.3 – 2.9 million)

Children under 15 years 510 000 (460,000 – 600,000)

5

Global Action to Address the HIV/AIDS Epidemic

2000 2001 2003 2004

Word Bank multi-sectoral AIDS Project (MAP)

United Nations General Assembly Special Session on

AIDS (UNGASS)

Global Fund for AIDS, Malaria, and Tuberculosis

U.S. President’s Emergency Plan for AIDS Relief (PEPFAR)

World Health Organization's call to provide treatment to 3 million people by 2005 (WHO 3x5)

(Announced December 1, 2003)

6

GOALS: Global

• Millennium Development Goal 6: Combat HIV/AIDS, malaria, and other diseases

Target 7: Have halted by 2015 and begun to reverse the spread of HIV/AIDS

• WHO 3 by 5 Goal: Universal access to antiretroviral therapy for all living with HIV/AIDS

Target: Treating 3 million people by 2005

7

Goals: US Presidents Emergency Plan

• Prevention of 7 million new infections

• Treat 2 million HIV-infected people

• Care for 10 million HIV-infected individuals and AIDS orphans

8

What Are the Goals of HIV/AIDS Programs?

• Prevent new HIV infections

• Extend and improve life for those already infected with HIV

• Mitigate the social and economic impacts of the epidemic

9

Key HIV/AIDS Program Areas

Prevention• Behavior Change and Communication

– e.g. sexual behavior, condom use, injecting drug use (IDU) behaviors• Medical Interventions

– e.g. PMTCT, VCT, blood safety, universal precautions, STI treatment etc.Care and Treatment

• Care and support to PLWHA and their families• Prophylaxis and treatment of opportunistic infections (including tuberculosis)• Treatment with antiretroviral therapy (ART)

Impact Mitigation• Support to Orphans and Vulnerable Children (OVC)• Reduction of stigma and discrimination• Addressing gender disparities

10

M&E Implications

11

M&E Implications

• High emphasis on accountability– Mandatory’ reporting on international indicators (MDG

and UNGASS)– Donor-reporting requirements linked to large influx of

money

• Drive toward standardization

12

The Three Ones: Principles for the coordination of national AIDS responses

• One agreed-upon HIV/AIDS Action Framework that provides the basis for coordinating the work of all partners.

• One national AIDS authority, with a broad-based multi-sectoral mandate.

• One agreed-upon country-level monitoring and evaluation system

Source: UNAIDS. 2004. Commitment to principles for concerted AIDS action at country level

13

M&E Challenges

• Complex multi-sectoral M&E plans

• Wide range of information needed

• Different approaches needed in concentrated versus generalized epidemics

• Each HIV/AIDS program component has specific M&E needs and challenges

14

M&E Challenges I

ALL COMPONENTS

•Rapid scale-up of new/routine systems

•Denominators – identifying eligible people

•Double counting in service statistics

15

M&E Challenges I

PREVENTION

•Quality of reporting of sensitive behaviors

•Identifying size of most-at-risk population

VCT & PMTCT

•Measuring impact

•Quality of services

•Service cascade

16

M&E Challenges II

CARE AND SUPPORT•Little M&E experience •Often community-based•Minimum care standards•Integration with TB-tracking referrals

ARV•Patient-level tracking systems•Adherence

17

M&E Challenges III

OVC•Little M&E experience•Often community-based•Minimum package of services•Psychosocial support measurement •Ethical & methodological issues in data collection

STIGMA & DISCRIMINATION•Definitions•Measurement tools – validity in different contexts•Selection bias – only disclosed PLWHA observed

18

Applying General M&E Principles to HIV/AIDS Programs

19

General M&E Principles

• Determine how data are to be used• Prepare M&E Plan

– M&E Framework– Indicators– Data sources and data-collection schedule– Evaluation-design / targeted-evaluations needs– Data reporting and utilization plan

• Implement M&E Plan• Use data for program decision-making and reporting

20

Information for Decision Making

• Global Level: Are we achieving global goals?• National Level: What should be the national policy on

PMTCT?• Program Level: Are we distributing services to meet the

need?• Facility Level: Are we providing enough services to meet

the need?• Provider Level: What is quality of a care for this patient?• Community Level: What are we doing as a community to

meet the need?• Individual Level: Where should I get treatment?

21

Frameworks for HIV/AIDS Programs

• Different types of frameworks can be used (e.g. Results framework, log frame)

• Different, inter-related frameworks for different program areas (e.g. VCT, PMTCT, care and support) likely to be needed for a comprehensive program.

• Output of one program activity may be the input to another

• HIV/AIDS frameworks based on relevant documents such as a national AIDS strategy.

22

HIV/AIDS Indicators

Program-level, National- and Global-level Indicators (Guides/Sources)

First-generation• 2000: General HIV/AIDS programsSecond-generation• 2004: HIV/AIDS Care and Support• 2004: Prevention of HIV in infants and young children (PMTCT)• 2004: HIV/AIDS programmes for young people • 2005 update: UNGASS (Millennium Development Goals)• 2005: ARV • 2000-2005: All UNAIDS and partner HIV/AIDS guides• To be released: OVC program guide (FHI), Concentrated

Epidemics M&E Guide, and revised GFATM Toolkit (2005/06)

Information sources for HIV/AIDS M&E

• Document review and key informant interviews– National Composite Policy Index– AIDS Program Effort Index

• Routine program information– Annual condom sales– Providers trained in VCT etc.

• Routine health information systems– No. client visits for VCT etc.– ARV drugs distributed etc.

• Medical records/patient tracking systems– ART adherence– No. patients on ART

• PLACE (Priorities for Local AIDS Control Efforts)– Identification and characteristics of sites where risk behaviors take place– Sexual partnership formation at sites

Information sources for HIV/AIDS M&E

• Facility surveys– Coverage of HIV/AIDS services (facility-based)– Readiness to provide quality HIV/AIDS services– Appropriate STI management

• General Population surveys– Sexual behavior – HIV seroprevalence

• Targeted Population Surveys/Behavioral Surveillance Surveys– Sexual and other risk behaviors – HIV seroprevalence

• Surveillance– HIV seroprevalence

• Vital Registration– AIDS mortality

25

Information Storage: CRIS

Country Response Information System• Purpose:

To enable the systematic – storage– analysis– retrieval– dissemination of collected information on a country’s response to HIV/AIDS

• Structure: Integrated system with 3 modules– Indicator – Project / resource tracking– Research Inventory

Reporting Schematic

Linkages 1 March 15, 2005

National AIDS Council

NASCOP Intervention-specific databases

Persons with HIV/AIDS

Health Facility offering HIV/AIDS related services Community-based

organization offering HIV/AIDS related services

Households affected by HIV/AIDS

Orphans and vulnerable children

DASCO

CACC

DTC

PASCO

Community-Based Routine Information System

HMIS

Routine Program Information (e.g., mass media, training, etc…)

Periodic surveys (DHS, SPA, BSS)

ANC surveillance

Research

Information from Line Ministries

CRIS, MDG, UNAIDS Stakeholders (e.g., funders, implementers, users)

NASCOP: National AIDS and STI Control Program, MOH PASCO: Provincial AIDS and STI Control Officer DASCO: District ASCO DAT: District AIDS Team CACC: Constituency ACC

Reporting from National AIDS Council Information flow

General Population

ARV, OI prophylaxis and treatment

VCT, PMTCT, PEP, STI

27

EXAMPLE: VCT

28

Illustrative Questions for VCT programs

• Are VCT services being provided as planned?• Do services meet minimum quality standards?• Is utilization of services increasing?• Are there reductions in riskier behavior (among those

seeking services)?• Are there increases in use of care, support, and treatment

services (among those seeking services)?

29

30

Illustrative Process and Output Indicators

• Number of people trained in providing VCT according to national and international standards

• Number of people who receive counseling (by sex) • Number of people who receive testing (by sex)• Number of clients who test positive for HIV• Number of HIV+ clients referred to treatment, care, and

support services

31

Percentage of Facilities Providing VCT Services

15

11

0

2

4

6

8

10

12

14

16

%

Ghana (n=428) Uganda (n=226)

Source: 2002 Ghana HIV Service Provision Assessment and 2002 Uganda HIV Service Provision Assessment

32

Percentage of VCT Sites With Selected Inputs for Quality Counseling

2

81

19

21

39

2

47

All items

Condoms in facility

Condoms at service site

Stop AIDS poster

Visual aids for HIV/AIDS

Guidelines

Visual and auditory privacy

Source: 2002 Ghana HIV Service Provision Assessment

Percent

33

Percentage of VCT Sites With Selected Inputs for Quality Counseling

96

52

65

62

63

78

Trained staff

Trained lab technician

Client register

HIV test kits

Visual aids for VCT

VCT guidelines

Percent

Source: 2002 Uganda HIV Service Provision Assessment

34

Illustrative Outcome Indicators

• Percentage of people in the community who know about the VCT services

• Proportion of people counseled and tested who report positive behavior change to avoid HIV infection/transmission

• Proportion of people tested and found to be HIV+ who report positive or negative reactions from others

35

HIV testing among women in Zambia

05

101520253035404550556065707580

Urban Rural All

Desires to be tested Knows source of testing Has been tested

Source: 2001 Zambia DHS

36

HIV testing among men and women in Zambia

05

1015202530354045505560657075

Men Women

Desires to be tested Knows source of testing Has been tested

Source: 2001 Zambia DHS

37

HIV testing experience among sentinel groups in Zambia

0

5

10

15

20

25

30

35

Female sex workers Male long-distance truck drivers

VCTEver tested

Source: BSS, Zambia 2000

38

• Multi-center randomized trial: 1995-1997

• Three sites: Kenya (N=1515), Tanzania (N= 1427) & Trinidad (N=1357)

• Randomized to receive VCT (N=2152) or Health Information (N=2141)

• Traced and interviewed at 6 & 12 months

• Cross-over at 6 months so that the original Health Information group now had access to VCT

Multi-centre study Design of the VCT Efficacy Study

39

Multi-centre study Unprotected intercourse with non-primary partners decreased significantly more among VCT participants

percent

0

10

20

30

40

HI Males

VCTMales

HIFemales

VCTFemales

Baseline 6 Months 12 Months

40

Multi-centre study: Unprotected intercourse with commercial sexual partners decreased significantly more among VCT participants

0

2

4

6

8

10

12

14

HI males

VCT males

HIfemales

VCTfemales

Overall HI

OverallVCT

Baseline

6 months

41

HIV Program Impact

42

Incidence vs. Prevalence

• Incidence = number of new infections in a time period

susceptible population (HIV negative) in the time period

• Prevalence = number of infected people at a given point in time

total population (HIV negative & HIV positive) at that point in time

43

Prevalence, the faucet and sink…

Number of HIV infected people

New HIV Infections

Deaths

44

Impacts Indicators

• Prevent new HIV infections– Percentage of young people aged 15-24 who are HIV-infected (UNGASS and

Millennium Development Goal)– Percentage of HIV-infected infants born to HIV-infected mothers (UNGASS)

• Extend and improve life for those already infected with HIV– AIDS incidence and prevalence– Quality of life measures – Case fatality rate for HIV/AIDS– Proportion of mortality attributed to AIDS– General population life expectancy

• Mitigate the social and economic impacts of the epidemic– Gross national product or other economic indicators– Infant and child mortality rates

45

Impacts: HIV sero-prevalence among 20-24 year old antenatal women in Uganda: 1990-2001

0

5

10

15

20

25

30

35

40

Nsambya Fort Portal Mbale

1990 1991 1992 1993 1994 1995 1996 1997 19981999 2000 2001

Source: STD/AIDS Control Programme, Ministry of Health, Uganda 2002

46

Impacts: HIV prevalence estimates based on ANC sentinel-site surveillance vs. estimates based on population surveys

Estimated HIV Prevalence in General Population Aged 15-49

05

10152025

Kenya Mali SouthAfrica

Zambia

Pe

rce

nta

ge

ANC SurveillancePopulation Survey

Sources: Boerma, et. al. 2003; Central Bureau of Statistics 2004; Republic of Kenya 2001

47

SAVVY - Sample Vital Registration with Verbal Autopsy

• A package:– “Sample vital registration” conducted in surveillance

communities– “Verbal autopsy” conducted as an interview with the family of the

deceased to ascertain the likely cause of death

• Purpose is – to provide measurement of vital events when alternative sources

(vital registration) are not available or not complete– To provide information on the cause of death when death

certificates may not be accurate

• May be used to monitor AIDS related deaths, use of health services, etc.