Unit 9: Evaluating a Public Health Surveillance System

# 1-9-1

Warm Up Questions: Instructions Take five minutes now to try the Unit 9 warm

up questions in your manual.

Please do not compare answers with other participants.

Your answers will not be collected or graded.

We will review your answers at the end of the unit.

# 1-9-2

What You Will Learn

By the end of this unit you should be able to:

list tasks for evaluating a surveillance system

develop a plan for evaluating your own country’s surveillance system

# 1-9-3

Why Evaluate?

Evaluation ensures that your HIV/AIDS surveillance system remains effective as the epidemic shifts over time.

If your system is no longer effective, you will not have the right information to control HIV/AIDS.

# 1-9-4

Purpose of Evaluation

Appraise and prioritise the disease events to be kept under surveillance

Assess how the system can detect and report these diseases

Assess the quality of the epidemiologic information produced

# 1-9-5

Purpose of Evaluation, Cont.

Assess how the system can respond to these diseases

Assess how surveillance results affect disease control and policy

Identify which elements of the system can be enhanced in order to improve the quality of information

# 1-9-6

Figure 9.1. Elements of an Evaluation

Evaluation

Document current state of the surveillance system

Identify strengths and weak points

Recommend improvements

Define training requirements or gaps

Justify resources

Additional funding

More and better training

Improved surveillance for better disease control

possibleoutcomes

# 1-9-7

Six Evaluation Tasks

1. Engage the stakeholders in the evaluation.

2. Describe the surveillance system to be evaluated.

3. Focus the evaluation design.

4. Gather credible evidence regarding the performance of the surveillance system.

# 1-9-8

Six Evaluation Tasks, Cont.

5. Justify and state conclusions and make recommendations.

6. Ensure use of evaluation findings and share lessons learned.

# 1-9-9

For more detail, refer to the Updated Guidelines for Evaluating Public Health Surveillance Systems (Centers for Disease Control and

Prevention, 2001), available at www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm

Task 1: Engaging Stakeholders in the Evaluation

Stakeholders include: Public health practitioners Healthcare providers Data providers and users Community representatives Governments at the district, provincial and

national levels Governmental, non-governmental, professional

and private non-profit organisations

# 1-9-10

Task 1: Engaging Stakeholders in the Evaluation, Cont.

Examples of ways to engage stakeholders include: Hold a community meeting to discuss plans

for the evaluation. Hold one-on-one meetings with the key

people listed above. Invite participants to join the evaluation

team.

# 1-9-11

Class Discussion: Task 1

Task 1 is to ‘engage stakeholders.’

Who would this be, in your case? Why is this task important?

# 1-9-12

Task 2: Describe the Surveillance System to be Evaluated

The evaluation should describe:

The public health importance of the health-related event under surveillance

The purpose and operation of the system

The resources used to operate the system

# 1-9-13

Class Discussion: Task 2

Task 2 is to ‘describe the surveillance system.’

In your case, where would you find the type of information listed in the task?

# 1-9-14

Task 3: Focus the Evaluation Design

Determine the specific purpose of the evaluation.

Identify stakeholders who will receive findings.

Consider what to do with the information generated from the evaluation.

Specify the questions that will be answered.

Determine the standards for assessing the performance of the system.

# 1-9-15

Class Discussion: Task 3

For task 3:

What are some ways you can think of to design an evaluation?

How would you know what to evaluate?

# 1-9-16

Task 4: Gather Credible Evidence Regarding the Performance of the

Surveillance System

Simplicity Flexibility Data quality Acceptability Positive predictive

value

Representativeness Timeliness Stability Data use/

dissemination

Describe the surveillance system in terms of its:

# 1-9-17

Class Discussion: Task 4

Task 4 concerns gathering data.

Should there be a time limit on gathering data?

What would you do if you found that some clinics didn’t have data or had incomplete data?

# 1-9-18

Task 5: State Conclusions and Make Recommendations

Conclusions can be justified through appropriate analysis, synthesis, interpretation and judgment of the gathered evidence.

Recommendations should address modifications and continuations of the public health surveillance system.

# 1-9-19

Class Discussion: Task 5

In task 5, you are asked to make recommendations.

How is this typically done in your district or region?

# 1-9-20

Task 6: Ensure Use of Findings and Share Lessons Learned

Develop strategies for communicating the findings from the evaluation.

Tailor recommendations to relevant audiences.

Recommendations for improvements should be distributed to all partners and sites involved in sentinel surveillance.

# 1-9-21

Class Discussion: Task 6

Task 6 concerns communications.

How would you decide whom to send your final report to?

In your case, would the report go to subject matter experts, perhaps Ministry of Health HIV/AIDS experts?

If not, how could you make sure that the people reading the report would understand what you were saying?

# 1-9-22

In Summary Evaluating your HIV/AIDS surveillance system is

important, to ensure that remains effective as the epidemic changes over time.

The tasks that you should implement during this evaluation include: engaging stakeholders describing the surveillance system focusing the evaluation design gathering evidence on the system’s performance stating conclusions and recommendations sharing the lessons learned

# 1-9-23

Warm Up Review

Take a few minutes now to look back at your answers to the warm up questions at the beginning of the unit.

Make any changes you want to.

We will discuss the questions and answers in a few minutes.

# 1-9-24

Answers to Warm Up Questions

1. List three stakeholder groups that should be engaged during the evaluation of the surveillance system.

# 1-9-25

Answers to Warm Up Questions

1. List three stakeholder groups that should be engaged during the evaluation of the surveillance system.

public health practitioners healthcare providers government officials representatives of affected communities non-profit and donor organizations etc.

# 1-9-26

Answers to Warm Up Questions, Cont.

2. If there is a high probability that cases identified by the surveillance system are actually cases of HIV infection, the system is said to have high:

a. sensitivityb. representativenessc. acceptabilityd. positive predictive value

# 1-9-27

Answers to Warm Up Questions, Cont.

2. If there is a high probability that cases identified by the surveillance system are actually cases of HIV infection, the system is said to have high:

a. sensitivityb. representativenessc. acceptabilityd. positive predictive value

# 1-9-28

Small Group Discussion: Instructions

Get into small groups to discuss these questions.

Choose a speaker for your group who will report back to the class.

# 1-9-29

Small Group Reports

Select one member from your group to present your answers.

Discuss with the rest of the class.

#1-9-30

Case Study: Instructions

Try this case study individually.

We’ll discuss the answers in class.

# 1-9-31

Case Study Review

Follow along as we go over the case study in class.

Discuss your answers with the rest of the class.

#1-9-32

Questions, Process Check

Do you have any questions on the information we just covered?

Are you happy with how we worked on Unit 9?

Do you want to try something different that will help the group?

# 1-9-33