DEVELOPING A MONITORING AND EVALUATION FRAMEWORK … · training support, psychosocial support and...

DEVELOPING A MONITORING AND EVALUATION FRAMEWORK FOR

IMPLEMENTING OPERATION CHILD SUPPORT ACTIVITIES AT REACH OUT

MBUYA PARISH HIV/AIDS INITIATIVES

FELLOW: MASABA DAVID WANALOBI BA (ARTS)

MA. DEVELOPMENT STUDIES

January 2010

ii

Table of Contents

Table of Contents .......................................................................................................................... ii List of figures ................................................................................................................................ iv

Figure 1: ROM Organization Hierarchy ................................................................................... iv Figure 2: Conceptual framework ............................................................................................... iv Figure 3: OVC Support module ................................................................................................. iv Figure 4 Process of developing the module ............................................................................... iv Figure 5 supported children’s unique identification numbers .................................................... iv

Declaration..................................................................................................................................... v Declaration..................................................................................................................................... v Acknowledgments ........................................................................................................................ vi List of Acronyms ......................................................................................................................... vii Executive Summary ................................................................................................................... viii 1.0 Introduction and Background ............................................................................................... 1

1.1 Introduction ........................................................................................................................... 1 1.2 Background ........................................................................................................................... 2

Reach Out Mbuya .................................................................................................................... 2 2.0 Literature review .................................................................................................................... 4

2.1 M&E framework ................................................................................................................... 4 2.2 Why M&E ............................................................................................................................. 4 2.3 Constraints ............................................................................................................................. 4 2.4 Human resource capacity ...................................................................................................... 5 2.5 Tools, data storage and use ................................................................................................... 5

3.0 Statement of the problem, justification, conceptual framework ........................................ 6 3.1 Statement of the problem ...................................................................................................... 6 3.2 Justification ........................................................................................................................... 6 3.3 Conceptual framework for developing an M&E framework ................................................ 6

4.0 Project objectives .................................................................................................................... 8 4.1 General objective ................................................................................................................... 8 4.2 Specific objectives ................................................................................................................. 8

5.0 Methodology ............................................................................................................................ 8 5.1 Project implementation .......................................................................................................... 9

5.1.1 Developing a Monitoring and Evaluation framework ........................................................ 9 5.1.2 Developing and selecting suitable performance indicators .............................................. 11 5.1.3 Reviewing existing data collection tools .......................................................................... 11 5.1.4 Training staff in the implementation of the framework ................................................... 12 5.1.5 Developing an electronic Relational Database ................................................................. 12

6.0 Ethical considerations ........................................................................................................... 14 7.0 Dissemination ........................................................................................................................ 14

iii

8.0 Sustainability of the project ................................................................................................. 14 References .................................................................................................................................... 15 Appendix I ................................................................................................................................... 17 Appendix II .................................................................................................................................. 24

iv

List of figures

Figure 1: ROM Organization Hierarchy

Figure 2: Conceptual framework

Figure 3: OVC Support module

Figure 4 Process of developing the module

Figure 5 supported children’s unique identification numbers

v

Declaration

I Masaba David Wanalobi do hereby declare that this programmatic report titled DEVELOPING A

MONITORING AND EVALUATION FRAMEWORK FOR IMPLEMENTING OPERATION

CHILD SUPPORT ACTIVITIES AT REACH OUT MBUYA PARISH HIV/AIDS

INITIATIVES has been prepared and submitted in fulfillment of the requirements of the MUSPH-CDC

HIV/AIDS Fellowship Program and has not been submitted for any academic qualification.

Signed ……………………………………………. Date……………………………….. Masaba David Wanalobi, Fellow Signed………………………………………….. …Date ………………………………… Dr. Alamo Stella-Talisuna Host Mentor Signed………………………………………….. Date ……………………………….. Dr. John Ssempebwa Academic mentor

vi

Acknowledgments

I acknowledge the support provided to me by my mentors: Dr. Alamo Stella-Talisuna, Mr.

Ssebyatika Elly, Mr. Wamara Moses and Dr. John Ssempebwa (academic mentor) for the

guidance and technical support provided to me while preparing this proposal.

I am indebted to the Fellowship programme office for the support visits.

Special thanks to all Operation Child Support (OCS) team for supporting me in activities at the

host institution.

Special regards to all the ROM staff for the support they rendered to me.

And my fellow Fellows for the constant support they render to me.

vii

List of Acronyms

AIDS Acquired Immune-Deficiency Syndrome

FBO Faith Based Organization

HIV Human Immunodeficiency Virus

M&E Monitoring and Evaluation

NSP National Strategic Plan for HIV/AIDS activities in Uganda

OCS Operation Child Support

OSF Operation School Fees

OVC Orphans and Vulnerable Children

PHA People Living With HIV/AIDS

PMMP Performance Measurement and Management Plan

ROM Reach Out Mbuya

Ss Social Support

TWG Technical Working Group

UAC Uganda AIDS Commission

UNAIDS The Joint United Nations Program on HIV/AIDS

WHO World Health Organization

viii

Executive Summary To date Reach Out Mbuya (ROM) is supporting over 1,000 OVC, through its OVC project

Known as Operations Child Support (OCS). The OCS project provides education and vocational

training support, psychosocial support and Child Protection. There is an M&E department at

ROM whose main role is to design and implement the M&E strategy involving: guiding strategic

planning, articulation of the information needs and data collection system, report writing,

monitoring and evaluating programme progress in all departments and ensuring continuous

quality improvement.

Although ROM has an M&E system with most of its components functional, a situational

analysis conducted in April 2009 as part of the Fellowship project showed some need for

improvement Some of the challenges found were attributed to the sudden increase in the number

of OVC supported and activities involved in. Among the gaps identified were difficulties of

collecting accurate data on the actual number of OVC supported, M&E functions were not

integrated into the OVC program, there was limited involvement and appreciation of the

significance of M&E by the OVC project implementers(Staff), indicators that existed lacked

Means of Verification (MOV), there was limited documentation of goals and objectives and lack

of a uniform understanding of the problem that the OVC project was addressing and the

strategies being used to address the problem . All these factors affected the effectiveness of the

decision making process, decisions were often based on haphazardly collected field

information. Therefore this project was undertaken to create a common frame work that will

ensure guided comprehensive data collection, supportive trucking of project progress,

compliance to set standards, efficiency and stakeholders involvement in M&E for enhanced

program performance and organizational learning.

The methodology and approach used in the implementation of this project were participatory and

phased as it entailed wide consultation at all stages. Project meetings, discussions and mini-

workshops were held involving the Executive Director, senior managers, team leaders and OCS

staff members. These consultations were aimed at enhancing ownership that would in effect

bring effective accountability at the stage of implementation. The first phase focused on

ix

developing an M&E framework that clearly defines goals and objectives of OCS interventions in

order to structure the M&E process. A log frame was developed, indicators identified,

assumptions and means of verification defined. A project Monitoring and Evaluation Plan was

developed based on the critical M&E questions in planning of (what, why, when, for whom, who

and how).

Following the project implementation the Fellow learnt that stakeholders, especially program

implementers must be involved in M&E management for they uphold the functioning of the

M&E system. For organizational effectiveness and efficiency the M&E functions should be

integrated into all relevant program areas and the M&E unit oversees the entire M&E system of

the organization and it is important to clearly define how the M&E system operates

1

1.0 Introduction and Background

1.1 Introduction In Uganda, the number of children under age 18 who have lost one or both parents stands at 2.3

million of which approximately 46% are due to HIV/AIDS (UNAIDS, UNICEF & USAID,

2004). Many more children live with one or more chronically ill parents.

Children who have lost one or both parents to HIV face challenges in their long-term health. In

addition to having an increased risk of illness or injury, orphaned and vulnerable children (OVC)

often receive inadequate food or shelter, may live with caregivers that ignore, exploit, or abuse

them, and have to cope with the trauma of seeing their parents get sick and die (USAID, 2007).

Despite recognition of the magnitude and negative consequences of this problem, there is little

evidence on “what works and what does not work” to improve the well-being of children

affected by HIV/AIDS. Given the lack of information on the impact of care and support

strategies for OVC, there is an urgent need to learn more about how to improve the effectiveness,

quality, and reach of these efforts. (USAID, 2007)

In an attempt to fill these knowledge gaps, the government of Uganda in partnership with other

stakeholders developed a country-level monitoring and evaluation system, with the formulation

of the National Performance Measurement and Management Plan (PMMP). This replaced the

National M&E framework which was ridden with short-comings (UAC, 2007).

The PMMP for the National Strategic Plan (NSP) for HIV/AIDS activities 2007/08 to 2011/12 is

a major step in ensuring that there is one country-level monitoring and evaluation system. The

PMMP outlines how the Government of Uganda plans to track the performance of the NSP.

The goal of the PMMP is to ensure collection and reporting of all national level HIV/AIDS

indicators. The purpose of the PMMP is to guide coordinated and efficient collection,

collation, analysis, interpretation and dissemination of information for HIV/AIDS

programs. This PMMP is designed to serve as a guide for baseline and subsequent annual

reports on national and district indicators for HIV/AIDS in Uganda.

2

Monitoring will be carried out through sector management information systems and

evaluations will include the Uganda Demographic and Health Survey, AIDS Indicator

Survey, Antenatal clinic sentinel surveillance, Most-At-Risk-Population Surveys, cohort

analyze of patients on ARVs as well as longitudinal studies for determination of HIV

incidence.

There are 53 national level indicators in the National HIV/AIDS PMMP. Of these, there are

4 OVC support indicators (indicators Nos. 39, 40, 41 and 43) and these are: percentage of

OVC whose households received emotional support in the past 12 months, percentage of

disadvantaged groups that have received vocational training the last 12 months, ratio of

current school attendance among orphans to that among non-orphans aged 6-14 in the past

year and percentage of OVC whose households received free basic external support in

caring for the children in the last 12 months (UAC, 2007).

1.2 Background

Reach Out Mbuya

Reach Out Mbuya (ROM) parish HIV/AIDS Initiatives is a faith based non governmental

organization under our lady of Africa Church, Mbuya Parish, Kampala District. Since its

inception in 2001 ROM has provided a holistic model of care with interventions designed to

target the mind, body, soul and the community to 3100 people living with HIV and over 1000

children infected and affected by HIV. ROM currently serves a targeted population of 60,000

poor residents of seven communities of Nakawa, Giza Giza, Banda, Acholi quarters, Kinawataka

1 and 2 and has more recently expanded to Kasaala in Luwero (ROM, 2008).

As a result of HIV and poverty, many clients of ROM are unable to keep their children in school.

Those who pass on leave children more vulnerable, such vulnerability leads to children being

stigmatized and neglected. To reduce this vulnerability, the OVC project, known as operation

Child Support (OCS) was started and this is a component of the social support department

activities. The social support department is one of the six departments that make up ROM, see

organogram in figure 1.

3

Figure 1: ROM Organization Hierarchy

This project was implemented under the social support department, which houses the OVC

project, known as OCS, under OCS we have the education support section known as operation

school fees (OSF) and the Psychosocial support section.

OCS has been in operation for 8 years. This project provides education support, material support,

child monitoring at school, home visits, and community based child friendly recreation activities,

psychosocial support and child protection. However, since the inception of this project, there

have been difficulties of collecting accurate data on the actual number of OVC supported, M&E

functions were not integrated into this project, there was limited involvement of implementers

(Staff) in M&E operations, the project implementers never appreciated the significance of

M&E, indicators that existed lacked Means of Verification (MOV), there was limited

documentation of goals and objectives and lack of a uniform understanding of the problem that

the OVC project was addressing and the strategies being used to address the problem . To

resolve the above challenges, the Fellow gave support to the OCS project team particularly to

streamline the M&E issues in the section to enable evidence based planning and decision making

using reliable and well articulated data.

Executive Director

Bread of Life (Micro finance)

M&E Dept

Roses of Mbuya (tailoring)

Operation Child Support Section

Operation School Fees Psycho-Social Support

Finance Dept

E .D’s Dept

Social Support Department

Medical Dept

Community Dept

4

2.0 Literature review

2.1 M&E framework A Monitoring and Evaluation framework is a tool that provides a roadmap for how the M&E

process will be conducted over the life of the project. Few countries in the world today have

effective and comprehensive systems in place to gather this data (WHO, 2004).

2.2 Why M&E

With the availability of resources, more Programs targeting OVC are being implemented. This

calls for increased efforts to monitor and evaluate these efforts at various levels to ensure the cost

effective use of these resources. Decision-makers need to be able to measure whether policies

and programs are working, and whether progress is being made towards set goals that have been

set. Donors are also placing more emphasis on performance, linking the release of funds to

performance based measures. In all this data/information is central and forms the basis of

decision making. Decision-making is critically dependent on the timely availability of sound

data/information (Zahr, 2005). Therefore a framework to guide collection, analysis, use and the

provision of information that will enable tracking successful interventions or best practices, as

well as constraints to OVC interventions is key to the sustainability of programs.

Investment in M&E can have multiple benefits, including: supporting evidence based planning

and allocating resources, enhancing transparency and helping to demonstrate results as part of

accountability to key stakeholders (WHO, 2004).

2.3 Constraints

Organizations and managers are often aware that in order to be effective they need to know on a

regular basis how well their organizations are doing; in reality, however, they often base

decisions on personal and staff judgment, anecdotal data, or haphazardly collected field

information. An assessment conducted by Ministry of Gender Labour and social Development

(MGLSD) in 2005 to establish OVC data needs and indicators being used to monitor OVC

interventions revealed that managers commonly tend to place less emphasis on monitoring and

evaluation because they perceive measuring performance as complex and time-intensive, and

they do not see a benefit to investing in M&E systems. Many organizations and their

5

management consider M&E to be a requirement of the funding agencies that support them,

because they required organizations to provide M&E plans for their proposed projects, an

external rather than internal necessity (MGLSD 2005). For instance, few managers consider

M&E to be a strategic system that they can adapt to assess their organizational capacity, judge

their economic effectiveness, or predict their organization’s future sustainability (McCoy et al.,

2005). Increasingly, very few resources are committed to capacity building and where they are

provided, it mostly target skills related to management and finance handling. Such factors and

many others, inhibit good M&E practices and improvement of project effectiveness and

efficiency.

2.4 Human resource capacity

One of the most important aspects of M&E is to ensure that the people in charge of M&E are

skilled enough to report accurately. Organizations that have interest in training health service

managers also recognize the need for more skills in M&E (Gladwin, 2000). Training should

therefore be a priority to ensure that the personnel have the necessary skills to capture data

(WHO, 2005). The staff directly involved in the generation of the data should be informed of the

importance of having accurate information as this will enable proper planning and

accountability.

2.5 Tools, data storage and use

One of the temptations in M&E is to get carried away with the technical aspects of data

collection, storage and analysis, rather than focusing on the more fundamental issues of making

sure the data is correct in the first place. This entails having in place correct tools for capturing

data first. There is no doubt that computerization of data storage and analysis has sped up data

handling considerably, but it has also drawn attention away from other more critical, and more

challenging aspects of M&E, such as coding and personnel capacity. In terms of data use,

countries have been consistently poor at supporting workers who are collecting data to use it

locally for planning and management purposes. Some observers speculate that facilitating greater

local use of data could improve data quality overall, as those doing the data collection should.

The developing countries and organizations should take action to strengthen M&E using the

available ICT. Though this due to poor economic and communication infrastructure, have been

6

limited to the national and higher levels, there should be effort to extend this to the lower

facilities (Simba, 2004).

3.0 Statement of the problem, justification, conceptual framework

3.1 Statement of the problem The orphans and vulnerable children’s project at ROM has been in operation for 8 years. However, since its inception there has been no guide that clearly defines what data needs to be collected, why the data is being collected, when the data should be collected, for whom the data needs to be collected (consumer), Who is responsible for data collection and how this data is reported). This has hindered the capacity of M&E to quantify the impact that the OVC project has had on the target population, justifying the results of the OVC projects interventions and the existence of the OVC project. This situation is further complicated by the existence of multiple funding partners, each demanding different data for different needs.

3.2 Justification The M&E framework based on the critical M&E questions in planning of (what, why, when, for

whom, who and how) will work to:

• Guide and align donor support to ROM priorities

• Outline how ROM plans to track the performance of its OVC project intervention,

• Provide guidance to the implementing staff in developing annual operations plans and

aligning them to ROM goals and targets

• Act as an instrument to assist with the mobilization and allocation of resources to the

OVC project

The framework intends to set out for ROM staff the tools and formats for the monitoring and

evaluation of the OCS interventions. It focuses on partner and internal reporting and some

additional instruments to track impact.

3.3 Conceptual framework for developing an M&E framework The Conceptual framework for developing an M&E framework for implementing OCS

interventions was based on the results chain model.

One level of objective should lead to the next level of objective as follows:

7

Figure 2: Conceptual framework for developing an M&E framework for implementing OCS interventions

‐National OVC policy, NSPPI, NSP, PMMP and ROM strategic plan -M&E technical working group & resource persons - Other relevant OVC and M&E materials -Energies - Financial resources

-Review existing documents

-Review the ROM OVC project

-Facilitate the establishment of the M&E

technical working group

- Develop and gain consensus on results

frame matrix & Indicators

- Review and where necessary design

new appropriate data collection tools

Ongoing discussions and feedback with

the M&E technical working group to

agree on content, draft documents,

materials and emerging issues

-Consultations & Consensus building

meeting

-Review workshops

-Train staff

-Develop an operational manual

M&E framework

focusing on

education, child

protection and

psychosocial

interventions at

ROM.

Inputs Processes Outputs

Effective and efficient

collection, analysis, use

and provision of

information that will

enable tracking

successful interventions

or best practices, as well

as constraints to OVC

interventions.

Improved quality of ROM OVC programming

Outcomes Impact

4.0 Project objectives

4.1 General objective To develop a comprehensive Monitoring and Evaluation framework that will effectively guide

data collection, analysis, use and the provision of information that will lead to improved quality

of OVC programming at ROM.

4.2 Specific objectives To develop a Monitoring and Evaluation framework that clearly defines goals and objectives of

Operation Child Support interventions.

1. To develop and select suitable performance indicators for output and outcomes of the

Operation Child Support interventions.

2. To review existing tools and make suggestions on how best they can be improved and

aligned to the organizations goals and objectives

3. To train staff in the implementation of the framework

5.0 Methodology Project implementation mainly followed a participatory approach involving all the relevant

stakeholders. The project implementation methods included:

1. Technical discussions with the technical working group (TWG), there was on going

consultation with the academic mentor who helped the Fellow to conceptualize the

project idea, Host mentors and some of the senior managers at ROM.

2. Worked through a project team: The Fellow worked with a project team which comprised

of the M&E Coordinator, Social support Coordinator, OCS Team Leaders and Assistants.

The team members were vital in guiding the activities under their respective areas of

specialty.

3. Used a phased implementation strategy/approach: The project was implemented

following a two phased approach i.e. in the initial stages an M&E framework was

developed and in the later stages of the project the capacity of the OVC section staff was

built in the use of the framework through training.

9

4. Mini-workshop, meetings and discussions: a mini-workshop was held with the senior

Managers to review structure, sections, content, flow, format, volume and consistency of

the draft M&E Framework.

5. Worked with an external facilitator: to guide the project team through the process of

developing the Log frame and the M&E matrix.

5.1 Project implementation

5.1.1 Developing a Monitoring and Evaluation framework This process involved holding of a two days workshop in which we had an external facilitator

hired to guide the project team through the process of developing the project goal, and objectives

which are the core of any M&E frame. The Fellow was largely involved in facilitating the

workshop, synthesizing the workshop discussions into a draft document, reviewing the

organisations documents, streamlining the objective hierarchy, defining the means of verification

and assumptions. A draft log frame was developed and discussed. A mini workshop was held to

discuss the draft log frame with senior Managers and directors.

Figure 1: The project team in the process of developing the project goal, and objectives-

guided by an External Facilitator

Figure 2: Some of the photos taken during the workshops

5.1.2 Developing and selecting suitable performance indicators This process involved reviewing old indicators, developing new indicators, and collecting all

these into a summary sheet, while doing this the fellow focused on the impact, outcome, output

and processes of all the indicators (See Appendix 2 –M&E Matrix). This was the starting point

for the fellow and the project implementers to clarify what exactly the OCS team was to measure

prior to the generation of the indicators. Other donors supporting OCS have specific indicators to

be monitored. These plus indicators from OCS were merged and indicated in the M&E Matrix.

Emphasize was put on the frequency of data collection, the sources of data and the responsible

persons. The indicators were arranged as per the objectives hierarchy i.e. impact, out come,

output and process indicators.

5.1.3 Reviewing existing data collection tools

Data collection tools were developed and standardized to capture all the data that is needed for a

particular OVC program.

The forms developed address all reporting requirements by ROM and other stakeholders.

Particularly, the following tools were developed and approved:

• OVC Assessment form

• OVC enrollment form was improved to accommodate unique identification number and

other important variables like date of birth and Site.

• Supported Children’s follow-up form was developed.

• Attendance form for all OCS activities: In order to ensure that the number of children

and youth involved in OCS activities are consistently collected, a standardized attendance

form was designed to record participants of any OCS activity carried out. This Form will

be attached to the minutes or narrative report.

• Activity report form for psychosocial activities: Every time a psychosocial support

activity is carried out, this form will be completed. The form will include information on

the types of activities carried out, the content of activities and who was involved, as well

as details of the content of any training.

12

• Support/Material distribution form: Whenever any materials are distributed or issued,

this form will be filled by the responsible person and will be recorded and captured

against the OVC unique number

5.1.4 Training staff in the implementation of the framework Capacity building of the staff in the use of the framework through training was done. Lecturing,

coaching, facilitation, discussions, questions and answers, brain storming, presentation and

discussions were the main methods used.

5.1.5 Developing an electronic Relational Database As part of the project, the fellow managed the process of developing an electronic Relational

Database with tables that store the data and interfaces were developed using Visual Basic to

access data from MYSQL database. SQL (Structured Query Language) is used to generate

reports from the database. This will improve monitoring and evaluation of OVC activities at

ROM, given the fact that most of the data is on paper

Figure 3: OVC Support module

13

The database was installed on the server and a workstation was created and is being used to

capture the data through the computer network to the centralized database. The following logical

structure was used in the Process of developing the module:

PROCESS OF DEVELOPING THE MODULE

Figure 4 Process of developing the module

Supported children are allocated unique numbers to facilitate follow up through the system.

Data can now be entered online from the workstations connected to the server and reports can be

generated easily.

Figure 5 supported children’s unique identification numbers

14

6.0 Ethical considerations

The proposal was reviewed by the Makerere University School of Public Health Higher Degree

Research and Ethics Committee and obtain clearance from the Uganda National Council of

Science and Technology (See appendix for Copies of clearance letters).

Efforts were made to minimize bias by incorporating an external facilitator as opposed to using

pre-set notions by the TWG of what the process should be.

7.0 Dissemination Out puts of this activity were disseminated at consensus building meetings and technical working

group reviews.

8.0 Sustainability of the project ROM management was deeply involved in this project because this is the main reason why they

asked for a fellow. All members of the OCS project were fully involved right from the beginning

of the project, in the absence of the Fellow these documents will continue to be used.

References

Abou Zahr, C.; Boerma, T. More than money is needed to improve health information systems.

Bulletin of the World Health Organization (WHO): the International Journal of Public Health

(2005). Available online at: http://www.who.int/entity/bulletin/volumes/83/8/578.pdf

Agency for International Development (USAID) and MEASURE Evaluation Providing tools to

reliably measure the well being of vulnerable children, the U.S. http://www.cpc.edu/measure/

(2007)

Agency for International Development (USAID) and MEASURE Evaluation, Improving global

M&E of programs for orphans and vulnerable children, the U.S. http://www.cpc.edu/measure/

(2007)

Gladwin Jean, Robert A. Dixon and Tom D. Wilson Managing misinformation - introducing a

new health management information system in Uganda .University of Sheffield, UK / id21

Development Reporting Service (2002)

http://www.id21.org/health/h1jg4g2.html

Gladwin J, Dixon R.A, and Wilson T.D Strengthening health information management in an east

African country using eternal training materials. Health information Journal Vol 6 No. 1, 9-14

(2000)

McCoy K. Lynn et’al, Patricia Njeri Ngari Edwin E. Krumpe. Building monitoring, evaluation

and reporting systems for HIV/AIDS programs (2005)

Ministry of Gender Labour and Social Development the district capacity and M&E assessment

report (2005), unpublished

Reach-Out Mbuya 2008 ANNUAL REPORT

16

Simba DO, (2004) Application of ICT in strengthening the Health Information system in the

developing countries in the wake of globalization. African Health Sciences Dec; 4(3): 195-199.

(2004).

Uganda AIDS Commission (UAC). National Strategic Plan for HIV/AIDS 2007/08 to 2011/12,

Moving Towards Universal Access. (2007)

Uganda AIDS Commission, The National HIV/AIDS Performance Measurement and

Management plan (PMMP)for the National Strategic Plan for HIV/AIDS 2007/08 to

2011/12, Moving Towards Universal Access.(2007)

United Nations Children’s Fund (UNICEF). The Framework for Protection, Care and Support of

Orphans and Vulnerable Children Living in a World with HIV and AIDS. New York: UNICEF,

the Joint United Nations Program on HIV/AIDS (UNAIDS); 2004.

WHO Improving the use of information for health care decision-making: what is needed World

Health Organization / World Health Organization (WHO) (2005)

WHO Practical principles on building and improving health management information systems.

Regional Office for the Western Pacific / WHO Regional Office for the Western Pacific (WHO-

WPRO) (2004)

WHO Practical principles on building and improving health management information systems.

Regional Office for the Western Pacific / WHO Regional Office for the Western Pacific (WHO-

WPRO) (2004)

17

Appendix I Project Log frame

Project Title: OCS Start date: April 2010 End date: March 2015

NARRATIVE SUMMARY What does the project want

to achieve?

INDICATORS How can we

tell if we have

achieved it?

MEANS OF VERIFICATION

Where can we get information that will tell us

this?

ASSUMPTIONS What else must happen if it is to succeed?

Project goal IMPROVED QUALITY OF LIFE OF CHILDREN INFECTED AND AFFECTED BY HIV&AIDS IN MBUYA PARISH Objective1:0 Evaluation Indicators (contribution)

TO INCREASE THE PROPORTION OF CHILDREN OF SCHOOL GOING AGE WHO COMPLETE SCHOOL FROM (50%-75%) BY 2015.

% of children who complete school

SP-mid-term and end of term review report

Ministry of Education and other stakeholders remain committed in ensuring quality education

Output 1:1 Monitoring Indicators (attribution) Children Enrolled in schools

# of children enrolled in school

OVC Educational Support registry

School Authorities will be cooperative

Activity 1:1:1 Carry out House Hold assessment # of House Holds assessed HH assessment Report Criteria will be followed during assessment

Activity 1:1:2 Categorization of beneficiaries into Primary, secondary and Vocation

# of children per category established

OVC Educational Support registry per category

Activity 1:1:3 Identify and Enroll intended beneficiaries into formal education

# of children identified & enrolled into formal education

List of identified & enrolled children and project reports

Parents/guardians will be cooperative

Activity 1:1:4 Identification of suitable schools # of suitable schools identified & selected

List of suitable schools identified & project reports

Parents/guardians/school authorities will be cooperative

Activity 1:1:5 Placement in schools # of children placed in schools List of children placed in schools & project reports

Parents/guardians/school authorities will be cooperative

18

Project Log frame



to achieve?


tell if we have

achieved it?



this?


Project goal IMPROVED QUALITY OF LIFE OF CHILDREN INFECTED AND AFFECTED BY HIV&AIDS IN MBUYA PARISH Output 1:2 Monitoring Indicators (attribution)

School fees for OVC attending school paid # of children whose fees has been paid

OVC Educational Support registry, Financial reports & beneficiary lists

Continued commitment of donors

Activity 1:2:1 Collection of previous terms reports to verify the child’s attendance

Previous terms reports on Childs file

Childs file

Activity 1:2:2 Develop school fees lists for active supported children

Active supported children’s school fees lists in place

Beneficiary lists & project reports

Activity 1:2:3

Processing payment of school fees

# of children whose fees has been paid

Financial reports & beneficiary lists, interview of beneficiaries


Activity 1:2:4 Negotiate and sign MOU with the school authorities Signed MOU in place MOU Terms of the MOU will be adhered to by the schools

Output 1:3 Monitoring Indicators (attribution)

Short term non-tuition materials support and other basic needs provided

# of children benefited Beneficiary lists, Distribution forms on Childs file & project reports


Activity 1:3:1 Development of beneficiaries lists beneficiaries lists in place Beneficiary lists file Willingness of the staffs to develop these lists

Activity 1:3:2 Preparations of procurement requests # of requisitions raised Financial reports & procurement records

Activity 1:3:3 Request and distribution of non-tuition materials # of children benefiting Beneficiary file & interview of beneficiaries


Activity 1:3:4 Narrative and financial report writing Narrative and financial report in place

Report File

19

Project Log frame



to achieve?


tell if we have

achieved it?



this?


Project goal IMPROVED QUALITY OF LIFE OF CHILDREN INFECTED AND AFFECTED BY HIV&AIDS IN MBUYA PARISH Output 1:4 Monitoring Indicators (attribution)

Children in school followed to ascertain their attendance and performance

# of children followed Internal audit & performance assessment report

Supported children will be willing to go to school

Activity 1:4:1 Develop schedule for the follow-up Schedule for school follow-up in place

Work plan Willingness of the staffs to develop the schedule

Activity 1:4:2 Carry out actual school visits # of school visits/Child follow-up forms on file

School visit reports staff will comply

Activity 1:4:3 Assessing the performance of the children at the end of the term

# of children whose performance has been assessed

Performance assessment report, School records and project reports

staff will comply

Activity 1:4:4 Follow up with home visits for the children who are at risk of dropping out of school

# of home visits conducted Child follow-up forms on file

Home visit reports staff will comply

Objective 2:0 Evaluation Indicators (contribution)

TO REDUCE THE RISK OF CHILD ABUSE AMONG CHILDREN WITHIN OUR CARE BY 2015

% level of OVC abused or exploited reduced

Evaluation and Baseline studies

The stakeholders will be supportive

Output 2:1 Monitoring Indicators (attribution) Awareness about child protection issues increased

# of awareness campaigns conducted

Attendance list & project reports

The community will adhere to the campaigns and put the message in practice

Activity 2:1:1 Carry out child safety assessment child safety assessment report on file

Assessment report & children’s fillies


Activity 2:1:2

Mobilization of stakeholders

Stakeholders mobilization report on file

Attendance lists, project reports, attendance reports and meeting minutes

20

Project Log frame



to achieve?


tell if we have

achieved it?



this?


Project goal IMPROVED QUALITY OF LIFE OF CHILDREN INFECTED AND AFFECTED BY HIV&AIDS IN MBUYA PARISH Activity 2:1:3 Develop and disseminate training manual guidelines Training manual in place resource center

Activity 2:1:4 Train staff, volunteers, children and the community of mbuya in child related advocacy

# of staff trained Attendance list & training reports

staff, volunteers, children and the community will practice the acquired skills


Basic assistance after abuse provided # of cases assisted, Basic assistance report on Childs file

On file The stakeholders will be supportive

Activity 2:3:1 Report child abuse cases to relevant authorities # of child abuse cases reported Report on child abuse cases on file

Project child abuse records


Activity 2:3:2 Refer child abuse cases to other networks # of cases referred Project child abuse records


Output 2:4 linkages established with other service providers to scale up child protection

# of organizations networked with

Activity 2:4:1 Identification of other service providers Other service providers Inventory in place

Other service providers Inventory

willingness of other service providers to cooperate

Activity 2:4:2 Create collaborations/networks # of organizations networked with

Report File

Activity 2:4:3 Refer child abuse cases to these networks # of cases referred Report File Objective3:0 Evaluation Indicators (contribution)

TO REDUCE THE PROPOTION OF CHILDREN PRESENTING WITH PSYCHOSOCIAL ISSUES IN MBUYA PARISH 2015

% of children and families reporting behavior change among children who previously presented themselves with PSS issues

Evaluation and Baseline studies

Behavior change will take place

21

Project Log frame



to achieve?


tell if we have

achieved it?



this?


Project goal IMPROVED QUALITY OF LIFE OF CHILDREN INFECTED AND AFFECTED BY HIV&AIDS IN MBUYA PARISH Output 3:1 Monitoring Indicators (attribution) Psychosocial support system strengthened

Activity 3:1:1 Develop and disseminate training manual (implication Tools)

Training manual in place resource center

Activity 3:1:2 Conduct refresher trainings for staff and volunteers to enable them identify, support and mentor CATTS in the provision of appropriate psychosocial services.

# of staff trained Attendance list & training reports

Availability of funds on time

Activity 3:1:3 Mobilize and offer PSS and life skills to OVC # of OVC supported per category

Beneficiary lists & project reports

Activity 3:1:4 Develop and disseminate PSS service delivery SOPs

PSS service delivery SOPs in place

in the resource center


IGAs to sustain the project strengthened # of children trained

Activity 3:2:1 Training in Brass band # of children who are trained & are active members of the brass band # Troops in place

Skills adopted will be used

Activity 3:2:2

Training in MDD

# of children who are trained & are active members of the MDD groups # Troops in place

The skills will be put to proper use

22

Project Log frame



to achieve?


tell if we have

achieved it?



this?


Project goal IMPROVED QUALITY OF LIFE OF CHILDREN INFECTED AND AFFECTED BY HIV&AIDS IN MBUYA PARISH Activity 3:2:3

Training in Art and crafts # of children trained & Items produced



Issues presented by OVC addressed appropriately # of OVC with improved emotional well being (i.e. more self-reliant, skilled and assertive)

Activity 3:3:1 On-going emotional support, from trained facilitator or a peer

# of sessions conducted # of children accessing counseling

Beneficiary interviews & Reports

Activity 3:3:2 Conduct Health education sessions # of children equipped with appropriate value based life skills

project report and PSS records


Activity 3:3:3 Behavior change program # of sessions conducted The right massage will be adopted by the children

Activity 3:3:4 Recreation activities Output 3:4 Monitoring Indicators (attribution)

Community involvement strengthened

Activity 3:4:1 Support to parents/caretakers to plan for the future of their children

Total number of care takers supported to plan for the future of their children

Attendance list, Willingness of these parents to collaborate

Activity 3:4:2 Meeting parents to discuss their children’s performance

Total number of parents/care takers met to discuss their children’s performance

Attendance list,

23

Project Log frame



to achieve?


tell if we have

achieved it?



this?


Project goal IMPROVED QUALITY OF LIFE OF CHILDREN INFECTED AND AFFECTED BY HIV&AIDS IN MBUYA PARISH Activity 3:4:3 Sensitizing parents on the importance of taking

children to school Total number of parents/care takers who attend the meeting

Attendance list,


Linkages established with other service providers # of organizations networked with

project & networking meetings reports

Willingness of these organizations to collaborate

Activity 3:5:1 Identification of other service providers Other service providers inventory in place

On file

Activity 3:5:2 Create collaborations/networks # of organizations networked with

project & collaboration meetings reports

Willingness of these organizations to collaborate

Activity 3:5:3 Refer child abuse cases to these networks # of cases referred project & referral reports

24

Appendix II

OCS M&E Matrix (2010-2015) Log Frame Reference Data Collection & Analysis Reporting & Information Use

Indicator Primary Data Source Frequency of Data Collection

Who is Responsible Reports Produced

Used by Who Feed Back To

Objective 1:0

% of children who complete school

SP-mid-term and end of term review report

Semi and end of SP period

M&E & Social Support Coordinator

Detailed Directors, senior Managers & OCS Supervisor

Other staff, CDC, Partners & MGLSD

Output 1:1 # of children enrolled in school

OVC Educational Support registry

Quarterly/Annually Social Support Coordinator & OCS Supervisor

Detailed quarterly Progressive report

Directors, senior Managers & OCS Supervisor


Activity 1:1:1

# of House Holds assessed

HH assessment Report After the activity has been carried out

OCS Supervisor & OSF Team leader

Detailed HH assessment Report


Activity 1:1:2

# of children per category established

OVC Educational Support registry per category

Annually OSF Team leader Detailed Annual Progressive report

OCS Team

Activity 1:1:3

# of children identified & enrolled into formal education

List of identified & enrolled children and project reports

After the activity has been carried out

OSF Team leader & Assistants



Activity 1:1:4

# of suitable schools identified & selected

List of suitable schools identified & project reports


Social Support Coordinator , OCS Supervisor & OSF Team leader

Detailed List of suitable schools identified

Activity 1:1:5

# of children placed in schools

List of children placed in schools & project reports


OCS Supervisor , OSF Team leader & Assistants

Detailed List of children placed in schools

25





Output 1:2 # of children whose fees has been paid

OVC Educational Support registry, Financial reports & beneficiary lists





Activity 1:2:1

Previous terms reports on child’s file

child’s file OCS Assistants

Activity 1:2:2

Active supported children’s school fees lists in place

OSF Team leader

Activity 1:2:3

# of children whose fees has been paid

OVC Educational Support registry & Finance records

Social Support Coordinator & OCS Supervisor

Activity 1:2:4

Signed MOU in place Social Support Coordinator

Output 1:3 # of children benefited OVC Material Assistance Register




Activity 1:3:1

beneficiaries lists in place

OCS Assistants

Activity 1:3:2

# of requisitions raised OSF Team leader

Activity 1:3:3

# of children benefiting

OVC Material Assistance Register

After the activity has been carried out OCS Assistants

]Activity 1:3:4

Narrative and financial report in place

Review of project records Social Support Coordinator & Finance department

Output 1:4 # of children followed Review of project records Quarterly/Annually Social Support Coordinator & Supervisor

Directors, senior Managers & OCS


26





Supervisor

Activity 1:4:1

Schedule for school follow-up in place

Review of project records OSF Team Leader

Activity 1:4:2

# of school visits/Child follow-up forms on file

OVC Educational Support Register


OCS Assistants

Activity 1:4:3

# of children whose performance has been assessed

OCS Supervisor and OSF Team Leader

Activity 1:4:4

# of home visits conducted Child follow-up forms on file


OCS Assistants

Objective 2:0

% level of OVC abused or exploited reduced


M&E Coordinator & Social Support Coordinator



Output 2:1 # of awareness campaigns conducted




Activity 2:1:1

child safety assessment report on file



Activity 2:1:2

Stakeholders mobilization report on file

After the activity has been carried out Team Leaders and

Assistants

Activity 2:1:3

Training manual in place


Activity # of staff trained After the activity has Social Support

27





2:1:4 been carried out Coordinator & OCS Supervisor

Output 2:3 # of cases assisted, Basic assistance report on child’s file

Quarterly/Annually




Activity 2:3:1

# of child abuse cases reported Report on child abuse cases on file

OCS (Legal support register) OCS Team

Activity 2:3:2

# of cases referred OCS Team

Output 2:4 # of organizations networked with




Activity 2:4:1

Other service providers Inventory in place PSS Team Leader

Activity 2:4:2

# of organizations networked with OCS Team

Activity 2:4:3

# of cases referred OCS Team

Objective 3:0

% of children and families reporting behavior change among children who previously presented themselves with PSS issues


M&E Coordinator & Social Support Coordinator



Output 3:1

Quarterly/Annually




28





Supervisor

Activity 3:1:1

Training manual developed and in place

Training document


Activity 3:1:2

# of staff trained

Training register


Activity 3:1:3

# of OVC supported per category PSS Team Leader

Activity 3:1:4

PSS service delivery SOPs in place PSS SOPs Document SP mid-term review PSS Team leader

Output 3:2 # of children trained

OVC Training Register Quarterly/Annually




Activity 3:2:1

# of children who are trained & are active members of the brass band # Troops in place OVC Club Register PSS Team leader

Activity 3:2:2

# of children who are trained & are active members of the MDD groups # Troops in place PSS Team leader

Activity 3:2:3

# of children trained & Items produced PSS Team leader

Output 3:3 # of OVC with improved emotional well being (i.e. more self-reliant, skilled and Quarterly/Annually




29





assertive) Supervisor

Activity 3:3:1

# of sessions conducted # of children accessing counseling PSS Team leader

Activity 3:3:2

# of children equipped with appropriate value based life skills PSS Team leader

Activity 3:3:3

# of sessions conducted PSS Team leader

Activity 3:3:4

0

Output 3:4 0

Quarterly/Annually




Activity 3:4:1

Total number of care takers supported to plan for the future of their children

After the activity has been carried out PSS Team leader

Activity 3:4:2

Total number of parents/care takers met to discuss their children’s performance


Activity 3:4:3

Total number of parents/care takers who attend the meeting


Output 3:5 # of organizations networked with

Quarterly/Annually




30





Activity 3:5:1

Other service providers inventory in place PSS Team leader

Activity 3:5:2

# of organizations networked with


Activity 3:5:3

# of cases referred


DEVELOPING A MONITORING AND EVALUATION FRAMEWORK … · training support, psychosocial support and...

Documents

Transcript of DEVELOPING A MONITORING AND EVALUATION FRAMEWORK … · training support, psychosocial support and...