May 8, 2012MWP-K Learning Event

Monitoring, evaluation, and learning (MEL) framework for the Millennium Water Program, Kenya

Background

Collaboration between: Millennium Water Alliance MWP-Kenya partners (CARE, CRS, FH, Living Water,

Lifewater, World Vision) Center for Global Safe Water, Emory University

Objectives: Improve effectiveness and increase sustainability

through more data-driven decision making Build foundation of learning and collaboration through

MEL to support future continuation of MWP-Kenya

Challenges Addressed by MEL Framework

Monitoring usually limited to project activities and outputs (e.g. # latrines built, # trainings completed)

Final evaluation is only time seized to assess achievement of outcomes (e.g. behavior change)

Want to use monitoring to track progress toward intended behavior change

outcomes to enable a change of course if needed

Challenges Addressed by MEL Framework

Minimal information sharing & reflection on monitoring data or other lessons throughout project life cycle.

Want to intentionally budget time and persons responsible for guiding data sharing and reflection to maximize

lessons learned to improve practice

Challenges Addressed by MEL Framework

Sustainability seldom evaluated after project life cycle due to close of project, budget limitations, etc.

Want to incorporate M&E indicators into tools used during project life cycle to

assess likelihood of future sustainability (are factors in place to improve longevity

of benefits after project is finished?)

Components of MEL for MWP Kenya

1. Key program indicators2. Evaluation: baseline and follow-up3. Monitoring: systematic, regular data

collection Includes process and tools to guide “Data for decision-

making”: Sharing, feedback, and action; Monitoring tools to assess progress towards outcomes

4. Focused topical learning Includes partners’ leadership in special learning

topics; Emory support for some sub-studies (e.g. sanitation

assessment)

1. Key Program Indicators

Emory assisted partners in developing logical framework in May 2010

Indicators include: Outputs (activities completed by partners) Outcomes = central focus (changes that beneficiaries

adopt in response to program) Includes indicators that suggest future sustainability

E.g. School has specific WASH budget; school management committee is involved in WASH program

2. Evaluation

Emory facilitated baseline assessment planning and implementation: Led development of evaluation tools and protocols

Household, school, and health facility surveys Analyzed data

Each organization collected data prior to implementation

3. Monitoring

Output-level indicators Responsibility of each partner Tracked through internal monitoring system

Outcome-level indicators Measured throughout course of project

To engage implementers to critically reflect on and address programmatic challenges in real-time

Monitoring activities conducted quarterly or once per school term

Data for Decision-Making

Sharing of experiences and lessons learned between partners

Data Reflection Tool (DaRT) Summarizes findings from each monitoring and

learning activity Project expectations Progress toward achieving indicators Plans to address lack of progress Potential threats to sustainability

Shared and discussed within partnership on a quarterly basis before Program Management Group meeting

Data for Decision-Making Quarterly Work Flow

Complete monitoring activities quarterly

Summarize findings into

Data Reflection

Tool (DART)

MWP Coordinator reviews DARTs

Meeting:Internal

Learning/

Reflection

Meeting:Pre-PMG Learning/ Reflectio

n

Meeting:

PMG

1. Flag issues for follow-up

2. Use to guide agenda for Pre-PMG meeting

3. Distribute copies at pre-PMG meeting

Discussion of:

1.What’s going well/ not as well as expected2.Possible course changes3.Additional info needs

Discussion of:1.Highlights from DARTs2.Lessons learned, best practices3.Group feedback4.Issues to raise at PMG

Discussion of:

1.Issues flagged at Pre-PMG meeting2.Lessons learned & best practices

Send

Decide:

Needed course

changes

Additional learning

needs

= Responsibility of each organization= Responsibility of Coordinator= Responsibility of all partners together

Summary of Monitoring Tools

Tool Objective

Rapid household observationsObserve and inquire about WASH practices in households in the intervention area to track changes toward achievement of intended outcomes and understand the trends in behavior change.

Water point observations and management committee key informants interview

Observe repair or maintenance needs; observe practice of integrated water resources management (IWRM); understand patterns of use for the water point, financial management practices, maintenance practices, and management challenges.

Water point user interviewsDetermine functionality, use patterns, and perceptions of water point and management committee from community perspective. Provides a double check on some information gathered in the management committee interview.

School monitoring visit Determine whether and how the school enacts the intervention on a day-to-day basis and assess threats to sustainability that might need to be addressed.

School management committee key informant interview

Understand the degree to which the SMC is involved in supporting school WASH activities, and encourage their support by inviting them to join you in your monitoring visit observations.

Pupil interviews Gain pupil perspective on WASH provision and education at school and preferences. Provides a double check on some information gathered in the school assessment.

4. Focused Topical Learning

Proposed Learning Leader

Topic

CARE Improving pupil handwashing behavior change in schools

CRS Improving utilization of household latrines

FH Improving community handwashing behavior in ASAL, nomadic areas

LWI Improving financial management of water sources

WVI Improving water supply sustainability

CARE Improving pupil handwashing behavior change in schools

Partners expressed interest in specific topics for further investigation

As learning leader, each org. should organize one meeting with partners to discuss lessons on topic

Summary

Thoughtful, timely collection of relevant dataData used to drive decision-making:

Guide targeting, technology choice, and program approaches to improve effectiveness and sustainability

Feed back findings into continued program activities in a timely and meaningful way

Engage donors, beneficiaries, and government stakeholders in the water, sanitation, and hygiene (WASH) sector

Embrace benefits of working in partnership Collective learning and reflection on best practices &

lessons learned