Livestock and Fish monitoring, evaluation and learning framework

Keith Child

Livestock and Fish Monitoring, Evaluation and Learning planning meeting, Nairobi, 27-28 November 2013

Agenda

1. MEL Process and Framework Goals

2. Parts of the Framework

Purpose of the MEL Framework

A comprehensive, yet concise narrative of why the M&E system is important, how it operates, what kinds of data it will collect and by whom

• Provides a common vision of what an M&E system will look like

• Provides a benchmark for measuring progress

• Provides a mandate and set of responsibilities

What makes a good framework?

Clarity of Purpose: a ‘framework’ not an instruction manual;A Compelling Vision: how will we achieve success;Relevant: Appropriate to the CRPEconomical: Achievable at an affordable costConcision: as long as necessary, as short as possible;Professional: must look and read like a formal document.

Background

• November 2012: Incomplete rough draft

• June 2013: Senior Scientist (Impact and Learning)

• July-August: MEL Framework consultations

– 4 formal meetings, many informal consultations• Impasse: RBM vs. Hybrid visions of the MEL Framework

• September 2013: Plan to finalize Framework presented to PPMC

• November 2013: Presentation of draft Framework to MEL Meeting

Forward

• Post MEL Meeting revisions to transform draft into Working Paper

• December 2013: Presentation to PPMC in Tanzania

• Post PPMC Meeting revisions to transform Working Paper into Finalized Framework

• April 2014: Presentation to PPMC in Penang for Finalization

Proposed Content of MEL Framework

MEL Framework Introduction

• Introduction to Framework

• Background: CGIAR and Livestock and Fish

• Challenges

• Structure of MEL Framework

• Integrated Phases

• Attribution versus Contribution

Integrated Phases

Implementation Phases and Corresponding MEL Research Questions

Dev

elop

men

t Ph

ase

(100

,000

s be

nefic

iari

es)

Scal

ing-

Out

Ph

ase

(mill

ions

of

bene

ficia

ries

)

Rese

arch

Pha

se

(10,

000s

be

nefic

iari

es)

Ong

oing

Fu

nctio

ns

Years 12-20 (SLOs)Years 5-8 (Progress towards IDOS) Years 8-12 (IDO data collection)

What Works? Should it work?

How and Why did it work?For whom will it work?

What level of attribution can be claimed?

How to Scale-Up?Will it continue to work?

Will it work somewhere else?How has implementation contributed to the results?

Are Program benefits sustainable?

How to Scale-Out?Is the rationale for why it worked still sound?

Can outputs be transferred/generalized to different settings?

How have innovations been adapted to local contexts?

International Public Goods

Program Monitoring

Attribution versus Contribution

• Attribution: whether or not and how much of a particular change can be attributed to an intervention

• Contribution: whether or not and how an intervention contributes to the change

Body: 5 Components

1. Organizational Perfomance

2. Learning and Reflection

3. Outcome and Impact Monitoring

4. Knowledge Management

5. Research Agenda

Component One: Organizational Performance

• Evaluation

– CRP Independent External Evaluations– CRP Commissioned External Evaluations– Subtheme and Project Evaluations

• Routine Portfolio Monitoring

• Performance Indicator Matrix

Evaluation Strategy• Nested approach in which lower-level evaluations feed

into and inform higher level evaluations

Project Evaluations

• Guidelines for type and frequency determined by total value of project over lifetime

Component Two: Learning and Reflection

• Theory of Change and Impact Pathways

– Evidence Base

• Best Bet Selection Criteria

• Implementation Theory

• Ex ante Impact Assessments

• Ex post Impact Assessments

Theory of Change

• Calls for theory based approach that relies on development of explicit ToC for each VC, by CRP Phase:

– Identify critical linkages between program inputs and impacts;

– Identify critical conditions for success (e.g., contextual factors such as implementation theory, policy and economic conditions, etc.);

– Identify alternative explanations of change;

– Facilitate the identification of research questions that need to be tested in order to confirm the original program ToC.

• Step from ‘does it work’ to understanding what it is about the program that makes it work

Component Three: Outcomes and Impact Monitoring

• Development Indicators

– Harmonizing within the CRP and CGIAR• Baselines and Benchmarking

• Targeting

• International Public Goods

Baselines and Benchmarking

• Baselines will be conducted for specific donor-funded projects, including projects that are to be piloted during the Research Phase.

• IDO data will be collected through ‘Benchmarking’

– benchmarking exercises in which the best possible data will be used in order to establish estimated values for IDO indicators; in some cases, this may involve statistical modelling

Component Four: Knowledge Management

• Information Management

– CRP Monitoring Information System

– Performance Indicator Matrix

– Development Indicator Bank

– Evidence Base

• Communications

Component Five: Research Agenda

• Project Research (e.g., epIAs, project evaluations, project ToCs, etc.)

– Responsibility of Project Managers

– Funded by projects

• Program Research (e.g., epIEs, IEEs, CCEEs, VC ToCs, etc.)

– Responsibility of MEL Steering Committee and CRP Management

– Funded by CRP

• Research Quality and Ethics

Conclusion

• The MEL Team

– MELCoP

• Appendix One: Terminology

• References

CGIAR is a global partnership that unites organizations engaged in research for a food secure future. The CGIAR Research Program on Livestock and Fish aims to increase the productivity of small-scale livestock and fish systems in sustainable ways, making meat, milk and fish more available and affordable across the developing world.

CGIAR Research Program on Livestock and Fish

livestockfish.cgiar.org