Evaluation of the Results-Based Approach and M&E of Finnish Development Cooperation Interventions

Prepared by

Ministry for Foreign Affairs of Finland

Purpose of the evaluation

• To deepen the understanding of the results-based approach and management requirements

Overall Objective

• Desk study• Review of project

documentation and interviewsScope

2

Methodology

From TOR to Evaluation Framework

Topic list of interview questions

Staff survey76/120

Management survey10/18

Document review checklist

Rating assessment of 17 projects

3

RBM Conceptual framework

4

Strategic Results Framework• Priorities•Objectives• Strategy

Credible measurement & analysis•Measure & assess results•Assess contribution to strategic objectives

Programme Results Framework•Results chain/theory of change•Align with strategic framework•Performance indicators

Credible performance reporting•Relevant, timely & reliable reporting

Use results to improveperformance•Adjust the programme•Develop lessons & good practices

5

How to manage

for results

Reporting on performance

against expectations

Deliberately learning from evidence and

analysis

Understanding the theory of

change

Setting out performance expectations

Measuring and analysing results & contributions

to outcomes

Developing a culture of results

0

1

2

3

4

5

6

An organisation with a results culture…

Engages in self-reflection

& self-examination

Engages in results-based

learning

Encourages experimentation

& change

10

Factors that help build such a culture

Demonstrated senior management leadership and commitment Informed demand for results informationA results-oriented accountability frameworkSupportive organizational systems, practices and proceduresResults measurement and results management capacityA capacity to learn and adapt

11

Findings and recommendations

Finding 1There is no statement or document guiding the adoption of results-based management for the MFA.

Recommenda

tion 1

A formal approach should be established for adopting results-based management.It should set out principles, objectives and accountability

13

Finding 2Policy documents do not contain a strategic results framework

Recommenda

tion 2

Develop a SRF with clear objectives to guide regional and country strategies

14

Finding 3Planning is methodical and systematic but lacks clear outcomes and indicators at country level

Recommenda

tion 3

Reorganise country-level planning with measurable objectives and indicators

15

Finding 4Despite clear and comprehensive guidelines, good standards of project design are not consistently applied.

Recommenda

tion 4

Improve quality by:a) Accessible guidelines & checklistsb) Provide good practice examplesc) Apply to all interventionsd) More effective QA

16

Finding 5The Monitoring, Reporting & Evaluation system is established but lacks added-value

Recommenda

tion 5

Redesign reporting to be more analytical at 3 levels:a) Project reporting & the supervisory boardb) Semi-annual reporting by embassiesc) Unit scorecards

17

Finding 6Meta-analysis by EVA-11 has scope to consolidate evaluation findings

Recommenda

tion 6

This should be continued with more analysis of trends of performance

18

Finding 7Management culture is risk averse and makes little use of past results to set policy

Recommenda

tion 7

Improve institutional culture[Build on new HR policy]

19

Finding 8HR systems lack incentives for delivering results

Recommenda

tion 8

[Build on new HR policy]Train managers to be more effective in RBM

20

Finding 9Information management is inadequate for document retrieval and comparative analysis

Recommenda

tion 9

Improve information management to enable analysis of performance

21

The End