Measuring policy influence: like

measuring thin air?

John Young: j.young@odi.org.ukArnaldo Pellini: a.pellini.ra@odi.org.uk

RAPID

Power, Politics and evidence use

Evidence production and communication

Knowledge intermediaries and interactions

Policy processes are...

Monitoring and Evaluation

Agenda Setting

DecisionMaking

Policy Implementation

Policy Formulation

Civil Society

DonorsCabinet

Parliament

Ministries

Private Sector

The Cynefn Framework

4

Policy change

• Discursive: Client-focused services

• Attitudinal: Farmers have good ideas

• Procedural: Participatory approaches to service development

• Content: UU20, UU25. New guidelines

• Behavioural: Approach being applied in practice

5

Focus on behaviour change

Inputs Activities Outputs

Other Actors

Project Team

Outcome ImpactOutcomes ImpactOutcomes Impact

BehaviourChange6

RAPID Outcome Mapping Approach

Academic research communications

Develop a network or partnership

Online communications

Media strategy

More research

Policy advocacy coalition

Why do M&E?

• To learn about what works• To manage better• To account:

– to donors– to recipients

8

Methods

• Classical case studies (IDRC, IFPRI)

• Episode studies (ODI/RAPID)

• Stories of Change (Denning)

• Micro-Narratives (Snowden)

• Impact matrices (Davies)

• Peer evaluations (CHSRF)

• Systematic reviews?

• RCTs?

9

Outcome Mapping

OUTCOME MAPPING:Building Learning and Reflection into Development ProgramsSarah Earl, Fred Carden, and Terry Smutylo

http://www.idrc.ca/en/ev-9330-201-1-DO_TOPIC.html

10

Social Network Analysis

11

RAPID Outcome Assessment

12

1. Strategy and direction –are you doing the right thing?

2. Management –are you doing what you planned to do?

3. Outputs – are the outputs appropriate for the audience?

4. Uptake – are people aware of your work?

5. Outcomes and impacts –are you having any impact?

A systematic approach

1. Strategy and direction

2. Management

3. Outputs

4. Uptake

5. Outcomes and impacts

13

M&E in ODI

Impact Logs & SoC

15

3ie PIM Project Evaluation Framework

16

TTI Evaluation Framework