Results-Based Management in UNDP

74
Results-Based Management Patrick Grémillet Bratislava Regional Center Management Practice 2011

description

Patrick Grémillet, UNDP Bratislava Regional Center, Management Practice 2011

Transcript of Results-Based Management in UNDP

Page 1: Results-Based Management in UNDP

Results-Based Management

Patrick GrémilletBratislava Regional Center

Management Practice2011

Page 2: Results-Based Management in UNDP

Session Schedule

Results-based Management (RBM) Principles

Introduction to RBM Setting a Results Management Strategy Problem Analysis and Results Mapping UNDP planning instruments Results Matrix

– Understanding the RBM Typology– Differentiating outcomes and outputs– Indicators, baseline and targets

August 2011 Management Practice - BRC 2

Page 3: Results-Based Management in UNDP

What is a Result ?

August 2011 Management Practice - BRC 3

Results are changes in a state or condition that derive from a cause-and-effect relationship. There are three types of such changes (intended or unintended, positive

and/or negative) that can be set in motion by a development intervention – outputs, outcomes and

impacts.

- UNDG agreed RBM terminology

Page 4: Results-Based Management in UNDP

IMPACT

OUTCOME

OUTPUTS

ACTIVITIES

INPUTS* using UN harmonized terminology (based on OECD/DAC Glossary of Key Terms in Evaluation and Results-based Management)

RESULTSare changes in a state

or condition that derive from a cause-

and-effect relationship.

What are Result?*C

ON

TR

OL

& R

ES

PO

SIB

ILIT

Y

MA

KIN

G A

DIF

FE

RE

NC

E

Page 5: Results-Based Management in UNDP

August 2011 Management Practice - BRC 5

A key component of RBM is performance monitoring which is to

objectively measure how well resultsare being achieved, and

report on measures taken to improve them.

What is results based management?

RBM is a management strategy by which all actors ensure that their processes, products and services

contribute to the achievement of desired results (outputs, outcomes and higher level goals or impact)

Page 6: Results-Based Management in UNDP

Why RBM?

Stated rationale/intended gains:

Improved focus on results instead of activities

Improved transparency

Improved accountability

Improved measurement of programme achievements (performance rather than utilization)

Enhanced strategic focus

Industry standard

To get more funds!!

August 2011 Management Practice - BRC 6

Page 7: Results-Based Management in UNDP

Results-Based Management in practice

In your table groups, discuss and agree…

What are the 3 main problems you have seen in trying to monitor and assess results as part of your work?

- one concern/problem per card/post-it

August 2011 Management Practice - BRC 7

Page 8: Results-Based Management in UNDP

General problems when applying RBM?

Difficult to apply

Difficult to learn

Difficult to integrate

Difficult to revise (... or reluctance to revise? )

Difficult to measure

Difficult to ‘attribute’ (at outcome level, the UN is accountable but not fully responsible)

August 2011 Management Practice - BRC 8

Page 9: Results-Based Management in UNDP

Common elements of RBM Problem analysis to understand causes

Stakeholder analysis

Structuring of programmes around a chain of desired results - addressing causes

Causality in the chain of results (if… then logic)

Use of ‘change language’ (future conditional)

Reliance on indicators to measure performance

Costing of results rather than isolated activity budgeting

August 2011 Management Practice - BRC 9

Page 10: Results-Based Management in UNDP

The RBM life-cycle approach

August 2011 Management Practice - BRC 10

Page 11: Results-Based Management in UNDP

Session Schedule

August 2011 Management Practice - BRC 11

Results-based Management (RBM) Principles

Introduction to RBM Setting a Results Management Strategy Problem Analysis and Results Mapping Results Matrix

– Understanding the RBM Typology– Differentiating outcomes and outputs– Indicators, baseline and targets

Page 12: Results-Based Management in UNDP

Making Strategic Choices

August 2011 Management Practice - BRC 12

VALUE SUPPORT CAPACITY & COMPARATIVE ADVANTAGE

Solving this problem would bring significant value to the

country/region.1.It is a global, regional, national priority2.It supports the country/region in achieving an MDG or other major development priority3.There is regional or national ownership of the issues.

Would we have support to work towards solving this

problem:1.It is in line with our mandate (Executive Board or Senior Management Support)2.We can count on government and others to partner with us.3.We can count on the support of those with decision-making power and resources.

Would we have the capacity and comparative advantage

to work on the problem:1.We have or can put in place capacity quickly to address the problem.2.We have the skills and resources to provide support more effectively or efficiently than others3.We have unique resources and/or attributes (e.g., neutrality, legitimacy, reputation, convening role)

Page 13: Results-Based Management in UNDP

Strategic Priority Setting

August 2011 Management Practice - BRC 13

UNDP comparativ

e advantage

Major national

challenge

Alignment of key actors to

support action

1. “Just do it”/strategic priority

2. Potential high priority, if consensus can be built

3. Potential high priority, if others cannot meet demand and internal capacity development is feasible

4. Lower priority: does not meet major national challenge

Value

Support

Capacity

Page 14: Results-Based Management in UNDP

Session Schedule

August 2011 Management Practice - BRC 14

Results-based Management (RBM) Principles

Introduction to RBM Setting a Results Management Strategy Problem Analysis and Results Mapping Results Matrix

– Understanding the RBM Typology– Differentiating outcomes and outputs– Indicators, baseline and targets

Page 15: Results-Based Management in UNDP

What problem?

August 2011 Management Practice - BRC 15

To solve any problem in society we need to understand its causes so we can take appropriate action to

address it. First, however, the problem must be clearly defined.

33-35

Page 16: Results-Based Management in UNDP

Formulate problem in a neutral manner

“Minorities and marginalized groups do not have the right to vote.” Versus

“Minorities and other marginalized groups do not participate in elections.” Or

“Low levels of participation by minorities in elections.”

August 2011 Management Practice - BRC 1633-35

Page 17: Results-Based Management in UNDP

Cause-effect analysis

August 2011 Management Practice - BRC 17

Manifestations of problem

Immediate

Underlying

Root

Underlying

Root

Immediate

Underlying

Root

Underlying

Root

Effects

Causes

Are there capacity constraints? – policy level, institutional, individual

Are there Social or Cultural Constraints?

Page 18: Results-Based Management in UNDP

Negative outcomes, manifestations of problems, unfulfilled rights

Root causesBeliefs, attitudes, culture, traditionsNatural resources, natural disasters

Political and economic systems, ideologies, conflict

Underlying causes and capacity issues

Policies, laws, budgetsSystems for service delivery

Behaviors and practices, low household incomes

Immediate causes

Affecting individuals and households

Causality analysis – Why?

August 2011 Management Practice - BRC 18

Page 19: Results-Based Management in UNDP

Problem Tree

August 2011 Management Practice - BRC 19

Malnutrition &

death

Inadequate dietary

intakeDisease

Insufficient

food security

Inadequate Maternal

& Child Care services

Insufficient health services

& unhealthy environment

Political, Ideological,

Economic structures

Resource Control

+

Organizational structuresRoot

causesRoot

causes

Underlying

causesUnderlying

causes

Immediate

causesImmediate

causes

ManifestationsManifestations

Page 20: Results-Based Management in UNDP

Results Map

August 2011 Management Practice - BRC 20

Women gain marketable skills in traditional and non-

traditional areas

Women learn good work attitudes and workplace

behavior

Women develop self-confidence

Some women receive traditional schooling

Potential employers educated and committed to programme

Women trained in vocational skills

Poor rural women earn meaningful wages and have improved life skills

Women enroll

Small stipend provided to women

Child care services provided

Training agency, child care agency, NGO, and National Women’s Bureau etc collaborate to design and deliver programmes

Long-term Outcome

Intermediate Outcomes

Partnerships

Outputs

Programme info provided to women

Advocacy and sensitization initiative

Page 21: Results-Based Management in UNDP

Session Schedule

Results-based Management (RBM)

Introduction to RBM– Key Principles– RBM Life cycle

UNDP planning instruments Results Matrix

– Understanding the RBM Typology– Differentiating outcomes and outputs– Indicators, baseline and targets

August 2011 Management Practice - BRC 21

Page 22: Results-Based Management in UNDP

Planning and Monitoring instruments

August 2011 Management Practice - BRC 22

Country Analysis

UN Development Assistance Framework (UNDAF)

Country Program Action Plan (CPAP)

Project documentAnnual Work Plan

(AWP)

Country Programme Document (CPD)

National plans/strategies; PRSP; …

Joint programmes

PL

AN

NIN

G

MO

NIT

OR

ING

Simplification• UNDAF and CPD

outcomes conflated into one

• Results matrix streamlined

Page 23: Results-Based Management in UNDP

UNDAF BEFORE….National Priorities/Goal(s)

UNDAF Outcome

Country Programme Outcomes

Country Programme Outputs

Good Governance and the protection of Human Rights

By 2013, achieve effective participation of citizens, and government accountability and integrity

Increased participation of civil society and citizens in decision making

Strengthened community participation in the planning and implementation of local development activities

Laws on public demonstrations and freedom of association improved in accordance with international standards.

Capacity of civil society strengthened through civic education and engagement with Parliament

Advocacy programmes conducted for accession to the UN convention against corruption.

Government assisted to meet its obligation under the International Covenant on Economic, Social and Cultural Rights.

Corruption in government management

significantly reduced Advocacy programmes conducted for the adoption of a national

anti-corruption law

Effectiveness of decentralized government

structures improved to deliver basic

services

Capacities of communes for decentralized planning, management and delivery of public goods and services further strengthened.

August 2011 Management Practice - BRC 23

Page 24: Results-Based Management in UNDP

UNDAF NOW…Single outcome layer, Outputs definition optional, integrated M&E data, no more separate M&E framework

National Development Priorities: Contributes to realization of National MDG targets 3.A: Ensure gender-representativeness at the level of no less than 30-70 percent in representative bodies and high-level executive authorities; 3.B: Halve the gap in incomes between women and men

Outcomes Indicators, Baseline, Target Means of Verification Risks and Assumptions

Role of Partners Indicative Resources (US$)

Outcome 1 More effective and accountable public institutions respond to the needs of all persons within the jurisdiction of the country, especially the most vulnerable

August 2011 Management Practice - BRC 24

Strategic focus• Single Outcome to be strategic but specific

enough to country context and UNDP programming needs

• Approach supports cross-practice work

Page 25: Results-Based Management in UNDP

NATIONAL PRIORITY OR GOAL: (Related to MDGs and/or other internationally-agreed development goals)

COUNTRY PROGRAMME / UNDAF OUTCOME #1: (UNDAF outcome that involves UNDP, copied verbatim from UNDAF) Outcome indicator: (The UNDAF outcome indicators copied verbatim from the UNDAF) Related Strategic Plan focus areas: (e.g. Energy and environment)

GOVERNMENT PARTNER CONTRIBUTION

OTHER PARTNER CONTRIBUTIONS

UNDP CONTRIBUTION

INDICATOR(S), BASELINES AND TARGET(S) FOR UNDP CONTRIBUTIONS

INDICATIVE COUNTRY

PROGRAMME OUTPUTS

INDICATIVE RESOURCES

BY OUTCOME (US$)

Approach that national partners will take to achieve this UNDP-specific UNDAF outcome

Major contributions from other actors key to achievement of UNDP-specific UNDAF outcome, incl. civil society

UNDP contribution to the UNDAF outcome, by developing key national capacities, incl. for promoting gender equality

SMART indicators that will enable measurement of UNDAF outcome-level change to which UNDP is making a specific contribution. Metrics and related information can draw on harmonized partner arrangements where applicable. See Handbook on Planning, Monitoring and Evaluation

Outline some (not all) possible outputs to be finalized with national counterparts, other partners and UNCT during preparation of UNDAF action plan (or CPAP).

Regular

Other

Regular

Other

UNDP specific Indicators and Targets

contributing to UNDAF Outcome

• Only one Outcome layer.

• Verbatim from UNDAF

New UNDP Country Programme Document(CPD) Results MatrixEffective October 2010

August 2011 Management Practice - BRC 25

Page 26: Results-Based Management in UNDP

Session Schedule

Results-based Management (RBM)

Introduction to RBM– Key Principles– RBM Life cycle

UNDP planning instruments Results Matrix

– Understanding the RBM Typology– Differentiating outcomes and outputs– Indicators, baseline and targets

August 2011 Management Practice - BRC 26

Page 27: Results-Based Management in UNDP

Ways how to depict the Results chain

LogframeNarrativeIMPACTA. XXXXXXXX

OUTCOMEA.1. XXXXXXXXA.2. XXXXXXX XXX

OUTPUTSA.1.1.XXX XXX XXXXA.1.2. XXXXXXXA.2.1. XXXXXXXX

Results Tree Results Pyramid

Page 28: Results-Based Management in UNDP

Principles of RBM

“If-Then” causality between levels of results

Common results language to describe changes

Collective accountability increases as you move up the chain of results towards outcomes and impacts (Key message No agency can do it alone!)

A results matrix is a mean not an end

A results matrix is contextual

August 2011 Management Practice - BRC 28

Page 29: Results-Based Management in UNDP

Results Measurable Indicators

Means of Verification

Risks and Assumptions

Goal

Purpose

Outputs

Activities

August 2011 Management Practice - BRC 29

Our counterparts may use different language (i.e. logframe), but the basis is the same…

Page 30: Results-Based Management in UNDP

CPAP Template

August 2011 Management Practice - BRC 30

UNDAF/CPD outcome #1: Extracted verbatim from the UNDAF/CPD

Relevant UNDP Strategic Plan result: Extracted verbatim from the UNDAF/CPD

Outcome-level indicators, baselines and targets for UNDP contribution to

UNDAF/CPD outcomes Country programme outputs

Implementation modality(ies) and implementing partner(s)

Indicative Resources by outcome (per year, US$)

Year Year Year Year Year Total

Output 1:

Annual Targets (recommended for multi-year outputs)

· Gender Marker Rating and

Motivation[1]

State the implementing modality (i.e. NIM, UN Agency, NGO, IGO or DIM); And

Government and/or UN

Implementing Partner[2]

Regular Resources

Output 2:

Annual Targets (recommended for multi-year outputs)

· Gender Marker Rating and Motivation

State the implementing modality (i.e. NIM, UN Agency, NGO, IGO or DIM); And

Government and/or UN Implementing Partner

Other Resources

UNDAF/CPD outcome #1: Extracted verbatim from the UNDAF/CPD

Relevant UNDP Strategic Plan result: Extracted verbatim from the UNDAF/CPD

Regular Resources

Other Resources

Page 31: Results-Based Management in UNDP

 

August 2011 Management Practice - BRC 31

Intended Outcome as stated in Country Programme:

Outcome indicators as stated in the CPAP, including baseline and target.

Partnership Strategy

Project Title and number

Intended Output Output Targets for (year)

Indicative Activities

Inputs

Output 1BaselineIndicators

Output2BaselineIndicators

UNDP ProjectsResults & Resources Framework

Page 32: Results-Based Management in UNDP

A few caveats

The results matrix is a storyline

It is a description of a programme or project strategy

Shows the intended paths we think are required to achieve desired results

The storyline comprises results, indicators & targets and assumptions & risks

August 2011 Management Practice - BRC 32

Page 33: Results-Based Management in UNDP

Results chain problems

Results not logically linked Results not sufficiently specific Results are composites of several results Results don’t express change (e.g. support provided

to strengthen….) Results statements are too wordy Confusion between levels of results Indicators

Not logically linked to the result Not measurable

Are new results

August 2011 Management Practice - BRC 33

Page 34: Results-Based Management in UNDP

Results Chain

August 2011 Management Practice - BRC 34

InputsInputs OutputsOutputs outcomes outcomes ImpactImpact

Experts; equipment; funds

People capacities improved; Laws/policies drafted

Policies adopted; Laws enacted;

Conditions improved health/ longevity

>

Partnerships and

other interventions

Page 35: Results-Based Management in UNDP

Results Chain

August 2011 Management Practice - BRC 35

Page 36: Results-Based Management in UNDP

Results Chain

August 2011 Management Practice - BRC 36

The likely or achieved short-term andmedium-term effects of an intervention’soutputs.

Positive and negative, primary and secondarylong-term effects produced by adevelopment intervention, directly or indirectly,intended or unintended.

Impact:

Outcome:

Outputs: Tangible products and services which result from the completion of activities within a development intervention.

Human Change

Products & ServicesSkills & Abilities

Institutional & Behavioural Change

Page 37: Results-Based Management in UNDP

Results Chain

August 2011 Management Practice - BRC 37

The likely or achieved short-term andmedium-term effects of an intervention’soutputs.

Positive and negative, primary and secondarylong-term effects produced by adevelopment intervention, directly or indirectly,intended or unintended.

Impact:

Outcome:

Outputs:

Changes in the lives of people

Tangible products and services which result from the completion of activities within a development intervention.

Institutional Change: values, ethic, rules, laws – associated with/to institutional performance, access.. Behavioural change: knowledge, skills acquisition,

practices (individual level)

Human Change

Products & ServicesSkills & Abilities

Institutional & Behavioural Change

Page 38: Results-Based Management in UNDP

Results Chain

August 2011 Management Practice - BRC 38

The likely or achieved short-term andmedium-term effects of an intervention’soutputs.

Positive and negative, primary and secondarylong-term effects produced by adevelopment intervention, directly or indirectly,intended or unintended.

Impact:

Outcome:

Outputs:

Changes in the lives of people

Tangible products and services which result from the completion of activities within a development intervention.

Institutional Change: values, ethic, rules, laws – associated with/to institutional performance, access.. Behavioural change: knowledge, skills acquisition,

practices (individual level)

Reduced infant and maternal mortality by 2018

National Public Works Agency has the management systems, equipment, and skills to

provide sanitation services to rural communities

Improved provision of public sanitary services to rural communities by 2015

Human Change

Products & ServicesSkills & Abilities

Institutional & Behavioural Change

Page 39: Results-Based Management in UNDP

A Typology for RBM

August 2011 Management Practice - BRC 39

HIV incidence reduced

Leadership empowered

Skills of Nat’l Aids Comm. strengthened

Train 250 district AIDS officers

Results Like… Focus @

Timeframe

<1 yr

<3 yrs

5 yrs

5-10 yrs

more

less

Collective

Accountability Institutional/

Behavioural

Operational/ skills, abilities,

products & services

Human!

UNOutcome

Impact

Output

Activity

if

if

if

then

then

then

Assumptions

UNOutcome

Impact

Output

Activity

UNOutcome

Impact

Output

Activity

Page 40: Results-Based Management in UNDP

A Typology for RBM: Governance

August 2011 Management Practice - BRC 40

Outcome

Impact

Output

Activity

More transparent governance

National capacity to implement governance

reforms increased

Legislative code for local self-governance

developed

- Training legislators - National consultation- Assessment of laws..

Results Like… Focus @

Timeframe

<1 yr

<5 yrs

5 yrs

5-10 yrs

more

less

Collective

Accountability Institutional/

Behavioural

Operational/ skills, abilities,

products & services

Human!

if

if

if

then

then

then

Page 41: Results-Based Management in UNDP

A Typology for RBM: Poverty Reduction

August 2011 Management Practice - BRC 41

Impact

Output

Activity

Poverty reduced

Regulatory environ. proposed to drive small enterprise

development

- Economic assessment - Training for chambers

of commerce - National consultation

Results Like… Focus @

Timeframe

<1 yr

<5 yrs

5-10 yrs

more

less

Collective

Accountability

Operational/ skills, abilities,

products & services

Human!

OutcomeEmployment and

income generation increased

Institutional/

Behavioural

5 yrs

then if

if

if

then

then

Page 42: Results-Based Management in UNDP

A Typology for RBM: Environment

August 2011 Management Practice - BRC 42

Impact

Output

Activity

Loss of environmental resources reversed

CBOs better abled to engage and mobilise

communties

- CBO trained - Community orientation

conducted- Small grants provided

Results Like… Focus @

Timeframe

<1 yr

<5 yrs

5-10 yrs

more

less

Collective

Accountability

Operational/ skills, abilities,

products & services

Human!

Outcome Protected areas are designated

Institutional/

Behavioural

5 yrs

then if

if

if

then

then

Page 43: Results-Based Management in UNDP

Outcomes

Let’s look at some examples…

Page 44: Results-Based Management in UNDP

How to articulate Outcomes

August 2011 Management Practice - BRC 44

Outcomes:

Outcomes are actual or intended changes in development conditions that interventions are seeking to support.

Some guides: 1. Avoid action verbs – ”Strengthening”, “enhancing”, etc

2. Avoid intentions – “To assist the government…”,

3. Use completed verbs: “…reduced”, “improved”, “have greater access to”, etc

4. Must signal that something has changed

5. The something which has changed must be important to the country/region/community, not just UNDP.

6. Avoid UN speak: gender mainstreamed

Page 45: Results-Based Management in UNDP

Typical pitfalls Wordy (..and no change language)

To promote equitable economic development and democratic governance in accordance with international norms by strengthening national capacities at all levels and empowering citizens and increasing their participation in decision-making processes

Too ambitiousStrengthened rule of law, equal access to justice and the promotion of rights

Containing multiple resultsThe state improves its delivery of services and its protection of rights—with the involvement of civil society and in compliance with its international commitments

August 2011 Management Practice - BRC 45

Page 46: Results-Based Management in UNDP

Typical pitfalls Wishy-washy (ie. Support provided to improve..)

Support to institutional capacity building for improved governance

So general, they could mean anythingTo promote sustainable development and increase capacity at municipal level

Overlapping with National goals/ MDGs (impacts) Substantially reduce the level of poverty and income inequality in accordance with the MDGs and PRSP

Confusing means and endsStrengthen the protection of natural resources through the creation of an enabling environment that promotes sound resources management

August 2011 Management Practice - BRC 46

Page 47: Results-Based Management in UNDP

Examples

Legal and regulatory framework reformed to provide people with better access to information and communication technologies.

The poor in x region have better access to capital and other financial services.

Reduction in the level of domestic violence against women by 2016

Increased regional and sub-regional trade

Higher and more sustainable employment and income for urban slum dwellers.

August 2011 Management Practice - BRC 47

Page 48: Results-Based Management in UNDP

Examples

By the end of 2010, user-friendly and sustainable health care and nutrition services are provided in compliance with international standards at national and sub-national levels.

Increased access to and completion of basic education, especially for girls.

By 2012, increased and more equitable access to and utilization of quality, integrated and sustainable basic services by the poor and vulnerable.

August 2011 Management Practice - BRC 48

Page 49: Results-Based Management in UNDP

Outputs

Tangible, deliverable, promises

Page 50: Results-Based Management in UNDP

Outputs: Definition & key features

Outputs are deliverables/end-project results Operational changes: new skills or abilities, the availability

of new products and services

Must be achieved within the project period

Managers have a high degree of control

If the result is mostly beyond the control or influence of the programme or project, it cannot be an output

Failure to deliver is failure of the project

3 to 6 outputs per agency outcome

Unless under a joint programme, outputs are NOT collective results

August 2011 Management Practice - BRC 50

Page 51: Results-Based Management in UNDP

Measuring Outputs

Easier to measure than outcomes (tangible!)

Indicators usually coming from existing data, assessments, analysis or from routine progress reports

Indicators can be ‘yes-no’

Or qualitative

1 to 4 indicator enough - the fewer the better to reduce costs

August 2011 Management Practice - BRC 51

Page 52: Results-Based Management in UNDP

Refining results…

August 2011 Management Practice - BRC 53

To strengthen the capacity of civil servants to do X by undertaking Y,…

The capacity of civil servants is strengthened to do X by undertaking Y,…

The capacity of civil servants in the4 poorest districts is strengthened to

do X by undertaking Y,…

The capacity of civil servants in the 4 poorest districts is strengthened to do X

by undertaking Y,…

Civil servants in the 4 poorest districts are better able to X

Let’s use results language to emphasis the future condition

we want to achieve.

All civil servants, everywhere? Can you be more specific? Are

there particularly weak or under-resourced civil servants we

should emphasise?

We can take out information that relates to either strategy

or activities.

Now, let’s try bringing the subject of change to the front, and shifting from

passive to active language.

Page 53: Results-Based Management in UNDP

Typical pitfalls

WordyUnclear logic and confused indicatorsOver-ambitiousPassive voice and wishy-washy wording

(ie. Support provided to improve, including but not limited to..)

Overlapping with Country Program outcomes or repeating activities

August 2011 Management Practice - BRC 54

Page 54: Results-Based Management in UNDP

Wordiness

Output 2.2.4 Strengthened capacities of central public authorities to develop, implement and monitor long term and linked mid term policies and programmes, including investment promotion and debt management

Suggest…

Output 2.2.4Central level authorities are

better able to make and monitor investment promotion

and debt management policies

August 2011 Management Practice - BRC 55

Page 55: Results-Based Management in UNDP

Good Outputs examples (?)

Market-based vocational training programme developed

A legislative framework and code for local self governance is drafted

National budget process more effectively incorporates inputs from local governance structures

All immunization centers have a functioning cold-chain, and adequate supply of vaccines and Vitamin A.

August 2011 Management Practice - BRC 56

Page 56: Results-Based Management in UNDP

August 2011 Management Practice - BRC 57

Good Output examples (cont’d)• National electoral body has adequate personnel, equipment

and skills to administer free and fair national and local level elections by 2012.

• Study of environment-poverty linkages completed• Police forces and judiciary trained in understanding gender

violence• National, participatory forum convened to discuss draft

national anti-poverty strategy• National human development report produced• Revised electoral dispute resolution mechanism established • Business processes reengineered• Compliance mechanisms established

Page 57: Results-Based Management in UNDP

Indicators

Measures of performance

Monitoring = How are we doing?

Indicators = How do we know?

Page 58: Results-Based Management in UNDP

Indicators

Indicators describe how the intended results will be measured - accountability

Objectively verifiable, repeatable measures of a particular condition

They force clarification of what is meant by the result …….the fine print!

Must be accompanied by baselines and targets

August 2011 Management Practice - BRC 59

Page 59: Results-Based Management in UNDP

Baseline, Target and Achievement

August 2011 Management Practice - BRC 60

Baseline

Commitment

CurrentLevel of

Achievement

Achievement

Performance

Achievement At end of period

Target

PlannedLevel of

Achievement

Page 60: Results-Based Management in UNDP

What to specify for each indicator?

Unit of analysis;Existing baseline information;Target for subsequent comparison;Expected perceptions or judgments of

progress by stakeholders;Detailed description of expected

conditions or situations to be observed;1-4 indicators for each result are

adequate – fewer the better to reduce cost.

August 2011 Management Practice - BRC 61

Page 61: Results-Based Management in UNDP

Indicators ‘indicate’ that change is happening or not happening

They can– Clarify the scale and scope of a result in the results

framework– Demonstrate progress when things go right– Provide early warning when things go wrong– Assist in identifying changes that need to be made in

strategy and practice– Inform decision making– Facilitate effective evaluation

August 2011 Management Practice - BRC 62

Page 62: Results-Based Management in UNDP

Indicators, Baseline, Target and Source of Data

August 2011 Management Practice - BRC 63

Legal and regulatory framework reformed to provide people with better access to information and communication technologies

Indicators: - # and proportion of the

population with access to the Internet, disaggregated by

gender- % of population with computers

and telephones-Extent to which key policies on

information technology are revised and promulgated

Source of Data:- National

Statistics- Monitoring

reports- SurveysBaseline:

-(based on analysis, capacity assessment, reports, national

statistics etc)Target: actual value

corresponding to each indicator, defined per year

Page 63: Results-Based Management in UNDP

Types of indicators

Factual indicators Factual (yes/no)

Existence (yes/no) Classes (x/y/z)

Policy recommendation submittedConstitution passed by ParliamentChamber of Commerce established Existence of free electronic media: free, partly free, not free

Numeric indicators Number

Percentage

Ratio

No. of government officials trainedNo. of regional networks on aid effectiveness createdNo. of regional CSOs attending regional conference on human rights

% of government budget devoted to social sectors% of population with access to basic health care

Ratio of female to male school enrolmentRatio of doctors per 1.000 people

August 2011 Management Practice - BRC 64

Page 64: Results-Based Management in UNDP

Qualitative indicatorsProcess changeBehavioral changesAttitudinal changes

ExamplesInclusive and participatory policy making processes in place in 10 countries in the regionPresence of a functional network of local governance practitioners in the regionPro-poor policies formulated Increased level of awareness on human rights among CSOs and governments in the region Quality of judiciary processes improved Improved perception of public on existence of free electronic mediaExtent of involvement of CSOs representing indigenous groups in national planning processesQuality of public-private partnership in the preparation of national plan of action on SME development

August 2011 Management Practice - BRC 65

Page 65: Results-Based Management in UNDP

Indirect (Proxy) Indicators

Direct indicators are preferable. However, they do sometimes not exist, are too expensive or inefficient to obtain. In such cases, indirect

indicator have to be used.

Purpose Direct indicator Proxy indicator as substitute

Improvements in the national judicial system

Quality of judicial cases improved -> difficult and expensive to obtain

Number of judicial cases challenged in the higher courtsRatio of cases filed to the cases processed in the judicial courts

Increase in per capita family income in the poorest state

household income -> household income survey needed -> not frequent enough

changes in local retail sales Increase in the total number of cell phone users in the state

To increase per capita income of small farmers

Crop sales and food consumption -> survey needed -> too expensive

purchase of typical consumer items improvement in buildings (e.g. roof) of life style (consumption of meat per week)

August 2011 Management Practice - BRC 66

Page 66: Results-Based Management in UNDP

August 2011 Management Practice - BRC 67

OUTPUT INDICATORS- District school teachers have

improved teaching skills- Number of teachers trained by end 2010- % of teachers trained that were rated as more

effective in doing their jobs 1 year later

- Quality national human development report produced, disseminated, and used to inform public policy.

1. Number of copies of NHDR distributed2. Percentage of Parliamentarians who receive copy3. Extent to which findings of NHDR were used to

inform high-level policy discussions and decisions4. Extent to which NHDR findings have influenced major

new programmes/initiatives or influenced budget allocation

- Civil society and community organizations in xxx regions have resources and skills to contribute to monitoring of local poverty reduction strategies

1. Number of NGO staff completing training courses in poverty analysis by end 2009

2. Percent of NGO staff who believe they are more effective in undertaking poverty analysis after they received training.

3. Percentage of districts with functioning Monitoring Committees (MCs)

- Draft New Policy on Climate Change formulated and submitted to Parliament

1. Progress made in drafting new policy.2. Stakeholder satisfaction with quality of draft new

policy.

Page 67: Results-Based Management in UNDP

Indicators and Means of Verification

1. A key test in ensuring whether an indicator is really good is to define “means of verification” for each indicator;

2. Are data sources available?

3. What does it take to obtain the data? Who needs to be involved? How much will it cost to obtain the data?

4. If means of verification are unclear or unrealistic, the indicator and, possibly, the result need to be revisited;

5. The definition of results, indicators and means of verification is an iterative process.

August 2011 Management Practice - BRC 68

Page 68: Results-Based Management in UNDP

Baselines and Targets

The indicator should be neutral – The target is what signals how much

change and in what direction

The baseline and target should use the same unit of measurement as the indicator

August 2011 Management Practice - BRC 69

Page 69: Results-Based Management in UNDP

The “SMART” way…

Specific Measurable

AttainableRelevant

Trackable

August 2011 Management Practice - BRC 70

Page 70: Results-Based Management in UNDP

What are good indicators?SPECIFIC The indicator needs to be as specific as possible in terms of quantity, quality,

time, location, target groups, baseline and target for the indicatorMEASURABLE Will the indicator show desirable change? Is it a reliable and clear measure of results? Is it sensitive to changes in policies & programmes? Do stakeholders agree on exactly what to measure?ACHIEVABLE Are the result(s) realistic and based on risk assessment, partnership strategy

and other factors contributing to the underlying resultRELEVANT Is it relevant to the intended result? Does it reflect the expectations and success criteria for change in the

target groups?TRACKABLE Are data actually available at reasonable cost & effort? Can proxy indicators be used? Are data sources known?

August 2011 Management Practice - BRC 71

Page 71: Results-Based Management in UNDP

Exercise for 10 mins

Page 72: Results-Based Management in UNDP

Input Activity Output Outcome Impact

Sector-specific expertise in disaster preparedness developed

Income of low-income families increased

To train community members in book keeping

By 2013, livelihoods of urban slum dwellers are more secure

International consultants to conduct study

Capacities created for an extended response to HIV/AIDS epidemic

New businesses and jobs are created in targeted, poor rural and urban areas

Study tour

National and local policies and plans are responsive to gender issues

By 2016, HIV/AIDS adult prevalence rate is less than 10 percent

Low-income families acquired the skills necessary to sustain micro-enterprises

Policy and regulatory environment improved for small enterprises

Pro-poor policies draftedAugust 2011 Management Practice - BRC 73

Exercise: Result Chain

Page 73: Results-Based Management in UNDP

Input Activity Output Outcome Impact

Sector-specific expertise in disaster preparedness developed

Income of low-income families increased

To train community members in book keeping

By 2013, livelihoods of urban slum dwellers are more secure

International consultants to conduct study

Capacities created for an extended response to HIV/AIDS epidemic

New businesses and jobs are created in targeted, poor rural and urban areas

Study tour

National and local policies and plans are responsive to gender issues

By 2016, HIV/AIDS adult prevalence rate is less than 10 percent

Low-income families acquired the skills necessary to sustain micro-enterprises

Policy and regulatory environment improved for small enterprises

Pro-poor policies drafted

August 2011 Management Practice - BRC 74

Exercise: Result Chain

Page 74: Results-Based Management in UNDP

Thank you!

Resource PersonPatrick Grémillet

Management Practice CoordinatorUNDP Bratislava Regional Center

[email protected]

This presentation is using slides/inputs from training materials developed by the UN Staff College and RBM

consultants

August 2011 Management Practice - BRC 75