PM&E of PRSPs final

8/4/2019 PM&E of PRSPs final

1/52

World Bank Workshop: Voices and Choices at theMacro Level

Action Learning Programme on Participatory Processes forPRSPs

3-5 April, 2001. Washington DC

Presentation by Rosemary McGee and John GaventaInstitute for Development Studies

Brighton, UK

PM&E for PRSPs

Challenges and Lessons


2/52

The Context of PM&E

The growing importance of monitoring andevaluation

Moving participation down the project cycle:from whose reality counts? To who counts

reality?

Who measures matters


3/52

From Projects to Policies

Increasingly PM&E Is Used

By citizens, to hold larger programmes andinstitutions to account

As a tool for learning, for continuousimprovement for programmes and policyimplementation

As a tool for dialogue, amongst differingstakeholders across power and otherdifferences


4/52

Recent Examples of PM&E of Large-

scale Policies and Programmes

In the US, citizen learning teams monitored

the federal empowerment zone programme inrural areas

In the UK, a number of local organisations

are monitoring the affect of nationalneighborhood renewal programmes on localpeople


5/52

Examples

In the Philippines, the BATMAN coalition isusing PM&E to develop indicators of goodlocal governance, and to evaluate the

impacts of the decentralisation In Colombia, ACIN, an association of

indigenous councils, uses PM&E as part of aregional planning and development process

The NGO working group of the world bank,involved its members globally to monitor andevaluate the banks policies of participation


6/52

In Differing Parts of the World PM&E ProcessesOften Use Similar Steps

Decide who

Participates

EstablishGoals

DevelopIndicatorsAnalyse

Results Gather

Information

PM&E

Learning

Cycle

Take

Action


7/52

Steps of the Learning Cycle

Selecting participants: who will be involved? Intended beneficiaries? Primary stakeholders? Other stakeholders?

Choosing goals Which goals of the policy are most important and to whom? Whats the goal of the PME process itself: accountability?

Learning? Proving impact?

Developing indicators Indicators of success, even of similar goals, will vary by

stakeholders. Negotiation and flexibility if diverse voices areto be included


8/52

Analyzing and reflecting on what has been

learned A key part of the process, which if done

together, can build dialogue and learningacross stakeholders

Taking action Who needs to know for change to occur? How will the results be communicated and

used?

Gathering information A range of methods may be used, from

qualitative to quantitative, participatory toconventional, depending on the key purpose


9/52

Applying the Learning Cycle:

Key Lessons for PM&E of Policy Who measures matters - involving more

stakeholders will open up questions of what ismost important and to whom

PM&E is a social learning process, as much as atechnical process, involving trust building,negotiation and dialogue, all of which take time

Possibilities for PM&E of policy will varyaccording to the space for participation in a givencontext. The PM&E process itself can create

space by bringing differing stakeholders together


10/52

Key Lessons (Continued)

The methods for PM&E must fit the purpose andthe audience - is the PM&E for: Learning lessons for improvement? Building dialogue and capacities for stakeholders?

Proving impacts? Tracking accountability?

Norms of validity will vary by purpose

Capacity to do PM&E requires not only individualskills but institutional change: flexibility, ability tolisten and learn, openness to change


11/52

How Do These Lessons

Apply to PM&E of the PRSPProcess?


12/52

World Bank PRSP Sourcebook States:

But: Monitoring can be undertaken by civil society groups

Appropriate goals, indicators and methods can allowcivil society a more significant role in monitoring

Outcome monitoring can be achieved through household

surveys

Impact monitoring is a "demanding and complexmethodology generally carried out by researchers

outside the agencies that monitor outcomes

Civil society is seen to have a role in helping set goals anddisseminate information


13/52

Decide who

Participates

Establish

Goals

Develop

IndicatorsAnalyse

Results Gather

Information

PM&E

Learning

Cycle

Take

Action


14/52

Step One: Who Should BeInvolved in Monitoring and

Evaluation?

Mapping the population: stakeholder and

institutional analysis

Analysis of whether the required skills,relationships and resources for PM&Ecurrently exist


15/52

Stakeholder Analysis

The poor

Organisations representing poor people: communitygroups, religious leaders, trade unions, farmers unions,traditional authorities

The better-off

Central and local government

Politicians and political parties Academic researchers and analysts

The press and broadcast media

Donor agencies


16/52

Institutional AnalysisWhat existing relationships and networks existbetween stakeholder institutions?

A. Formally institutionalised groups

B. Livelihood/ interest/ identity groupsWhich institutionalised groups (A) representwhich livelihood and identity groups (B)?

What groups, networks and communities shouldbe involved in the PM&E process to make it asinclusive and workable as possible?


17/52

PM&E of PRSP Policy ProcessesRequires:

Skills Collaborative networks within & between civil

society and state and other responsibleinstitutions

Relationships of trust, accountability & learningbetween stakeholders

Spaces and policies which enable engagementand monitoring

Resources and time Structures for representation

Civil society


18/52

PM&E of PRSP Policy Processes

Requires:

Transparency and openness

Responsiveness and accountability Openness to institutional learning to enable

participation

Relationships of trust, listening andaccountability with civil society

Time

Government


19/52

PM&E of PRSP

by multiple civil

society

stakeholders

PM&E of PRSP

by 'external' civil

society

stakeholders, such

as International

NGOs

Weak CapacitiesStrong Capacities

What form of PM&E is possible given the

existing capacities, relationships and resources?


20/52

PM&E of PRSP

by multiple civil

society

stakeholders

PM&E of PRSP

by 'external' civil

society

stakeholders, such

as International

NGOs

Weak CapacitiesStrong Capacities

What form of PM&E is possible given the

existing capacities, relationships and resources?


21/52

Forming a Steering Committeeand Structure for PM&E

It is vital that these different activities are co-ordinated toenable:

Analysis of findings

A formal mechanism for engagement with responsible

institutions Scope for public debate on findings

A vision of the national picture: inclusions & exclusions

Communication and information

PM&E by multiple civil society stakeholders involves manymonitoring activities being undertaken in different areas andissues and at different levels


22/52

Zambia: Developing a MonitoringSteering Committee

Civil society groups were invited by government toparticipate in the process of formulating the PRSP.

Co-ordinated by the jesuit centre, civil society groupsestablished a network and steering committee of 14csos: 'civil society for poverty reduction'(CSPR).

The responsiveness of government to the CSPR

monitoring and evaluation is enabled through aCSPR seat in the government PRSP working group.


23/52

Poverty monitoring

steering committee

Research &

analysis group

Routine data

collection group

Surveys &

census group

Dissemination

group

All of these groups have civil society representation.In addition, wider scale participation of civil society and poor people

in monitoring through PPA mechanism facilitated through the

Research and Analysis group

Tanzania: Structure for Monitoring and Evaluation


24/52

Key Lessons:

Successful multi-stakeholder PM&E requires certainskills and capacities

There are a range of different forms & structures for

civil society engagement in monitoring PRSPs It isimportant to allow space for different stakeholders toparticipate in monitoring in different ways

Appropriate and accountable institutional mechanismslinking civil society monitoring to state and othermonitoring processes are crucial to ensure that civilsociety monitoring is not sidelined


25/52

Decide who

Participates

Establish

Goals

Develop

IndicatorsAnalyse

Results Gather

Information

PM&E

Learning

Cycle

Take

Action


26/52

Step two: Establishing goals & intermediate goals

Key Questions:

What type of goals are to be monitored? Reduction in income poverty? Improvements in service delivery? Principles? Processes?

Who defines the goals and intermediate goals? Ideally, goals are defined through dialogue between multiple

stakeholders. The defining of goals through partnership is central to

PM&E.

What mechanisms are there to hold responsible parties accountablefor particular goals?

Each goal should be backed up with agreed accountability mechanisms

W ld B k PRSP S b k


27/52

Finalgo

als

In

termediate

goals

Monitoring inputs Where are resources being allocated? Public expenditure reviews

Monitoring outputs What goods and services are generated by the project?

What is the quality and scope of these outputs?

Monitoring outcomes Who are the beneficiaries? Do the poor have improved access to goods and services?

Are these goods and services appropriate, & for whom?

Monitoring impacts What is the impact on poverty & meeting different needs? Who is healthier, better educated, better employed?

World Bank PRSP Sourcebook:Guidelines on Monitoring


28/52

These are all poverty monitoring goals but do

not address PRSP principles or the participatoryprocess.

By most contemporary definitions, poverty is morethan material wellbeing (income levels, health,education provision). Powerlessness and

voicelessness are also dimensions of poverty.

Monitoring of the PRSPs principles and of the

participatory process, focuses attention onempowerment, voice and influence as goals andindicators of poverty reduction .


29/52

Monitoring

Poverty

Monitoring

Principles

Monitoring

the

participator

y process

What type of

goals to monitor?


30/52

Monitoring

Poverty

Monitoring

Principles

Monitoring

the

participator

y process

What type of

goals to monitor?


31/52

Tanzania: Poverty monitoring goals

Through regular PPAs, civil society is offeredopportunities to express its perspectives on:

Nature of changes in poverty

Causes behind changes in level & nature of poverty

Impact of different policies and strategies on the poor

Tanzanias inclusion of PPAs in the overall Poverty

Monitoring Strategy ensures that civil societyparticipates in monitoring progress towards PRSPtargets


32/52

Monitoring

Poverty

Monitoring

Principles

Monitoring

the

participator

y process

What type of

goals to monitor?


33/52

Monitoring PRSP principles

Aim:To hold government institutions andInternational Financial Institutions to account

for the principles underpinning the PRSP Country ownership

Results-orientated

Comprehensive Long-term in approach

Based on partnership


34/52

Example: Monitoring country ownership principleBroad: Multiple sets of stakeholders feel that they share

ownership of the processes and outcomes

Goals: Government & donors engage in a systematic learningprocess, through which they consciously step back fromtheir traditional dominant positions

Different sections of civil society perceive policy to be adomain over which they can and have a right to own andinfluence

Intermediate: Capacity of civil society & government to jointly own the

Goals PRSP is strengthened through institutional development,(3-5 years) relationship-building & reflection processes

The meaning of 'country ownership', and attaining thisgoal, is agreed between different stakeholders, andmechanisms for accountability to this goal are defined


35/52

Monitoring

Poverty

Monitoring

Principles

Monitoring

the

participator

y process

What type of

goals to monitor?


36/52

Monitoring the participatory process

Aim:

To evaluate whether participatory PRSPprocesses lead to the assumed benefits


37/52

Monitoring the participatory process

The process itself

Form, level, quality and inclusiveness of participation

Focusing on:

The developmental benefits of the processThe impact of participation on self-development

The instrumental benefitsThe impact of participation on policy, decision-makingand other broader changes


38/52

Good quality and inclusive participationitself contributes to poverty reduction by:

Making better policy

Empowering participants & enabling them todemand more responsive institutions

Building participants' capacities to dialogue,negotiate & advocate


39/52

Indicators for monitoring policy process- COGS

Framework, Yorkshire Council UK.

Principles BENCHMARKSInfluenceEnsuring thatparticipation leads toreal influence

Local stakeholders:Valued as equal partnersRepresented on decision-making bodiesOpportunities to participateAccess and control over resources

Evaluation includes local stakeholder agendaInclusivityValuing diversity &addressing inequality

Diversity of local stakeholder interests reflected in policyand process

CommunicationClear, transparent &

accessible policies &procedures

Two-way information strategyPolicy & strategy are clear & accessible

CapacityUnderstandingdifferent partners &their capacities

Local stakeholders - resources to participateUnderstanding, knowledge & skills are developed to makea working partnership


40/52

Key lessons:

Different stakeholders are interested in differentgoals and have different capacities It is crucial

that space is given for different stakeholder

groups to define different goals

If goals differ among stakeholders, processes ofnegotiation over goals are required at the outset

to identify a number which can feasibly bemonitored, and to ensure that the outputs will be

carried through to action


41/52

Decide who

Participates

Establish

Goals

Develop

IndicatorsAnalyse

Results Gather

Information

PM&E

LearningCycle

Take

Action


42/52

Step three: Developing indicators

What goals can be measured, through which indicators?

What indicator data is it practicable to collect and analyse?

Is quantitative or qualitative information needed?

Which stakeholders have the capacity to collect & analyse?

Who chooses the indicators?Different stakeholders may choose different indicators to monitor thesame goal

Will these indicators be effective for holding government andcivil society stakeholders accountable?If monitoring is to have influence, different stakeholders must agreeon the validity of indicators, and be willing to account for their actionsas measured by them


43/52

Key lessons:

Not everything can be monitored

Indicators should be carefully chosen and related to priority goals

Identifying indicators for intangible goals is not an exact science

If people attach high priority to intangibles, they can probably deviseways of measuring their attainment

UNDP Tanzania statement on choosing indicators:

"Agreement has to be reached among the stakeholders on a priority list ofindicators for poverty monitoring. There is a balance to be struck betweencomprehensiveness and affordability. In the case of Tanzania, middleground has to be found between the extensive list of poverty and welfareindicators and the much shorter core list presented in the PRSP".


44/52

Example: Indicators to monitor principle ofcountry ownership

INPUT State resources allocated to increase quality & scope ofIndicators participation, transparency and accountability.

Civil society seeks resources to support theirparticipation and advocacy

PROCESS Initiation of meaningful participation beforePRSP goals are set

Indicators Civil society demands responsiveness and accountabilityas conditions for their participation

OUTPUT Meaningful participatory arenas opened by governmentIndicators across country and include all stakeholders who want to

participate

Civil society facilitation of participatory analysis & monitoring withconfidence that government will accountable for the results

OUTCOME PRSP includes needs and priorities of wide range of stakeholdersIndicators Increased civil society and government capabilities and

opportunities to lead PRSP and negotiate with donors and creditorsover it


45/52

Decide who

Participates

Establish

Goals

Develop

IndicatorsAnalyse

Results Gather

Information

PM&E

LearningCycle

Take

Action


46/52

Step four: Gathering information

An information-gathering strategy needs to bedeveloped, including how resource needs willbe met

A baseline is needed point of comparison fortracking changes over time

Qualitative or quantitative information?

Which information gathering tools?

Formal or informal approaches?


47/52

Decide who

Participates

Establish

Goals

Develop

IndicatorsAnalyse

Results Gather

Information

PM&E

LearningCycle

Take

Action


48/52

Step five: Analysing and reflecting onwhat has been learnt

Analysis of findings feeds into critical review of whatever isbeing monitored reflection on PM&E process feeds into a

better PM&E process next time round

The PRSP was conceived as a dynamic, iterative framework inwhich both content and process could be continually reviewedand improved on

This places considerable, ongoing demands on leaders andparticipants of PRSPs and implies a need for sustained

investment in capacity-building for diverse stakeholders


49/52

Decide who

Participates

Establish

Goals

Develop

IndicatorsAnalyse

Results Gather

Information

PM&E

LearningCycle

Take

Action


50/52

Step Six: Taking action for improvement

Improvements to PRSPs content, the participatory process oradherence to underlying principles

Links between the civil society PM&E process and national PRSPSteering Committee will be crucial for achieving this

Improvements in PM&E process may take various forms:Indigenouscivil society in a more central role

More skilled application of information-gathering tools

More strategic ways found to spread and apply the lessons learntCivil society actors can take action themselves but cannot forceGovernments or donors to do so

How committed are government and donors to learning and changing

as a result of PM&E?


51/52

Example: Bolivia

One proposal: to establish a civil society monitoring system, parallel tothe Governments (concern over accountability and transparency of

government institutions)

Alternative proposal: broad, long-term effort to address these concernsby gradually building capacity in participatory approaches among

government and civil society

Proposed 3-year programme of support to National Working Group onParticipatory Approaches:

To strengthen the capacity of the National Working Group to facilitate and

support participation of local people and other actors in planning, decision-making and project activities related to poverty reduction at the local, regionaland national levels

Slow, expensive, work-intensive


52/52

Key lesson:

There is no quick fix to establishing PM&E ofPRSPs

participatory processes take time, cost money

and demand skills

If poverty reduction is about increasing poorpeoples voice, empowering them to address their

own problems, and making government and donorinstitutions more accountable to them, the benefitsof PM&E outweigh the costs and go beyond whatconventional M&E can offer

PM&E of PRSPs final

Documents

Transcript of PM&E of PRSPs final