Participation in Evaluation Practice: Idealistic or Achievable?

Dr. Ian Patrick, Ian Patrick & Associates

What is participatory evaluation?

Information Sharing Input Seeking

Group Problem Solving / Decision Making

Source: Kaner et al 2008

Participatory monitoring & evaluation (PM&E) is a process through which stakeholders at various levels:

• Engage in M or E of a particular initiative• Share control over the content, the process

and the results of the M&E activity• Engage in taking or identifying corrective

actionsWorld

Bank, 2003

Participation

Continuum of Participation

NoParticipation

Information Consultation Partnership Control

Self Mobilisation

Interactive Participation

Functional Participation

Participation forMaterial Incentive

Participation by Consultation

Passive Participation

Manipulative Participation

-

+

• Evaluations are often labelled as ‘participatory’, but often only involve consultation.

Relevance

Predominant Perspectives

Effectiveness

Efficiency

Impact

Sustainability

APPROACH

STRATEGIES

Control of the Evaluation Process

Stakeholder Selectionfor Participation

Stage / Depthof Participation

Participation in Evaluation

Evaluation DesignData CollectionData AnalysisDeveloping RecommendationsReporting of FindingsDissemination of Findings

Weaver and Cousins, 2004 & Cullen, 2009

Argument Against PE

“Simply amateur evaluation, with the only professional involved being self-excluded from exerting any control over conclusions drawn” (Scriven, 2005)

Elements of Participatory Evaluation

Values Commitment

Approach

Stages Methods

Time

Resources

Acceptability

Evaluator as Facilitator

Community / Organisation

Take StockSet GoalsDevelop StrategiesDocument Progress

EmpoweredOutcomes

Empowering Process

Empowerment Evaluation

Unambiguous values orientation - principle drivenEnd point is not assessment of program's worthFoster improvements and self-determinationOn-going process - program improvement & people's empowerment

Evaluation is not neutral- for one group to force social change- for another to reinforce the status quo

Most Significant

Change(MSC)

16

CollectStories

Changes inpeople's lives

Reviewand Select

Identifyfindings

Use Stories

RefineAction

- Knowledge, meaning and value ascribed through experience- Reality is interpreted differently by different people

Process reinforces group cooperation, learning and consensus buildingLearning and agreement about significant change, project achievement and appropriate directionsAction is based on what people truly valueComplements other monitoring and evaluation approaches

Outcome MappingVision

BoundaryPartner

BoundaryPartner

BoundaryPartner Boundary

Partner

OutcomeChallenge

ProgressMarkers

OutcomeChallenge

ProgressMarkers

OutcomeChallenge

ProgressMarkers

OutcomeChallenge

ProgressMarkers

ProgramImplementation

Unit

Mission

Progress Markers

Arguments for Participatory Evaluation

RightsRelevanceAccuracyEffectivenessProcess

Who would argue against participation?

Issues in PE• Increased time and resource demands• Difficulty managing multiple stakeholders• Lack of stakeholder skill or knowledge; and possible

low priority• Empowerment focus – ideological motivations or

evaluative?• Destabilising – unleash demands• Rigour and objectivity• Few studies of efficacy

Considerations when deciding on a PE• Why is participation being advocated?• Is there support for a participatory approach?• Whose voice counts and which stakeholder would

need to be included?• In what capacity could they participate?• How will you ensure that decision making about the

evaluation is inclusive?• Are there sufficient resources available (time,

funding and capacity)• Who is likely to benefit from the participatory

evaluation process, and from its outcomes?

PE depends ability of the evaluators to :

• Assess interest in & scope for a participatory approach to the evaluation

• Advocate for participation and its benefits• Establish positive working relationships• Establish ways in which participatory

methods will be used• Use strong facilitation skills to manage the

process and identify/mediate competing interests as they appear

• Ensure the process communicates findings