Masterclass Performance Measurement Framework

Performance Measurement Framework

Shared framework of indicators

ImpactAreas GuidingQuestion Indicator

LivelihoodandWellBeingArethebasicneedsofthe

farmersbeingmet?

FoodSecurity:Accesstosufficientfood

Income

AssetsPerceivedWell-Being

GenderWhatarewomen’srolesand

benefitsinthiscrop?

Women’sparticipationincrop

EquitableAccesstoTraining

ParticipationinDecision-Making

EnvironmentalPerformance

Isthelandwellstewarded? Adoptionofconservationpractices

FarmProductivity

Arefarmersrealizingthepotentialoftheirfarm?

Adoptionofgoodagpractices

EstimatedProductivity

CropRevenueorNetIncome

AccesstoServices

Dofarmershaveaccesstoservices?

Accesstocredit,trainingandinputs

Arefarmersusingtheseservices?

Useofcredit,training,andinputs

TradingRelationshipsArefarmersexperiencinggoodtradingrelationships?

TBD

What is a shared approach?

• A common framework of indicators and metrics to help guide the practitioner.

• A suite of indicators to pick what’s appropriate. Not a single set of indicators.

• Shared approach implies committing to use the same indicators and metrics when asking the same questions.

Why a shared approach?

From shared questions about farmers to shared approach for:

• Greater efficiency and effectiveness

• Reduced burden on suppliers and farmers

• More effective community learning

Criteria leading to framework

• Fewest questions (indicators) that give “sufficient” insight into livelihoods and performance.

• Affordability and scalability vs scientific robustness

• Simplicity vs nuance

• Embedding approaches for regular monitoring and reporting

Characteristics of Performance Measurement

Indicators and metrics appropriate for:

• Surveys under 30 minutes

• Minimally trained enumerators or even self reporting

• Across a wide range of supply chain types

Yields actionable data

Cost efficient enough to scale

Grew from overlap in a theory of change many share

Out of the ToC grew common learning questions

the Learning Questions dictated the Shared Framework of indicators

ImpactAreas GuidingQuestion Indicator

LivelihoodandWellBeingArethebasicneedsofthe

farmersbeingmet?

FoodSecurity:Accesstosufficientfood

Income

AssetsPerceivedWell-Being

GenderWhatarewomen’srolesand

benefitsinthiscrop?

Women’sparticipationincrop

EquitableAccesstoTraining

ParticipationinDecision-Making

EnvironmentalPerformance

Isthelandwellstewarded? Adoptionofconservationpractices

FarmProductivity

Arefarmersrealizingthepotentialoftheirfarm?

Adoptionofgoodagpractices

EstimatedProductivity

CropRevenueorNetIncome

AccesstoServices

Dofarmershaveaccesstoservices?

Accesstocredit,trainingandinputs

Arefarmersusingtheseservices?

Useofcredit,training,andinputs

TradingRelationshipsArefarmersexperiencinggoodtradingrelationships?

TBD

Scope of the Shared Approach Framework

Core reporting metrics

(a subset for reporting &

communication)

Shared PM Framework (detailed enough to be

“actionable”) Full Measurement Study

(will usually address additional questions)

Made-to-fit

Each organization will have it’s own purpose and goals for

measurement, but when we share learning questions in common, we can align on

indicators and increase the value of our data by gathering it

in a common way.

M&E in the Programme Cycle

Financing and

contracting

Final Evaluation

Implementation

Scoping

Project Design M&E

Strategy/Framework

Operational Planning M&E

Plan/Matrix

Partici

pation

Monitoring and

Evaluation

Impact Assessment

Year 1 Year 2 Year 3 Year 4 Year 5

M&E systems

Deep dive

Deep dive

Year 0

Baseline Impact Assessment

Interventions

Goal: Evaluate impact of specific changes (interventions) so that outcomes can be attributed to the specific interventions. Valid control groups (counterfactuals) are needed.

Performance Monitoring


Performance Monitoring

Chain wide indexes

Goals: Assess “status” and track change over time. Outcome change cannot be attributed to specific causes but trends can be tracked and correlations observed.

Mixing methods


(2) Performance measurement to track broad annual change on few key indicators (short survey by technician, 10 min)

Baseline

(1) Baseline to key issues, get stakeholder buy in, and tailor survey to specific context. Research + focus groups

Interventions

(3) Focused impact Assessment on small sample to answer specific causation question. Trained interviewers plus control group

Example of thinking process

Learning questions: Are the farmers consistently food secure in our supply chain? Is food security improving?

Indicators: Food security

Specific Metric: % of farmers with 2 months or more of food insecurity

Approach: Youth in coop run short interviews during annual meeting using Ipod-based system.

Survey question: Are there times in the year where you have to reduce meal size, skip meals, or change diet? Which times in the year? (number of months)

What you want to know

What you ask a farmer

Appropriate methodology depends on purpose (and budget!)

Purpose: Why do you want to know? What will you do with the results? Who is the audience?

Learning questions & population: What questions are you trying to answer? About whom?

Indicators, Metrics, and Questions: What are the appropriate indicators to track? What specific survey questions are appropriate and effective?

Approach: How and to whom are you going to ask the questions?

Embedding: How could this be part of an ongoing system?

Purpose

Learning Questions

ToC

Approach Indicators/

survey

Embedding

Of course -- depends on purpose, specific questions, and resources

Key Methodology Questions

How do we collect the data?

From whom do we collect data?

Who collects the data?

Self reporting Group setting Indirect Household

Group gathering point

Rough sampling

Voluntary individual

Statistical sampling w/control

Coop TA Youth Self Certifier Researcher

Of course -- depends on purpose, specific questions, and resources

Key methodology Questions





Rough sampling




household survey for rough baseline


Of course- depends on purpose, specific questions, and resources

Key Methodology Questions





Rough sampling





Masterclass Performance Measurement Framework

Business

Transcript of Masterclass Performance Measurement Framework