Attributing Variation in a Regional Climate-change Modelling Experiment
How to evaluate ultimate impact of value chain interventions? Mixed methods design for attributing...
-
Upload
irene-taylor -
Category
Documents
-
view
219 -
download
2
Transcript of How to evaluate ultimate impact of value chain interventions? Mixed methods design for attributing...
How to evaluate ultimate impact of value chain interventions? Mixed methods design for attributing indirect interventions to farmers’ income.
The case of maize in Bangladesh
Gideon Kruseman and Shovan Chakraborty25 March 2013
Klik op het pictogram als u een afbeelding wilt toevoegen
Overview
Case
Evaluation questions
Design (conference theme)
Impact logics
Mixed methods design
Results
Conclusions
Communication and results (conference theme)
Lessons learned
Case description
2004 Katalyst introduced maize through
Retailer training
Extension officers training
Contract farming
2011 impact evaluation indicated strong income impact but method lacked before-after and with-without
2012 new impact evaluation based on rigorous scientific methods
Questions
1. What ultimate impact can be expected 8 years later?
2. How can that be measured?
No baseline possible in VC (uncertain who will benefit ex-ante)
Difficult to determine comparison group with notorious spill-over effects
3. What is the impact?
Outreach
Impact
Attribution
Q3 challenging
-> Mixed
methods
Q1&2 uncommon
-> Multi-
disciplinary team with
knowledge of value chains
Design of IE
What expectations underpinned choices for design?
1. There is impact that is attributable to the intervention
2. Measurement of impact is limited by lack of baseline, no clean control groups and many external factors
3. Attribution is strongest low in impact chain (with retailers, officials and companies, not with farmers)
4. If intervention is unique, attribution is easier to establish
Impact logics and the IE (Q1)
Level of attribution of Katalyst interventions to
outcomes
Demonstration Effect: other farmers grow maize
Ultimate outcomesKatalyst’s impact)
Intermediate outcomes
Immediate outcomes
InterventionPartner with company to
promote quality seeds and info (2008-2010)
Client farmers get quality seeds and info on cultivation
techniques and improve practices
167 Partners (120 retailers, rest dealers, agrovets, MSVs,
farmers) promote quality seeds & cultivation techniques
Client farmers get high yields and income
Partner with company to establish contract farming
(2008 - 2011)
Contracted farmers get info on cultivation techniques, buy-
back, finance and inputs (seeds) and improve practices
50 contractors (retailers/traders) start contract farming
Contracted farmers get high yields and income
Partner with government to improve extension services
(2009 – 2011)
Trained farmers get info on cultivation techniques and
improve practices
75 extention workers start promoting maize and cultivation techniques
Trained farmers get high yields and income
Other sources of quality seeds
Other sources of quality seeds
Macro-economy and climateMacro-economy and climate
Other information sourcesOther information sources
Other information sourcesOther information sourcescopy other contractorscopy other contractors
Retailer training Contract farming Maize-based cropping
Focu
s as
sess
men
t
Design (Q2)
Approach
For each question test assumptions of the impact logic at every step
Core methods
Large n farmer survey data
Small n in-depth interviews with farmers, contractors and retailers, treated and non-treated
Maize sector study
Analysis
Production cost-revenue analysis
Factor analysis
Qualitative analysis of in-depth interviews
Mixed methods (Q3)
What outreach?
In-depth contractors interviews
Validation with in-depth farmers interviews
What income effects?
Large n farmer survey for production cost analysis and quantitative cohort comparison
Validation with in-depth farmers interviews
To what extent can impact be attributed?
Qualitative analysis of all in-depth interviews
Factor analysis of large n survey
Results: outreach
Farmers affected by contract farming
char new char old
mainland new
mainland old Total
Direct outreach 867 2203 966 1753 5.789
Indirect outreach 1185 17675 510 4058 23.428
Results: income effect
Relevant assumption to be tested: farmers have higher yields than the benchmark
char new char oldmainland new
mainland old copy
yield (maund/dec) 35 87 5 9 18
Total Rev. Impact not considering Land size Change (BDT) 17,410 42,980 1,938 3,879 7,947
Total Rev. Impact including Land size Change (BDT) 38,523 80,757 4,742 10,399 16,902
Total income increase (1 yr after intervention) 33,399,450 177,908,515 4,580,293 18,230,162 395,988,876
Results: contribution of Katalyst
Contract farming has started as a result of Katalyst interventions.
●Only contractors are those involved in the intervention
●These contractors have growing number of contract farmers
Conclusion: True
Knowledge passed on through Katalyst training of contractors is crucial for contractors
●Knowledge comes from many sources including Katalyst
Conclusion: True / False
Results: contribution of Katalyst
Knowledge passed on from contractors to farmers is crucial for farmers
●Knowledge comes from many sources including Contractors
●Contractors are main source of information
Conclusion: partly true
Service provision by contractors is important to contract famers
Conclusion: true
There is a spill over effect of maize cultivation from contract farmers to neighbouring farmers
Conclusion: true
Results: contribution of Katalyst
Knowledge passed on through Katalyst training of Retailers is crucial for retailers
●Knowledge comes from many sources including Katalyst
Conclusion: True / False
Knowledge passed on from retailers to farmers is crucial for farmers
●Knowledge comes from many sources including retailers
Conclusion: True / False
Conclusions
There is impact of contract farming especially because of service provision specifically related to this production form
The impact of contract farming is 100% attributable to Katalyst
Knowledge on maize cultivation comes from many sources, knowledge is vitally important but relative importance of information sources cannot be attributed to any single source.
There is a contribution to the knowledge base through Katalyst interventions
Communication and use of IE
Communication is focussed on:
Underpinning and justifying estimated impact
To which interventions could this be attributed
Some interventions (contract farming) have attributable impact
Some interventions (retailer training) do not
Results are used for:
Fine-tuning current interventions
Design of improved monitoring of interventions and immediate and intermediate results to ensure more robust future impact evaluations (counterfactual analysis)
Lessons learned
Importance of impact logic framework
Importance of defining hypothesis based on impact logic
Importance of design based on hypotheses and application of mixed methods to overcome threats to validity of conclusions
Separation of impact and attribution/contribution
Need to inform donors on costs benefits of IE