2012 EES Peace Precarious Workshop full slides
Transcript of 2012 EES Peace Precarious Workshop full slides
![Page 1: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/1.jpg)
M&E and evaluation for development
in peace precarious situations
Catherine Elkins, PhD, MALD Belling the Cat LLC–Duke University EES Workshop 01 October 2012
![Page 2: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/2.jpg)
Assisting development: Is our territory cognita?
2
![Page 3: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/3.jpg)
Our day’s agenda
• Sessions and schedule in handouts – Strategic contingency approaches – Case studies over time – Six-dimensional analytical model – Lessons for peace precarious situations and lessons
from peace precarious situations
![Page 4: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/4.jpg)
What is ‘normal’ development?
• “Think–pair–share” • Define
– elements of typical development context – typical development goals – typical development stakeholders
– typical development implementers / partners
![Page 5: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/5.jpg)
Peace Precarious Situations
photo © Joe Holmes 5
![Page 6: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/6.jpg)
What is ‘peace precarious’ development?
• Differences we experience in
– context – goals – stakeholders
– implementers / partners
![Page 7: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/7.jpg)
Peace precarious situations
7
• May overlap circumstances labeled in other discussions as fragile, low-level violence, conflict, post-conflict, failed states
![Page 8: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/8.jpg)
Peace precarious situations
8
• Power disputes • recent, ongoing, or chronic • violent at least sometimes
• Destabilized but not obliterated core institutions
• No directional assumptions • No ‘state’ assumptions
![Page 9: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/9.jpg)
Introductions: Catherine Elkins
• http://linkedin.com/in/seaelkins • Professor, Duke University; Consultant • About 20 years’ experience working in
developing countries • Afghanistan, Cambodia, Eritrea, Guinea,
Guyana, Iraq, Kazakhstan, Lebanon, Mauritania, Pakistan, Rwanda, South Africa, Uganda, West Bank, Zimbabwe
• Belize, Jordan, Senegal, Tanzania, Thailand
![Page 10: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/10.jpg)
Introductions: Colleagues
• Name, main occupation • Evaluation (or M&E) experience • Development experience • Peace-precarious experience
![Page 11: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/11.jpg)
Introductions: Cases 1, 2
11
![Page 12: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/12.jpg)
Introductions: Case 3
![Page 13: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/13.jpg)
Development in peace precarious situations
USAID 2006, Provincial Reconstruction Teams in Afghanistan: an Interagency Assessment (PN-ADG-252) . 13
• Extreme cases: International development efforts ongoing despite widespread internal or regional areas characterized by a “mid-range of violence”
• “Instability still precludes heavy NGO involvement, but […] is not so acute that combat operations predominate”
![Page 14: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/14.jpg)
Critical dimensions
14
![Page 15: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/15.jpg)
Key aspects for M&E/Evaluation • Volatility • Internal and external ‘constants’ are
potentially/probably variable • And especially: Multiple parameters can
change at once
Peace precarious situations
![Page 16: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/16.jpg)
Activity
• Afghanistan – external evaluation – What do we know about the 6 dimensions? – What else do we need to know?
– Ideas for evaluation plan • In principle • 20 questions
![Page 17: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/17.jpg)
Focus on stakeholders
17
![Page 18: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/18.jpg)
Stakeholder complexity & sensitivities
Sources for volatility • Who has a stake in project failure?
• Who has mixed interests, hedged bets?
• What additional political or other concerns do those supporting success need to consider?
18
![Page 19: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/19.jpg)
Lunch
19
![Page 20: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/20.jpg)
Theories of change
• Hypothesized relationships among influential factors shaping conditions or behavior of interest
• Probabilistic causal expectations • Observable, measurable, falsifiable model • Mechanisms expected to produce results
20
![Page 21: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/21.jpg)
Implementation – or so we like to think
21
![Page 22: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/22.jpg)
Theories of change and peace precarious development
• “Normal” development interventions – Complicated – Complex
• “Peace precarious” interventions – Complicated, complex, and volatile – Relationships between good program design &
good M&E/evaluation design in peace-precarious situations
22
![Page 23: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/23.jpg)
Peace precarious stakeholders and planning
• From initial exercise • From Afghan case exercise • From own experiences
• Relationship dynamics and contingencies
![Page 24: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/24.jpg)
Activity
• Guinea governance M&E – Dynamics over time
– http://www.reuters.com/article/2010/06/27/us-guinea-events-idUSTRE65Q0ZP20100627
Project spr’08
Army mutiny sum‘08
Coup Dec’08
Massacre Sep’09
Coup attempt Dec’09
2 intra-coup
transitions Jan-Feb’10
Pres election June’10
Run-off results final
Nov’10
Legislative elections fall ’11** (not yet!)
![Page 25: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/25.jpg)
Activity (cont’d)
• Guinea – Initial design – Big disruption – Ongoing disruptions
![Page 26: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/26.jpg)
M&E strategy
• Guinea governance M&E – Model and measurement
– Elasticities
![Page 27: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/27.jpg)
Afghanistan and Guinea
• M&E and evaluation dynamics – Same or different challenges?
– Documentation
![Page 28: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/28.jpg)
Model (theory of change) and measurement
28
‘Normal’ vs peace precarious challenges • If–then hypotheses of predicted
relationships – Falsifiable using empirical evidence – Discriminating paths among multiple influences – Quasi-experimental counterfactual; may be very
loose “other things being equal”
• Baseline, implementation, results
![Page 29: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/29.jpg)
Fidelity and slippage
• Contextual constants and variables • Cost and implementation elasticities • Articulation of expectations, execution, M&E,
and evaluation including changes from design through close-out
• Learning systems
29
![Page 30: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/30.jpg)
Egypt, girls’ education, and “revolution”?
globalgiving.org
![Page 31: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/31.jpg)
31
Punctuated equilibria • Full or partial disruptions, interruptions,
unpredictable timing • Expansions and curtailments of components,
flexible scope • Nature and timing of specific shocks or factor
changes (context, institutions, stakeholder/counterpart composition)
Recap: Exploring the cases
![Page 32: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/32.jpg)
Exploring the cases – coping under stress
32
Cost factors • Fluctuation in budget • Fluctuation in security expenses • Staffing through interruptions, evacuations • Recruiting, screening, incentives, and turnover
![Page 33: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/33.jpg)
Measurement risk factors
33
Data quality and precision • Human capital and stress effects on timeliness,
completeness, accuracy • Perceived risk and risk attitudes
– Risks to participants, beneficiaries, enumerators
– Transparency and accountability risks • Existence, accessibility, reliability of secondary
data sources
![Page 34: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/34.jpg)
Information uses
Lower confidence in data and…
• Potential for political or other unintended uses of project results and evaluation findings?
• Likelihood that certain actors and audiences will misunderstand or misuse findings?
• Potential damage to any stakeholders from use or misuse of results or findings?
34
![Page 35: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/35.jpg)
Information and stakeholders in context
• International context and global dynamics – Goals, uses, audiences, information/security
dimensions affecting development results information
– Six dimensions across our cases
– Appropriate, useful M&E/evaluation? • Possible? • Likely? • Under what conditions?
35
![Page 36: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/36.jpg)
Cross-cutting conclusions and challenges
36
![Page 37: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/37.jpg)
Gaps analysis and theory building
• What are we missing?
• What else can we bring to bear?
37
![Page 38: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/38.jpg)
Strategy
38
Where program and evaluation concerns meet • Design evaluation around challenges to project
and ways challenges are most likely to affect evaluation
• Identify specific decision points – More than one theory of change? – Client understanding or learning curve? – Variance across stakeholder interests? – Data needs for methodology options? – Budget requirements for strategic options?
![Page 39: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/39.jpg)
Design for development
39 handout
![Page 40: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/40.jpg)
Useful M&E/evaluation (peace precarious)
• Questions: – Adequacy of development models for (some)
peace-precarious situations – Adequacy of M&E/evaluation approaches for
(some) peace-precarious situations – Managing expectations and validating deliverables – Ethics and other issues
40
![Page 41: 2012 EES Peace Precarious Workshop full slides](https://reader031.fdocuments.in/reader031/viewer/2022030310/58f05a251a28abd95d8b45a7/html5/thumbnails/41.jpg)
Final Questions?
41
Thanks!