1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns...
-
Upload
moses-hawkins -
Category
Documents
-
view
219 -
download
1
Transcript of 1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns...
![Page 1: 1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns Human Development Network November 2006.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649ebb5503460f94bc3151/html5/thumbnails/1.jpg)
1
Measuring Results and Evaluating Impact:
What You Need to Know
Paul Gertler and Barbara BrunsHuman Development Network
November 2006November 2006
![Page 2: 1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns Human Development Network November 2006.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649ebb5503460f94bc3151/html5/thumbnails/2.jpg)
2
How do we turn this teacher…
![Page 3: 1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns Human Development Network November 2006.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649ebb5503460f94bc3151/html5/thumbnails/3.jpg)
3
…into this teacher?
![Page 4: 1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns Human Development Network November 2006.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649ebb5503460f94bc3151/html5/thumbnails/4.jpg)
4
What do we need to know??
• How many teachers • How many are performing like teacher A, like
teacher B, or absent altogether• How many kids:
– enrolled in school– attending school– repeating grades– dropping out – completing primary school
• Trends over time: are things moving in right direction?
![Page 5: 1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns Human Development Network November 2006.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649ebb5503460f94bc3151/html5/thumbnails/5.jpg)
5
What do we need to know??
• Student learning – which kids are learning and how much? – who is benefiting?
• Costs (per student) • What inputs, policies and incentives have greatest
impact on learning? – remedial teachers, multi-grade teacher training,
deworming pills– higher teacher standards, curriculum focused
on early grade reading– contract teachers, school-based management,
vouchers• Which are most cost-effective?
![Page 6: 1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns Human Development Network November 2006.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649ebb5503460f94bc3151/html5/thumbnails/6.jpg)
6
Message 1: Monitoring and Evaluation is our core business
• All TTLs need tools and techniques for– Better monitoring of Bank/IDA operations:
• defining project development objectives clearly• credible results chains • selecting indicators
– Building client capacity for M&E – Impact evaluation
• Both are important, and some complementarities– Standardized student learning data and good
administrative data (health facility surveys, school census data) lower costs and improve quality of impact evaluations
![Page 7: 1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns Human Development Network November 2006.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649ebb5503460f94bc3151/html5/thumbnails/7.jpg)
7
Message 2: We’re not doing a very good job at present
• All Networks struggling with need for better monitoring and more impact evaluation– INF study: over 200 different indicators in water supply
projects, measuring similar things, most w/only 1 obs.
– OPCS study: 78 education projects exiting in FY04-05, • Only 3 reported MDG (primary completion rate)• Only 5 had any data on student learning – and none
comparable• Less than 50% tracked enrollments (mix of GER,
NER, raw numbers, gender)– In S Asia HNP projects, avg. operation had 26 indicators,
only 33% with clear definition, baseline and collection schedule
![Page 8: 1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns Human Development Network November 2006.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649ebb5503460f94bc3151/html5/thumbnails/8.jpg)
8
Little Progress of M&E Over Time(from Ben Loevinsohn and Aaka Pande)
0
10
20
30
40
50
60
70
80
90
Indicatorsmeasurable
Clear target Baseline datacollected
Follow-up datacollected
Data collectionroughly in
keeping withPAD
Before 2000
After 2000
![Page 9: 1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns Human Development Network November 2006.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649ebb5503460f94bc3151/html5/thumbnails/9.jpg)
9
Message 3: HD under special pressure with “results agenda”
• Greater accountability for major global goals– 2/3 of MDGs are HD outcomes– HD outcomes important in IDA 14 results
• build country capacity to measure these outcomes
• link PDOs to “higher level” sectoral or MDG outcomes
• Greater role in “selling” aid – OECD electorates focused on HD issues – selling IDA 15 on “results”, esp. HD results
![Page 10: 1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns Human Development Network November 2006.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649ebb5503460f94bc3151/html5/thumbnails/10.jpg)
10
Message 4: there is help….• Tools
– Checklist for M&E in HNP operations, also education– Guidance on indicators (AFR Generic Results
Framework for HIV projects)– Common instruments for classroom observation and
student learning assessment • Technical advice
– “on demand” impact evaluation clinics– FPSI “on demand” PDO/monitoring clinics
• Financing– BNPP funding to launch prospective impact evaluations
• Training
![Page 11: 1 Measuring Results and Evaluating Impact: What You Need to Know Paul Gertler and Barbara Bruns Human Development Network November 2006.](https://reader035.fdocuments.in/reader035/viewer/2022062409/56649ebb5503460f94bc3151/html5/thumbnails/11.jpg)
11
Plan for next 3 daysTODAY• Understanding the value of impact evaluation• Understanding the results agenda from an institutional perspective• TTL experience in improving M and E• 4 hour module on Improving “M” -- results focus in projects
TUESDAY • TTLs will meet in MC-5-100• Researchers will meet in IFC (F-5-P100)• All day on impact evaluation – methods, operational issues
WEDNESDAY• Everyone will meet in MC-4-100• Teams organized along sector lines will prepare evaluation designs
for projects of their choice & write concept notes• Teams present in evaluation clinics