Group 4 ‘Shiny happy evaluators’

ALES AN JAN OLGA URSKA ZBYNEK

☺ ☺ ☺ ☻ ☺ ☺

Case 1: Social safety net program

• Benefits distribution – Officials– Neighborhood committee

• Timeframe> Between 1998-2008 (evaluation 2004)

• From 1 urban area to 40 by mid term – 80 urban areas at the end >>> national policy

• Independent from other allowances’ programs (e.g. other public services – education, health)

MoneyMoney DataData Hardware sys.Hardware sys. HR: staff + NC + trainersHR: staff + NC + trainers

•We have enough financial resources

•Data are relevant and reliable•Sys. of distribution established

• Fair and transparent NC

Training of staff + NCTraining of staff + NC Development of data process tools

Development of data process tools

Identification of eligible participants

Identification of eligible participants

•All staff working in the system receive training + understand the process

Identified eligible participants

Identified eligible participants

Trained staff + NCTrained staff + NC Functional databaseFunctional database

Distribution of benefitsDistribution of benefits• All eligible participants receive benefits

Poverty reduction in urban area

Poverty reduction in urban area

• Benefits are used wisely

Group 4 – An, Aleš, Jan, Olga, Urška, Zbynek

We’re almost half way…

Descriptive question

1. How were the beneficiaries selected?– 1.B. What were the criteria for selection ?

Measure or indicators: Comparison with evaluation of similar project; Triangulation

Data Sources: Rules and procedures on who qualifies, - internal manual

Design: One shot, Case study

Sample: One time

Data collection Instrument: Semi structured interviews, desk study

Data Analysis: Qualitative

Descriptive part of Evaluation Report

• Chapter 1: Role of the Stakeholders– Neighborhood committee

• Training of committee’s members• Terms of Reference

– Selection of beneficiaries• Determination of threshold

– Officials• Training for officials• Amount of officials needed

Descriptive statistics

0

5

10

15

20

25

30

35

40

45

50

zone 1 zone 2 zone 3 zone 4

Neighbourhood committees

Trained officials

Applicants (in 100)

Beneficiaries (in 100)

Normative question

2. Was the program effective in distributing benefits to eligible beneficiaries?

– 2.B. Have errors in distribution occurred?

(in other words: were eligible applicants excluded and some beneficiaries wrongly included?)

Measure or indicators: Threshold criterion; (or error : wrong inclusion or exclusion)

Data Sources: Sample of appraisal documents, semi-structured interview

Design: Causal tracing strategies

Sample: Random sample

Data collection Instrument: Semi structured interviews, secondary data analysis

Data Analysis: Quantitative and Qualitative, descriptive statistics

Applications in zone 1

Non eligibleapplicants

False beneficiaries

True beneficiaries

Cause & Effect question

4. To what extent did the program influence the economic activities of the eligible group?

4. A What was the effect of benefits distribution on unemployment rate among the eligible group?

Measure or indicators: Unemployment rate

(in areas with and without intervention)

Data Sources: Database of the regional authorities

Design: Interrupted Time series with comparison group

Sample: Census

Data collection Instrument: Secondary data analysis, Expert judgment

Data Analysis: Quantitative, descriptive, interpretation of association

Unemployment rate (%)

6

8

10

12

14

16

18

1998 2004 2007

zone A (pilot)

zone B (mid term)

zone C (nointervention)

Correlation analysis of side effects

  Benefits UnemploymentMigration

rate Birth rate

Support 1      

Unemployment 0,866 1    

Migration rate 0,945 0,982 1  

Birth rate 0,756 0,982 0,928 1

Were beneficiaries happier after the program?

• Suicide rate boring database analysis

• Amount of smiles/parties hidden camera

• http://vodpod.com/watch/140462-rem-muppets-furry-happy-monsters

THANK YOU FOR YOUR ATTENTION !