Lessons from Meta-Evaluation of MGNREGA From a Gender and Equality Lens

Ranjani.K.MurthyEngendering Policy Through Evaluation

Project, ISST, New Delhihttp://gendereval.ning.com/

Objectives of meta-evaluation

Sensitive to gender and social

equity?

Preparation

Methodology

Reports

Design, Impact,

Institutional arrangement

Framework Meta-Evaluation

UN System Wide Approach Evaluation

Performance Indicator

Adapted

Parameters: gender and equity sensitivity

Evaluation preparation

• 1. Evaluability

• 2. Stakeholder identification process

• 3.Team expertise and balance

Evaluation methodology

• 4 Evaluation criteria,

• 5.Evaluation questions

• 6 Evaluation approach,

• 7.Evaluation indicators,

• 8.Evaluation methods

• 9 Data analysis

Evaluation report and use

• 10. Evaluation Report

• 11. Validation process

• 12 Government response

• 13 Disseminations

Evaluation findings

• 14 Findings on design, implementation impact and institutional arrangement

Rating system each evaluation

Not available N/A

Missing

0

Partially met

1

Fully met 2

Meta scores

0-0.5 • Missing

0-.51 to 1.25

1.26-1.75

• Meets

>1.76 • Exceeds

Approaches

Scope

• Professional Institute Network Studies: 21

• Concurrent Evaluation of MGNREGA: 1

• Total: 22

MNREGA Act, 2005

Provide for the enhancement of the livelihood security of the households

One hundred days of wage employment in every year to every household

Minimum and equal wages; atleast 33% women; <less than 5 km; conditional creches

Subsequent orders/recommendations gender and social equity

Pregnant/

differently abled

Trolley

Toilets

ICDS

50% mates

Single women

Bank account

33% for SC/ST/OBCs

Land and irrigation

development

Fourth Edition of Guidelines, 2013

Empowerment of the socially disadvantaged, especially women, Scheduled Castes (SCs) and Schedules Tribes (STs), through the processes of a rights-based legislation

Evaluation preparation – Score 1

Evaluability

1

86% studies - 3 yrs

Design-partial integration

Data- partially available

Team

0.8

Women 28% of team

40% of team leaders

Muslim none

71% - expertise ‘approaching’

Stakeholder identification

1.33

Workers

Women workers

PRIs, Mates

Different castes

Local government

Gram Rozgar Sevaks

Government

Less, Muslims, single women, transgender,

pregnant and lactating women

Evaluation methodology (1.22)

Evaluation approach (1.32)

CE: Quantitative; sampling good and poor performance. “Beneficiary and mandays”

Quantitative & qualitative methods used (62%). Household: 34% Purpose sampling of women, SCs 22%

Evaluation questions (1.18)

CE: Implementation (work, wages, amenities) and impact, PIN: Implementation (100%), impact, context, design,

planning and institutional (14%)

Evaluation criteria (1)

CE: 9 parameters -nothing gender/social specific PIN studies 17 parameters, 1 refers to women’s

empowerment and SC/ST inclusion

Evaluation methodology

Data analysis (1.38)

CE: Wide variation in women’s participation 24% adopt gender analysis- construction of ‘worker’, visibility, strategic or practical, identity, and GEWI index, empowering

Methods and tools (1.23)

CE: Questionnaire with workersPin: 81% Questonnaire, 19% with non worker-others

FGDs: 52%, case studies: 29%, PRA: 14%. Ethics?

Evaluation indicators (1.23)

CE: % workers women/WHH, work created for women & SC/STs, and access to creches

Pin: Same as CE (80%), nutrition and LFPR, appropriateness of tools, worker’s committee, child labor and VAW (<20%),

Evaluation report and use

Report (1.27)

•CE: GSE integrated into findings,. No section on recommendations and conclusions

•Pin: GSE Integrated into findings, followed by conclusion, recommendations and method

Validation (1.02)

•PIN: 29% with district/state government, 10% PRIs and NGOs, and 5% taking it back to workers

Management Response (0.67), Dissemination (1.05)

•One PIN study reports that a government official agreed to look into a recommendation-recording of BPL status in muster roll

•All reports on govt. webpage, 33% ppt, . PIN Copy right, no budget to take back findings to women workers.

Assessment of impact on women's empowerment and gender equality by 12 PIN studies (1.08)

33%

42%

25%

Empowered

Mixed

Not empowered

Reasons

Positive Mixed/negative

31-46% of employment to women??

Expansion of women’s mobility, and increase in income

Formation of women only gangs in some places

Migration of women reduced

Enabled women experiencing domestic violence to stand up (north east)

Bihar- lesser productivity norms for women

Worker assumed to be a male: Productivity norms, amenities, timings

Elderly women, pregnant and lactating women, differently abled women need different norms; single women less represented in most states

Invisibility of women’s work:Pair work, payment to men/joint accounts, undervaluation of lifting/throwing, women proxy workers, over working of water vendors

Receipt of equal wages varied- 40% to 99%; men perceive it below dignity to work on equal wages; demanded money for consumption of alcohol

Little sharing of work; women already undernourished

No provision for development of land of WHHs or women’s control over CPRs created; Caste discrimination- tools, land development, wells wages, water, work sites

Mates 0-33% women- Sexual harassment in work place noted in 2/3 studies

Women less found in social audits, gram sabha meetings and VMCs

Meta-score: 1.15 (prep 1.22, method: 1.22, report & use: 1.02, impact, 1.08)

Ap

pro

ach

ing • Management response to

report (0.67)

• Evaluability (1)

• Evaluation criteria (1)

• Dissemination of report (1.05)

• Performance on GE/SE (1.08)

• Validation of findings (1.09)

• Evaluation questions (1.18)

• Evaluation indicators (1.23)

• Evaluation methods (1.23)M

eeti

ng • Evaluation report (1.27)

• Evaluation team (1.29)

• Evaluation approach (1.32)

• Stakeholder analysis (1.33)

• Data analysis (1.38)

Recommendations: Substantive Equality

Preparation Methodology Report /use Impact

- Strengthen design from a GSE lens- Ensure disaggregated data is available - Ensure all evaluation organisations (EOs)have GSE expertise.-At-least 50% of the EO heads and staff may be women

-TOR may spell out that GSE is a cross cutting theme-- GSE questions and indicators to cover design, implementation, impact and institutional arrangements-Changes in gender relations in various institutions should be captured-The present questionnaire may be revised from a GSE lends - Qualitative methods for GSE assessment may be elaborated

- Validate with marginalised workers-Government response on web-site-Budget for dissemination to all

- Individual JC/guar- Transgender- Different norms- Not male timings- Anemia tablet- Pair work-

individual payment- Individual account- Women’s gang- Maternity benefit- Toilet/sanitary nap- Land transfer- Collective DM,

monitoring and mgt

- Other harassment- Spouse

sensitisation- Mates and staff- Unions

End

We have to persistently negotiate spaces within evaluations and in our

interaction with laws/programmes to push the agenda of gender and

social equality in

Evaluations and legislation/

programmes are gendered and reflect other hierarchies