Post on 15-Jul-2015
Measuring and analyzing decision-making in
development research:
An Introduction
Workshop Session #1, Part 1
Peterman, de la O Campos and Olney
December 2, 2014: Bioversity, Rome
Empowerment and decision-making: The premise
Women’s empowerment seen as a development goal in itself (increased
gender equity) – as well as a means to achieving other favorable poverty-
related outcomes.
Many program designs and development outcomes have been shown to
depend on women’s ability to negotiate favorable allocations of resources
within the household (Doss 2013).
Thus, women’s empowerment measures are important to measure and
analyze as:
① Program impacts/outcome measures (endline or change over program
period)
② Program moderators/facilitators of program impacts (baseline or initial
conditions/heterogeneous effects).
Why decision-making?
Malhotra and colleagues (2002) review women’s empowerment indicators in 45 studies, which they characterize as “vast and interconnected” – across domains (socio-cultural, economic, familial, legal, political, psychological) at different levels of aggregation (individual, household, community, institutional).
The most commonly used definition of women’s individual empowerment use some version of Kabeer’s (2001) description of “the expansion in people’s ability to make strategic life choices in a context where this ability was previously denied to them.”
Decision-making and autonomy indicators are preferred by many researchers because they represent direct measures of empowerment – rather than indirect (proxy measures: e.g. education, earnings, age).
Standard decision-making questions in
quantitative household surveys
“Who in your household usually has the final say” …
Own health
Own earnings
Children’s health
Children’s education
Small daily household (food) purchases
Large household (asset) purchases
Use of family planning
Collected in the Demographic and Health Surveys and other large multi-topic surveys
Typically asked only to women
Standard response options
Respondent herself
Her partner
Respondent and partner jointly
Respondent and others in the
household jointly
Others in the household
Could also enter one or more
IDs of household members
Variations on the ‘standard module’
Specificity in domains –aimed at programmatic or culturally specific
important areas of agency and decision-making (Burkina Faso):
“Can you make the decision to purchase… toiletries such as soap and
toothpaste? Special foods for your children?”
Asking men or other household members (WEAI)
Interactive/qualitative assessments of decision-making using ranking
tools (Rwanda)
Changes in questionnaire wording/design:
“In an ideal situation, who in your household would make the decision?”
(Ecuador, Yemen).
To what extent do you feel you can make your own personal decisions
regarding [livestock raising] if you want(ed) to? (WEAI 2.0)
Decision-making indicator construction and
analysis
Indicator construction:
Sole decision-making
Sole and joint decision-making
Summation and individual questions versus factor analysis
Treatment of not applicable domains
Gap’s in decision-making (if collecting both men and women)
Analysis:
Multivariate frameworks controlling for household
demographics (impact analysis)
Interaction terms and stratification (heterogeneous effects)
How well are we doing?
Despite a limited number of qualitative studies examining validation
of questions, there is scant quantitative research examining robustness
of quantitative measures both in survey design and analysis of
indicator construction. Many studies still conflate status (static) with
empowerment (process) (Heckert and Fabric 2013).
In many cases, although there is evidence that favorable outcomes are
associated with bargaining power, empirical evidence cannot
rigorously identify causality due to study design and data limitations --
it is therefore difficult to identify specific policies that increase
women’s bargaining power in development settings (Doss 2013).
Review of programming and impacts on women’s
empowerment (van den Bold and colleagues (2013))
Type of intervention Quantitative
evidence
Qualitative
evidence
1 Conditional Cash Transfers (CCTs) Mixed +
2 Unconditional Cash Transfers (UCTs) Mixed More evidence
needed
3 Microfinance Mixed Mixed
4 Agricultural interventions Mixed/More
evidence needed
Mixed/More
evidence needed
“Hence, while many development initiatives seem to target women specifically, or have
women’s empowerment as one of their objectives, no sufficient body of evidence
overwhelmingly points to success in terms of improving women’s empowerment, or
improving nutrition through women’s empowerment (pp. 29)”
Session objectives and organization
Review the measurement of decision-making indicators and
their use in nutrition and agriculture research (Amber)
Case studies:
① Quantitative evaluation in Burkina Faso (Deanna)
② Qualitative evaluation in Rwanda (Ana Paula)
Discuss strengths and weakness of standard quantitative
decision making indicators with application to Ecuador, Uganda
and Yemen (Amber)
Application to ongoing program evaluation work
Works cited
Doss C. 2013. Intrahousehold Bargaining and Resource Allocation in Developing Countries. World Bank Research Observer, 28(1).
Heckert J and MS Fabric. 2013. Improving Data Concerning Women’s Empowerment in Sub-Saharan Africa. Studies in Family Planning, 44(3): 319-344.
Kabeer N. 2001. “Reflections on the measurement of women’s empowerment.” In Discussing Women’s Empowerment – Theory and Practice. Sida Studies No. 3. Novum Grafiska AB: Stockholm.
Malhortra A, Schuler SR and C Boender. 2002. Measuring Women’s Empowerment as a Variable in International Development. Background paper prepared for the World Bank Workshop on Poverty and Gender: New Perspectives.
van den Bold M, Quisumbing A and S Gillespie. 2013. Women’s Empowerment and Nutrition: An Evidence Review. International Food Policy Research Institute (IFPRI) Discussion Paper #01294. Washington DC.
Indicator choice and survey design
experiments from cash and food transfer evaluations in
Ecuador, Uganda and Yemen
Workshop Session #1, Part 2
Peterman, Schwab, Roy, Hidrobo & Gilligan (in progress)
IFPRI – Poverty, Health & Nutrition Division
December 2, 2014: Bioversity, Rome
Review of literature: Transfers and decision-making
Review of programs aimed at empowering women and linkages with
nutrition found mixed and thin evidence base for quantitative
measures of CCTs/UCTs impacts on empowerment – however more
promising evidence from qualitative studies (van den Bold et al. 2013).
Progresa: Qualitatively showed empowerment of women, however
quantitatively only 1 domain out of 5 (spending of own income) (Adato et al. 2000; Handa et al. 2009).
Bolsa Familia: Quantitatively showed impacts on 1 out of 8 domains
(contraceptive use) – concentrated in urban areas (de Brauw et al. 2013).
Zambia Child Grant Program: No quantitative impacts across 9
domains (AIR 2013).
Kenya Hunger Safety Net Program: Mixed quantitative and
qualitative findings (OPM and IDS 2012).
Objectives: Transfers and decision-making in cross-
country transfer evaluations
① Whether relative rankings of decision-making are sensitive to
differences in indicator construction and survey design
experiments.
② Tests correlation between the various decision-making indices
and other proxy (indirect) measures of women’s status
(women’s education and age) or development outcomes
(household dietary diversity and food consumption).
③ Whether the transfer programs had a measurable impact on
decision-making outcomes for women.
Introduction: WFP partnership and cross country study
While the conceptual issues underpinning these issues are well-understood, there is little rigorous evidence on the relative impacts of different modalities
Partnership with the World Food Programme (WFP) to provide new evidence on impacts and cost effectiveness of alternative modalities in Bangladesh, Ecuador, Niger, Yemen and Uganda.
From 2008 to 2011, number of WFP programs using alternative modalities increased 10-fold from 5 projects in 2008 to 51 in 2011 ($208 million in programming).
Target for 2012 was nearly 1/3 of operations in cash, vouchers and “digital foods.”
The Intervention: Context in Ecuador (Hidrobo et al. 2014).
Cash, food, food voucher
transfers
Colombian refugees and poor
Ecuadorians
7 urban centers in the Northern
provinces of Carchi (highland)
and Sucumbíos (lowland)
6 monthly transfers of $40
Targeted towards women (76%)
Conditional on nutrition training
The Intervention: Context in Yemen (Schwab 2013)
Cash and food transfers
Unconditional seasonal safety net
Rural areas of Hajjah and Ibb
Governorates
Bi-monthly transfers for 6
months equal to $25 per month
Targeted towards head of
household
The Intervention: Context in Uganda (Gilligan and Roy 2013)
Cash and food transfers
Targeted to households with child
aged 3 – 5 attending UNICEF
sponsored ECD center
Three rural districts in
Karamojong sub-region (NE)
$12 every 6 weeks, for 12
months
Targeted towards primary
caregiver/mother of child
Overview: Evaluation designs
Ecuador: 80 neighborhoods and 145 clusters in 2 stage randomization.
Neighborhoods randomly assigned to treatment or comparison, thereafter,
clusters within treatment neighborhoods randomized to food, cash or
voucher (N = 2,357 households).
Yemen: 136 food distribution points randomized food or cash. Comparison
household drawn from same areas who were just above the proxy means cut
point for program qualification (N = 3,540 households).
Uganda: 98 ECD centers randomized cash, food or comparison (N = 2980
households).
All countries: Before and after household level surveys, additional
facility/community and biomarkers vary by country. Decision-making
modules to one woman per household, interviewed in private.
Context: Gender and Development
Human Development Index (2012): Ecuador (89), Uganda (161) and
Yemen (160) out of 189 ranked countries.
Gender Inequality Index: Ecuador (83), Uganda (110) and Yemen (148) out
of 148 ranked countries.
Ecuador: Although equitable frameworks (inheritance, asset ownership etc.)
exist, gender-based violence is high (35% lifetime physical partner violence)
and culture of machismo.
Yemen: Entrenched gender discrimination, no legal age at marriage, no law
criminalizing spousal rape, restrictions on women’s movement without male
guardians.
Uganda: Recent progress in legal status of women, however gaps remain in
implementation. Gender-based violence is high (56% lifetime partner
violence) and age at first marriage is 17.9 years (2011 UDHS).
Data: Decision-making indicators
Standard measures: Sole, sole and joint
Underlying threat points: Who makes the decision, or who
would make the decision in the case of a disagreement or
dispute?
Division of tasks/preferences: Who would ideally make the
decision?
Social desirability bias: “There are many women in Uganda who are
able to exert control over decisions in their household and can influence
important aspects of their lives.”
Data: Women’s status and Household-level well-being
Women level:
Age
Education (in years)
Household level:
Dietary Diversity Index (DDI): Number of unique foods consumed in
the last 7 days (1-47)
Value of per capita monthly food consumption (includes food
consumed inside and outside of the household)
Total per capita monthly consumption (food and non-food)
0.54
0.72
0.4
0.46
0.61
0.34
0.44 0.47
0.42 0.44 0.4
0.37
0.51
0.67
0.23
0.34
0.56
0.19
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Own work for pay Own health Child's education Child's health Daily food
purchases
Large asset
purchases
Percentage of women's reported sole decision-making
accross domains
Ecuador
Yemen
Uganda
Women report making:
4.5 (out of 9) sole decisions in Ecuador (50%)
2.5 (out of 6) sole decisions in Uganda and Yemen (42%)
Highest: Own health, daily food, purchases
Lowest: Child’s education and large asset purchases
4.47
7.52
6
5.31
9
2.54 3.1
2.43 2.28
6
2.51
4.41
6
0
1
2
3
4
5
6
7
8
9
10
Sole Sole or joint Sole after
disagreement
Ideal decisionmaking Total possible
Comparison of women's decision-making indicators
Ecuador
Yemen
Uganda
Including jointness increases decision-making in all countries (more in Ecuador and
Uganda)
Women report higher decision-making in Ecuador after disagreement, lower in
Yemen
Ideal decision-making is not markedly higher than actual in Ecuador and Yemen
Relative rankings of decision-making using factor analysis
Ecuador (N = 1,174) Sole Sole or joint Ideal
After
disagreement
Sole 1.00
Sole or joint 0.31 1.00
Ideal 0.52 0.47 1.00
After disagreement 0.65 0.36 0.43 1.00
alpha statistic 0.91 0.86 0.89 0.90
Households in Ecuador and Uganda show low correlation between different
constructions of indicators (majority do not exceed 0.50)
Households in Yemen show higher correlations, however differences still exist (0.74 –
0.89)
Relative rankings of households may differ based on how questions are asked
Results: Associations and impacts
Associations with women’s status and household well-being
Age: Positive association
Education (in years): No association
Dietary Diversity Index (DDI): Mixed
Value of per capita monthly food consumption: Positive
association/mixed
Total per capita monthly consumption: Positive association/mixed
Impacts of transfers on decision-making
No impacts of transfer or transfer type in Ecuador or Uganda
Impacts driven by food transfers in Yemen
No differential rankings by social desirability introduction in Uganda
Summary: What do we know, what can we do better?
Phrasing matters: Explore the wording of questions which most reflect local
perceptions of how decisions are made, as well as program goals—particularly in local
languages, which often have limited vocabulary for nuances. Formative research is
particularly helpful for both these points when studying a new context with different
cultural and gender norms.
Ask about the right domains: Specific to the level of influence one might expect the
program to change or depend on for leveraging benefits – specific to context.
Analysis: Pay attention to response options which reflect possible decision-making
arrangements, and think through how indicators will be constructed. We should not
simply assume that sole decision-making is preferable to joint decision-making,
depending on household structure and power dimensions within the household.
Qualitative work: Triangulation of evidence may be necessary to uncover certain
domains of empowerment, not able to be captured with quantitative survey methods.
More research!: We need to continue advancing the frontier of how to most
accurately capture and analyze decision-making and women’s empowerment.
Acknowledgements
In country partners for data collection and survey management: Centro de
Estudios de Población y Desarrollo Social (CEPAR), Yemen Polling Company and
Makerere University.
IFPRI colleagues including John Hoddinott, Nancy Johnson, Amy Margolies,
Hazel Malprit, Vanessa Moreira and Agnes Quisumbing for helpful
discussions at study conception and contributions through work on the larger
food and cash transfer evaluation. Caroline Guiriec for assistance in
administration of the grant.
WFP (Rome, Quito, Kampala and Sana’a) for excellent collaboration and
program implementation.
Funding from the Government of Spain for the impact evaluations and to
the CGIAR Research Program on Agriculture for Nutrition and Health
(ANH) led by IFPRI for the analysis and writing of this paper.
Works cited
American Institutes for Research (AIR) 2013. Zambia’s Child Grant Program: 24-month impact report. Washington, DC: AIR.
de Brauw A, Gilligan DO, Hoddinott J and S Roy. 2013. The Impact of Bolsa Familia on Women’s Decision Making Power. World Development, 59: 487-504.
Gilligan DO and S Roy. 2013. Resources, stimulation and cognition: How transfer programs and preschool shape cognitive development in Uganda. Agricultural & Applied Economics Association’s 2013 AAEA & CAES Joint Annual Meeting: Washington DC.
Handa S, Peterman A, Davis B and M Stampini. 2009. Opening up Pandora’s Box: The effect of Gender Targeting and Conditionality on Household Spending Behavior in Mexico’s ProgresaProgram. World Development 37(6): 1129-1142.
Hidrobo M, Hoddinott J, Peterman A, Margolies A, and V Moreira. 2014. Cash, food, or vouchers? Evidence from a randomized experiment in northern Ecuador. Journal of Development Economics, 107: 144-156.
Oxford Policy Management (OPM) and Institute of Development Studies (IDS). 2012. Kenya Hunger Safety Net Programme: Monitoring and Evaluation Component – Impact Analysis Synthesis Report. Oxford, UK: Oxford Policy Management: Brighton, UK: IDS.
Schwab B. 2013. In the form of bread? A randomized comparison of cash and food transfers in Yemen. Paper presented at the Agricultural & Applied Economics Association’s 2013 AAEA & CAES Joint Annual Meeting: Washington DC.