RESEARCH METHODS FESTIVAL 2012: BRINGING THE

LAB TO THE FIELDANANDI MANI, UNIVERSITY OF WARWICK & CAGE

Types of Experiments (Harrison-List JEL)

______AFE_________FFE_________NFE______________________________

Lab [field experiments] NE, PSM, IV, STR, etc.

Conventional lab experiment (Lab) employs a standard subject pool of students, an abstract

framing, and an imposed set of rules Artefactual field experiment (AFE)

same as a conventional lab experiment but with a non-standard subject pool

Framed field experiment (FFE) same as an artefactual field experiment but with field context in

the commodity, task, information, stakes, time frame, etc. Natural field experiment (NFE)

same as a framed field experiment but where the environment is the one that the subjects naturally undertake these tasks, such that the subjects do not know that they are in an experiment

Motivation

Development Economists have been doing Field Experiments using Randomized Control Trials (RCTs) for over a decade now, addressing a wide range of questions, e.g. Do Cameras in Schools improve Teacher Attendance &

Student Outcomes? Does Microfinance spur Business Investment among the

poor? Some of these RCTs come under criticism for a lack of

light on the Mechanisms underlying the observed findings (Deaton(2009))

Lab Experiments may help identify Potential Reasons for Certain Outcomes Observed in

Survey Data Mechanisms Underlying Some Field Experiment Findings,

which would help increase the External Validity

Advantages of Lab Experiments

Better Control: Lab Experiment Design makes it feasible to generate results ceteris paribus Testing alternative theoretical mechanisms Test Institutions (e.g. Auction formats)

Scope for Replication & Comparison across Cultural Settings

Cheaper Market Design Pilots

Outline of Talk

Applications:(A) GENDER DIFFERENCES IN PRODUCTIVITY & PAY Fact 1: Capital Returns (de Mel et al -- 2009): Lower returns of

Women owned firms associated with less supportive spouses Q: Could Intra-Household Decision-Making Play a role in this?

Fact 2: Women in the US earn 75% of what men do on the labor market – and education, experience, hours worked don’t explain more than 50% of it Q: Gender Differences in Competitive Behavior explain this gap?

(B) POVERTY & DECISION-MAKING Fact 3: Poor seem to make irrational decisions on Savings, Human

Capital Investment Q: Could Poverty Affect Stress Levels & Cognitive Ability?

(C) PITFALLS OF LAB EXPERIMENTS & POSSIBLE SOLUTIONS

Intra-HH DecisionsHow does decision- making work within the family?

Laws about Property Rightsand Inheritance

Entrepreneurship/Income Generation

Programs

Schemes to encourageHuman capital

investment

Does it amplify the inequities of market outcomes for its members, or does it mitigate them?

HH Decision-Making: Experiment Questions

Assuming that HH members do not share a common set of preferences…

Q1: Is HH decision-making efficient – i.e. do members maximize HH (Investment) returns ? OR …

Q2: If not, why do they sacrifice HH income? Is it …for economic reasons – for instance, greater

bargaining power/ control over HH resources (how much?)

…or for other socially influenced reasons? …and do they do this only when their spouse won’t

know?

Arguments for an Experimental Approach

One-phrase Summary of Survey Based Empirical Studies of HH resource allocation decisions: Can’t be sure!

Allocation decisions directly observed real-time No need to infer decisions from data reported ex-post Survey responses may “adjusted to fit” local social and

cultural norms(Bertrand-Mullainathan(2001)), whereas... Actions speak better than words

Focus on Investment rather than Consumption Decisions No scope for effects arising from possible

substitutions outside the experiment

Experiment Location & Sample

Anantapur district, Andhra Pradesh (2nd most drought prone)

300 couples, from 32 villages -- all wives members of Self-Help Groups (SHGs) run by a single NGO

Promised Participation Fee (Rs.50 – about 62p), roughly equal to daily wages, with scope to make more based on their performance

Experiment Protocol

3-4 villages participating daily (10 day experiment)

Participating couples from each village brought in to NGO location

Separate waiting area for men and women Three couples taken to six separate rooms,

where.. Experiment explained and options presented

by a coordinator Data recorded by two independent data entry

staff Separate waiting areas for male and female

participants who completed the experiment Individual payment to participants upon

completion Participants taken back to village when all

payments completed

p 11

Experiment Tasks

No Tradeoff

Investor Tradeoff between higher HH income and own control over it

o4 Investment Decisions for each Spouse, individually presented in random orderoTask: Allocate Rs.50 (seed money) across two Investment options Blue and Red oEfficient Investment Allocation = Rs. 50 in Blue

Investment Means

INVESTOR CONTROL OVER INCOME

ALL MENWOMEN

N=502N=250 N=252

(1) (2) (3)        Fixed Share 44.95 42.2 47.68

  (11.42)(13.24

) (8.44) Low Control 36.63 34.94 38.32

(19.27)(18.33

) (20.06) Medium Control 38.30 35.62 40.98

  (18.77)(18.85

) (18.35) High Control 43.37 41.36 45.36

  (14.93)(15.92

) (13.63)

Overall Mean Investment - (across 4 decisions) 40.81 38.53 43.09

(16.75)(17.03

) (16.18)

p 12Efficient Investment: Rs.50 in Blue option Both Men and Women are Inefficient.

They’re willing to sacrifice HH income, to gain more control over it for themselves.

But Men don’t maximize HH returns even if their share of control is Fixed! WHY??

Fraction of Efficient HHs

A third of men are inefficient even when their share of control is Fixed – i.e. they undercut their own income (and their wife’s) rather than maximize HH returns

“Irrationality” not explained by low education/confusion, lack of experience with financial decisions, longer term effects on bargaining power within HH

Why are Men inefficient under Fixed Shares

Men don’t like it when their wife’s share exceeds theirs

When her share>50% They are willing to

undercut their own income..

to ensure she does not earn too much more than themselves.

30 40 50 60 700.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

40.00

45.00

50.0047.68 48.48 47.39

43.5246.59

42.40 41.1943.80

37.50

32.32

Wife's Invt. (mean)Husband's Invt.(Mean)

Wives of “Spiteful” Husbands are more inefficient in other three decisions, where Control over HH income depends upon investment allocation. CONCLUSION: Consistent with de Mel et al(2009) finding, Less cooperative spouses => Lower Productivity on Women’s Businesses

Gender Earnings Gap: Motivation

POTENTIAL SOURCES OF GENDER GAP IN LABOR MARKET OUTCOMES Occupation choice Experience & Continuity in labor market participation Discrimination Psychological factors

Stereotype threat Claude Steele (1997): Additional anxiety causes choking under

pressure when performing a task Ambady et al (1999), Psychological Science

Self-Confidence

Competitive Behavior

Performance under Competition

Gender Differences: Gneezy-Niederle-Rustichini(2003, QJE)-Summary

Lab Experiment conducted in Israel with students from Technion Participants Task: Solving Mazes on a computer Studied Participants’ performance under three payment schemes

(a) Non-competitive (Piece rate compensation) (b) Competitive (Winner-take-all tournament) (c) Random pay setting (One person in Group of 6 is paid, rest are not)

Main findings: Men’s performance improves considerably going from (a) to (b), whereas

women’s performance does not change Women’s performance is much worse when their tournament group

includes men than when it has only women

Do Women prefer to Compete less?

Niederle-Vesterlund( Aug 2007, QJE) Women may choose lower powered jobs for multiple reasons:

Responsibility & Time demands of such jobs, given family considerations Discrimination may discourage attempts to obtain these jobs Competitive Pressure of such jobs?

Experiments allow choice of tasks with similar time demands, where innate abilities do not differ among men and women, and discrimination is ruled out

Theories (about why women shy away from high-profile jobs): They may Dislike Competition Lack Confidence, relative to men Be Risk Averse Have Feedback Aversion (They’re more discouraged by negative feedback).

Experimental Design makes it possible to distinguish among various channels

Experiment Details

Lab experiment with students at University of Pittsburgh, groups

Task: Addition of sets of five 2 digit-numbers, for five minutes

Information to Participants: Only on own absolute performance, no information on others’ performance. Information provided real time, as task is performed.

Studied Payment Scheme Choice of Men vs. Women: Piece-rate vs. Winner-take all (Competitive) scheme, given information above.

4 participants per group, two male and two female (20 groups)

Experiment Design Task 1: Piece rate (PR) of $0.5 per correctly solved addition Task 2: Tournament (T; winner take all) rate of $2 per correctly solved addition

At a 25% chance of tournament win, both payment schemes generate the same expected payoff.

Tournament payoff is in per task terms to avoid guesswork about what would be a high enough fixed payment to induce tournament entry among high performers

Task 3: First choose payment scheme (PR or T) and then do addition task Participants evaluated against others’ performance in Task2 –why?

Eliminates effects of beliefs about others’ choice on decision 10,000 (feasible) groups made with replacement from the data, avg. across 100 trials to

determine individual success probability in tournament.

Task 4: Choose payment scheme (PR or T) for (previous) Task 1; No new task To separate the preferences for competition from other factors such as risk aversion

& feedback aversion , on tournament entry decision Ask participants to guess their rank in task 1 and task 2 in their group of four

To measure effects of self-confidence on tournament entry and performance

Main Findings

Men and Women are equally good at Addition Task under Piece Rate and Tournament. Despite this,

being a woman reduces probability of selecting Tournament payment scheme in Task 3 by 38% Not explained by individual performance in previous

rounds (T1,T2) or current round (T3) itself. For women, total expected cost of under-entry is

much larger than cost of over-entry; for men it’s the reverse

Despite accounting for differences in Self-confidence, being female still reduces Tournament entry probability by 27.8%

Taking the Lab Design to the Field

Potential Concerns with the above 2 Experiments: Experiment 1: Performance could be influenced by Task specific

differences in ability (men have advantage in spatial ability and arm-throwing capacity) -- so mazes may not be to women’s advantage.

Experiment 2: Women’s observed Preferences for Competition may be due to being socialized to believe they are worse competitors than men – or that their behavior should be “ladylike” (less aggressive) ?

Gneezy-Leonard-List (2007) address both these concerns – How? Socialization: Repeat similar experiment design in one Matrilineal &

Matrilocal tribe and one Patriarchal tribe Task: Task unfamiliar to people in both tribes

Lab-in-the-Field: Design

Maasai (Tanzania): Patriarchal “Men treat us like donkeys”

Maasai woman (Hodgson (2001))

Khasi (NW India): Matrilineal “We are sick of playing the

roles of breeding bulls and baby-sitters” Khasi man (Ahmed (1994)

Subjects in 2 groups, randomly paired with 1person from other group (paired subject identity/sex not known) Task: Throw tennis ball into bucket 3 metres away (10 chances per subject) Payment Scheme: X per “success” irrespective of paired subject performance OR 3X per “success” if own performance better than paired person X=Rs.20 in India; X = 500 shillings in Tanzania

Maasai

Maasai men choose to compete at twice the rate that women do

Similar to findings in Western settings

Maasai vs. Khasi

Khasi women choose to compete at twice the rate that men do

And even at a rate slightly higher than Maasai men

Authors’ Conclusion: Any number of subtle influences on children or adults can cause differences in attitudes to competition -- even if the behavior is broadly framed by genetic endowment

Poverty, Stress & Cognitive Capacity

USING

SUGARCANE

HARVESTS

TO UNDERSTAND

THE PSYCHOLOGY OF POVERTY

Poverty, Cognitive Capacity & Decisions-1

A fundamental assumption of Economics is the Scarcity of Resources… Yet the Rational Model assumes that Mental

Capacity is Infinite ! But Decision-Making Takes Mental Effort,

and its Tiring!

Question: Does the State of Being Poor affect Cognitive Capacity? (Mani-Mullainathan-Shafir)

Sugarcane Harvests

Long Cycle Crop – about 11 months Farmers are down to the wire a few weeks before

Harvest Receive Lump sum Returns a few weeks after Harvest

Sugar Mills assign Cutting Dates to individual farmers, hence farmers don’t have control over when their Income arrives

Methodology: Compare Individual Farmers’ before vs. after Harvest on Measures of: Stress: Blood Pressure, Heart Rate (Round 1-- 2009) Cognitive Capacity & Attention: IQ(Raven’s) tests, Stroop

tests (Round 2 – 2011)

Raven’s IQ Test

Stroop Tests

Coffee

House

Train

Window

Monkey

Brick

Stroop Tests

Green

Stroop Tests

Red

Stress Measures: Blood Pressure

IQ Measure: Raven’s Test

Cognitive Depletion: Stroop

Summary of Findings

Main Findings: Poverty in the Pre-Harvest Period Raises Stress Levels Lowers IQ & Cognitive Capacity

Comments: These findings are not driven by Adverse Nutritional changes pre-harvest Learning Effects post-harvest (for IQ tests)

Potential Pitfalls – and some Solutions

Lack of Anonymity May elicit more pro-social behavior when observed Solution: Double blind experiments, Outcome measure unclear

Context and Framing Label “Wall Street” game vs. “Community” game affects play Solution: Neutral wording; Collect Background data on subjects

Self-Selection in Participants Biased sample -- Could be a problem in all Field Experiments Solution: Conduct experiment in different settings

Low Stakes may elicit non-serious behavior Solutions: Vary stakes, Treat results as lower/upper bound, Use suitable

subjects Relevance of Lab decisions to “real” behavior?

Track correlation b/w the two (e.g. Karlan(2005) AER) – Trust game outcome and Repayment of Microfinance Loan a year later

THANK YOU!