For Peace and Goodwill?

For Peace and Goodwill? Using an Experimental Game to Analyse the Effect of the Desarollo y Paz

Programmes in Colombia.

David PHILLIPSInstitute for Fiscal Studies, London

(also Orazio Attanasio and Luca Pellerano)

LACEA Annual ConferenceBuenos Aires, 1st October 2009

Introduction

• Increasing emphasis on evaluating programmes with socio-cultural aims:– values, aptitudes, perceptions, motivations, capabilities, empowerment– relationships, networks, social capital

• Rigorous impact evaluation in this domain is extremely challenging as:– Expected socio-cultural changes are often not clearly specified (Theory of Change)– They are difficult to measure– As the analysis moves in the domain of the “intangibles”, the evaluation must be

driven my a multi method and multidisciplinary approach.

• Attanasio, Pellerano and Phillips (2009):– We evaluate the effect of a Social Fund type intervention in Colombia on social capital

and conflict management using a Public Goods game.

1. The Setting and the Programme2. The Experimental Games3. The Data4. Descriptive Statistics5. The Evaluation Strategy6. Results 7. Conclusions

The Setting. Conflict in Colombia.

• Colombia has been affected by a complex civil conflict along the whole course of its recent history.

• Two main illegal factions: Guerrilla(s) and Paramilitary groups (+ new emerging groups after process of desmovilización)

• Despite progress, violence still affects many rural areas of the country in a very clustered way.

• Strong association between the presence of illegal groups and violence levels, local political instability, inequality (Sanchez et al, 2003)

The Peace and Development Regional Programmes (PRDPs)

• Community-driven development / peace-building initiative

• Networks of grassroots organizations formed from existing civil society (labour unions, the church, private foundations) in poor and violent regions

• Claim peace is bottom up and based on development and community action.

• Support a wide range of development processes informed by the principles of protecting life (“Primero la vida”), equity, solidarity and participatory democracy

• The PRDPs work as an “umbrella fund”, financing a series of projects along three main axes:

1. Productive Development2. Institutional Development and Governance3. Human Rights and “Culture of Peace”

The Peace and Development Regional Programmes

• Started in Magdalena Medio in the early 1990s (Jesuits)

• 6 PRDPs have been supported by the UNDP, the World Bank (LIL1 and LIL2 grants, 1998-2003; Paz y Desarrollo Loan, 2004-2008) and the European Commission (Laboratorios de Paz I, II and III, 2002-2010)

• Total investment of more than 180 million USD

6 Regions9 Departments

125 Municipalities

1363 projects800 grassroots organizations

180,000 estimated beneficiaries

On average:300 beneficiaries

per project2 years project

duration120,000 USD project

value

The Peace and Development Regional Programmes

Experimental Games

Social capital

• Competing definitions and competing measuring tools• Standard survey techniques

– Structural approach (membership) versus motivational/value based approach (trust, reciprocity)

– People tend to respond to value based survey questions according to social expectations and social norms (Glaeser et al., 2000).

• Our definition: social cooperation (act collectively for mutual gain)• Our measure: a behavioural measure of contribution to a local public

good in a field experimental setting

Experimental Games

• Based on the theoretical framework of the experimental economics: players’ decisions should reveal their preferences/motivations

• The “controlled” experiments are designed in such a way that decisions only have monetary consequences (no reputation effects)

• As the decisions imply a real monetary payoff (on average the equivalent of 3 USD) participants are expected to reveal their preferences more similarly to how they would behave in a real life situation

Experimental Games

Voluntary Contribution Mechanism – VCM (Marwell and Ames, 1979)• The experiment is a simple public good game in which subjects decide to

invest in a ‘public’ or a ‘private’ account• Played in group (40 participants on average)• Every token invested in the public account yields benefits for the whole

group, while the token invested in the private account only yields benefits to the owner of the account

• Investing in the public pot is socially optimal• However an individual is always better off by investing in the private account

(Dominant Strategy)

)(100)1(2000 n

jjii GGS

The game is played in Two Rounds: First Round. Anonymous decision Second Round. 10 minutes communication

contribution to local public goodcommunication

The Data

Two main data sources

1. (Late) Baseline for the evaluation of PyD and LP (Nov 2006 – Feb 2007).• Treated Locations. Beneficiary and Non-Beneficiary Households• 18 months of exposure to the activities of the PRDPs on average• High variability in exposure because of the roll out of the PRDPs both within and

across municipalities

2. Second follow up for the evaluation of Familias en Acción (Nov 2005 – Apr 2006)• Control Locations• Sample for the evaluation of the famous CCT programme.• Targeted to the poor (SISBEN1), mainly women

0.1

.2.3

.4.5

Fra

ctio

n

0 50 100 150Months

PRDP Project Beneficiaries Members of Other Civic Organizations

Sample of Treated Locations

904 Non-Beneficiaries

(Control Group 2)

782 Beneficiaries

(Treatment Group)

Beneficiaries with higher exposure

Beneficiaries with lower exposure

The Data

Sample of Control Locations

Sample of 2472

(Control Group 1)

PRDP Municipalities (37)FA Municipalities (67)

FA Data PRDP Data

The Data. Experimental Game

Control

Locations

Treatment

Locations Total

Number of participants 2242 1518 3760

Number of sessions 67 45 112

Size of the group for the smaller session 21 11 11

Size of the group for the biggest session 40 90 90

Average session size 36.90 35.96 36.52

Average proportion of women in the session 0.54 0.88 0.74

Average proportion of beneficiaries in the session 0 0.45 0.18

SD of the proportion of beneficiaries in the session 0 0.11 0.23

Descriptive Evidence

Municipality CharacteristicsAltitude 1285.766Municipality development index 37.243Coca crops extension 56.472Murder rate (per 1000 inh.) 0.926

Total population 92067.281Proportion urban population 0.543Outgoing displaced population (per 1000 inh.) 27.98Voting rate (local elections) 0.485Voting rate (presidential elections) 0.768Voting majority (local election) 0.408Per capita public expenditure 0.308Per capita public investment 0.246Primary school students (per 1000 inh.) 148.43Secundary school students (per 1000 inh.) 243.632Judiciary processes initiated (per 1000 inh.) 17.962Judiciary processes terminated (per 1000 inh.) 18.723

Descriptive Evidence

• High degree of heterogeneity in beneficiary characteristics according to project types

Individual Characteristics

Female 0.528 Membership in Juntas de Acción Comunal 0.367Age (years) 39.85 Leadership role in grassroots organizations 0.421Less than primary 0.297 Voted in last elections (0-6) 3.956Full primary 0.207 One can trust the majority of people in the comm. 0.348Some secondary 0.159Full secondary + 0.253

Household Characteristics

Female head of the household 0.204 Monthly income (COP) 368165Family members younger than 18 1.568 Monthly food expenditure (COP) 253129Urban 0.495 Value of self produced food (COP) 73619Own house 0.652 Any savings during the last year 0.111Number of rooms 2.973 Any member sufferd HHRR violation (prev.year) 0.101Water by pipe 0.711Sewage system 0.476Rubbish recollection 0.449Gas by pipe 0.485Phone (landline) 0.228Top distribution assets 0.118Displaced (self declared) 0.313Months living in the neighborhood 239Other support 0.298Laboratorios de Paz 0.422

Evaluation Strategy

• No random assignment -> Potential endogenous programme placement (selection and self selection)– Across locations– Within locations

• No pre-programme information (apart from municipality characteristics)

• Identification strategya) within locations (beneficiaries vs. non-beneficiaries in treated locations) b) across locations (treated vs. control matched municipalities)c) within treated households (based on the intensity of exposure)

• May provide insights on the presence and magnitude of spill-over effects

• Multivariate regression models. All models are based on the “selection on observables” assumption

Evaluation Strategy. Across Municipalities

• We match treatment and control locations on a complete set of pre-programme municipality characteristics

• 2 control locations dropped because of missing information• 20 control locations and 16 treated locations are dropped because they fall outside

the common support• 2,427 households (58% of the original sample) are on the common support)

0 .2 .4 .6 .8 1Propensity Score

Untreated: Off support Untreated: On supportTreated: On support Treated: Off support

Evaluation Strategy. Across Municipalities

Control

Locations

Treatment

Locations

Matched

Control

Locations

Matched

Treatment

Locations


Female 0.839 0.545*** 0.837 0.538***Age (years) 41.567 40.592 41.542 41.575Urban 0.580 0.557 0.624 0.518Water by pipe 0.658 0.728 0.65 0.706Sewage system 0.290 0.504*** 0.307 0.438*Rubbish recollection 0.350 0.480** 0.375 0.412Gas by pipe 0.097 0.505*** 0.102 0.532***Phone (landline) 0.079 0.252*** 0.078 0.238***Less than primary 0.451 0.306*** 0.446 0.305***Full primary 0.135 0.198*** 0.133 0.200***Some secondary 0.146 0.163 0.155 0.159Full secondary + 0.053 0.250*** 0.053 0.238***

Municipality Characteristics

Altitude 817.422 1285.766*** 1028.791 1146.092Municipality development index 34.962 37.243 34.589 34.883Coca crops extension 8.193 56.472 12.618 6.955Murder rate (per 1000 inh.) 0.599 0.926* 0.660 0.648

Total population 27496.389 92067.281*** 32226.170 33416.668

Proportion urban population 0.504 0.543 0.520 0.504Outgoing displaced population (per 1000 inh.) 15.799 27.98 20.959 18.246Voting rate (local elections) 0.516 0.485 0.503 0.526Voting rate (presidential elections) 0.861 0.768 0.821 0.833

Voting majority (local election) 0.410 0.408 0.413 0.432Per capita public expenditure 0.305 0.308 0.301 0.315Per capita public investment 0.243 0.246 0.236 0.259Primary school students (per 1000 inh.) 161.537 148.430** 156.607 156.547Secundary school students (per 1000 inh.) 85.490 243.632 87.676 75.788**Judiciary processes initiated (per 1000 inh.) 13.106 17.962 14.033 13.067Judiciary processes terminated (per 1000 inh.) 13.117 18.723* 15.082 14.489

* difference significant at 10%; ** significant at 5%; *** significant at 1%

Evaluation Strategy. Within Treated Municipalities

Descriptive statistics

Control

Households

Treated

Households

Below

Median

Exposure

Above

Median

Exposure


Female 0.56 0.528 0.535 0.517Age (years) 41.233 39.85 38.817 41.712**Urban 0.611 0.495** 0.47 0.542Water by pipe 0.742 0.711 0.72 0.697Sewage system 0.529 0.476 0.459 0.509Rubbish recollection 0.508 0.449 0.433 0.48Gas by pipe 0.523 0.485 0.49 0.472Phone (landline) 0.272 0.228 0.195 0.292**Less than primary 0.314 0.297 0.309 0.277Full primary 0.189 0.207 0.234 0.159**Some secondary 0.166 0.159 0.152 0.166Full secondary + 0.247 0.253 0.23 0.295*Female head of the household 0.229 0.204 0.187 0.236Family members younger than 18 1.4 1.568* 1.557 1.583Displaced (self declared) 0.228 0.313** 0.319 0.306Months living in the neighborhood 246.238 239.297 236.096 243.727Owns house 0.685 0.652 0.64 0.672Number of rooms 2.883 2.973 2.949 3.018Top distribution assets 0.143 0.118 0.096 0.159*Other support 0.271 0.298 0.301 0.295Laboratorios de Paz - - 0.4 0.465

Municipality Characteristics

Altitude 1278.455 1294.217 1352.762 1185.251Municipality development index 37.336 37.135 36.978 37.382% Coca crops 51.79 61.884 43.925 96.055Murder rate (1000 inhabitants) 0.94 0.911 0.949 0.839

Treatment Locations

* difference significant at 10%; ** significant at 5%; *** significant at 1%

Results. Experimental Game

• There is no significant difference in the contribution rates between treated and control households within treated locations.

Effects within treated municipalities

Round 1 Round2

Treated Household -0.004 -0.003 -0.012 0.030 0.023 0.022

[0.037] [0.037] [0.036] [0.043] [0.046] [0.046]

Round 1 Decision YES YES YES

Location Controls YES YES YES YES YES YES

Household Controls YES YES YES YES YES YES

Session Composition Controls YES YES YES YES

Potentially Endogenous Vars. YES YES

Observations 1485 1485 1485 1485 1485 1485

Marginal Probit. Errors are Clustered at the Session level. * significant at 10%; ** significant at 5%; *** significant at 1%


• The level of contribution to the public good is much higher in municipalities where the PRDPs operate, than in other municipalities in the country with comparable characteristics.

• Results hold if CS is imposed based on matching on individual characteristics

Effects across municipalitiesRound 1

All

LocationsMatched Locations

All Locations

Matched Locations

All Locations

Matched Locations

Treated Location 0.256*** 0.221*** 0.150* 0.185* 0.135* 0.141[0.059] [0.071] [0.083] [0.102] [0.082] [0.098]

Location Controls YES YES YES YES YES YESHousehold Controls YES YES YES YES YES YESSession Composition Controls YES YES YES YESPotentially Endogenous Vars. YES YES Observations 3782 2243 3782 2162 3778 2162Marginal Probit. Cluster at the Session level. * significant at 10%; ** significant at 5%; *** significant at 1%Round 2

All

LocationsMatched Locations

All Locations

Matched Locations

All Locations

Matched Locations

Treated Location 0.332*** 0.313*** 0.305*** 0.288*** 0.300*** 0.263**[0.065] [0.075] [0.099] [0.110] [0.100] [0.104]

Round 1 Decision YES YES YES YES YES YESLocation Controls YES YES YES YES YES YESHousehold Controls YES YES YES YES YES YESSession Composition Controls YES YES YES YESPotentially Endogenous Vars. YES YES Observations 3782 2243 3674 2162 3674 2162Marginal Probit. Cluster at the Session level. * significant at 10%; ** significant at 5%; *** significant at 1%


• We find some light evidence that, ceteris paribus, increased exposure to the PRDPs lead to higher contribution to the public good, particularly in the long run.

Exposure effects

Round 1 Round 2

Exposure (7-14 months) 0.045 0.010 -0.047 -0.058

[0.048] [0.047] [0.047] [0.043]

Exposure (>14 months) 0.085** 0.064 -0.014 -0.032

[0.043] [0.042] [0.048] [0.043]

Months of Exposure 0.003 0.002 -0.001 -0.001

[0.003] [0.003] [0.003] [0.003]

Months of Exposure (Municipality Max)

0.003** 0.002* -0.001 -0.001

[0.001] [0.001] [0.001] [0.001]

Round 1 Decision YES YES

Location Controls YES YES YES YES

Household Controls YES YES YES YES

Session Composition Controls YES YES

Observations 684 684 684 684

Errors are Clustered at the Session level. * significant at 10%; ** significant at 5%; *** significant at 1%


• Exploiting knowledge networks, social capital effects seem to propagate within treated municipality through a spill-over mechanism.

• The fact of knowing the PRDP could however be endogenous to social capital.

Examining the case for Spill Over effects.

• The previous findings suggest that there might be some contamination effect in social capital formation at the community level between treated and control households within the same municipality.

Round 1 Round 2

Knows PRDP 0.078** 0.094*** 0.001 0.013

[0.033] [0.034] [0.031] [0.032]

Treated Household -0.044 -0.048 0.03 0.016

[0.038] [0.037] [0.046] [0.048]

Round 1 Decision YES YES




Observations 1472 1472 1472 1472

Marginal Probit. Errors are Clustered at the Session level. * significant at 10%; ** significant at 5%; *** significant at 1%


• Because of the sampling procedures, the proportion of beneficiaries in the session doesn’t reflect coverage rates in the municipality

• In the second round, the group effect seems to be operative for control households only

• This suggests that it is mainly driven by reputation factors• Complementary evidence suggests that PRPD beneficiaries are actually

considered more “trustworthy” than the rest of the population

Critical Mass or Conditional Cooperation? Round 1 Round 2

Proportion of Treated in the Session 0.521** 0.614*** 0.584** 0.634**

[0.214] [0.223] [0.264] [0.256]

Treated Household 0.025 0.060 0.248* 0.265**

[0.145] [0.153] [0.129] [0.124]

Interacted (proportion * treated) -0.082 -0.143 -0.518* -0.564*

[0.316] [0.328] [0.314] [0.301]

Decision in Round 1 YES YES




Observations 1485 1485 1485 1485

Errors are Clustered at the Session level. * significant at 10%; ** significant at 5%; *** significant at 1%


Exploring the “critical mass” hypothesis

.5.5

5.6

.65

.7.7

5lo

wes

s r_

b1a

mea

n_tip

o_

pers

02

46

Den

sity

.2 .3 .4 .5 .6 .7mean_tipo_pers

Density lowess r_b1a mean_tipo_pers

.65

.7.7

5.8

.85

low

ess

r_b

2a m

ean_

tipo

_pe

rs

02

46

Den

sity

.2 .3 .4 .5 .6 .7mean_tipo_pers

Density lowess r_b2a mean_tipo_pers

Conclusions

• PRDPs are grassroots-driven programmes designed to improve social capital and hence entrench peace and drive development.

• Survey-based evaluations has problems so an experimental Public Goods game is used instead: contribution as measure of social capital

• Difficult to identify an effect at individual level.

• Much more contribution in treatment versus control municipalities.

• Increased duration of exposure, knowledge of programme increase contribution.

• Proportion of beneficiaries in game increase contribution

• Suggest reputation is important and critical mass and intensity and coverage are key

For Peace and Goodwill? Using an Experimental Game and a Social Dilemma to Analyse the Effect

of the Desarollo y Paz Programmes in Colombia.

more information on the PRDPs can be found at:http://www.redprodepaz.org/

http://www.accionsocial.gov.co/

THANKS

LACEA ConferenceBuenos Aires, 1st October 2009

http://www.redprodepaz.org/

http://www.accionsocial.gov.co/

For Peace and Goodwill?

Documents

Transcript of For Peace and Goodwill?