Evaluation: Experiments, Matching and D-in-D Lecture 5.
-
Upload
stella-martha-crawford -
Category
Documents
-
view
213 -
download
0
Transcript of Evaluation: Experiments, Matching and D-in-D Lecture 5.
Evaluation: Experiments, Matching and D-in-D
Lecture 5
Agenda
• Work our way through a bit of Kremer and duFlo …– Experiments– Matching– DD
• Migration, Mothers and Money …
Migration, Mentoring and Mothers: The Effect of Migration on Children’s
Educational Performance in Rural China
Scott Rozelle and Xinxin ChenStanford University
QQ Huang (U. of Minn.) and Linxiu Zhang (CCAP)
July 2007
Introduction/Motivation
Migration is one of the main ways of alleviating poverty in developing countries
Migration itself, however, is not costless.
• For example: There may be an adverse effect of migration on the educational achievement of the children of migrants (McKenzie et al. on Mexico)
Overall Increase in Off-farm Work
0%
20%
40%
60%
80%
100%
1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000
Year
off-farm busy season part time farm only
In 2000: 45% of rural labor force have jobs off the farm … more than 80% of households have at least 1 person working off the farm
In 1980: only 4% worked full time off the farm
51%
2005
Migration-fastest growing segment
0%
4%
8%
12%
16%
20%
Year
Per
cent
of T
otal
Wor
kfor
ce
Migrant Wage Earners
Self employed
TVE/Local wage earning local
migrant
Summary: Migration in China
• Migration is rising fast, surpassing 100 million individuals (deBrauw et al., 2002)
• Migrants also are moving further away from home and leaving for a longer period of time (Rozelle et al., 1999).
• Most of China’s migration is by individuals instead of entire households, in most cases the school-aged children of the migrant parents are being left.
• Work in China: Migration higher income / poverty alleviation
Introduction/Motivation
Migration is one of the main ways of alleviating poverty in developing countries
Migration itself, however, is not costless.
• For example: There may be an adverse effect of migration on the educational achievement of the children of migrants (McKenzie et al. on Mexico)
Results from current literature
• School performance of the migrant children is being adversely affected by migration since parental care falls with migration (Wang and Wu, 2003; Tan and Wang, 2004; Li, 2004; Zhou and Wu, 2004).
• These results are all based on casual observation
• Are they true?
• Is there anything about migration that can offset this effects?
The Trade-off:
Parents can stay at home and can take care of their kids …
But often good jobs are hard to find and people are very poor … sometimes they can
not supply their kids with warm clothes and enough nutrition
Although migrants to China’s cities typically earn more money than those left in the
village (Giles and deBrauw / Park and Du)
Parents are often far from their kids …
… and, their children are left behind … sometimes with their grandparents … and sometimes at boarding schools …
Objectives Examine the effect of migration activities of men
and women on the educational performance of their children.
– Compare the distribution of children’s scores for different types of rural households and describe how the grades vary over time.
– Examine whether migration negatively affects the school grades of rural children.
– Explore how migration will affect children’s educational performance in different types of households in terms of wealth or demographic composition.
Limitations / Contribution
• Only one region of China • Counterfactual limited … do not observe
children that migrate with their parents …• May be mainly looking at correlations
• Still say something about what happens to grades of children of migrants when they leave them in the countryside
Data
• A data set collected in 2006, with information of changes in school performance of children before and after their parents outmigrated.
• 1649 fifth grade students in 36 primary schools in 6 counties in Shaanxi province
• Random sample of towns and schools within the counties and classes within the schools … but surveyed ALL students within each class …
The Sample (6 counties)
Two Key Variables
• Grades of school achievement – math and Chinese language scores
– scores from 2001/2 (first grade) to 2005/6 (fifth grade)
• From records kept by students + schools – standardized scores (second term scores)
[all scores from standardized tests corrected by joint grading panel of teachers]
• Migration status– migration histories of each parent between 2002 and 2006
Time lines of academic calendars from 2001/2 to 2005/6
200620042002 2003 20052001
Grade 1 second term scores
Grade 5 second term scores
Never Migrant Households
200620042002 2003 20052001
Grade 1 second term scores
Grade 5 second term scores
New Migrant Households
New migrant = parents were both home in 2002; at least one or both parents outmigrated by 2005
Different types of new migrant households by migration status
Any Parent Migrated households: households in which both parents lived at home in 2002 and at least one parent –either the father; mother or both parents—outmigrated in 2006;
Father Migrated Only (or mother-stayed-at-home) households: households in which only the father outmigrated in 2006 but was at home in 2002;
Father Migrated (Unconditional ) households: households where the father was at home in 2002 but outmigrated in 2006 (including households in which the mother was either at home or not at home in 2006);
Mother Migrated Only (or father-stayed-at-home) households: households where only the mother outmigrated in 2006 but was at home in 2002; Mother Migrated (Unconditional) households where the mother was at home in 2002 but outmigrated in 2006 (including households in which the father was either at home or not at home in 2006);
Both Parents Migrated households: households where both parents were at home in 2002, but outmigrated in 2006.
Never Migrant
households.
New Migranthouseholds
Migration status in 2006
Migration Status in 2002
(1)
Number of
Migrants/
Non-migrants
in 2002 a
(2)
Father Migrated Only
(mother stayed home)
(3)
Mother Migrated Only
(father stayed home)
(4)
Both Parents
Migrated
(5)
Return migrants
(rows 1-3) / Never
Migrant (row 4)
(1)
Father Migrated Only
(mother stayed home)
149
94 d
55 c
(2)
Mother Migrated
Only (father stayed
home)
18
9 d
9 c
(3)
Both Parents
Migrated
69
7
5
40 d
17 c
(4)
New Migrants (col. 2,
3 and 4) / Never
Migrant (col. 5)
1358
131 b
35 b
54 b
1138
(5)
Total number of
households
1594
232
49
94
1219
131 35 54 1138
Patterns of Migration for Sample Households in China, 2002 and 2006
Migration status, 2002
Migration status, 2006
Never migrants, 2002
Never migrants, 2006
1594
Total sample size
Figure 1 Average Yearend Test Scores in China, 2002 and 2006
73.4074.54 73.97
70.1972.38
71.29
50.00
55.00
60.00
65.00
70.00
75.00
80.00
Chinese Math Average
2002
2006
School Performance
The average grades of all children fell between 2002 and 2006 … one interpretation is that grades fell as more migration … alternative
explanation: fifth grade is graded “harder” than first grade?
Figure 2. Differences in Yearend Test Scores between First Grade Students from
Migrant and Non-migrant Households in Rural China, 2006
70.87
69.95
72.01
70.57
68.50
69.00
69.50
70.00
70.50
71.00
71.50
72.00
72.50
Never Migrant Both ParentsMigrated
Father Migrated(Unconditional)
Mother Migrated(Unconditional)
The effect of migration on school performance seems to be complicated and that care needs to be exercised
in any interpretation. If one went out and found a family in 2006 in which neither parent outmigrated in 2002 and both parent outmigrated in 2006 … and compared to Never Migrant … might be inclined to blame migration!
Figure 2. Differences in Yearend Test Scores between First Grade Students from
Migrant and Non-migrant Households in Rural China, 2006
70.87
69.95
72.01
70.57
68.50
69.00
69.50
70.00
70.50
71.00
71.50
72.00
72.50
Never Migrant Both ParentsMigrated
Father Migrated(Unconditional)
Mother Migrated(Unconditional)
The effect of migration on school performance seems to be complicated and that care needs to be exercised
in any interpretation.
Figure 3. Kernel Density Plots of Distributions of Average Test Scores in Never Migrated Households and Both Parents Migrated Households,
2002 and 2006
0.0
1.0
2.0
3.0
4D
ensi
ty
0 20 40 60 80 1002002 Average Test Scores(both Chinese and Math)
Both Parents Migrated
Never Migrant0
.01
.02
.03
.04
Den
sity
0 20 40 60 80 1002006 Average Test Scores(both Chinese and Math)
Both Parents Migrated
Never Migrant
the gap was narrowing
Figure 4. Kernel Density Plots of Distributions of Average Test Scores in Households that Vary by Wealth Category and Household Composition in China, 2002 and 2006
0.0
1.0
2.0
3.0
4D
en
sity
0 20 40 60 80 1002002 Average Test Scores(both Chinese and Math)
Poorer Households
Wealthier Households
0.0
1.0
2.0
3.0
4D
en
sity
0 20 40 60 80 1002006 Average Test Scores(both Chinese and Math)
Poorer Households
Wealthier Households
Panel A. Average Test Scores by Wealth Category
0.0
1.0
2.0
3.0
4D
en
sity
0 20 40 60 80 1002002 Average Test Scores(both Chinese and Math)
Households with no Siblings
Households with Siblings
0.0
1.0
2.0
3.0
4D
en
sity
0 20 40 60 80 1002006 Average Test Scores(both Chinese and Math)
Households with no Siblings
Households with Siblings
Panel B. Average Test Scores by Household Composition
Methodology (1)
Difference in Difference (DID)
Model (1), Restricted & Unadjusted: ΔScorei = α + δMIGi + εi
Model (2), Unrestricted & Unadjusted: ΔScorei = α +δMIGi +γScore_02i + εi
Model (3), Restricted & Adjusted: ΔScorei = α +δMIGi +βXi + εi,
Model (4), Unrestricted & Adjusted: ΔScorei = α +δMIGi +γScore_02i +βXi + εi
where, i is an index for the student, ΔScorei is the change of the second term score of st
udent i between 2002 and 2006 (that is the final grade from the fifth grade minus the final grade from the first grade); MIGi is the treatment variable (which makes δ the parameter of inte
rest). Finally, the term Xi is a vector of covariates that are included to capture the characterist
ics of students, parents and households and also includes a set of 12 town indicator or dummy variables.
Model #2: Difference-in-differences
• Compares before-after changes of participants vs. before-after change of non-participants
• Any common trends get differenced out.
• Limitation: only common trends between two groups get differenced out– We control for base value of Yi and observables
i T i D i i i D iDD iY T D T D X
Our equation of choice (the full model)
• Model (4),
• Unrestricted & Adjusted:
ΔScorei = α +δMIGi +γScore_02i +βXi + εi
• Propensity Score Matching (PSM)----Basic matching
----Multi-dimensional matching
• Difference in Difference Matching (DDM)----Basic matching
----Multi-dimensional matching
Methodology (2)
Results
• DD results
• PSM results
• DDM results
Table 2. DD Regression Results Analyzing the Effect of Migration on School Performance of Students in China
Dependent Variable = Changes in Second Term Test Scores between 2002 and 2006 (ΔScore)
(1) (2) (3) (4)
Treatment Variable (MIGi)b Restricted &
Unadjusted
Unrestricted &
Unadjusted
Restricted &
Adjustedc
Unrestricted &
Adjustedc
(1) Any_Parent_Migrated 3.183 2.327 2.169 1.164
(3.72)*** (3.03)*** (2.58)** (1.65)*
Characteristics of the students in 2002
-0.460 -0.627 (2)
Student score in the second term in
2002 (Full score is 100) (14.93)*** (18.04)***
0.826 -0.383 (3)
Gender dummy (=1 if male and 0 if
female) (1.28) (0.75)
Age of the student in 2002 (Years) 0.097 -1.322 (4)
(0.26) (4.39)***
-2.754 1.168 (5)
Cadre dummy (=1 if the student was
a student cadre in 2002 and 0 if not) (3.83)*** (1.93)*
-1.051 -0.972 (6)
Mentor dummy (=1 if the student
had a mentor in 2002) (0.99) (1.26)
0.438 0.443 (7)
Sibling dummy (=1if the student had
no siblings in 2002) (0.55) (0.71)
Characteristics of the parents in 2002
-0.066 -0.053 (8) Age of the father (Years)
(0.85) (0.85)
-0.200 -0.044 (9)
Level of education of the father
(Years of schooling) (1.06) (0.35)
0.114 0.274 (10)
Level of education of the mother
(Years of schooling) (0.77) (2.39)**
Characteristics of the household in 2002
0.031 0.037 (11)
Size of total household land holding
in 2002 (mu) (0.36) (0.57)
0.078 0.251 (12)
Number of household members in
2002 (Person) (0.25) (1.01)
0.056 -0.037
(13)
House value dummy (=1 if the
house is worth more than 5000
yuan) (0.08) (0.07)
(14) Number of Observations 1575 1575 1549 1549
(15) R-squared 0.01 0.27 0.10 0.43
Dependent Variable = Changes in Second Term Test Scores
between 2002 and 2006 (ΔScore)
Treatment Variable (MIGi)
(1) (2) (3) (4)
Restricted & Unadjusted
Unrestricted &
Unadjusted
Restricted & Adjusted
Unrestricted & Adjusted
Any_Parent_Migrated 3.183 2.327 2.169 1.164
(3.72)*** (3.03)*** (2.58)** (1.65)*
Table 2. DD Regression Results Analyzing the Effect of Migration on School Performance of Students in China
Table 2. DD Regression Results Analyzing the Effect of Migration on School Performance of Students in China
Dependent Variable = Changes in Second Term Test Scores between 2002 and 2006 (ΔScore)
(1) (2) (3) (4)
Treatment Variable (MIGi)b Restricted &
Unadjusted
Unrestricted &
Unadjusted
Restricted &
Adjustedc
Unrestricted &
Adjustedc
(1) Any_Parent_Migrated 3.183 2.327 2.169 1.164
(3.72)*** (3.03)*** (2.58)** (1.65)*
Characteristics of the students in 2002
-0.460 -0.627 (2)
Student score in the second term in
2002 (Full score is 100) (14.93)*** (18.04)***
0.826 -0.383 (3)
Gender dummy (=1 if male and 0 if
female) (1.28) (0.75)
Age of the student in 2002 (Years) 0.097 -1.322 (4)
(0.26) (4.39)***
-2.754 1.168 (5)
Cadre dummy (=1 if the student was
a student cadre in 2002 and 0 if not) (3.83)*** (1.93)*
-1.051 -0.972 (6)
Mentor dummy (=1 if the student
had a mentor in 2002) (0.99) (1.26)
0.438 0.443 (7)
Sibling dummy (=1if the student had
no siblings in 2002) (0.55) (0.71)
Characteristics of the parents in 2002
-0.066 -0.053 (8) Age of the father (Years)
(0.85) (0.85)
-0.200 -0.044 (9)
Level of education of the father
(Years of schooling) (1.06) (0.35)
0.114 0.274 (10)
Level of education of the mother
(Years of schooling) (0.77) (2.39)**
Characteristics of the household in 2002
0.031 0.037 (11)
Size of total household land holding
in 2002 (mu) (0.36) (0.57)
0.078 0.251 (12)
Number of household members in
2002 (Person) (0.25) (1.01)
0.056 -0.037
(13)
House value dummy (=1 if the
house is worth more than 5000
yuan) (0.08) (0.07)
(14) Number of Observations 1575 1575 1549 1549
(15) R-squared 0.01 0.27 0.10 0.43
Highest R-square:
> 0.40
Table 3. DD Regression Results Analyzing the Effect of Migration on School
Performance of Students in China by Household’s Migration Status
Dependent Variable = Changes in Second Term Test Scores
between 2002 and 2006 (ΔScore)
(1) (2) (3) (4)
Treatment Variable (MIGi)
b Restricted &
Unadjusted
Unrestricted
& Unadjusted
Restricted &
Adjustedc
Unrestricted
& Adjustedc
3.183 2.327 2.169 1.164 (1)
Any_Parent_migrated,
(3.72)*** (3.03)*** (2.58)** (1.65)*
4.634 3.812 3.630 2.356 (2)
Father_Migrated_Only
(mother stayed home) (4.27)*** (4.09)*** (3.45)*** (2.73)***
3.812 2.879 2.984 1.508 (3)
Father_Migrated
(Unconditional) (4.10)*** (3.52)*** (3.24)*** (1.98)**
0.839 0.156 -0.861 -0.121 (4)
Mother_Migrated_Only
(father stayed home) (0.45) (0.08) (0.45) (0.07)
0.903 0.444 -0.147 -0.541 (5)
Mother_Migrated,
( Unconditional) (0.73) (0.37) (0.12) (0.48)
1.367 0.615 1.040 -0.536 (6) Both_parents_migrated
(0.79) (0.38) (0.58) (0.35)
Table 4. PSM and DDM Estimators and the Effect of Migration on the School Performance of Students in Rural China, 2002 and 2006
Propensity Score Matching
Difference-in-Difference
Matching Treatment Variable c d
Average Treatment
Effect for the Treated
t-value/
z-value b
Average Treatment
Effect for the Treated
t-value/
z-value b
(1) (2)
(1a) Basic Matching 1.16 (1.02) 0.31 (0.28) Any_parent_migrated
(1b) Multi-dimensional Matching 1.57 (1.60) 2.12 (1.86 )*
(2a) Basic Matching 2.04 (1.36) 1.12 (0.77) Father_Migrated_Only
(mother stayed home) (2b) Multi-dimensional Matching 3.59 (2.96 ) *** 3.12 (1.93 )**
(3a) Basic Matching 1.57 (1.20) 2.35 (1.93)** Father_migrated,
(Unconditional) (3b) Multi-dimensional Matching 2.19 (2.04 ) *** 2.52 (1.99 )***
(4a) Basic Matching -0.63 (-0.22) -1.1 (-0.39) Mother_Migrated_Only
(father stayed home) (4b) Multi-dimensional Matching -0.94 (-0.43) 1.93 (0.58)
(5a) Basic Matching -0.45 (-0.26) -1.51 (-0.88) Mother_migrated
(Unconditional) (5b) Multi-dimensional Matching -0.46 (-0.32) 0.82 (0.48)
(6a) Basic Matching -0.22 (-0.09) -0.56 (-0.23) Both_parents_migrated
(6b) Multi-dimensional Matching -0.28 (-0.13) 0.97 (0.43)
Table 4. PSM and DDM Estimators and the Effect of Migration on the School Performance of Students in Rural China, 2002 and 2006
Propensity Score Matching
Difference-in-Difference
Matching Treatment Variable c d
Average Treatment
Effect for the Treated
t-value/
z-value b
Average Treatment
Effect for the Treated
t-value/
z-value b
(1) (2)
(1a) Basic Matching 1.16 (1.02) 0.31 (0.28) Any_parent_migrated
(1b) Multi-dimensional Matching 1.57 (1.60) 2.12 (1.86 )*
(2a) Basic Matching 2.04 (1.36) 1.12 (0.77) Father_Migrated_Only
(mother stayed home) (2b) Multi-dimensional Matching 3.59 (2.96 ) *** 3.12 (1.93 )**
(3a) Basic Matching 1.57 (1.20) 2.35 (1.93)** Father_migrated,
(Unconditional) (3b) Multi-dimensional Matching 2.19 (2.04 ) *** 2.52 (1.99 )***
(4a) Basic Matching -0.63 (-0.22) -1.1 (-0.39) Mother_Migrated_Only
(father stayed home) (4b) Multi-dimensional Matching -0.94 (-0.43) 1.93 (0.58)
(5a) Basic Matching -0.45 (-0.26) -1.51 (-0.88) Mother_migrated
(Unconditional) (5b) Multi-dimensional Matching -0.46 (-0.32) 0.82 (0.48)
(6a) Basic Matching -0.22 (-0.09) -0.56 (-0.23) Both_parents_migrated
(6b) Multi-dimensional Matching -0.28 (-0.13) 0.97 (0.43)
Table 4. PSM and DDM Estimators and the Effect of Migration on the School Performance of Students in Rural China, 2002 and 2006
Treatment Variable
(MIGi)
Propensity Score Matching
Difference-in-Difference Matching
ATT t-value/ z-value
ATT t-value/ z-value
Any_parent_migrated
(1a) Basic Matching
1.16 (1.02) 0.31 (0.28)
(1b) Multi-dimensional Matching
1.57 (1.60) 2.12 (1.86 )*
Summary
• There is no evidence that migration in our sample of households has hurt school performance.
• In fact, when the father outmigrates (either by himself or with others), migration appears to have a small, positive effect on the school performance of migrant children.
Heterogeneous effects
• Heterogeneous Effects from Wealth
Model (5): ΔScorei = α +δ1MIGi +δ2*MIGi *poor+γScore_02i +βXi + εi,
• Heterogeneous Effects from Household Composition
Model (6): ΔScorei = α +δ1MIGi +δ2*MIGi *nosibling+γScore_02i +βXi + εi,
Table 5. DD Regression Results with Heterogeneous Effects from Wealth and Household Composition
Dependent Variable = Changes in Second Term Test Scores between 2002 and 2006 (ΔScore)
Panel A
Heterogeneity Effects from Wealtha
Panel B
Heterogeneity Effects from Household compositionb
Treatment Variable(MIGi) a Treatment Variable(MIGi) b
2.397
1.118
Any_Parent_Migrated (2.82)***
Any_Parent_Migrated (1.28)
-2.271 0.195 Any_Parent_Migrated * Poor
(1.79)* Any_Parent_Migrated * Nosibling
(0.15)
2.958
2.028
Father_Migrated_Only
(mother stayed home) (2.83)***
Father_Migrated_Only
(mother stayed home) (1.87)*
-1.170 0.965 Father_Migrated_Only*Poor
(0.72) Father_Migrated_Only*Nosibling
(0.57)
2.668
1.516
Father_Migrated
(Unconditional) (2.85)***
Father_Migrated
(Unconditional) (1.60)
-2.139 0.028 Father_Migrated * Poor
(1.54) Father_Migrated * Nosibling
(0.02)
2.285
-0.828
Mother_Migrated_Only
(Father stayed home) (1.59)
Mother_Migrated_Only
(Father stayed home) (0.40)
-3.783 1.680 Mother_Migrated_Only* Poor
(1.28) Mother_Migrated_Only* Nosibling
(0.48)
1.349
-0.403
Mother_Migrated
(Unconditional) (0.99)
Mother_Migrated
(Unconditional) (0.29)
-3.369 -0.174 Mother_Migrated * Poor (1.62)
Mother_Migrated * Nosibling (0.08)
1.720
0.155
Both_Parents_Migrated (0.87)
Both_Parents_Migrated (0.08)
-3.982 -1.457 Both_Parents_Migrated * Poor
(1.38) Both_Parents_Migrated* Nosibling
(0.48)
Conclusion• We can reject the hypothesis that migration harms the
grades of their children .
• In fact, the migration of some migrant households has a statistically significant and positive effect on the performance of the children.
• There is neither a systematically different effect of migration between the children of more wealthy and less wealthy households nor between the children from families that have one and more than one child.
Policy implications
• It is not that migrants do not need to have special attention in education … their grades are lower (but they are always lower) … increased education will raise their productivity (other studies) …
• Point of our paper: migration by itself does not cause this … although we have not identified that exact mechanism, may be that the income effect of migration is offsetting the parental care effect
• So should build better schools … have high quality boarding facilities … increase mentoring inside schools (e.g., by small classes) … and promote the admittance of rural students to urban schools (for low or no tuition) … but, don’t do it because believe migration leads to lower education achievement … there is no evidence from our sample