Introduction and Motivationcee.lse.ac.uk/conference_papers/25_03_2002/costas_meghir.pdf ·...
Transcript of Introduction and Motivationcee.lse.ac.uk/conference_papers/25_03_2002/costas_meghir.pdf ·...
The Impact of Financial Incentives on Education Choice1
Lorraine Dearden, Carl Emmerson, Christine Frayne
and Costas Meghir
Institute for Fiscal Studies
March 2002
Handout prepared for CEE conference
Preliminary and incomplete – Incomplete referencing
1 ACKNOWLEDGEMENTS: The funding for this work has been provided by the DfES under the project for evaluating the Education Maintenance Allowance. We also thank our colleagues at the National Centre and CRIS for our collaboration on this project.
Introduction and Motivation
Education participation rates post the UK’s compulsory school leave age of 16 have
been traditionally quite low. The resulting low education levels for a large proportion
of the working population has been considered a source of skill shortage and a reason
for persistent poverty. The issue was considered even more serous as a result of the
increases in unemployment for low skill workers that have occurred since the early
1980s. These low levels of education are considered all the more surprising given the
relatively high returns to education. Thus the question is whether the low levels of
education were due to costs or perhaps liquidity constraints.
Over the years the participation rates in post-compulsory education increased
substantially reaching levels of 60% or 70% and far more in some areas. Moreover,
the percentage of individuals ending up with some formal school qualification also
increased, signifying possibly an increase in the educational level of those leaving
school at 16. Nevertheless, this still left a substantial proportion of individuals with
very low levels of schooling, which has been considered inefficient. The presumption
of the policy makers has been that these low levels of education are due to constraints
rather than to the outcome of an informed choice in an unconstrained environment.
There is little direct evidence on the importance or otherwise of liquidity constraints,
and this is not completely clear-cut (see Cameron and Heckman (1998), Cameron and
Taber (2000) and Dale and Krueger (1999)). Moreover, there is certainly no evidence
in the UK on this issue. Hence the UK government decided to pilot a scheme that
would offer grants to pupils from low-income families if they continued school post
16. We have collected data on this programme based on interviews with parents and
children in the pilot areas as well as in a set of carefully selected control areas. The
evaluation of this programme cannot necessarily provide information on the
importance of liquidity constraints on education, since it changes the relative costs of
remaining in school. However, it can provide valuable information on the sensitivity
of schooling choices on education costs. This is a key input in any policy that seeks to
increase participation in education. Moreover, in contrast to US studies that consider
the impact of tuition subsidies on College participation, this programme targets
younger, less educated pupils, i.e. the equivalent to those deciding whether to
continue or to drop out of high school. The group the government wishes to target are
thoe equivalent of US high school dropouts.
In this paper we use the data we have collected to evaluate the impact of education
subsidies on post-compulsory schooling choices. We consider the choice at 16 (to
continue or to drop out) and a year later. Thus we are able to measure the impact of
the subsidy on high school completion. The programme was not randomly allocated
to school districts (Local Education Authorities, LEAs). It was implemented fully in a
small number of areas. We had to pick a number of control areas in which we
collected information to define eligibility as well as a host of other characteristics.
Given the non-randomness of the programme we use propensity score matching to
balance the characteristics of the treatment and control sample. Of course our
assumption is that there is no selection on unobservables. This assumption is quite
credible in our context because of the large array of observable characteristics and
because there has been no opportunity of moving between areas to take advantage of
the reform. We show in this paper that the impact of the subsidy is quite substantial
both in the first year and subsequently. Of course we are not yet in a position to
measure the labour market returns for those who changed their behaviour due to the
programme, nor to assess whether the potential General Equilibrium effects will
reduce the impact of the programme if it is implemented on a national scale.
The paper proceeds as follows. In Section 2 we describe the programme and its
variants. In section 3 we describe the data and we compare the characteristics of the
treatment and control areas. In section 4 we discuss briefly the evaluation
methodology and in section 5 we discuss the results. In section 6 we offer some
concluding remarks.
The Education Maintenance Allowance
The Education Maintenance Allowance (EMA) pilots were launched in September
1999 in 10 Local Education Authorities. The scheme pays a means-tested benefit to
16-18 year-olds who remain in education after year 11. The payments consist of
weekly allowance (during term time only), a retention bonus every term and an
achievement bonus at the end of the course. The benefit can be claimed for up to 2
years (or 3 for young people with special needs). The pilot schemes ran for two
cohorts, so young people reaching the school leaving age in both 1999 and 2000 could
apply for the benefit.
Four different variants of the EMA are being tried out, with differences in the
generosity of the scheme and whether it is paid to a parent or the young person.
Details of the different variants are given in the table below. The basic EMA variant 1
was piloted in 3 urban areas and 1 rural area. Variants 2, 3 and 4 were all piloted in 2
urban areas.
Table 1. The Four Variants of EMA Variant Maximum
weekly EMA award
Weekly payment paid to
Retention bonus (per term)
Achievement bonus
1 £30 Young person £50 £50 2 £40 Young person £50 £50 3 £30 Parent £50 £50 4 £30 Young person £80 £140 In each area the full amount of EMA could be received by young people whose
parents incomes were £13,000 below or less.2 This was tapered away linearly so that
2 Income is defined as the taxable income of the biological parents in the previous tax year.
individuals whose parents income was £30,000 were eligible for £5 a week of EMA
with those whose parents earned more than £30,000 not eligible for any award.
The Data - Characteristics of the Pilot and Control Areas – Pre-reform
The programme was announced in the spring, just before the end of the school year.
The data used to evaluate the programme is based on a face-to-face interview with the
parents and the children. The data set was constructed so as to include both eligible
and ineligible individuals in both pilot and control areas. The first interview was
conducted at the beginning of the school year when the subsidy became available and
once those willing to participate in school could have implemented their decision. In
the following year the first cohort was followed up using a telephone interview.
Moreover a new cohort was interviewed in the same way as the first, with a face-to-
face interview. The sample sizes for the first and second cohort are give in Table 2.
There has been some attrition in the second wave of cohort 1 where about 76% of the
original sample was successfully followed up. In Table 3 we provide characteristics of
those who were interviewed in both waves and those who we only saw in wave 1. It is
clear from this information that those who dropped out of the sample were of lower
socio-economic background. Looking at the pilot areas only we find that the
participation rate for those who dropped out of the sample was 18% points lower than
those who were successfully reinterviewed. Controlling for observable characteristics
this diffeence declines to 7% percentage points with a standard error of 0.02. Thus
the impact of attrition may not be fully accounted for by observable characteristics
and we need to keep this in mind when interpreting the results. Interestingly the
attrition rate is similar in pilot and control areas, thus mitigating any problems that
attrition may create in the evaluation.
Table 2. Sample sizes Rural Urban Total Cohort 1, Wave 1 2,769 7,034 9,803 Cohort 1, Wave 2 2,231 5,226 7,457 Cohort 2, Wave 1 2,707 7,120 9,827
Table 3. Characteristics of those who attrit and those who did not attrit from the sample Non-attriters Attriters All Number of observations 7,457 2,346 9,803 Proportion staying on in ft education 0.782 0.582 0.734 Proportion going to work 0.125 0.199 0.143 Proportion in a pilot area 0.611 0.623 0.614 Weekly family income 417.00 311.64 391.78 Family receives means-tested benefit 0.21 0.34 0.24 Mother and Father figure present 0.67 0.48 0.62 Father figure present 0.79 0.66 0.76 Owner occupier 0.76 0.55 0.71 Council or Housing Association 0.19 0.35 0.23 Has statemented special needs 0.09 0.12 0.10 Mothers age 40.76 37.02 39.86 Fathers age 32.41 24.52 30.52 Mother has a levels or higher 0.29 0.18 0.27 Mother has o levels or equivalent 0.26 0.23 0.25 Father has a levels or higher 0.27 0.14 0.24 Fatther has o levels or equivalent 0.19 0.14 0.18 Father manager or professional 0.21 0.12 0.19 Father clerical or similar 0.25 0.20 0.24 Mother manager or professional 0.16 0.10 0.14 Mother clerical or similar 0.31 0.23 0.29 Father variables missing 0.32 0.48 0.36 1 or 2 parents in work when born 0.87 0.76 0.85 Attended 2 primary schools 0.28 0.26 0.27 Attended more than 2 primary schools 0.09 0.11 0.09 Received childcare as a child 0.92 0.87 0.91 1 set of Grandparents around when child 0.33 0.31 0.32 1 sets of Grandparents around when child 0.41 0.43 0.41 Grandparents provided care when child 0.29 0.30 0.29 Ill between 0 and 1 0.22 0.23 0.22 Number of older siblings 0.91 1.07 0.95 Number of younger siblings 0.91 0.98 0.93 Older sibling educated to 18 0.33 0.26 0.31 White 0.93 0.90 0.92 Father in full-time work 0.56 0.39 0.52 Father in part-time work 0.02 0.02 0.02 Mother in full-time work 0.35 0.28 0.33 Mother in part-time work 0.35 0.26 0.33 Achieved A–C GCSE Maths 0.51 0.32 0.46 Achieved A–C GCSE English 0.62 0.42 0.57 Multiple deprivation score 32.68 36.28 33.54 Income 26.60 29.03 27.18 Employment 14.74 15.67 14.96 Health Deprivation and Disability 0.76 0.89 0.79 Education, Skills and Training 0.50 0.73 0.56 Housing 0.24 0.45 0.29 Geographical Access to Services –0.29 –0.41 –0.32 Child poverty 38.54 42.30 39.44 per cent not staying on post 16 64.17 64.60 64.27 per cent not going to university 86.07 85.40 85.91 Class sizes in 1999 21.42 21.32 21.40 Authorised absences 8.51 8.75 8.57 % getting 5 GCSE A–C in 1999 39.78 37.55 39.25 % getting 0 GCSE A–G in 1999 5.68 6.33 5.83 School has 6th form? 0.46 0.43 0.46 Distance to nearest year 12 provider 2536.65 2242.65 2466.29
We now turn to a description of the pilot and control areas. The areas do differ in
some respects and it is important that this is taken into account. Tables 4 and 5
provide some pre-reform statistics on pilot and control areas, while Table A1 in the
appendix provides definitions of the deprivation indices we report (which are
computed by the government). When considering rural areas the pilot are clearly more
deprived and have lower school participation rates than the control areas. This is
partly due o the fact that the rural pilot area is Cornwall and it is very hard to find
areas sufficiently similar. However the urban pilot and control areas are much more
similar; nevertheless if there are any differences there again the pilot areas are more
deprived.
Table 4. Local area based variables – rural areas only
Variable Pilot Control Total Number of observations 2,076 1,936 4,012 Measures of local deprivation (index) Multiple deprivation score 27.61 23.29 25.53 Income 23.96 20.40 22.24 Employment 13.53 10.78 12.20 Health Deprivation and Disability 0.52 0.15 0.34 Education, Skills and Training –0.03 0.29 0.13 Housing 0.22 -0.19 0.02 Geographical Access to Services 0.25 0.19 0.22 Child poverty 35.68 29.95 32.92 Education participation rates per cent not staying on post 16 56.76 60.77 58.70 per cent not going to university 82.95 85.03 83.95 Nearest school data Class sizes in 1999 21.84 21.86 21.85 Authorised absences 8.08 8.18 8.13 % getting 5 GCSE A–C in 1999 52.41 46.15 49.39 % getting 0 GCSE A–G in 1999 3.12 4.21 3.65 School has 6th form? 0.57 0.59 0.58 Distance to nearest year 12 provider 4,022.72 4,237.83 4,126.52
Table 5. Local area based variables – urban areas only Variable Pilot Control Total Number of observations 7,266 3,607 10,873 Measures of local deprivation (index) Multiple deprivation score 40.43 38.88 39.91 Income 31.52 31.28 31.44 Employment 17.26 16.92 17.15 Health Deprivation and Disability 1.10 1.03 1.07 Education, Skills and Training 0.90 0.80 0.87 Housing 0.56 0.45 0.53 Geographical Access to Services –0.56 -0.62 -0.58 Child poverty 45.81 44.94 45.52 Education participation rates per cent not staying on post 16 69.53 67.16 68.74 Per cent not going to university 89.28 88.24 88.93 Nearest school data Class sizes in 1999 21.24 21.25 21.24 Authorised absences 8.78 9.05 8.87 % getting 5 GCSE A–C in 1999 34.06 33.92 34.01 % getting 0 GCSE A–G in 1999 7.08 6.84 7.00 School has 6th form? 0.44 0.34 0.40 Distance to nearest year 12 provider 1,578.23 1,958.04 1,704.23
The evaluation Methodology – Matching
Since the allocation to treatment and control was not random we have to be concerned
about possible systematic differences between treatment and control groups. The main
concern is of course that those in pilot areas differ in a systematic way from those in
the control area. One important advantage here is that there is limited or no scope for
moving as a response to the reform. This is because the reform was announced
relatively late in the year and the samples drawn soon after that. However this in no
way implies that the distribution of characteristics is similar in the treatment and
control area, but it may well imply that if we condition on a well-chosen set of
observables, similar (probably) to those used by policymakers to identify the pilot
areas we are likely to remove the effects of any composition bias.
Within each pilot area there are three types of individual: Those whose family income
makes them ineligible for the subsidy, those whose family income makes them
eligible for the full amount and finally, those on the taper, who receive some positive
subsidy which is less than the full amount. Thus in general the subsidy that
individuals receive (and hence the intensity of the treatment) will vary with household
income. One would wish to quantify the impact of the programme as a function of this
intensity. On the other hand household income may well be correlated with
unobservables affecting education choices. Consequently it may be difficult to
compare outcomes at different levels of income since the differences may be
confounded by heterogeneity. Thus, we follow two different approaches. In the first
approach we simply identify eligible and ineligible individuals and we allow the
impact of the treatment to vary by observable characteristics. We then also consider
differences between those with the full subsidy and those on the taper, but we do not
necessarily attribute a causal interpretation to the differences between the groups. The
approach allows for heterogeneity of treatment impact. Obviously, part of the
heterogeneity here is due to the fact that the treatment itself varies by the level of
income.
The outcome of interest in this paper will be participation in post-compulsory school,
i.e year 12 and year 13. Suppose the outcome of an individual with characteristics Xi
who is exposed to the EMA is Y . The same individual would have outcome Y were
she/he not to be exposed to the treatment. Obviously, either one or the other outcome
is observed.
1i
0i
The impact of the policy for the ith individual (Y ) is thus not observed. The
main evaluation parameter that we will estimate is the impact of treatment on the
treated, i.e. , where P is one for individuals in the pilot areas and
zero in the control areas. What we do observe is , which is the average
participation rate for those exposed to the EMA. To construct the counterfactual
we assume that which means
01ii Y−
|( 1ii PYE
() 0iYE=
)1|( 01 =− iii PYYE
)1
)1=
| iP =|( 0 =ii PYE ),0,1|( 0iiii XXPYE =
that given the observable characteristics the allocation to treatment and control is
random. Under this assumption it is now well known (see Rosenbaum and Rubin,
1983) that where
is the propensity score and is simply the probability of being allocated
to the pilot given observed characteristics. It follows that we can estimate the
counterfactual by the sample analog of
))|,1Pr(,0|())|,1Pr(,1|( 00iiiiiiii XPPYEXPPYE =====
))]|,1Pr(,0|([ 01 iiiiF XPPYEE ==
)|,1Pr( ii XP =
)1|( 0ii PYE ==
1FE
,
where denotes an expectation with respect to the distribution of the propensity
score in the treatment sample.
Implementing this involves the following steps. In the first step the propensity score is
estimated. In the second step we estimate the conditional expectation of the outcome
in the control areas given the propensity score using a kernel method. At this point we
are careful to ensure that all observations whose value of the propensity score is
outside the range of the propensity score in the treatment sample are deleted. This
imposes common support avoiding a major source of bias (see Heckman, Ichimura
and Todd, 1997). Finally the overall average is constructed using as weights the
distribution of the propensity score in the pilot areas.
The comparisons we make are the following
• EMA eligible urban young men in pilot areas with eligible urban young men in
control areas; EMA eligible urban young women in pilot areas with eligible urban
young women in control areas;
• EMA eligible rural young men in pilot areas with eligible rural young men in
control areas; and
• EMA eligible rural young women in pilot areas with eligible rural young women
in control areas.
One important issue in such policy experiments is the extent to which peer effects will
have an impact on ineligible individuals. This depends on a number of unknown
factors including of course who the peers are. If there is positive sorting by family
income this is not going to be much of an issue, particularly since the yaper will mean
that the incentives to participate for just eligible individuals and just ineligible ones
will be similar. However we can investigate this issue by considering the impact of
the programme on the entire population, rather than just on the ineligible ones; this is
an additional comparison that we carry out.
This separate matching is necessary if the differential impact of EMA on these groups
is to be estimated (to ensure the composition of the control is directly comparable).
All the data have been weighted3 by the appropriate pilot area population weights
derived from the FRS. This helps to overcome problems associated with non-random
panel attrition and sampling problems in our data. In addition when individuals are
weighed to the relevant sub-population, we are able to combine the effects in the rural
and urban areas. When the urban and rural data is not weighted in this way, our rural
results would be given too much weight since they are over-sampled in the EMA data.
The variables used to match the samples in the pilot and control are those given in
Table 3 (except of course the first three). These include both individual and household
level variables, characteristics of the ward in which the individual resides4,
characteristics of the nearest school as well as characteristics of the local education
authority.
The results
Impact of EMA on Year 12 Destinations
Table 2.1 shows estimates of the overall impact of EMA on young people’s initial
decisions to remain in full-time education, to move into employment or to become
NEET (Not in Education, Employment or Training). These results combine Cohort 1
and Cohort 2 and young men and young women together. The EMA effect has been
3 This is done using analytic weights. 4 The ward is a very small area of a local authority and will typically be quite homogeneous in terms of housing, levels of income and socio-economic background.
estimated separately for urban and rural areas and using both matched and unmatched
samples.5 The effects were estimated using pilot population weights.
5 Unmatched samples refer to just the mean of the relevant variable across the entire subgroup of interest. For example in unmatched samples the average participation in education among cohort 1 urban women in the pilot areas is compared with the mean in the control areas. It differs from the descriptive analysis later in this report since that compares only individuals from areas which have been deemed to have broadly similar characterstics.
Table 2.1 Impact of EMA on Year 12 Destination: all Eligible Young People
from Cohort 1 and Cohort 2, by Location – Pilot Weights
Per cent Unmatched sample Matched sample Pilot Control Pilot Control Increase Urban Results: FT Education 69.5 65.3 69.9 64.0 5.8 (S.E) (0.5) (0.8) (1.2) Work 14.4 16.2 14.3 17.8 -3.6 (S.E) (0.4) (0.6) (1.0) NEET 16.1 18.5 15.8 18.1 -2.3 (S.E) (0.4) (0.6) (1.0) Sample size 7266 3607 7111 7111 Population size 48,498 50,855 48,498 48,498 Rural Results: FT Education 83.1 74.5 83.7 77.6 6.1 (S.E) (0.8) (1.0) (3.4) Work 8.5 13.6 8.3 10.5 -2.3 (S.E) (0.6) (0.8) (2.4) NEET 8.4 11.9 8.0 11.8 -3.8 (S.E) (0.6) (0.7) (2.7) Sample size 2076 1936 1812 1812 Population size 5,804 5,628 5,804 5,804 Total: FT Education 71.0 66.3 71.3 65.5 5.9 (S.E) (0.5) (0.6) (1.1) Work/Training 13.8 15.9 13.7 17.1 -3.4 (S.E) (0.4) (0.5) (0.9) NEET 15.3 17.8 15.0 17.4 -2.4 (S.E) (0.4) (0.5) (0.9) Sample size 9342 5543 8923 8923 Population size 54,301 56,484 54,301 54,301
Note: Bootstrapped standard errors are reported based on 1,000 replications. EMA has had a positive and significant effect on post-compulsory education
participation among eligible young people. The overall estimate for both rural and
urban areas, combining Cohorts 1 and 2, is 5.9 percentage points.
Distinguishing between urban and rural areas is potentially important because the
local labour market opportunities are likely to differ. As we can see from the table this
difference may be reflected in the different schooling participation rates in urban and
rural areas, with the latter having much higher participation. It is of course a priori
unclear how this is likely to impact on the effect of the policy.
It turns out that the effect of the policy is broadly similar in both rural (6.1 percentage
points) and urban areas (5.8 percentage points). However, it is interesting to point out
that in earlier results, when we did not match on local area characteristics the findings
were different, leading to the impression of a much larger rural impact.
It is clear that this increase in post-compulsory education participation has drawn
young people from both employment and the inactivity group (NEET) in both urban
and rural areas.
Another important dimension of difference are men and women. Women have higher
post-16 school participation rates. As can be seen from Table 2.3 the impact for
women is lower. However, the difference is not significant (t-stat of 0.87)
Table 2.2 Impact of EMA on Year 12 destination: eligible urban young
males, by cohort, pilot weights.
Per cent Unmatched sample Matched sample Pilot Control Pilot Control Increase All urban men:
FT Education 67.3 62.4 67.4 60.6 6.9 (S.E) (0.8) (1.1) (1.7) Work/Training 17.2 19.1 17.1 21.9 -4.8 (S.E) (0.6) (0.9) (1.5) NEET 15.5 18.5 15.4 17.5 -2.1 (S.E) (0.6) (0.9) (1.3) Sample size 3613 1802 3548 3548 3548 All urban females: FT Education 71.8 68.2 72.3 67.5 4.8 (S.E) (0.7) (1.1) (1.7) Work/Training 11.6 13.2 11.6 13.9 -2.3 (S.E) (0.5) (0.8) (1.3) NEET 16.6 18.6 16.2 18.6 -2.4 (S.E) (0.6) (0.9) (1.5) Sample size 3653 1805 3563 3563
Note: Bootstrapped standard errors are reported based on 1000 replications.
A potentially important concern is the way that the programme impact evolves over
time. One concern may be that the programme may have had a low effect in the first
year of implementation because it was announced late in the academic year and pupils
may have committed to an alternative course of action. Moreover, the possibility of
continuing in Full time education is enhanced by good exam results at the end of
compulsory schooling. This presumably requires effort and preparation. On the other
hand there may be other differences between cohorts, including a change in the
economic environment, which could lead to different results. Finally it should be
pointed out that in some of the pilot areas the take up of the subsidy by those eligible
and participating in school was not 100%. Apart from the administrative issues in the
local authority which underlie some of the take up issue, this could also indicate that
pupils were not fully aware of the programme. However, there are differences
between the cohorts that would imply that the effect could go the other way [provide
table here]. First, since the eligibility threshold was fixed nominally (as well as the
subsidy) individuals who were fully eligible for the programme fell (from 3/5 to a
half). A larger proportion ended up being on the taper. This in itself could reduce the
response are the subsidy on average is now lower. Another key difference between the
cohorts is the sampling of individuals classed as vulnerable by the Department of
education (DfES). In the first cohort these were interviewed. However, halfway
through the process we were stopped from approaching these pupils. Cohort 2 does
not include such individuals.
Table ADD1 suggest that the effect of the EMA on education participation at year 12
is larger in the first cohort (7.1 percentage points) compared to the second cohort (4.6
percentage points), although this difference is not statistically significant. There seems
to be an increase in participation in education between the cohorts and a
corresponding fall in the proportion in work in the control areas.
Table ADD1 Impact of EMA on all Eligible Young People by cohort– pilot
weights
Per cent Pilot weights Pilot Control Increase Cohort 1: FT Education 71.2 64.1 7.1 (S.E) (1.6) Work 14.1 18.5 -4.5 (S.E) (1.4) NEET 14.7 17.4 -2.6 (S.E) (1.3) Sample size 4464 4464 Population size 27,000 27,000 Cohort 2: FT Education 71.5 66.9 4.6 (S.E) (1.6) Work 13.3 15.7 -2.4 (S.E) (1.3) NEET 15.2 17.4 -2.2 (S.E) (1.3) Sample size 4459 4459 Population size 27,297 27,297
Note: Bootstrapped standard errors are reported based on 1000 replications.
Eligibility for full or partial EMA awards
Only just over half of young people in Cohort 2 and three-fifths in Cohort 1 were
eligible for the maximum amount of weekly EMA available in their area. The
remainder of successful applicants for EMA would have received an amount below
the maximum to a minimum of £5 per week. This section distinguishes between
partial and full eligibility (rather than receipt) to see if the impact of EMA differs by
whether a person was fully or only partially eligible. For this analysis young people
were also matched within eligibility group, which results in the sample being slightly
smaller than for our earlier results. Of course it is important to note that when we
compare between eligibility groups more than the amount of the subsidy changes.
Crucially, overall family income is lower as well as other characteristics
Consequently, although the incentive may be higher for those on full eligibility other
household and local factors may limit the impact.
Among those who were estimated to be eligible for a full EMA award, EMA
increased full-time education participation in Year 12 by 7.5 percentage points for
young men and 4.7 percentage points for young women (Table 2.6). For those
estimated to be eligible for only a partial award, the corresponding figures are 3.2
percentage points and 5.5 percentage points. Thus from men the response of those
fully eligible is much larger than in the population who are facing the taper.
Nevertheless, the difference is not very significant.
Table 2.6 Impact of EMA on Year 12 destination: all eligible young people
from cohort 1 and cohort 2, by gender and amount of EMA, pilot weights.
Per cent
Males Females Pilot Control Increase Pilot Control Increase Fully eligible: FT Education 67.4 59.9 7.5 70.2 65.5 4.7 (S.E) (2.2) (2.1) Work/Training 15.5 20.9 -5.4 11.0 13.8 -2.8 (S.E) (1.9) (1.6) NEET 17.0 19.1 -2.1 18.8 20.7 -1.9 (S.E) (1.8) (1.7) Sample size 2437 2437 2490 2490 Population size 16,932 16,932 17,347 17,347 Taper: FT Education 71.1 67.9 3.2 80.0 74.5 5.5 (S.E) (3.6) (3.3) Work/Training 17.9 22.0 -4.1 11.2 12.0 -0.8 (S.E) (3.1) (2.3) NEET 11.0 10.1 0.9 8.8 13.5 -4.7 (S.E) (2.6) (2.7) Sample size 1706 1706 1648 1648 Population size 9,957 9,957 10,060 10,060 All Eligibles: FT Education 68.8 62.9 5.9 73.8 68.8 5.0 (S.E) (1.9) (1.8) Work/Training 16.4 21.3 -4.9 11.1 13.1 -2.0 (S.E) (1.6) (1.3) NEET 14.8 15.8 -1.0 15.1 18.0 -2.9 (S.E) (1.5) (1.5) Sample size 4143 4143 4138 4138 Population size 26,889 26,889 27,407 27,407 Note: matched samples only. Bootstrapped standard errors are reported based on 1000 replications.
Impact of EMA on eligible and ineligible young people
In all the results discussed so far, the focus has been solely on young people who were
eligible for EMA because of parental income. It is of policy interest, however, to
know what impact EMA has had on the entire population of young people, including
those who were ineligible on income grounds. This may turn out to be important if
the increased participation among the eligibles induces the ineligibles to do so as well.
This may be driven by peer effects as well as by a realisation that they need to keep
up if they are to be able to compete in the labour market.
The results of comparing the entire population of pilot and control areas is given in
Table 2.7. The EMA is estimated to have increased male participation rates by 4.3
percentage points 3.0 percentage points for young women and 3.7 overall. However,
comparing the magnitude of the results for the eligible population with the magnitude
of the effect on both the ineligible and eligible groups combined provides no evidence
of any positive spillover effects.
Table 2.7 Impact of EMA on Year 12 destination: all eligible and ineligible
young people from cohort 1 and cohort 2, by location and gender, pilot weights
and population weights
Per cent
Pilot weights Males Females Overall All: FT Education 4.3 3.0 3.7 (2.2) (2.2) (1.6) Work/Training -3.1 -1.3 -2.2 (2.0) (1.5) (1.3) NEET -1.2 -1.7 -1.5 (1.4) (1.8) (1.1) Sample size 5550 5508 11058 Population size 37,419 37,289 74,708 Note: Bootstrapped standard errors are reported based on 1000 replications.
Impact of EMA on Year 13 Destinations
So far the analysis has concentrated on the impact of the EMA on initial destinations
in Year 12, the first post-compulsory year. The subsidy is available to year 13 pupils
as well. A key policy question is the impact of the EMA on continuing school into
year 13, on overall educational achievement and, ultimately, on labour market
success. In this section, we focus on Cohort 1 and examine their destinations in Year
13, one year after the introduction of EMA.
The key problem for this analysis is attrition. We have already shown that differential
attrition rates by education attendance cannot be completely explained away by the
observables, although a large part can. However, it seems that attrition is similar in
nature in the pilot and control area an consequently unrelated to the existence of the
EMA. Consequently we can hope that the attrition bias balances out between
treatment and control and that no substantial bias remains.
To describe the sequence of events we now define four mutually exclusive outcomes:
• education in Year 12 and education in Year 13;
• education in Year 12 and other activity in Year 13;
• other activity in Year 12 and education in Year 13; and, finally,
• other activities in both year 12 and year 13.
Table 2.9 shows the impact of EMA based on the division of the population into the
four mutually exclusive groups described above. The important conclusion that omes
from this table is that whenever the EMA has been effective it has led to increase in
both year 12 and year 13 attendance and thus it is shown to have long term effects. It
also shows how EMA has affected education retention rates, defined as the proportion
of those in full-time education in Year 12 who were still in full-time education in
Year 13. EMA increased retention rates by 5.2 percentage points in urban areas (from
69.4 per cent to 74.6 per cent) and 10.8 percentage points in rural areas (from 76.2 per
cent to 87.0 per cent). This was despite the higher education participation rates
experienced in Year 12 as a result of the EMA.
Table 2.9 Impact of EMA on Year 12 and Year 13 destinations, Version II:
all eligible young people from cohort 1 who were re-interviewed in wave 2, by
location, pilot weights.
Per cent Pilot Control Increase Urban: Education Y12 → Education Y13 60.5 53.6 6.9 (S.E) (2.2) Education Y12 → Other activity Y13 14.1 15.8 -1.7 (S.E) (1.8) Other activity Y12 → Education Y13 2.2 2.4 -0.2 (S.E) (0.7) Other activity Y12 → Other activity Y13 23.2 28.2 -5.0 (S.E) (2.1) Retention Rate (for those in Edn in Y12) 74.6 69.4 5.2 (S.E) () Sample size 2497 2497 Rural: Education Y12 → Education Y13 75.9 61.6 14.3 (S.E) (6.6) Education Y12 → Other activity Y13 11.1 14.6 -3.5 (S.E) (4.2) Other activity Y12 → Education Y13 1.5 2.8 -1.3 (S.E) (2.3) Other activity Y12 → Other activity Y13 11.5 21.0 -9.5 (S.E) (5.8) Retention Rate (for those in Edn in Y12) 87.0 76.2 10.8 (S.E) () Sample size 708 708 All Areas: Education Y12 → Education Y13 62.1 54.5 7.6 (S.E) (2.2) Education Y12 → Other activity Y13 13.8 15.7 -1.9 (S.E) (1.7) Other activity Y12 → Education Y13 2.1 2.4 -0.3 (S.E) (0.7) Other activity Y12 → Other activity Y13 22.0 27.4 -5.4 (S.E) (1.9) Retention Rate (for those in Edn in Y12) 75.9 70.2 5.7 (S.E) () Sample size 3205 3205 Population size Note: Bootstrapped standard errors are reported based on 1000 replications.
Comparing the Different Variants of EMA
We now consider differences across variants. This is interesting because the variants
differ in the amount of the benefit offered and interestingly Variant 3 differs in that
the subsidy is paid to the mother. Moreover we have differences in the retention
bonus and the achievement bonus. These are important aspects of the programme that
are not reduced when the main subsidy is tapered off; hence they may impact on those
who have a reduced benefit due to family income. Conditional on the matching
assumption, by comparing across variants we should be able to estimate the impact of
the changing the structure of the programme.
To compare the variants we take as the base all young people in the sample in the
urban control areas. These are then matched to young people in the four different
urban Variants. This enables comparisons to be made across Variants on the basis of
the same set of control young people.
Table 2.14 reports the results (standard errors are yet to be computed) of this
matching exercise and suggests that:
Variant 1 increased education participation by 8.4 percentage points;
Variant 2 increased participation by 3.9 percentage points;
Variant 3 increased participation by 5.1 percentage points;
Variant 4 increased participation by 8.3 percentage points.6
However, the problem with these results is that the effects were calculated based on
different sub-samples of the controls. For example the individuals that were
successfully matched in variant 1 will have different characteristics to those
successfully matched in variant 2. As a result any difference in the EMA effect found
between variants could be due to the variant itself or the differences in individual
characteristics in the areas. For Variant 2 a large proportion of the control sample was
used whereas for Variant 4 a much smaller proportion was used.
6 The overall effect on education participation that we find by using this methodology is 6.2 percentage points. This is very slightly higher than the 5.9 percentage point EMA effect that we find in table 2.1. The reason for the difference is that table 2.14 is reporting the estimated effect of the EMA had it been introduced on the individuals living in the control areas whereas most of our analysis looks at the effect on the individuals living in the pilot areas. Since these individuals will, despite the matching process, have slightly different characteristics this will lead to slight differences in the effect found. The results are not statistically significantly different.
Table 2.14 Impact of the EMA on Year 12 destinations: all eligible young
people from cohort 1 and cohort 2, by variant, pilot weights.
Per cent Pilot Control Increase Variant 1: FT Education 74.6 66.2 8.4 Work/Training 14.2 16.1 -1.9 NEET 11.2 17.7 -6.5 Sample size 2875 2875 Variant 2: FT Education 69.8 65.9 3.9 Work/Training 17.7 15.9 1.8 NEET 12.5 18.2 -5.7 Sample size 3047 3047 Variant 3: FT Education 70.0 64.9 5.1 Work/Training 16.4 16.7 -0.3 NEET 13.7 18.4 -4.7 Sample size 2874 2874 Variant 4: FT Education 73.5 65.2 8.3 Work/Training 12.6 17.4 -4.8 NEET 13.9 17.4 -3.5 Sample size 2245 2245 All eligibles: FT Education 71.8 65.6 6.2 Work/Training 15.5 16.5 -1.0 NEET 12.8 18.0 -5.2 Sample size 11041 11041 Note: We have been unable to calculate standard errors due to problems of small samples in the bootstrapping process. This problem is overcome by presenting the results only for those control area young
people who have been found matches in all four of the Variants, thus respecting the
support conditions for the propensity score in all comparisons together. The result of
doing this is shown in Table 2.15 and suggests that the impact of:
Variant 1 on full-time education participation was 10.2 percentage points;
Variant 2 on full-time education was 4.7 percentage points;
Variant 3 on full-time education participation was 5.2 percentage points; and,
Variant 4 on full-time education was 7.1 percentage points.
The story emerging from both Tables 2.14 and 2.15 is relatively consistent. The
biggest impacts are found for Variant 1 (the core EMA Variant) and Variant 4 (where
larger retention bonuses were paid). However, Table 2.15 also shows that the sample
on which the common effect has been estimated was small and this could have
affected the precision of the results.
Conclusions
In this paper we use unique data collected for the purposes of evaluating the impact of
a schooling subsidy on school participation in England. Our results imply that
reducing schooling costs have significant and large effects on participation in school.
The elasticity of participation with respect to costs for 16 year olds is estimated
approximately at 1.5. This shows that the scope for affecting education decisions
using subsidies to education can be substantial, at least if we ignore General
Equilibrium effects.
More specifically, the results imply that the EMA has significantly raised post-16 full-
time education participation among eligibles in Year 12 by around 5.9 percentage
points and for the whole population (eligibles and ineligibles combined) by around
3.7 percentage points. The results also suggest that this participation gap for eligibles
widens between Year 12 and Year 13 and that this is largely driven by the significant
impact EMA has on retention in education in Year 13 for those eligibles who were in
education in Year 12. It also appears that the impact of EMA is only significant for
those receiving the full amount. When we compare the different variants there is
evidence that money paid to the child is more effective in both increasing education
participation, as well as retention, in education in Year 13 for those who were in full-
time education in Year 12. However, it is also clear that the most effective way to
increase retention is to increase bonuses. The retention outcomes for variant 4
individuals is significantly larger than for other groups. All of these measured effects,
however, vary by gender, rural/urban status and cohort. Despite this, our analysis
suggests that if the EMA were to be rolled out nationally, our estimated effects would
not alter significantly because of the different composition of pilot areas and England
as a whole.
Of course a number of important questions remain. First, we do not know whether
liquidity constraints are an important factor in driving the estimated factor. Second
and related to the previous question, we do not know what returns will be enjoyed by
those induced into staying on by the subsidy. Finally, we really have very little idea of
how these returns may change when the programme is rolled out nationally and the
supply of educated workers changes. This of course depends on many factors, not
least the production function. These are all important research and policy questions
that we will be pursuing.
Appendix Table A1. Indicators used in each deprivation score
Income Adults in Income Support households (DSS) for 1998 Children in Income Support households (DSS) for 1998 Adults in Income Based Job Seekers Allowance households (DSS) for 1998 Children in Income Based Job Seekers Allowance households (DSS) for 1998 Adults in Family Credit households (DSS) for 1999 Children in Family Credit households (DSS) for 1999 Adults in Disability Working Allowance households (DSS) for 1999 Children in Disability Working Allowance households (DSS) for 1999 Non-earning, non-IS pensioner and disabled Council Tax Benefit recipients (DSS) for 1998 apportioned to wards
Employment Unemployment claimant counts (JUVOS, ONS) average of May 1998, August 1998, November 1998 and February 1999 People out of work but in TEC delivered government supported training (DfEE) People aged 18–24 on New Deal options (ES) Incapacity Benefit recipients aged 16–59 (DSS) for 1998 Severe Disablement Allowance claimants aged 16–59 (DSS) for 1999
Health Deprivation and Disability
Comparative Mortality Ratios for men and women at ages under 65. District level figures for 1997 and 1998 applied to constituent wards (ONS) People receiving Attendance Allowance or Disability Living Allowance (DSS) in 1998 as a proportion of all people Proportion of people of working age (16–59) receiving Incapacity Benefit or Severe Disablement Allowance (DSS) for 1998 and 1999 respectively Age and sex standardized ratio of limiting long-term illness (1991 Census) Proportion of births of low birth weight (<2,500g) for 1993–97 (ONS)
Education, Skills and Training
Working age adults with no qualifications (3 years aggregated LFS data at district level, modelled to ward level) for 1995–1998 Children aged 16 and over who are not in full-time education (Child Benefit data – DSS) for 1999 Proportions of 17–19 year old population who have not successfully applied for HE (UCAS data) for 1997 and 1998 KS2 primary school performance data (DfEE, converted to ward level estimates) for 1998 Primary school children with English as an additional language (DfEE) for 1998 Absenteeism at primary level (all absences, not just unauthorised) (DfEE) for 1998
Housing Homeless households in temporary accommodation (Local Authority HIP Returns) for 1997–98 Household overcrowding (1991 Census) Poor private sector housing (modelled from 1996 English House Condition Survey and RESIDATA)
Geographical Access to Services
Access to a post office (General Post Office Counters) for April 1998 Access to food shops (Data Consultancy) 1998 Access to a GP (NHS, BMA, Scottish Health Service) for October 1997 Access to a primary school (DfEE) for 1999
Child poverty Percentage of children that live in families that claim means tested benefits (Income Support, Job Seekers Allowance (Income Based), Family Credit and Disability Working Allowance).
Source: Department of the Environment, Transport and the Regions (2001), Regeneration Research Summary: Indices of Deprivation 2000, (Number 31, 2000) (www.regeneration.dtlr.gov.uk/rs/03100/index.htm).
References Cameron, Stephen V & Heckman, James J, 1998. "Life Cycle Schooling and Dynamic Selection Bias: Models and Evidence for Five Cohorts of American Males Cameron and Taber (2000) Borrowing Constraints and the Returns to Schooling, NBER Working Paper, 7761 Heckman, Ichimura and Todd (1997) Matching as an Econometric Evaluation Estimator, Review of Economic Studies Heckman, JJ, Lance Lochner, Christopher Taber (1998) Explaining Rising Wage Inequality: Explorations with a Dynamic General Equilibrium Model of Labor Earnings with Heterogeneous Agents , NBER working Paper 6384
Stacy Berg Dale and Alan B. Krueger (1999) Estimating the Payooff to Attending a More Selective College: An Application of Selection on Observables and Unobservables NBER working Paper 7322 Rosenbaum P.R., Rubin D.B. (1983): The Central Role of the Propensity Score in Observational Studies for Causal Effects. Biometrika 70, 41-55