Diversity, choice and the quasi-market: an empirical analysis of England’s secondary education policy, 1992-

2005

Steve Bradley and Jim Taylor Department of Economics

Lancaster University Management School

How has education policy changed?

What have been the consequences of the policy reforms?

How can the impact on outcomes be estimated?

Pre-1990

o Local Education Authorities (LEAs) determined the distribution and use of school funding

o LEAs determined allocation of pupils (except for church schools and grammar schools)

o LEAs appointed and employed teaching staff

o Limited role for head of school

o Limited role for parents and governors

Early 1990s: the creation of a quasi-market in secondary education

o Motivation: general dissatisfaction with educational outcomes

o Aim: to improve educational outcomes

o Method: creation of quasi-market + targeting of ‘disadvantaged’ pupils

Three main strands:

1. Establishment of a quasi-market: competition between schools

2. Specialist schools programme: diversity to improve pupil-school ‘match’

3. Urban education policy: Education Action Zones for ‘disadvantaged’

Current policy

The quasi-market reforms: post-1990

Pre-conditions for quasi-marketsPre-conditions for quasi-markets

Policy reformsPolicy reforms Decentralised Decentralised decision-decision-makingmaking

ChoiceChoice VoiceVoice IncentivesIncentives InformationInformation

Local management of schoolsLocal management of schools ++

Opting-out of government Opting-out of government controlcontrol

++ ++

Parents on governing bodyParents on governing body ++ ++

Funding based on enrolmentsFunding based on enrolments ++ ++

Parental choice of schoolParental choice of school ++

Specialist schoolsSpecialist schools ++

Attainment Tables + OFSTEDAttainment Tables + OFSTED ++ ++

Purpose of the quasi-market

o Improve performance through greater competition for pupils (diversity + choice + local management of schools)

o Increase transparency and accountability

o Improve efficiency through direct funding - schools now responsible for 90% of recurrent expenditure - more efficient allocation of resources - increase in total educational product

o Induce private funding into state education - private funders can contribute to creation of new schools (academies) or take over ‘failing’ schools to raise performance

But will the quasi-market improve educational outcomes for all pupils?

o Choice may lead to more sorting/segregation:

- ‘poorly educated’ parents less able to utilise information flows

- better-off parents move to live within a ‘good’ school’s catchment area (allocation - lottery?)

- also better-off parents can afford travel costs leading to cream-skimming by popular schools

o Why is sorting harmful?

- may lead to loss of peer effects for lower ability pupils; efficiency losses if peer effects are non-linear

- long term - reinforces persistence of income disparities

Constraints on the quasi-market

o ‘Comprehensive’ schools cannot (ostensibly) choose pupils

o Entry and exit severely limited

o Excess demand for places in popular schools

o Accurate information needed for choice (5-yearly inspection reports, annual assessment tables, open-days, annual school reports). But information can be misleading (e.g. raw scores and value added)

o Choice severely limited in many school districts(non-metropolitan areas (20% of districts have 4 schools or less)

Number of specialist and non-specialist secondary schools in England

0

500

1,000

1,500

2,000

2,500

3,000

3,500

1992 1994 1996 1998 2000 2002 2004

Non-specialist schools

Specialist schools

Diversity: the Specialist Schools Programme

2006: 80% of schools now specialist

   Year first Year first introducedintroduced

Total in Total in 20062006

TechnologyTechnology 19941994 585585

LanguagesLanguages 19951995 221221

Arts Arts 19971997 421421

SportSport 19971997 350350

BusinessBusiness 20022002 229229

EngineeringEngineering 20022002 5757

MathsMaths 20022002 225225

ScienceScience 20022002 303303

HumanitiesHumanities 20042004 7272

MusicMusic 20042004 2727

TotalTotal   -- 24902490

Specialisms

Urban Education Programme

extra funding for schools in disadvantaged urban areas (28% of all schools) - 1999/05

(Education Action Zones)

o Support for gifted and talented pupils - learning mentors for individual pupils

o Support for the ‘hard to teach’ - learning support units (to improve attendance)

o Provision of high-tech equipment in poorly equipped schools

Estimating the impact of the educational reforms

o Have educational reforms been effective? (e.g. exam results, truancy)

o Have the reforms had any distributional consequences?

o Which policies have been the most effective?

Exam results at age 16: % with 5+A*-C grades

30

35

40

45

50

55

60

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Truancy rate (%)

1.00

1.05

1.10

1.15

1.20

1.25

1.30

1.35

1.40

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Days lost through unauthorised absence

30.0

35.0

40.0

45.0

50.0

55.0

60.0

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Specialist schools

Non-specialist schools

Proportion of pupils with ‘good’ exam results (5 or more A*-C grades)

Gap widened from 7 (2001) to 14 (2005)

Metropolitan v non-metropolitan schools

% 5 or more A*-C grades

30.0

35.0

40.0

45.0

50.0

55.0

60.0

19931994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004

2005

%

Metropolitan

Non-metropolitan

Gap narrowed from 7 (2001) to 3 (2005)

Truancy rate (%)

0.6

0.8

1.0

1.2

1.4

1.6

1.8

1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005

Non-metropolitan schools

Metropolitan schools

Truancy rate = % of half days unauthorised absence

Estimating the effect of the policy reforms on educational outcomes

Following Hanushek (1979, 1986), a school’s production function can be written as follows:

Yst = f(PUPst, FAMst, SCH,t) + errorst

Y = outcome (e.g. exam results, attendance) PUP = pupil characteristics (e.g. ability, gender, ethnicity)FAM = family background variables (e.g. household income, parental education) SCH = school inputs (e.g. school & teacher quality)

Extending this to include three separate measures of education policy:

Yst = f(PUPst, FAMst, SCHst, COMPst, SPECst, URBPROGst) + errorst

COMP = competition from other schools in the same districtSPEC = specialist schools policyURBPROG = Education Action Zone policy (low income areas)

Endogeneity problems with the OLS production function

o Single equation production function likely to produce biased results:

- Error term includes unobservables (e.g. parental attitudes towards education & innate ability of pupils)

- FAM and SCH are correlated (e.g. schools with a high proportion of rich children find it easier to recruit ‘good’ teachers)

- SCH is endogeneous (e.g. schools with ‘good’ exam results find it easier to recruit ‘good’ teachers)

o Hence: - school quality variables (e.g. pup/teach): underestimated - policy effects (SPEC and URBPROG): overestimated

An alternative approach: fixed effects model with panel data

Endogeneity problems less severe - control for unobservables

Model to be estimated:

Yst = αs + λCOMPst + ηSPECst + δURBPROGst + Xstβ + Ttλ + εst

Y = exam outcome COMP = exam outcome of other schools in district (lagged) SPEC = a specialist school dummy (policy-off / policy-on) URBPROG = inner city schools policyX = time-varying controls (e.g. pup/teach, % poor) T = year dummies αs = school fixed effects (time invariant) - FE model estimates effect of policy variables on within-school variation in Y over time

Estimated coefficientEstimated coefficient

Competition Competition 0.20***0.20***

Urban programmeUrban programme 1.8***1.8***

Specialist schools programmeSpecialist schools programme 0.9***0.9***

Pupil-teacher ratio Pupil-teacher ratio -0.001***-0.001***

Part-time / full-time teachersPart-time / full-time teachers 0.0080.008

Number of pupilsNumber of pupils 0.010***0.010***

Number of pupils squaredNumber of pupils squared 0.0000.000

% eligible for free school meals% eligible for free school meals -0.260***-0.260***

Y94 (Some year dummies)Y94 (Some year dummies) 1.81.8

Y95Y95 2.02.0

Y97Y97 3.43.4

y00y00 5.55.5

y02y02 6.66.6

y04y04 8.38.3

y05y05 10.610.6

ConstantConstant 0.2980.298

R-squaredR-squared 0.410.41

nn 4025140251

Fixed effects model: dependent variable = exam performance

Single-year OLS v fixed effects results

Controls = year dummies, pupil-teacher ratio, % pupils eligible for free school meals, etc.

No. of schools in districtNo. of schools in district Competition Competition Urban Urban education education programmeprogramme

Specialist Specialist schools schools programmeprogramme

OLS model for 2005OLS model for 2005 0.13***0.13*** 8.5***8.5*** 6.5***6.5***

Fixed effects model for 1992-2005Fixed effects model for 1992-2005 0.20***0.20*** 1.8***1.8*** 0.9***0.9***

Explanatory variableExplanatory variable With policy With policy effectseffects

Without policy Without policy effectseffects

CompetitionCompetition 0.200.20 --

Urban programmeUrban programme 1.81.8 --

Specialist schools programmeSpecialist schools programme 0.90.9 --

y94y94 1.81.8 2.12.1

y95y95 2.02.0 2.72.7

y96y96 3.33.3 4.04.0

y97y97 3.43.4 4.54.5

y98y98 4.14.1 5.65.6

y99y99 5.45.4 7.47.4

y00y00 5.55.5 8.48.4

y01y01 5.85.8 9.59.5

y02y02 6.66.6 11.111.1

y03y03 7.77.7 12.712.7

y04y04 8.38.3 13.913.9

y05y05 10.610.6 16.616.6

Note: Controls not shown

Effect of including policy variables on time trend of exam performance

Explanatory variablesExplanatory variables Estimated coefficientEstimated coefficient

CompetitionCompetition 0.20***0.20***

Urban programme: phase 2000 2.3***

Urban programme: phase 2001 1.4***

Urban programme: phase 2002 1.1***

ArtArt 1.1***1.1***

Business studiesBusiness studies 2.5***2.5***

EngineeringEngineering -0.7-0.7

LanguagesLanguages 0.00.0

MathsMaths 0.00.0

ScienceScience 0.7*0.7*

SportSport -0.2-0.2

TechnologyTechnology 1.6***1.6***

HumanitiesHumanities -0.3-0.3

MusicMusic 0.80.8

More detailed policy effects

Aggregate effect of education policies on exam results, 1992-2005

Main findings:o 10pp improvement in competitor schools is associated with a 2pp improvement for individual schools

– small (but significant) effect: overall effect around 3pp

o Specialist schools effect in arts, business studies, science and technology: but only 1pp overall

o Urban programme raised exam score by 1.8pp

Total policy impact: 6pp of the 16pp improvement in exam results (1993-2005) is ‘explained’ by the three policies.

What about the other 10pp? Grade inflation?

Distributional consequences of the quasi-market reforms

Have the reforms benefited some groups more than others?

Three tests:

1. Effect on different ability groups

2. Effect on different income groups

3. Effect on different ethnic groups

Do policy effects vary over the ability range?

Answer: • competition: effect is very small at top end of ability range• urban programme: effect is weakest at bottom end of ability range• specialist schools programme: effect is greatest at bottom end of ability range

Exam score quintileExam score quintile Competition Competition Urban Urban education education programmeprogramme

Specialist Specialist schools schools programmeprogramme

Schools with lowest exam scoresSchools with lowest exam scores 0.23***0.23*** 0.9***0.9*** 1.9***1.9***

Second quintileSecond quintile 0.24***0.24*** 1.7***1.7*** 1.5***1.5***

Third quintileThird quintile 0.26***0.26*** 2.9***2.9*** 1.2***1.2***

Fourth quintileFourth quintile 0.18***0.18*** 2.3***2.3*** 0.10.1

Schools with highest exam scoresSchools with highest exam scores 0.04*0.04* 2.4***2.4*** 0.5*0.5*

Do policy effects vary over the family income range?

Answer: Schools with highest poverty levels have benefited the most from education policy

Free school meals quintileFree school meals quintile Competition Competition Urban Urban education education programmeprogramme

Specialist Specialist schools schools programmeprogramme

Lowest % eligible for free meals (rich kids)Lowest % eligible for free meals (rich kids) -0.1-0.1 -1.1*-1.1* 0.20.2

Second quintileSecond quintile 0.13***0.13*** 0.90.9 0.8*0.8*

Third quintileThird quintile 0.25***0.25*** 1.5***1.5*** 1.1***1.1***

Fourth quintileFourth quintile 0.24***0.24*** 1.4***1.4*** 1.2***1.2***

Highest % eligible for free meals (poor kids)Highest % eligible for free meals (poor kids) 0.23***0.23*** 1.4***1.4*** 2.9***2.9***

Do policy effects vary according to a school’s ethnicity?

Answer: Biggest policy effects for schools with high % of ethnic minority pupils

EthnicityEthnicity Competition Competition Urban Urban education education programmeprogramme

Specialist Specialist schools schools programmeprogramme

Under 10% ethnic minority Under 10% ethnic minority pupilspupils

0.16***0.16*** 0.7***0.7*** 0.9***0.9***

10% to 50% ethnic minority 10% to 50% ethnic minority pupilspupils

0.15***0.15*** 1.7***1.7*** 0.50.5

Over 50% ethnic minority Over 50% ethnic minority pupilspupils

0.27***0.27*** 2.8***2.8*** 2.4***2.4***

% eligible for free school meals % eligible for free school meals (average 1992-2005)(average 1992-2005)

Lowest Lowest quintile quintile

Middle Middle quintilesquintiles

Highest Highest quintilequintile

ArtsArts 0.10.1 1.3**1.3** 2.3***2.3***

Business studies Business studies 1.11.1 2.3**2.3** 6.0***6.0***

EngineeringEngineering -2.7**-2.7** 1.51.5 -3.9*-3.9*

LanguagesLanguages -0.5-0.5 0.00.0 5.6***5.6***

Mathematics Mathematics -0.6-0.6 0.9*0.9* 2.22.2

ScienceScience 0.10.1 1.6***1.6*** 2.7***2.7***

SportSport 0.00.0 -0.1-0.1 -0.2-0.2

TechnologyTechnology 1.1***1.1*** 1.5***1.5*** 4.3***4.3***

Controls included?Controls included? YesYes YesYes YesYes

R-squared (within)R-squared (within) 0.420.42 0.390.39 0.500.50

nn 80918091 2414324143 80178017

Distributional consequences of the specialist schools programme: by specialism

Metropolitan v non-metropolitan schools

Why might the policy effect differ between metropolitan and non-metropolitan schools?

(i) Parental choice is greater in metropolitan areas

(ii) Greater competition for pupils in metropolitan areas

(iii) Extra resources for deprived urban areas since 1999 - Education Action Zones (virtually all schools in metropolitan areas + some other deprived areas)

Impact of competition, urban programme and specialist schools programme: metropolitan v non-metropolitan

Competition Competition Urban Urban education education programmeprogramme

Specialist schools Specialist schools programmeprogramme

Non-metropolitan Non-metropolitan areasareas

0.11***0.11*** 0.9**0.9** 0.5***0.5***

Metropolitan areasMetropolitan areas 0.39***0.39*** 1.1***1.1*** 1.7***1.7***

o Much stronger policy effects in metropolitan areas

Competition Competition Urban Urban education education programmeprogramme

Specialist schools Specialist schools programmeprogramme

Non-metropolitan areasNon-metropolitan areas -0.42**-0.42** -0.13***-0.13*** -0.05**-0.05**

Metropolitan areasMetropolitan areas -3.35***-3.35*** -0.22***-0.22*** -0.10**-0.10**

Impact of policy on truancy rate: metropolitan v non-metropolitan

Policy effects much stronger in metropolitan areas

Some conclusions

1. Effect of increased competition- Only around 3pp of the increase of 20pp can be attributed to the increased competition for pupils- But impact bigger in metropolitan schools

2. Specialist schools programme - accounted for only an extra 1pp in exam results- but variation between specialisms (up to 3pp in business studies/

enterprise)

3. Inner cities programme has accounted for an extra 2pp in GCSE results

4. Hence only one-third of the total improvement is accounted for by the three major policy initiatives

5. Estimated impact of policy has had important distributional benefits (biggest effects for low ability and low income groups)