Jane Friesen, Mohsen Javdani and Simon WoodcockSimon Fraser University

May 2009

Does public information about school quality lead to flight from

low-achieving schools?

IntroductionMany jurisdictions are developing strategies to improve

school quality at the K-12 level.

School choice policies are increasingly popular:• increase competitive pressure on schools• improve disadvantaged students’ access to good schools

School choice policies take two primary forms:• increase number of alternative choice schools• provide information about school-level achievement to

encourage parents’ to choose “good” schools

Most recent evidence

Direct provision of information about school-level achievement to parents increases probability that they choose higher-achieving schools

Hastings et al. (2007)Hastings and Weinstein (2008)

Will public information have a similar effect?

Information reaches a larger number of parents – preferred schools may hit capacity constraints• overall effect on choice may not scale up• disadvantaged children may be crowded out of access to

good schools

Children whose parents have poor access to media may not receive information

Information is updated:• parents could be misled or confused when education

authorities update public information if these measures are subject to substantial sampling variation.

Our questions

When released publicly, does news about achievement at a child’s school affect the likelihood of their changing schools?

Do some groups respond differently than others, particularly those who may have relatively poor access to private information (income, language)?

Does public information have persistent effects?

Outline of talk

Overview of approach and results

Empirical details

Implications

Overview – empirical approach• student-level longitudinal data • multiple cohorts of BC elementary school students• 1999-2004

our data span:the introduction of standardized testing, and the public dissemination of school-level results

We measure the effect of public information on probability a student separates from her school following Grade 4

“within-school” estimator – we compare changes in behaviour in schools that get “good news” to changes in behaviour in schools that get “bad news”

Overview – public information in B.C. Spring 2000: first Foundation Skills Assessment (FSA) exams

were written; reading, writing and numeracy; Grades 4 and 7; low-stakes

Fall 2000: first FSA exams were released to schools, who were instructed to share the information with parents on request.

October 2001: Results of the 1999/2000 and 2000/2001 FSA

exams posted on the Ministry website. Each subsequent FSA result posted in the fall following the exam.

June 2003: The Fraser Institute began issuing annual report cards on B.C. elementary schools. These include school scores based on FSA exam results, and rankings based on these scores. They receive very widespread media coverage.

Overview - resultsPublic release of information about school-level achievement

had a large effect on the inter-school mobility of some Grade 4 students.

Response is particularly large among English-language parents in low-income neighborhoods

English-language parents respond the first time that school-level achievement measures are placed in the public domain.

Do not respond in subsequent years.

Non-English language parents (especially Chinese) also respond strongly, but not until media provide widespread coverage of Fraser Institute’s school report cards.

Responses to public information

1999 2000 2001 2002 2003

99 FSAs released

00 FSAs released

99,00,01 FI scores released

02 FI scores released

English-languageparents in lowincome neighborhoods respond

Non-English-language parents respond

Institutional background: school access/choiceGuaranteed access to the neighborhood public schoolMay also choose a non-catchment neighborhood school, a public

magnet program, or a private school.

Access to non-catchment neighborhood school:• before 2003: required the permission of both the

catchment school and the preferred school• after 2003: Open Boundaries policy allows students to

attend any public school in the province provided space is available after catchment area students have enrolled

Rules for admission to oversubscribed schools are set by province and individual school districts.

Empirical specificationLinear probability model for whether a public school student

separates from her school at the end of the Grade 4 school year

1999-2003, pooled

School separations depend on:• individual characteristics• time-varying school characteristics• information • school fixed effects• year effects

Standard errors clustered school-by-year.

Empirical specification• individual characteristics

o gender, home language, special needs, own test scores, previous seps, French Immersion o census neighborhood chars, distance to school, distance to catchment school, # of schools in proximity to home

• school characteristics o %ESL, %home lang,%abor, %special needso school mean test scores (grade 4, current year)

Information variablesTest scores are released in the school year following the

school year in which tests are writteni.e. information is based on one-year lag of test scores

We allow information shocks to affect separations as:

“news”interaction between information measure and indicator

for year in which it is released

“oldnews”Interaction between information measure and indicator

for years subsequent to year of release

Empirical specification

Effects of school and individual characteristics vary across information regimes.

1999 – no test scores available2001, 2002 - school-level test score on Ministry’s website2003, 2004 - school report cards widely publicized

Identifying assumption

Previous years’ shocks affect separations only through their persistent effect on current test scores

Open boundariesCould be pent-up demand for separations from low-achieving

schools

Release of this pent-up demand under Open Boundaries could be confounded with first release of Fraser Institute scores

We account for this by interacting school mean test scores with indicator for whether open boundaries in effect

Interpretation of results before 2003: effect of information on seps under neighborhood enrollment

Interpretation of results after 2003: effect of information on seps under Open Boundaries policy

Sample Percent

School Separation Rate

All 100.0 9.3

Aboriginal 5.0 19.6

Non-English 32.6 12.8Chinese 13.5 8.4Punjabi 8.1 10.1Other 14.3 13.5

ESL 30.2 11.0

Catchment school 75.6 8.7French immersion 6.5 6.6N 74358

Sample Characteristics, Grade 4 Students Enrolled in Lower Mainland Public Schools, 1999/2000-2003/2004

English Chinese PunjabiSchool-Mean FSA Variables1999 FSA Score*(Yr=2000) -0.035** 0.026 0.006

(0.017) (0.035) (0.048)

2000 FSA Score*(Yr=2001) -0.027* 0.011 0.006(0.016) (0.026) (0.032)

Fraser Institute Variables1999-2001 FI Score*(Yr=2002) -0.005 -0.025*** -0.013

(0.007) (0.010) (0.013)

2002 FI Score*(Yr=2003) -0.000 -0.030*** -0.017*(0.004) (0.006) (0.010)

Effect of Information on Separation Probability, Grade 4 Students in Lower Mainland Public Schools, 1999/2000-2003/2004

English Chinese Punjabi“Old News” Variables1999 FSA Score*(Yr>2000) -0.008 0.056* -0.010

(0.017) (0.033) (0.039)2000 FSA Score*(Yr>2001) -0.012 -0.005 0.059

(0.016) (0.023) (0.038)1999-2001 FI Score*(Yr>2002) -0.004 0.007 0.008

(0.006) (0.009) (0.015)Current FSA Variables (Controls)Current Mean FSA Score -0.008 -0.019 0.003

(0.012) (0.021) (0.033)Current Mean FSA Score*(Yr>2001) 0.008 0.041 0.025

(0.017) (0.033) (0.048)Number of Schools 362 318 261Number of Observations 48149 10045 6041

Effect of Information on Separation Probability, Grade 4 Students in Lower Mainland Public Schools, 1999/2000-2003/2004

English Non-EnglishRich Poor Rich Poor

School-Mean FSA Variables1999 FSA Score*(Yr=2000) 0.030 -0.184***0.069 0.046

(0.024) (0.043) (0.055) (0.044)2000 FSA Score*(Yr=2001) -0.005 -0.043 -0.043 -0.002

(0.021) (0.033) (0.044) (0.038)Fraser Institute Variables1999-2001 Mean FI 0.004 -0.017* -0.020 0.003

(0.007) (0.010) (0.015) (0.012)

2002 FI Score*(Yr=2003) 0.000 -0.012 -0.039*** -0.017*(0.006) (0.010) (0.012) (0.010)

By Home Language and Quartile of Distribution of Mean Household Income

What have we learned?

Some have worried that only well-off families can take advantage of school choice. This does not appear to be true.

Families in low-income neighborhoods respond strongly to information.

What have we learned?

Publicity associated with Fraser Institute rankings appears to play an important role in ensuring that information reaches non-English speaking families.

What have we learned?

Annual updates of info don’t seem to confuse people – they don’t keep responding.

On the contrary, annual releases may be important for informing newcomers to the province.

What have we learned?Public information about test scores provides information

that parents use.

Authorities should take care to ensure that information brings competitive pressure to bear on schools that are ineffective, rather than on schools that serve disadvantaged populations.

Designing meaningful measures of school effectiveness is a challenge.