School accountability and school choice

Cassandra Hart, University of California-DavisDavid Figlio, Northwestern University & National Bureau of Economic

Research

Conference on Competition and Subnational GovernmentsApril 26, 2014

Cream-skimming in school choice literature: concern that children of most-informed parents will use programs and leave public schools

School report cards under accountability programs meant to help parents make more informed choices

Relevant literatures on ◦ parental use of information (Hastings, van Weelden, &

Weinstein, 2007; Billings, Brunner, & Ross, 2014)◦ information on capitalization of information in housing

markets (Figlio & Lucas, 2004) But little work on how increased information (non-

targeted) affects distribution of children across schools

Motivation: Sorting between schools

Potential for provision of new information to either polarize or equalize composition of schools◦ Salience of grades—clear evaluation of quality from

an authoritative source—may be especially high here Might have greater effect on low-SES families,

if they had less access to reliable gauges of quality previously

Might have greater effect on high-SES families, if they are better situated to act on new information

Motivation: sorting between schools

Look at how kindergarten class composition changes based on the provision of new information based on introduction of/changes to accountability regimes◦ A+ Accountability Plan roll-out (1999)

Differences in effects based on available alternatives

Effects on segregation indices

This paper

Introduced in Spring 1999◦ Graded schools A-F◦ Replaced system in which parents primarily had

access to information on raw test scores-provided “lumpy” evaluation of schools

Florida A+ Accountability Plan

Grade

Number of schools

Percent

A 116 8.3%B 191 13.6%C 643 45.7%D 402 28.6%F 54 3.8%

Newspapers FLDOE website

Publicity around 1999 grades

Front-page news in high-performing districts…

…and low-performing districts

Real estate agents School spaces DOE website

Publicity around 1999 grades

1999 school average scores by 1999 grades

Florida Education Data Warehouse individual student records (school enrollments, data on lunch status, etc.)

Birth records for birth cohorts from 1992 to 1999◦ Detail on parental and family characteristics

Maternal education Maternal age Maternal marital status at birth Birth conditions, (e.g. birth weight, multiple births,

etc.)

Data

School-level information on changes in composition of incoming kindergarten classes ◦ Maternal characteristics recorded during birth of

child Years of education Age Marital status (binary indicator for married)

◦ Family income in kindergarten year (share FRL)◦ Index of all of these◦ Size of incoming kindergarten class

Analytic plan

1999: new grade shock (for all grade levels separately). Equation for A grade:

Kst=school average characteristics of incoming kindergarten class (or log number students in some specifications)

Key IV: series of year indicators*1999Grade τs: school fixed effect. SchoolVars includes excess

absenteeism rate (share of students absent 21+ days), stability/mobility rate, suspension rates, and average FCAT test scores

Years 1997/98-2001/02 Identify grade effect by controlling for underlying

elements of school grade (Figlio & Lucas, 2004)

stsstsst choolVarsSearYAearYK

1999*

Analytic plan-1999 shock

Composition: Mean maternal education

*** p<.01, **, p<.05, *, p<.10,.

  A [compar:

B,C]

B [compar: C]

D [compar: C]

F [compar:

C,D]Grade X 1997 0.005 -0.028 0.050* -0.045  (0.040) (0.032) (0.026) (0.051)Grade X 1999 0.064** 0.004 -0.023 -0.090*  (0.030) (0.028) (0.024) (0.048)Grade X 2000 0.104*** 0.027 -0.064** -0.084  (0.033) (0.033) (0.026) (0.080)Grade X 2001 0.122*** 0.017 -0.018 -0.101  (0.040) (0.032) (0.029) (0.083)School Controls Y Y Y YSchool FE Y Y Y YUnique Schools 943 829 1,033 1,087School-Years 4,693 4,123 5,145 5,415Group Outcome Mean

13.450 13.189 11.308 11.110

Group Outcome SD 0.778 0.814 0.903 0.770

Composition: Mean maternal age

  A [compar:

B,C]

B [compar: C]

D [compar: C]

F [compar:

C,D]Grade X 1997 -0.017 0.031 0.053 0.180  (0.093) (0.081) (0.067) (0.120)Grade X 1999 0.107 0.065 -0.065 0.039  (0.083) (0.074) (0.056) (0.128)Grade X 2000 0.147 0.042 -0.094 0.210  (0.091) (0.087) (0.065) (0.146)Grade X 2001 0.184* 0.154* -0.068 -0.075  (0.103) (0.084) (0.069) (0.167)School Controls Y Y Y YSchool FE Y Y Y YUnique Schools 943 829 1,033 1,087School-Years 4,693 4,123 5,145 5,415Group Outcome Mean

28.459 27.973 25.209 24.655

Group Outcome SD 1.587 1.615 1.511 1.335*** p<.01, **, p<.05, *, p<.10,.

Composition: Fraction with married parents

  A [compar:

B,C]

B [compar: C]

D [compar: C]

F [compar:

C,D]Grade X 1997 -0.006 -0.002 -0.003 -0.003  (0.007) (0.006) (0.005) (0.011)Grade X 1999 0.006 -0.003 -0.002 0.009  (0.005) (0.006) (0.004) (0.011)Grade X 2000 0.015** 0.005 -0.002 0.016  (0.006) (0.006) (0.005) (0.013)Grade X 2001 0.010 0.004 0.002 0.021*  (0.007) (0.006) (0.005) (0.012)School Controls Y Y Y YSchool FE Y Y Y YUnique Schools 943 829 1,033 1,087School-Years 4,693 4,123 5,145 5,415Group Outcome Mean

0.819 0.788 0.459 0.326

Group Outcome SD 0.098 0.104 0.157 0.148*** p<.01, **, p<.05, *, p<.10,.

Composition: Fraction on subsidized lunch

  A [compar:

B,C]

B [compar: C]

D [compar: C]

F [compar:

C,D]Grade X 1997 0.002 0.010 -0.019** -0.028*  (0.008) (0.007) (0.007) (0.014)Grade X 1999 -0.009 -0.012** -0.002 -0.017  (0.007) (0.006) (0.006) (0.013)Grade X 2000 -0.038*** -0.011 0.012 -0.016  (0.009) (0.009) (0.008) (0.018)Grade X 2001 -0.019** -0.012 0.023** 0.011  (0.009) (0.009) (0.010) (0.019)School Controls Y Y Y YSchool FE Y Y Y YUnique Schools 943 829 1,033 1,087School-Years 4,693 4,123 5,145 5,415Group Outcome Mean

0.219 0.261 0.732 0.853

Group Outcome SD 0.142 0.158 0.168 0.132*** p<.01, **, p<.05, *, p<.10,.

Empirically driven Regressed grade-year standardized FCAT score

(average of math and reading) on:◦ Maternal education◦ Maternal age◦ Maternal marital status◦ Lunch status as of kindergarten◦ Race dummies (Black, White, Latino, Asian)◦ Maternal immigrant status

“Index” is the predicted third grade (standardized) test score based on these characteristics, fixed at birth or (for lunch status) K entry

Index creation

Composition: Advantage index  A

[compar: B,C]

B [compar: C]

D [compar: C]

F [compar:

C,D]Grade X 1997 -0.000 -0.005 0.012*** 0.008  (0.006) (0.005) (0.004) (0.009)Grade X 1999 0.014*** 0.003 -0.004 -0.003  (0.005) (0.005) (0.004) (0.008)Grade X 2000 0.025*** 0.005 -0.006 0.003  (0.006) (0.006) (0.005) (0.012)Grade X 2001 0.022*** 0.006 -0.009* 0.007  (0.007) (0.006) (0.005) (0.012)School Controls Y Y Y YSchool FE Y Y Y YUnique Schools 943 829 1,034 1,088School-Years 4,693 4,123 5,145 5,415Group Outcome Mean

0.280 0.239 -0.251 -0.398

Group Outcome SD 0.134 0.128 0.202 0.157*** p<.01, **, p<.05, *, p<.10,.

Advantage index: First-borns only

  A [compar:

B,C]

B [compar: C]

D [compar: C]

F [compar:

C,D]Grade X 1997 0.000 0.002 0.019*** -0.000  (0.008) (0.008) (0.007) (0.014)Grade X 1999 0.018** 0.014** -0.001 0.010  (0.008) (0.007) (0.006) (0.011)Grade X 2000 0.027*** 0.020** -0.007 0.012  (0.009) (0.008) (0.007) (0.019)Grade X 2001 0.031*** 0.020** -0.006 0.011  (0.010) (0.008) (0.007) (0.018)School Controls Y Y Y YSchool FE Y Y Y YUnique Schools 943 829 1,034 1,088School-Years 4,688 4,118 5,142 5,412Group Outcome Mean

0.290 0.246 -0.236 -0.403

Group Outcome SD 0.143 0.139 0.219 0.192*** p<.01, **, p<.05, *, p<.10,.

Ln(enrollment)  A

[compar: B,C]

B [compar: C]

D [compar: C]

F [compar:

C,D]Grade X 1997 -0.034 -0.007 0.032** 0.087***  (0.025) (0.019) (0.016) (0.026)Grade X 1999 0.057*** 0.049*** 0.022* 0.025  (0.019) (0.015) (0.012) (0.030)Grade X 2000 0.052** 0.057*** -0.045*** -0.058  (0.025) (0.019) (0.015) (0.037)Grade X 2001 0.033 0.073*** -0.094*** -0.046  (0.024) (0.020) (0.017) (0.037)School Controls Y Y Y YSchool FE Y Y Y YUnique Schools 943 829 1,034 1,088School-Years 4,693 4,123 5,145 5,415

*** p<.01, **, p<.05, *, p<.10,.

Ln(enrollment): First-borns  A

[compar: B,C]

B [compar: C]

D [compar: C]

F [compar:

C,D]Grade X 1997 -0.016 -0.036 0.013 0.053  (0.034) (0.027) (0.023) (0.052)Grade X 1999 0.068** 0.031 0.030 0.010  (0.028) (0.024) (0.019) (0.041)Grade X 2000 0.098*** 0.022 -0.023 -0.085  (0.032) (0.029) (0.021) (0.053)Grade X 2001 0.072** 0.050* -0.081*** 0.003  (0.031) (0.026) (0.023) (0.051)School Controls Y Y Y YSchool FE Y Y Y YUnique Schools 943 829 1,034 1,088School-Years 4,688 4,118 5,142 5,412

*** p<.01, **, p<.05, *, p<.10,.

Ln(enrollment): Effect of 1999 A  Mother:

CollegeMother: No

CollegeNot FRL FRL

Grade X 1997 -0.045 -0.066* -0.038 -0.074  (0.035) (0.037) (0.029) (0.056)Grade X 1999 0.077*** 0.021 0.070*** 0.016  (0.024) (0.025) (0.022) (0.038)Grade X 2000 0.067** -0.019 0.110*** -0.187***  (0.030) (0.032) (0.031) (0.060)Grade X 2001 0.057* -0.047 0.064** -0.062  (0.030) (0.036) (0.030) (0.058)School Controls Y Y Y YSchool FE Y Y Y YUnique Schools 945 945 945 944School-Years 4,704 4,704 4,704 4,669

*** p<.01, **, p<.05, *, p<.10,.

Does the effect of school grades depend on the performance/availability of neighboring schools◦ Availability=>can’t easily sort if nothing else

easily available◦ Performance=>benchmarking/signaling story

School grades and signaling

Lee

Measures of availability/quality of alternatives:◦ Any other public elementary within 3 miles (with

1999 grade)◦ Any other public elementary within 3 miles with

1999 grade C or lower◦ Had Public Choice grant (to support open

enrollment plans) in 1999

Effects: by available alternatives

Available alternatives: by grade

Weighted by 1998 student populations

  A B C  mean/sd mean/sd mean/sdAny Other Schools in 3 mi (%) 70.435 70.106 72.745  (45.854) (45.908) (44.563)Any Schools C or lower in 3 mi (%) 48.968 49.840 66.478  (50.231) (50.140) (47.245)District Had Public Choice Grant: 1999 (%)

57.851 54.430 49.589

  (49.598) (49.934) (50.037)

Index outcome: by alternatives  (1) (2) (3) (4) (5) (6)

Interactions None w/i 3 Any w/i 3 None C or lower w/i

3

Any C or lower w/i

3

No Open Enroll 1999

Open Enroll Grant 1999

A X 1997 -0.001 -0.000 0.001 -0.001 -0.008 0.006  (0.011) (0.007) (0.009) (0.008) (0.009) (0.008)A X 1999 0.011 0.019*** 0.013* 0.018** 0.011 0.016**  (0.010) (0.006) (0.007) (0.007) (0.007) (0.007)A X 2000 0.023** 0.029*** 0.023*** 0.028*** 0.012 0.033***  (0.011) (0.007) (0.009) (0.007) (0.009) (0.007)A X 2001 0.002 0.032*** 0.004 0.037*** 0.015 0.026***  (0.014) (0.008) (0.010) (0.009) (0.012) (0.007)Unique Schools 261 636 357 540 474 471School-Years 1,299 3,167 1,776 2,690 2,358 2,346

1997 diff p 0.964 0.871 0.2431999-01 joint p 0.116 0.040 0.255

IV: Within-city std dev of 1999 grades (standardized)

Outcome measures: Multigroup segregation index (Theil’s Information Index)◦ By race (White, Black, Latino, other)◦ By parental education (HS dropout, HS grad, some

college, college grad)◦ By lunch status (not FRL/FRL)

Segregation

  Racial Seg H Education H Not FRL Hstd_sd99GPA_cX1997 0.000 0.000 -0.001  (0.001) (0.001) (0.002)std_sd99GPA_cX1999 -0.000 0.001** 0.002  (0.001) (0.000) (0.001)std_sd99GPA_cX2000 0.000 0.001** 0.002  (0.001) (0.000) (0.002)std_sd99GPA_cX2001 -0.002 0.001* 0.000  (0.001) (0.001) (0.002)Year FE Y Y YControls Y Y YNumber of Cities 180 180 180City-Years 896 897 896Seg Measure Mean 0.076 0.038 0.075Seg Measure SD 0.071 0.031 0.074Seg Measure Min 0.001 0.000 0.000Seg Measure Max 0.426 0.146 0.467p-value ‘99, ‘00, ‘01 F test

0.329 0.067 0.462

Evidence of parental response to new provision of information (response to high-quality), especially among more educated parents◦ Modest evidence of benchmarking—response

somewhat greater when alternatives judged poor quality (or too far to be easy alternatives)

◦ Robust to district-by-district exclusion, exclusion of red-shirters and repeaters

◦ Some evidence of segregative effects (by parental education levels only)

Conclusion

Thank you!

Suggestions?

cmdhart@ucdavis.edu, figlio@northwestern.edu