Understanding Student Achievement: The Value of Administrative Data Eric Hanushek Stanford...
-
date post
22-Dec-2015 -
Category
Documents
-
view
217 -
download
0
Transcript of Understanding Student Achievement: The Value of Administrative Data Eric Hanushek Stanford...
Understanding Student Achievement: The Value of Administrative Data
Eric HanushekStanford University
Big Issues in School Policy Debates
Relating analysis to policy interests
Confidence in causation
Generalizability
Analytical designs
Random assignment experiments Natural experiments “Data solutions”
Trade-offs Credibility Expense Questions that can be addressed
UTD Texas Schools Project
Multiple cohorts followed 1993-2002
Annual achievement in grades 3-8 (TAAS math and reading)
Each cohort > 200,000 students in over 3,000 schools
Augmented with district data
Examples of Topics Teacher quality variations Charter schools
Not discussed School choice and mobility Special education Teacher mobility Racial composition Peer achievement
Existing Evidence on Teacher Quality
Substantial variation in teacher quality
Observable characteristics of teachers explain little of the variation
Salary and other factors affect teacher transition probabilities
No evidence on transitions and teacher quality
Questions Addressed
What is variation in teacher quality? Measurable characteristics?
Do urban schools lose their best teachers? Quality by transitions
Do districts hire the best teachers?
Basic model
standardized gain
( , )
isg
j
G
f X S
j j
Measurement Error and Calculation of Variance of Teacher Quality
Observe teachers in two years:
Correlation across years:
(1) (2),j j
12( ) var( ) / var( )E r
Estimated Variance in Teacher QualityLonestar District
Within districtWithin school
and year
unadjusteddemographic
controlsunadjusted
demographic controls
Teacher-year variation
0.210 0.179 0.109 0.104
Adjacent year correlation
0.500 0.419 0.458 0.442
Teacher quality variance / (s.d.)
0.105(0.32)
0.075(0.27)
0.050(0.22)
0.047(0.22)
Kernal Density Estimates of Teacher Quality Distribution: Standardized Average Gains by Teacher Move Status
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
-2 -1.8 -1.6 -1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8
Relative teacher quality (s.d.)
Stays at Campus Campus Change District Change Out of Public Education
Conclusions on Teacher Quality Very large differences among teachers
Differences within schools much larger than between schools
Conventional measures not good index of quality (master’s degree, certification test)
Observable characteristics First year of experience Teacher-student race match
Common assumptions about market for teachers not correct Best do not leave Districts with advantages do not use them
Popularity of charter schools
3,000 charter schools
40 states plus DC since 1991
1 percent of total students
10 percent of size of private school market
7+ percent rate of closure
Evaluation issues
Most analysis of entry and participation
No reliable information on performance
Difficulty of selection issue
Very political
Evaluation approaches
Model selection process [Heckman (1979)]
Instrument for attendance [Neal(1997)]
Intake randomization [Howell and Peterson (2002)]
Difficulties with traditional approaches
Difficult to find factors affecting attendance but not achievement
Cannot handle treatment heterogeneity
Empirical framework Mean differences in individual value-added
Identify charter school from individual entry-exit Consider time varying effects associated with
charter school movements
Heterogeneity across schools
Consumer responsiveness to quality
Charter enrollment
1997 2001
4th grade 0.2 % 0.8%
7th grade 0.2% 0.9%
Participation rates by race/ethnicity
1997 2001
Blacks 0.8% 2.2%
Hispanics 0.1% 0.6%
Whites 0.0% 0.4%
Low income 0.3% 0.8%
Charters by vintage (analytical)
1997 1998 1999 2000 2001 2002 Total
one 17 10 70 83 43 47 270
Charters by vintage (analytical)
1997 1998 1999 2000 2001 2002 Total
one 17 10 70 83 43 47 270two 2 16 9 69 78 40 214
Charters by vintage (analytical)
1997 1998 1999 2000 2001 2002 Total
one 17 10 70 83 43 47 270two 2 16 9 69 78 40 214Three 0 2 15 8 68 73 166Four 0 1 2 15 8 66 92Five+ 0 0 1 3 17 22 43
Charter school effect
Charter -0.17
Age 1 -0.33
Age 2 -0.25
Age 3 -0.08
Age 4 0.00
Age 5 or more 0.02
Demographically Adjusted School Quality
Residual-Based Quality Measure
Charter Non-Charter
-1.57744 .788794
0
2.32064
Do parents make good decisions?
Parents cannot see value added Considerable mobility/exiting
Models: Exit=f(quality, age, year, race, grade)
Parental Choice(linear probability of exit)
Student characteristics
Student + peer
characteristics
Student + peer
characteristics + peer
achievement
School quality
0.002 0.006 0.006
School quality x charter
-0.152 -0.142 -0.138
Parental Choice(linear probability of exit)
Student characteristics
Student + peer
characteristics
Student + peer
characteristics + peer
achievement
Student + peer
characteristics + peer
achievement
School quality
0.002 0.006 0.006
School quality x charter
-0.152 -0.142 -0.138
high income-0.187
low income-0.096
Conclusions on Charter Schools
Difficult start-up period Mean performance regular ≈ charter
after two years Heterogeneity in both markets Parents react to quality in charter
market Low income reaction one half upper
income
Administrative data
Pros Broader generalizability Understanding heterogeneity Perhaps less costly
Cons Requires structure (e.g., linearity, time pattern of
achievement) Regulatory problems (confidentiality) Data quality issues
Papers on Teacher Quality and Charter Schools
www.hanushek.net or www.nber.org
Hanushek, Eric A., John F. Kain, Daniel M. O'Brien, and Steve G. Rivkin. 2005. "The market for teacher quality." National Bureau of Economic Research, Working Paper No. 11154, (February).
Hanushek, Eric A., John F. Kain, Steve G. Rivkin, and Gregory F. Branch. 2005. "Charter school quality and parental decision making with school choice." National Bureau of Economic Research, (March).