HOW DOES YOUR KINDERGARTEN CLASSROOM AFFECT YOUR EARNINGS...

HOW DOES YOUR KINDERGARTENCLASSROOM AFFECT YOUR EARNINGS?

EVIDENCE FROM PROJECT STAR

Teresa Steininger, Srajal Nayak

EC 426 - Public Economics

November 13, 2017

Teresa Steininger, Srajal Nayak (EC 426 - Public Economics)Tennessee STAR Experiment November 13, 2017 1 / 27

Outline

1 Recap - STAR Experiment

2 Chetty et al. - QJE (2011)

3 Class Questions

Question 1

Question 2

Question 3

Question 4

Question 5

4 References


Recap - STAR Project

The Tennessee STAR project randomly assigned 11,571 students to Small(∼15) or Large (∼22) classrooms within their schools in grades KG - 3rd.

Teachers were also randomly assigned to classrooms.

Students wrote a standardized test - measuring their reading and mathskills - at the end of each year.

Students were intended to remain in the same class type (small vs large)through third grade.

However some students moved across class types (movement notcompletely random!!).

Substantial attrition (students changing schools/ getting retained in grade)

New students entered in participating schools in grades 1-3 (also randomlyassigned to clasrooms within school on entry).

To account for non-random sorting, assign treatment based on initialrandom assignment (intent-to-treat)


Chetty et al. - QJE (2011)

2 empirical strategies - First study the impact of “observable” classroomcharacteristics. Some of the main results of this analysis are:

KG test scores highly correlated with outcomes such as earnings at age27, college attendance, home ownership, and retirement savings.

Students in small classes significantly(1.8 %-points) more likely to attendcollege and exhibit improvements on other outcomes.

Students with a more experienced teacher in KG have higher earnings.

Students randomly assigned to ‘higher quality classrooms’ in grades KG-3have higher earnings, college attendance rates, and other outcomes.

The effects of class quality fade out on test scores in later grades.

Suggestive evidence of long-term impact through gains in non-cognitive skillsthat persist. Hence, turn to second empirical strategy - captures both“observable”and “unobservable” aspects of classrooms.


Class Questions

1 What does the regression of students’ long-term outcomes on the“leave-out” means of the class mates’ scores capture?

2 What potential impact of teachers and peers is this not capturing?

3 Can we distinguish between the impact of teachers and peers?

4 What if we observed students’ scores over two years with the teachersrandomly re-allocated to the different classes in the second year?

5 Discuss whether this analysis would be useful to guide policy on teacherrecruitment/retention and on assigning students to different classes?


Question 1: Model 1

Model 1 aims to answer:

Key Policy Question: Do classroom environments that raise test scoresalso improve adult outcomes?

How to model a good classroom environment?

End-of-year test scores are the best available measure in short run.

Model 1 tries to test for “class effects” (includes effects of teachers, peers,class-level shocks) on scores and earnings exploiting random assignmentto classrooms.

We discuss a few caveats of this model specification and always try to findways to circumvent it.


Question 1: Model 1

Assumptions: All students enter the same grade (later extend it to the casewith multiple entry grades).

sicn = dn +∑k

µSkZ

kcn + aicn (1)

yicn = δn +∑k

µYk Z

kcn + ρaicn + νicn (2)

sicn, yicn ... test scores/earnings for student i in class c at school n;

dn, δn ...School F.E.;

aicn ... intrinsic academic ability;

νicn ... intrinsic earning ability that is uncorrelated aicn;

Zcn = (Z1cn, ..., Z

kcn) ... vector of classroom characteristics like class size,

teacher experience etc.;

µSk , µ

Yk ... effects of class characteristic k on test scores/earnings;


Question 1: Model 1

sicn = dn + zcn + aicn (3)

yicn = δn + βzcn + zYcn + ρaicn + νicn (4)

zcn =∑k

µSkZ

kcn ... total impact of the bundle of class characteristics offered

ρ ... controls correlation b/w aicn and νicn;

zYcn ... CR components that affect only earnings without

affecting test-scores;

β ... correlation of class-effects on scores and class-effects

on earnings


Question 1: Getting a measure of zcn

zcn unobserved: hence we measure it using mean test score of the class.

scn =1

I

I∑i=1

sicn = dn + zcn +1

I

I∑i=1

aicn (5)

Assume: mean value of zcn across classes within a school is zero (zn = 0).

∆ scn = scn − sn = zcn +

[1

I

I∑j=1

ajcn −1

IC

C∑c=1

I∑j=1

ajcn

](6)

∆scn is a noisy observable measure of class quality zcn (noise arises fromvariation in student abilities across classes).

∆scn is consistent (as I →∞, ∆scn converges to true zcn)


Question 1: Possible regression model

Motivation for estimating the following regression:

yicn = αn + bM∆scn + εicn (7)

Identifying assumption: random assignment of students into classrooms -εicn ⊥ zcn, i.e. intrinsic abilities are orthogonal to class quality

For this reason we only include every student once: in their year of entryinto school n

Large sample properties: OLS estimate b̂M consistent estimate of β

Small sample properties: ∆scn correlated with aicn because class sizeis small

OLS estimate b̂M upward-biased in finite samples (high ability studentraises the average class score and also has high earnings)

→ Solution: Omit own score sicn from measure of class quality forindividual i


Question 1: Final Regression Model

Proxy for class-quality using leave-out mean (or jackknife) peer scoremeasure

∆s−icn = s−icn − s−in = zcn +

[1

I − 1

I∑j=1,j 6=i

ajcn −1

IC − 1

C∑k=1

I∑j=1,j 6=i

ajkn

](8)

Author’s main estimates are based on the following regression type:

yicn = αn + bLM∆s−icn + εicn (9)

”Own-observation-problem” solved: plimN→∞b̂M > 0 iff β > 0.


Question 1: Regression limitations

yicn = αn + bLM∆s−icn + εicn (10)

1 ∆s−icn noisy measure of class quality

attenuation bias: bLM biased towards 0 relative to β.

The authors use sample variance of test scores to estimate the degree ofthis bias at 23%.

2 Above analysis ignores variation in class quality due to peer-effects.

High-ability student raises peers’ scores (zcn ⊥ aicn violated).

This creates upward bias in bLM

But authors can tightly bound this ”reflection bias”


Question 1: Regression Results


Question 2: What potential impact of teachers andpeers is this not capturing?

yicn = δn + βzcn + zYcn + ρaicn + νicn

zYcn : Impact of classroom characteristics that matter for earnings but not

for grade.

Motivating teacher, Network effects

Communication skills, Creativity, Behavioural attributes

Question: How likely that there are strong classroom effects that affectearnings but are orthogonal to test scores given that good teacher willalso lead to good test scores?

Authors report estimate that only 1/5 of the variance of the class effecton earnings comes through class effects on test scores

Use random-effects estimator to see how classroom variation changes afterwe control for test scores


Question 3: Can we distinguish between the impact ofteachers and peers? (I)

Regression of interest

yicnw = αnw + β∆s−icnw +Xicnwδ + εicnw (11)

Intuitively, the measure ∆s−icn answers:

“How good are your classmates’ scores compared with those of classmatesyou could have had in your school?” (Chetty 2011, p.1635)

What will classmates’ end of year scores depend on?

Teacher quality: how well did the teacher communicate concepts/controlthe class/motivate the class relative to other teachers

Peer quality: how disruptive were they? How attentive? Did they helpeach other?

Shocks: were many students sick on test day, was there constructionoutside classroom


Question 3: Can we distinguish between the impact ofteachers and peers? (II)

Formally:

∆s−icn = s−icn − s−in = zcn +

[1

I − 1

I∑j=1,j 6=i

ajcn −1

IC − 1

C∑k=1

I∑j=1,j 6=i

ajkn

]

=∑k

µSkZ

kcn +

[1

I − 1

I∑j=1,j 6=i

ajcn −1

IC − 1

C∑k=1

I∑j=1,j 6=i

ajkn

]

∆s is a single index that captures all classroom characteristics affectingtest scores

So what are we measuring?

bLM gives an estimate of the “extent to which class quality in the initialclass of entry (weighted by the entry rates across the four grades) affectsoutcomes” (Chetty 2011, p.1636)


Question 3: Model 2 set-up

Why not do better and estimate teacher effects? Consider Model 2:“Fixed Effects Model”

yicn = αn + γcn +Xicnδ + εicn (12)

yicn ... outcome for student i, first entered class c in school n

αn ... school fixed effect

γcn ... class effect

Xicn ... vector of predetermined individual background characteristics

Estimation through a fixed-effects specification for the classeffects (include dummy for every γcn)

Identifying assumption: random assignment of students (evidently givenin this experiment, but big debate in literature (Chetty vs Rothstein)

Advantage: no longer have to explicitly control for class quality butdrawback: cannot identify whether classroom environments that raisetest scores also improve outcomes later in life


Question 3: Model 2 results (I)

Consider Model 2: “Fixed Effects Model”

yicn = αn + γcn +Xicnδ + εicn

H0 : no class effects, all γcn = 0 due to random assignment

Conduct F-test, reject null for y = KG end-of-year test scores, wageearnings but not for 8th grade test scores

NB: specification only uses KG sample (not enough power to detect classin later years due to small proportion of new entries)

Suppose now we tried to only include teacher specific fixed effects

Ever teacher only captured once in data, i.e. always observe teacher andclass jointly

→ Model indentical, results identical


Question 3: Model 2 results (II)


Question 4: Distinguishing b/w teacher and class effect?(I)

Fix 1: Observe teachers in multiple classrooms

yicn,t = αn + γjn +Xicn,tδ + εicn,t

γjn ... teacher j fixed effect

t ... time subscript

Can identify teacher fixed using variation if test scores across classroomswith same teacher

Identifying assumptions:

1 students randomly assigned to teachers

2 teacher quality constant across time (see Chetty (2014a) for relaxation ofthis assumption, model with drift)


Question 4: Distinguishing b/w teacher and class effect?(II)

Fix 2: Observe students’ scores over two years with teachersrandomly re-allocated to the different classes in the second year

yicn,t = αn + γjn +Xicn,tδ + εicn,t

γjn ... teacher j fixed effect

t ... time subscript

Identifying assumptions

1 teachers randomly assigned to students

2 teacher quality constant across time AND

3 teacher quality constant across grades it teaches

→ Both “fixes” will provide consistent estimates of γjn, but for bestestimates of teacher quality would ideally observe same teacher over manyyears


Question 5: Would this analysis be useful to guidepolicy on teacher recruitment/retention?

Recruitment/retention

There are class effects that fade-out in the short run, but affect long termoutcomes

If these effects are caused by the teachers, basing teacher retention policieson short-term metrics may not be efficient, particularly if the aspects ofteaching affecting short term and long term outcomes are complimentary

Introducing such policies, can lead to teachers teaching to the test

Hire good teachers/ sack out the bad ones:

Previous literature on teachers’ value added - Value added from past isgood predictor for future value added (Chetty et al. (2014)(a))

Does teachers’ valued added differ across student types? If yes, it couldinvalidate large parts of our analysis.


Question 5: Would this analysis be useful to guidepolicy on assigning students to different classes?

Assigning students to classes: Different goals of schools

High average performance i.e. focus on mean performance within school(Compensation for bad or good teachers to keep average quality constant)

focus on top or bottom tail in distribution (sorting of good students to goodteachers vs sorting of bad students to good teachers)

focus on narrow distribution

Possible extensions: Performance pay? Make pay proportional toteacher-value added!


References I

R. Chetty, J. N. Friedman, N. Hilger, E. Saez, D. W. Schanzenbach, andD. Yagan.

How does your kindergarten classroom affect your earnings? evidencefrom project star.

The Quarterly Journal of Economics, 126(4):1593–1660, 2011.

R. Chetty, J. N. Friedman, and J. E. Rockoff.

Measuring the impacts of teachers ii: Teacher value-added and studentoutcomes in adulthood.

The American Economic Review, 104(9):2633–2679, 2014.


References II

R. Chetty, J. N. Friedman, and J. E. Rockoff.

Response to rothstein (2014)‘revisiting the impacts of teachers.’.

Unpublished manuscript. Downloaded from http://obs. rc. fas. harvard.edu/chetty/Rothstein response. pdf on October, 13:2014, 2014.

A. B. Krueger and D. M. Whitmore.

The effect of attending a small class in the early grades on college-testtaking and middle school test results: Evidence from project star.

The Economic Journal, 111(468):1–28, 2001.

J. Rothstein.

Teacher quality in educational production: Tracking, decay, and studentachievement.

The Quarterly Journal of Economics, 125(1):175–214, 2010.


HOW DOES YOUR KINDERGARTEN CLASSROOM AFFECT YOUR EARNINGS...

Documents

Transcript of HOW DOES YOUR KINDERGARTEN CLASSROOM AFFECT YOUR EARNINGS...