PROF. ERIC BETTINGER STANFORD UNIVERSITY, NBER Postsecondary Remediation: How can we serve the needs...

PROF. ERIC BETTINGERSTANFORD UNIVERSITY, NBER

Postsecondary Remediation:Postsecondary Remediation:

How can we serve the needs of How can we serve the needs of Underprepared Students?Underprepared Students?

The Academic Gatekeeper: Remedial & Developmental

Postsecondary Courses2

Lack of academic preparation is a significant barrier to college success – How should we address this problem?

Nationally approx. 40% of 1st year students are placed into college remediation (55-60% at CCs)

The bulk of remediation is provided at non-selective publics colleges & universities, the point of entry for 80% of 4yr students and 99% of 2yr students

The remediation placement exam taken once arriving on campus has become the key gatekeeper to a college-level education (often a surprise to students)

3

CUNY phased remediation out of its 4yrs (1999)“CUNY university system currently devotes far too much money and effort to teaching skills that students should have learned in high school.”

Debates about Remediation

CSU: Fall 2001 “kicked out more than 2,200 students – nearly 7% of the freshman class – for failing to master basic English and math skills.”

Proponents: opportunities for underprepared, many of whom did not have the chance to take in HS

Critics: Provides disincentives for high school students; Double dipping; Not the appropriate place

$1 Billion each year at publics (conservative estimate)

The Continuing Debates about Remediation…

4

CT and AZ do not allow remediation at publics

8 states, incl. FL and IL, restrict remediation to 2yrs

VA: Some High Schools “guarantee” their diplomas – pay remedial expenses of their students

FL: Legislature elected to have students pay the full (unsubsidized) cost of their remediation (4 times the regular tuition)

Limits on funding of remediation: CA, TX, TN, & UT

Difficulties in Studying Remediation5

Comparisons are Different. How do you compare remedial to non-remedial students? Historically, researchers focused on descriptive

studies often without looking at college outcomes.

Policies are Often Inconsistent True over time and true across schools

Heterogeneity in Remediation Among students Among class offerings

Lack of Data

Why Don’t Raw Comparisons Work?6

Dramatic Differences Between Remedial and Non-Remedial Students

Average ACT Scores Ohio 4-Year Colleges Math: 23.3 for Non-Remedial , 17.4 for Remedial English: 22.8 vs. 15.8 Reading: 23.8 vs. 16.9

Average HS GPA Math: 3.3 vs. 2.5; English: 3.4 vs. 2.8

5-year Dropout Rate 31 Percent for Non-Remedial; 66 Percent for Math

Remediation; 67 Percent for English Remediation

How Inconsistent are the Policies?7

Ohio Non-Selective Colleges

SELECT IVE UNIVERSIT IES

Freq

uenc

y

ACT Math Score10 15 20 25 30 35 36

0

.120192

Diff

ere

nce

in L

og

from

Max

ACT Math Score10 15 20 25 30 35 36

0

100

NON-SELECT IVE AND UB UNIVERSIT IES

Freq

uenc

y

ACT Math Score10 15 20 25 30 35 36

0

.226811

Diff

ere

nce

in L

og

from

Max

ACT Math Score10 15 20 25 30 35 36

0

100




SELECT IVE UNIVERSIT IES

Freq

uenc

y

ACT Math Score10 15 20 25 30 35 36

0

.120192

Diff

ere

nce

in L

og

from

Max

ACT Math Score10 15 20 25 30 35 36

0

100

NON-SELECT IVE AND UB UNIVERSIT IES

Freq

uenc

y

ACT Math Score10 15 20 25 30 35 36

0

.226811

Diff

ere

nce

in L

og

from

Max

ACT Math Score10 15 20 25 30 35 36

0

100



Ohio 2-Year Colleges

COMMUNIT Y COLLEGES

Fre

qu

en

cy

ACT Math Score10 15 20 25 30 35 36

0

.256623

Dif

fere

nce

in

Lo

g f

rom

Ma

x

ACT Math Score10 15 20 25 30 35 36

0

100

How Heterogeneous are the Students?

10

0.0

05.0

1.0

15.0

2.0

25D

ens

ity

0 20 40 60 80National Percentile

kernel = epanechnikov, bandwidth = 3.3164

Kernel density estimate

Heterogeneity Among Students11

Recommended CurriculaBottom Half:

practice and apply estimation and computation using whole numbers and decimals

choose the appropriate method of computation to solve multistep problems (e.g., calculator, mental, or pencil and paper)

calculate length of a lineTop Half:

solve routine arithmetic problems that involve rates, proportions, and percents

do multistep computations with rational numbers calculate area and perimeter of triangles and rectangles;

use geometric formulas

Heterogeneity in Delivery12

Almost half taught by Adjunct (31%) or Graduate Students (17%)

63% taught by non-tenure track faculty42% taught by faculty with doctorates; 31%

have masters; remaining 27% by other degrees (mostly bachelors)

Lack of Data13

National data has low sample sizesSystem-wide data is important

15 percent “transfer down” 16 percent “transfer up” All of these students would look like dropouts

State Administrative Data to the Rescue

14

System-wide data from Florida, Texas, and Ohio have helped develop new strategies

Identify “bubble” students Discontinuities Similar students across campuses

15

Florida Administrative Dataset Sample: All first-time, degree-seeking CC

students who began in Fall 1997, 1998, 1999, or 2000 (over 100,000 records)

Term-by-term transcript data through Spr 2006

All test scores (CPT, SAT, ACT) plus other controls

Outcomes: passing Algebra/English 101, retention, certificate, associate degree, transfer to state 4-year college, credits earned (remedial and non-remedial)

RD Design: The IntuitionBeth and Becky are observationally

similar

Compare the outcomes of Beth & BeckyCompare the outcomes of Beth & Becky

Beth and Becky both take the College Placement Test (CPT)

Beth scores just above the cut-off score

Becky scores just below the cut-off score

Beth to college-level courses Becky to remediation

Crossover: Beth takes

remediation anyway

No Show: Becky never

enrolls in remediation

RD Design: The IntuitionBeth and Becky are observationally

similar

Compare the outcomes of Beth & BeckyCompare the outcomes of Beth & Becky

Beth and Becky both take the College Placement Test (CPT)

Beth scores just above the cut-off score

Becky scores just below the cut-off score

Beth to college-level courses Becky to remediation

Endogenous Sorting:

Beth retests to place out of remediation

Crossover: Beth takes

remediation anyway

No Show: Becky never

enrolls in remediation

18

Endogenous Sorting around the Cutoff

Density of Reading CPT for Institution E

0.0

1.0

2.0

3.0

4

-50 -40 -30 -20 -10 0 10 20 30

Expect to see larger number who barely exceed the cutoff than those who barely failed

discontinuity of the conditional density at the threshold

19

Passing First College-Level Course

(negative)

0.2

.4.6

.8

-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff

Estimated Discontinuity = -0.066(0.008)


0.2

.4



2 yr Degree Completion

3035

4045


Estimated Discontinuity = 1.527(0.447)

Total Credits Earned

0.2

.4.6

.8



Fall-to-Fall Retention

0.2

.4



Transfer to 4 yr

2025

3035

40



Total College-Level Credits Earned

Figure 4: Outcome by Reading CPT Score and Estimated Discontinuity

20

Total Credits Earned(positive)

0.2

.4.6

.8




0.2

.4



2 yr Degree Completion

3035

4045


Estimated Discontinuity = 1.527(0.447)

Total Credits Earned

0.2

.4.6

.8



Fall-to-Fall Retention

0.2

.4



Transfer to 4 yr

2025

3035

40



Total College-Level Credits Earned

Figure 4: Outcome by Reading CPT Score and Estimated Discontinuity

The Florida Remediation StudyConclusions – Overall Sample

21

Being assigned to remediation appears to increase the total number of credits completed for students on the margin of passing out of the requirement…

But it does not increase the completion of college-level credits or eventual degree completion.

Remediation might promote early persistence in college, but it does not necessarily help students on the margin of passing the cutoff to make long-term progress toward a degree

How do these results compare to other studies?

Texas Study

22

State with a single cutoffs and placement exam RD methodology similar to Florida study

Sample: students who took all three placement exams (math, reading, and writing) and passed the writing section

Remediation appears to have little effect on a wide range of educational and labor market outcomes. The estimates are small and statistically insignificant but suggest that students are neither harmed nor greatly benefited by remediation

The Ohio Study23

Nearly 66,000 first-time freshman in Fall 1998 (FT, traditional age, 4yr degree intent) for 6 years

Compares observationally-similar students: one placed into remediation because his nearby college has a stringent policy while the other student does not because his school has a lax policy

Students in remediation had better subsequent outcomes – Reduced the likelihood of dropping out and increases the likelihood of completing a degree

Discouragement effect from certain Majors

Reconciling the Results?24

Single placement exam and cutoff versus autonomy

Different locations of the cutoff (where should it be?)

Different student samples (all versus traditional-age, degree seeking) and institutions (only CCs vs. 2yrs and 4yrs)

Could the effects of remediation differ by type of student?

Does remediation work for students with far less preparation?

25

The FL, OH and TX studies focus on students on the margin of needing remediation -- they do not investigate the effects of remediation on students who are extremely under-prepared (i.e. don’t have an appropriate control group)

The Tennessee Case Remediation at two-years and four-years

Multiple cutoffs and changes in placement policy over time

Can investigate the effects of different types of remediation for students of different abilities

26

Policy Implications and Remaining Questions

Costs of remediation should be given careful consideration in light of the limited benefits.

Explore noncompliance and retesting practices and consider potential consequences

What is the best way to offer remediation? Characteristics of strong remedial programs?

What are the effects of limitations states impose on remedial course-taking (e.g., only at CCs, time limits)?

Early Placement Testing – a preventative measure?

PROF. ERIC BETTINGER STANFORD UNIVERSITY, NBER Postsecondary Remediation: How can we serve the needs...

Documents

Transcript of PROF. ERIC BETTINGER STANFORD UNIVERSITY, NBER Postsecondary Remediation: How can we serve the needs...