Sunny BeckerHuman Resources Research Organization

June 21, 2013

Relationship of High School Exit Exams to Post-High School Outcomes

A Collaborative Effort in California

California Independent Evaluator

2

2000 Present

CAHSEE Independent Evaluator

California Independent Evaluator

3

2000 Present

CAHSEE Independent Evaluator

2002: Census administration to sophomores in Class of 2004

2002

California Independent Evaluator

4

2000 Present

CAHSEE Independent Evaluator

2006: Passing CAHSEE becomes graduation requirement

20062002

California Independent Evaluator

5

2000 Present

CAHSEE Independent Evaluator

2010: Post High School Outcomes (PHO) Study was authorized

PHO Study

2006 20102002

Research Questions

1. What post high school outcomes can be linked to CAHSEE performance?

2. How well and in what ways does CAHSEE performance predict post high school performance?

3. How feasible is a collaborative effort among volunteer LEAs to analyze the relationships between CAHSEE performance and post high school outcomes?

6

Post High School Outcomes (PHO)

7

The Challenge: Student Outcomes

Ideal: Follow up with high school graduates– One small charter school did this

Senior Surveys– Good for all outcomes (postsecondary education, military,

civilian work, other)– Cost effective– Intentions as proxy for actual outcomes– Some LEAs already administered senior surveys that were used

as a basic template for LEAs who did not have them

8

The Challenge: Student Outcomes (cont’d)

Senior surveys (continued)– College enrollment plans, including college characteristics – Career fields – Highest degree– Work full time– Military enlistment by branch of service– and other things.

9

The Challenge: Student Outcomes (cont’d)

National Student Clearinghouse’s Student Tracker (NSC ST)– As of July 2012, more than 3,300 colleges and universities contribute

to the NSC database, approximating 96 percent of all students enrolled in U.S. public and private postsecondary institutions.

– “No record found”• These students may never have enrolled in a postsecondary

institution in the United States, or • The student may be among the 4% of students who enrolled in

one of the institutions for which NSC does not collect data, or • The student may have enrolled outside the U.S., or• The student may have requested that their data not be shared, or• The ST matching algorithm may have failed.

10

The Data We Collected

Student demographics CAHSEE ELA and Mathematics test scores Early Assessment Program (EAP) scores Senior Surveys

– Asked LEAs to administer a Senior Survey with common items to Class of 2011

– Some LEAs already had senior surveys with no individual ID

NSC ST– Some LEAs subscribed on own – We sampled an additional 20,000 students

• Sampled by LEA and graduating class (2007, 2009, 2011)

11

PHO Study: Full Sample & Analytic Sample

12 LEAs 114,000 graduates in Classes of 2006 through 2011

We limited analyses to three classes– Class of 2011: Short-term outcomes in Fall following graduation;

opportunity to collect data for this study (SSV)

– Class of 2009: Mid-range outcomes 2.5 years following graduation

– Class of 2007: Longer-term outcomes 4.5 years following graduation

68,000 graduates 16,000 senior surveys 44,000 Student Tracker records

12

Representativeness of Sample

13

Demographic Group Graduation Year

2007 2009 2011 2011 State

All Students 20,843 100.0% 27,870 100.0% 27,274 100.0% 382,558 100.0%

Females 10,503 50.4% 14,496 52.0% 13,663 50.1% 197,045 51.5%

Males 10,336 49.6% 13,374 48.0% 13,609 49.9% 185,513 48.5%

American Indian or Alaska Native 162 .8% 188 .7% 157 .6% 2,692 0.7%

Asian 4,175 20.0% 4,700 16.9% 4,406 16.2% 39,717 10.4%

Pacific Islander 245 1.2% 328 1.2% 320 1.2% 2,432 3.2%

Filipino 1,100 5.3% 1,835 6.6% 1,704 6.2% 12,104 3.2%

Hispanic or Latino 5,215 25.0% 10,177 36.5% 11,076 40.6% 167,886 43.9%

African American or Black 2,331 11.2% 2,222 8.0% 2,218 8.1% 24,917 6.5%

White, non-Hispanic 7,475 35.9% 8,265 29.7% 7,013 25.7% 124,863 32.6%

Multiple Races 60 .3% 92 .3% 186 .7% 5,311 1.4%

Economically Disadvantaged 6,316 31.5% 10,293 37.1% 13,325 48.9% 219,856 57.5%

English Learner 2,342 11.7% 3,109 12.6% 1,483 8.9% 60,280 15.8%

Reclassified Fluent English 4,190 21.0% 6,786 27.4% 4,374 26.4% N/A N/A

Special Education Students 1,516 7.5% 1,773 6.4% 2,121 7.8% 34,156 8.9%

Findings

Just a few sample findings Many more analyses in report

14

Findings from Early Assessment Program (EAP)

Compared CAHSEE performance to EAP performance EAP has more rigorous standards than CAHSEE ELA : Half of CAHSEE Advanced students were EAP

Exempt Math: One-quarter of CAHSEE Advanced students were

EAP exempt; two-thirds were conditionally exempt

15

Findings from Senior Surveys

Compared CAHSEE performance to future intentions

16

Post High School Plans

17

Post High School Plans by CAHSEE ELA Achievement

18

Planned Education Level by CAHSEE ELA Achievement

19

Immediate Plans by Socioeconomic Status

20

Findings from Student Tracker

Compared CAHSEE performance to actual postsecondary outcomes– We assumed students not found in Student Tracker followed another

path (e.g., military, civilian work)

21

First Enrollment Date by Graduating Class

22

Graduating Class First Enrollment Year

Total None 2006–07 2007–08 2008–09 2009–10 2010–11 2011–12 2007 Count 3,092 384 9,549 793 389 228 135 14,570

% within Class 21.2% 2.6% 65.5% 5.4% 2.7% 1.6% .9% 100.0% 2009 Count 3,150 0 0 495 10,340 654 247 14,886

% within Class 21.2% .0% .0% 3.3% 69.5% 4.4% 1.7% 100.0% 2011 Count 5,271 0 0 0 0 292 9,205 14,768

% within Class 35.7% .0% .0% .0% .0% 2.0% 62.3% 100.0% Total Count 11,513 384 9,549 1,288 10,729 1,174 9,587 44,224

% within Class 26.0% .9% 21.6% 2.9% 24.3% 2.7% 21.7% 100.0%

Enrollment Timing by Graduating Class

23

Graduating Class Timing of Post High School Enrollment

Total No Enrollment

Info Enrolled Immediately

after HS Enrolled Later

2007 Count 3,092 9,933 1,545 14,570 % within Class 21.2% 68.2% 10.6% 100.0%

2009 Count 3,150 10,835 901 14,886 % within Class 21.2% 72.8% 6.1% 100.0%

2011 Count 5,271 9,497* Not 14,768 % within Class 35.7% 64.3% applicable 100.0%

Total Count 11,513 30,265 2,446 44,224 % within Class 26.0% 68.4% 5.5% 100.0%

CAHSEE ELA and Mathematics performance are closely related to postsecondary enrollment rates.

24

An Important Finding

Lessons Learned

LEA recruitment is time-consuming and labor-intensive. Clear specifications of expectations are important to

facilitate full participation by the LEAs. Allowing dedicated time for discussion of the study was

paramount to its success.– Planning Workshop

– Preliminary Results Workshop

Some senior survey items were of limited value and might benefit from revision if a similar study were conducted in the future.

25

Answers to Research Questions

1. What post high school outcomes can be linked to CAHSEE performance?

– Links between CAHSEE performance and postsecondary academic pursuits – Links between CAHSEE performance and future intentions of high school seniors – These intentions show some promise for accurately predicting behavior.

2. How well and in what ways does CAHSEE predict post high school performance?

– Evidence that CAHSEE performance predicts near-term postsecondary academic pursuits

– Weaker evidence that seniors planning to work or join the military may well have done so, based on the absence of evidence that these students pursued higher education

– Robust relationship between the10 levels of CAHSEE achievement constructed for this study to postsecondary enrollment

3. How feasible is a collaborative effort among volunteer LEAs to analyze the relationships between CAHSEE performance and post high school outcomes?

– Very feasible approach to analyzing these sorts of research questions.– Some LEAs now plan to administer senior surveys on a routine basis. – CDE should consider providing a uniform questionnaire for LEA consideration.

26

A Heartfelt Thanks

HumRRO wishes to thank the representatives from the California LEAs who participated in the PHO Study—the individuals who attended workshops, the data analysts who prepared data, and the leaders who granted time and effort to contribute to this study. We appreciate their sharing ideas during the Planning Workshop, their sustaining the effort to provide data—in some cases, implementing new data collection processes such as senior surveys—and their working with us in the Preliminary Results Workshop to interpret and debug findings. Without their commitment and involvement this study would, quite literally, not be possible.

27

http://www.cde.ca.gov/ta/tg/hs/documents/cahsee2012evalrpt.pdf

28

Full Report

Contact Information

Sunny BeckerHumRRO

(443) 944-8226sbecker@humrro.org

Contact Information

Sunny BeckerHumRRO

(443) 944-8226sbecker@humrro.org