Finally. . . Informed Decision Making

Presented by

Stephanie Houston & Laura EvansColton-Redlands-Yucaipa ROP

NACTEI Conference May 14, 2004

Introduction

• What are Regional Occupational Programs/Centers?

• Where is CRY-ROP and who does it serve?

• What threats is CRY-ROP facing and how does it respond?

• What is CRY-ROP’s vision of using data to drive decision making?

Data and Student Academic Performance

• High School Performance

–No Child Left Behind

–API Scores

–SAT 9 Scores

–HSEE passing rates

What are the implications of the NCLB legislation, API, HSEE…etc?

• Emphasis on academic standards and course content

• Emphasis on data analysis

• Expanded attention to teaching strategies that reach all students

• Potential impact on student electives

• Expanded attention to assessment strategies

• Development of intervention and remedial opportunities for under performing students and schools

Prepare. . .

• Look at the NCLB legislation, HSEE & API as opportunities to show how ROP or other career & technical classes reinforce and support student achievement.

• Anticipate change--and prepare for it!

Focus on the positive…

• 2/3rds of the Stanford 9 reading comprehension questions are textual and functional reading items which are predominant in ROP courses

• ROP or other career & technical instructors can show how they support and strengthen academic skills through occupational applications

Connect…

• ROP or other career & technical courses can assist students who do not do well in traditional courses by teaching academic skills in context

• ROP or other career & technical courses may provide alternative graduation requirements in some districts

• ROP or other career & technical should be part of the solution in improving student performance

No Child Left Behind• Accountability utilizing scientifically based

research– Use of rigorous systematic and empirical methods– Adequacy of data to justify the general conclusions– Reliance on methods that provide valid data across

multiple measurements and observations– Use of control groups– Details for replication– Acceptance by a peer-reviewed journal or approved by

a panel of independent experts

Data Collection Methods• Registration forms• Attendance/Completion forms• Certificate request forms• Perkins core indicators• Student surveys

– Exit– Satisfaction– Follow-up– Student Transcript Data Collection Form

• Shared data files/databases

Tools of the trade

• Scantron

• Customizable databases

• Crystal Reports software

• SPSS software

• Partnerships with other schools

• Research/studies

Partnerships for Advancing Student Success (PASS) P-16 Council

Vision- To create a system of education that all students deserve, that our workforce expects, and that a democracy requires

P-16 Council

• Pre-school through post-secondary education• “Systems” thinking to encompass the

interdependent and interacting components of the education system

• System-wide coherence providing a cohesive education from childhood to adulthood

• Legal Memorandums of Understanding in place to ensure appropriate use of data

P-16 Council

• Data elements are shared and analyzed to strengthen program quality and guide decision making

• CRY-ROP is represented on the San Bernardino Consortium Data Oversight Committee

• CRY-ROP is represented on the CAL-PASS Steering Committee/Users Group

UCR CAROCP Study

• Completed by the University of California, Riverside - School Improvement Research Group (SIRG)

• Multi-dimensional research project to study the impact of ROCP courses on 11th and 12th grade students

UCR CAROCP Study Goals

• Documentation of the overall effectiveness of California’s Regional Occupational Centers and Programs by – Comparing ROCP students to a

demographically comparable “control” group– Examining 7 outcome domains

UCR Study Outcome Domains

• High school GPAs

• SAT-9 test performance

• Graduation rates

• School attendance rates

• Job placement

• Job opportunities

• Admissions to post-secondary education

UCR Study Design

• An electronic database to integrate information• Three data sources

– High school transcript records

– Telephone follow-up interviews

– High school satisfaction survey

• 11th & 12th graders from – ROC/Ps

– A control group of students NOT on a UC admission track, not in Special Day Classes

Inland Desert Tech Prep Consortium

• Partnership among 43 high schools and community colleges

• Perkins funded• Data collection automated• 23,399 high school students supported in 2003• 1,171 vocational students transitioned to 3

colleges and took vocational courses in 2003

HORIZONS Study

• Commissioned by San Bernardino County Superintendent of Schools School to Career Horizons Grant

• Conducted on P-16 data files

• Limited data available, study needs to be expanded

• Generalizations only – sample too small

HORIZONS StudyAverage 2000-2002 SAT9 Score by Category

400

450

500

550

600

650

700

750

CRA

URA

UCR

A RA UA

UCR

CR

R

UCA UR

U

UC CA

C

NONE

A

Category

Scor

e

HORIZONS StudyChanges in Average SAT9 Scores - 2000-2002

Must be in Category for All 3 Years

193

32

57

-16

0016

-12

17

-120

-70

-20

30

80

Reading Math Language Science Social Studies

Subject

Ch

ang

e in

Ave

rag

e S

core

CTE

N-CTE

HORIZONS Study2000-2002 High School Student Records Matched to 1998-2002/3

Community College Records

14.2%

19.3%

9.5%8.8%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

CTE&ROP CTE Only ROP Only No CTE or ROP

HORIZONS Study2000-2002 Category Matches To College Files

20.8%

14.9%

5.9% 5.6%4.6% 4.5%

0.0%

8.6%8.7%8.9%

13.6%

18.2%

20.2%18.8%

23.6%

0.0%

5.0%

10.0%

15.0%

20.0%

25.0%

UR

A

UC

RA

UR

UA

RA

UC

A

UC

R U R

UC

CR

C

Non

e A

CA

HORIZONS Study

Award Type Number of Awards Category NameAssociate of Arts (A.A.) degree 1 UCRACertificate requiring 6 to fewer than 18 units 13 UC Certificate requiring 6 to fewer than 18 units 9 UCertificate requiring 6 to fewer than 18 units 9 UCR Certificate requiring 6 to fewer than 18 units 5 UCRACertificate requiring 6 to fewer than 18 units 2 UACertificate requiring 6 to fewer than 18 units 1 CCertificate requiring 18 to fewer than 30 semester units 2 UCRCertificate requiring 18 to fewer than 30 semester units 1 UCAssociate of Science (A.S.) degree 5 UCRAAssociate of Science (A.S.) degree 1 UA

UCR CAROCP Study

• Common elements that distinguish ROC/P courses involved in the study from traditional 9-12 educational programs:– Yearly labor market analysis– Business advisory committee– Inclusion of academic standards necessary for

success in the career area taught– Student involvement in real-life practice of the

career skills

UCR CAROCP Study• 2002

– 7 ROC/Ps– 5 Career Programs: Auto, Business, Child

Development, Health & Marketing– 1545 ROC/P Students

• 2004– 22 ROC/Ps– 8 Career Programs: Auto, Business, Child

Development, Environment Tech, Health, Public Safety, Marketing & Technology

– 2282 ROC/P Students

UCR CAROCP Study Instrumentation

• Four original data collection and management instruments were developed– Student Transcript Data Collection Form– Telephone Follow-up Survey– High School Student Satisfaction Survey– Integrative relational database capable of

merging data form all sources, built in Microsoft Access

UCR CAROCP Study Major Findings

• Raised/comparable high school GPAs

• Improved class attendance

• Better wages

• More likely to be promoted on the job

• Good rates of further education

• Preference for ROC/P over high school

• Low appreciation of value of education

• Resistance to academic classes

UCR CAROCP Study

• Better Wages– 2002: Average salary of $7-$10 per hour, with

ROC/P students earning $1.44 more per hour

– 2004: Average salary of $7-$13 per hour, with ROC/P students earning $2.40 more per hour

UCR CAROCP Study• Raised high school GPAs

– 2002: ROC/P students entered training programs with academic records below Control Group. By the completion of 12th grade, ROC/P students raised their GPAs by more than three-tenths of a point – a better rate than the Control Group

– 2004: ROC/P students entered training programs with academic records below Control Group. By the completion of 12th grade, ROC/P students raised their GPAs by about three-tenths of a point – same pace as the Control Group

UCR CAROCP Study

Class Attendance

8.28

3.58

11.62

9.19

4.92

8.54

4.21

9.04

0

2

4

6

8

10

12

14

10th Grade 12th Grade 10th Grade 12th Grade

2002 2004

Day

s A

bse

nt/

Yea

r

ROCP Group

Control Group

UCR CAROCP Study

• On the Job Promotions– 2002: ROC/P students 28%, Control Group 21%– 2004: ROC/P students 25%, Control Group 23%

• Raises– 2004: ROC/P students 25%, Control Group 19%

UCR CAROCP Study

• Further Education-Comparable– 2002: ROC/P students 54% transition rate;

Control Group 53% transition rate– 2004: ROC/P students 51% transition rate;

Control Group 52% transition rate

UCR CAROCP Study

• Value Statements– ROC/P students “Very much” like career education

course work

– “Only a few” high school classes are relevant for ROC/P students in terms of getting jobs or preparing them for adult life

– ROC/P students report low levels of appreciation for Math, Science and Social Studies

– ROC/P students report matching enthusiasm for English Language Arts, Music, Art , Drama and ROC/P course work

Conclusions

• California’s ROC/Ps have a positive impact on students

• Positive impacts are evidenced in academic achievement, the labor market and continuing education

• Full implementation of an accountability system for career and technical education can document the social and economic payoff of programs nationwide

Sources & Links

CAROCP– http://www.carocp.org/

CAL-PASS – www.cal-pass.org

P-16– www.edtrust.org– www.csulb.edu/~acaproj/partnerships.html

NCLB– www.nochildleftbehind.gov

Questions?