B11 ACT — Cognitive and Non-Cognitive Factors that Determine Student College and Career Readiness

© 2012 by ACT, Inc. All rights reserved.

ACT — Cognitive and ACT — Cognitive and Non-Cognitive Factors Non-Cognitive Factors

that Determine Student that Determine Student College and Career College and Career

ReadinessReadinessApril Hansen, Postsecondary Director

George Schlott, Senior Consultant for Program Solutions

ACT, Inc. – Midwest Region-Lincolnshire IL

What is College and Career Readiness?

Ensuring that all students, graduating from high school, have acquired the skills in English,

mathematics, reading, and science that they need to be ready for entry-level college courses without

remediation

A Question for AllA Question for All

Planning for School

Improvement

Improving Course Rigor

Measuring Student Progress TowardCollege and Career Readiness

Research

EXPLORE

8th & 9th grade

curriculum-based

educational and career planning program

PLAN

10th grade curriculum-

based educational and career planning program

The ACT

11th & 12th grade

curriculum-based

measurement for

learning outcomes

ENGAGE

Middle and High School assessment

that measures factors of academic success

QualityCore

Research-driven

solutions for strengthening curriculum & instruction

CoreWork Diagnostics

Online service to diagnose and

improve content and practice areas

ACT College Readiness Standards

ACT College Readiness Benchmarks ACT Core Course

Standards

WorkKeys

Job skill assessments and portable credentials necessary for career

pathways

College & Career Readiness Information SystemCollege & Career Readiness Information System

Core Practice Audit

Online service to diagnose and

improve content and practice areas

ACT College & Career Readiness SystemACT College & Career Readiness System

Through collaborative research with postsecondary institutions nationwide, ACT has established the following college readiness

benchmarks*:

* Minimum score needed on an ACT subject-area test to indicate a 50% chance of obtaining a B or higher or a 75% chance of obtaining a C or higher in the corresponding first-year credit-bearing

college course.

Test College Course ACT

English English Composition 18

Math College Algebra 22

Reading Social Science 21

Science Biology 24

Test College Course EXPLORE 8 EXPLORE 9 PLAN 10 ACT COMPASS

English English Composition

Math College Algebra

Reading

Social Science

Science Biology

13

17

15

20

14

18

16

20

15

19

17

21

18

22

21

24

69

65

88

NA

ACT College Readiness BenchmarksACT College Readiness Benchmarks

Students who are college/career ready when they leave high school have a significantly higher likelihood of: – Enrolling in a postsecondary program,– Enrolling in credit bearing courses without the

need for remediation,– Succeeding in entry level postsecondary

course work,– Persisting in their postsecondary education, – Completing a postsecondary degree or

training program, and – Entering the job market with significantly

higher lifetime earning potential.Regardless of ethnicity and SES

Value of College ReadinessValue of College Readiness

© 2012 by ACT, Inc. All rights reserved. 6

• Many students are not prepared to meet the hurdles they face throughout the academic pipeline, and they don’t persist and succeed.

NCES 2010

• What can we do to improve student success and persistence?

Enter 9th grade

HS Graduat

e

Enter College

College Graduate

(Bachelor’s)

~100%

96.9% 74.9% 52.5% 29.0%

Leaky Educational Pipeline

Leaky Educational PipelineLeaky Educational Pipeline

Pop Quiz

What’s the number of American high school students who drop out of school, every day, bored, frustrated, or so far behind that they’ve given up?

6,000

Leaky Educational PipelineLeaky Educational Pipeline

Two-year schools -- 54 % Four-year private colleges -- 73%Four-year public -- 71% *

*Wes Habley, ACT, January 2009

By 2020: Cost of the average private--$60-70,000 per year Cost of the average public--$17,500-27,500 per year

In 2008:Out of approx. 1.8 million first year college students, 450,000 did not return to

the college or university where they began their college career.* That’s 25%.*Bryan Matthews, Retention Matters, Inside Higher Ed, Nov. 2, 2009

Attrition Costs MoneyAttrition Costs Money

If you have an incoming class of 1000, and lose even 10% of the class (100 students), that’s

100 x $30,000 = $3 million $3 million x 4 years = $12 million

Retaining students allows you to maintain revenue streams while keeping tuition increases down.

It’s the low hanging fruit.

Do the Math!Do the Math!

evaluate students’ personal, behavioral, and academic skills critical to high school and college achievement

determine their levels of academic risk

apply specific interventions to help them persist in their studies and achieve academic success

identify student strengths and areas for improvement in student motivation, social engagement, and self-regulation

predict college retention for each incoming freshman

What if you had an assessment that would:What if you had an assessment that would:

Research is the Foundation

1. The strongest predictors of college persistence and degree completion are: prior academic achievement and course selection (rigorous high school classes).

2. Prior academic achievement and cognitive ability surpass all other factors in their influence on student performance.

3. Non-academic factors can influence academic performance, retention and persistence, but cannot substitute for it.

Grades 6-9Grades 10-12

College Teacher

You Do!You Do!


• Create a comprehensive assessment system that predicts success in education and work

• Tailor assessments for critical transition points and context

• Measure characteristics amenable to change

• Help educators connect students’ needs to interventions

The ChallengesThe Challenges


Achievement

GradesCollege Work6 7 8 9 10 11 12

EXPLORE ® PLAN ® ACT ® COMPASS ®

ENGAGE and ENGAGE Teacher EditionENGAGEBehavior

WorkKeys ®

Talent

ACT Interest Inventory & Map of College MajorsCareer Fit

ACT’s Academic Achievement, Behavior & ACT’s Academic Achievement, Behavior & Career Planning SolutionsCareer Planning Solutions


• Students’ personal characteristics and psychosocial development influence their ability to stay in school and be successful

• ACT research shows that psychosocial/behavioral development can be structured into 3 broad domains:– Motivation – Social Engagement – Self-Regulation

• ACT’s academic behavior assessments measure these domains

ACT’s Academic Behavior AssessmentsACT’s Academic Behavior Assessments


Student Grade Levels

6 7 8 9 10 11 12

ENGAGEGrades 6-9

ENGAGE Grades 10-12

ENGAGECollege

ENGAGE Teacher Ed. Grades 6-9

ENGAGE TeacherEd. Grades 10-12

College

Student reported(surveys)

Teacher reported(teacher ratings)



• Surveys (Student reported)– ENGAGE Grades 6-9 (typically in 6th & 9th

grade)– ENGAGE Grades 10-12 (typically in 11th grade)– ENGAGE College (typically early in 1st

semester)

• Behavioral Measures (Teacher reported)– ENGAGE Teacher Edition for Grades 6-9 and

10-12 (completed by teachers 2-3x per year for each student)

COMING SOON!



Domain ENGAGE Grades 6-9ENGAGE Grades 10-12 & College

Motivation (Getting work done)

Academic Discipline Commitment to School Optimism

Academic Discipline Commitment to College Goal Striving General Determination Study Skills Communication Skills

Social Engagement(Getting along)

Family Attitude toward Education

Family Involvement Relationships w/ School

Personnel School Safety Climate

Social Activity Social Connection

Self-Regulation(Keeping your cool)

Managing Feelings Thinking Before Acting Orderly Conduct

Academic Self-Confidence Steadiness



• Measures students’ perceptions of themselves, their families’ commitment to education, school-related factors, and important behavioral data

• Developed to predict academic success and high school graduation

• Low stakes, self-report instrument, 4th-grade reading level

• 10 scales• Online administration

ENGAGE: Grades 6-9ENGAGE: Grades 6-9


• Measures students’ psychosocial attributes, determines level of risk, and helps identify interventions to promote successful transition to postsecondary studies

• Developed to predict academic success and retention in the first year of college

• Low stakes, self-report instrument, 6th-grade reading level

• 10 scales• Online administration

ENGAGE 10-12 & CollegeENGAGE 10-12 & College


• Academic Success and Graduation/Retention Indices

• Student profiles can be used to:– Identify relative strengths and needs– Inform student advising – Match students’ needs to interventions

• Aggregate reports • Roster reports• All reports available online 24/7

ENGAGE: ReportsENGAGE: Reports


Profile of scores

Success Indices(only on Advisor Report)

Interpretive feedback, sorted from strengths to needs

ENGAGE: Sample Advisor ReportENGAGE: Sample Advisor Report


ENGAGE: Sample Profile ReportENGAGE: Sample Profile Report


ENGAGE: Sample Intepretative ReportsENGAGE: Sample Intepretative Reports


ENGAGE: School/Institution Aggregate ReportENGAGE: School/Institution Aggregate Report


ENGAGE: Roster ReportENGAGE: Roster Report


• Grounded in research– Developed based on meta-analysis of predictors of

academic performance and persistence (Robbins et al., 2004)

– Validated across many studies (e.g., Allen et al., 2010; Casillas et al., in press; Robbins et al., 2006, 2008)

• Helps educators to consider the whole student– Provides profile of relative strengths and needs

• Practical and flexible– Administer online at any time– Reports available online 24/7– Identify areas of challenge for students– Create a crosswalk of ENGAGE scales to existing

services

Why ENGAGE?Why ENGAGE?


Percentage of students accurately identified

as having a 9th-grade GPA <2.0

Selection Method Hit Rate

Random 24%

EXPLORE Composite only

69%

ENGAGE Grades 6-9 only

80%

EXPLORE & ENGAGE 83%


Average Early High School GPA, by EXPLORE

and ENGAGE Grades 6-9 Scores

1.771.99

2.89

1.96

3.34

2.41

3.13

3.63

2.62

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

Bottom 25% Middle 50% Top 25%

EXPLORE Composite Score

Av

era

ge

Ea

rly

HS

GP

A

Low ENGAGE Moderate ENGAGE High ENGAGE


Percentages of Students Accurately Identified as Being at

High Risk for Dropping Out

Selection MethodHit

Rate

Random 17%

EXPLORE Composite only 37%

ENGAGE Grades 6-9 Grad. Index only

42%

EXPLORE & ENGAGE Grad. Index

52%


Persistence Rates, by EXPLORE and ENGAGE Grades 6-9 Graduation Index Levels

48

62

72

89

9692

9698

69

0

20

40

60

80

100

Bottom 25% Middle 50% Top 25%

EXPLORE Composite Score

Pe

rsis

ten

ce

Ra

te

Low Graduation Index Moderate Graduation Index High Graduation Index


Use of ENGAGE College to Identify Students Academically at Risk

Note. Students scoring in the bottom 5% of each of these populations were flagged.

Accuracy of Identification

Selection Method

Drop OutAcademic Difficulty

Random 10% 20%

ENGAGE Success Indices

24% 46%


• Designed to assess and monitor students’ development

• Students are rated by teachers who know them well

• Requires each student to be rated by at least one teacher

• Used to 2-3 times per year• Scales are “anchored” with behavioral

statements to enhance accuracy and reliability

• Same dimensions for Grades 6-9 and 10-12, but rating scales are made up of different items.

ENGAGE: TeacherENGAGE: Teacher


Domain Scales

Motivation

(Getting work done)

Initiative Planning & Organizing Sustained Effort Performance

Social Engagement

(Getting along)

Communication Working with Others

Self-Regulation

(Keeping your cool)

Managing Feelings Orderly Conduct

ENGAGE: TeacherENGAGE: Teacher


Combining Assessment and Interventions

to Promote Student Success

ENGAGE

ENGAGETeacher Edition

Support Resource

s &

Interventions

Reduced Risk

Continued Evaluation & Feedback


Example Crosswalk between Interventions and

Areas of Need Interventions

Areas of Need

Goal Setting& Time

Management

Mentoring

Cross-Age Tutoring

Community Service

Group Work &

Communication Skills

Self-Confidenc

e Workshop

How to Follow Rules

Motivation x x x x Social

Engagement

x x x x

Self-Regulation x x x


Behavioral Education Solutions

• Grade 6-12 curriculum– Units designed to help students build

behavioral skills– Based on carefully developed Behavioral

Learning Objectives– Fully developed and structured lesson plans

and activities

• ENGAGE Tool Shop (Postsecondary) – Several tools available for each ENGAGE scales

to address students’ needs– Can be completed by students on their own or

as part of FYE courses or other group settings

COMING SOON!


ENGAGE Tool Shop (College)


Intervention Research

• University of North Texas– Large university (36,000 students)– Administers ENGAGE to all incoming 1st-year

students– Identify at-risk students and follow up with

immediate individualized intervention• Northern Arizona University

– 4-year public institution serving 13,000 students

– Use ENGAGE to guide intrusive interventions with those students who need the most help


UNT Face-to-Face Meeting

• Advisor or student affairs staff meet with student during first 5-6 weeks of semester

• Present ENGAGE results by focusing first on strengths, then needs

• Focus resources to aid student in: – building on one strength and addressing one

area for improvement– using crosswalk of ENGAGE to campus

resources


Sample Crosswalk of Resources

SRI Scale Definition ResourcesAcademic Discipline

Effort put into school work and the degree to which students see themselves as hardworking and conscientious.

• Learning Center• Office of Exploring Majors• UNT Math Tutor Lab• UNT Student Writing Lab

Social Connection Feelings of connection and involvement with school and community.

• Off-Campus Student Services

• Transfer Center• Residence Life• Recreation Center

Academic Self-Confidence

Belief in ability to perform well in school.

• Learning Center• Counseling and Testing• Office of Exploring Majors


www.act.org/engage/index.html


www.act.org/engage/index.html

For each version of ENGAGE, website contains:– Features and Benefits– Testing Timeline– Program Materials– Research– Results and Reports

B11 ACT — Cognitive and Non-Cognitive Factors that Determine Student College and Career Readiness

Education

Transcript of B11 ACT — Cognitive and Non-Cognitive Factors that Determine Student College and Career Readiness