The Impact of Non-Cognitive Variable

Assessment in Academic Probation:

A Case Study in Improving Retention

Holly Schuck, Assistant DirectorÁlvaro Plachejo, Coordinator

Christopher Clendenen, Graduate AssistantFirst Year & Transition Programs

St. Cloud State University

Agenda

Academic Collegiate Excellence (ACE) Program

Non-Cognitive Variables (NCV)

Applying NCVs

Probation & Suspension

Findings

Conclusions

Academic Collegiate Excellence ACE Program

Provisional acceptance program for traditional 1st year students

Fluctuated between 75 – 650 students Over-represented by males (57%) and students of color

(29%)

Admission Process

Evolution

Admissions: Recruitment Focus

Consistent Issues with Retention (F-S, 1-2)

First Year & Transition Programs: Retention Focus

ACE Requirements C- or above in 4 credits of transition coursework

COLL 150: Discovering the College Experience

COLL 110: Reading & Study Strategies

GPA of 1.75/4.0 or higher

ACE students who do not meet these standards are suspended and must appeal to return for 2nd semester

Standards Enforcement - SAP

• Students who do not meet the minimum 1.75 GPA requirements during first term are placed on academic warning for the University.

• This policy applies to all undergraduate students who have cumulative credits between 0-44 and is a completely separate academic issue from the ACE program.

• Students in the ACE Program may be ACE suspended and St. Cloud State University academic warning.

ACE Suspension Processes

Pre-2012 Begin NCVCurrent

2012-2013

Suspension Email

& “Yes”

Begin use of NCV in

admission and appeal

processes, via interview

Expand on NCV

assessment and

predictions of success/strugg

le

Crisis:

• Need to Improve Retention Rates for ACE (provisionally admitted)

students

• 2012 Level: Less than 60% persisted from first to third semester

Our solution?

Non-Cognitive Variables

Variables relating to adjustment, motivation, and perceptions

“The goal of using noncognitive variables is not to substitute this approach for the cognitive focus more

commonly employed in assessments,

but to add to the range of attributes that we consider in making the many judgments

required of us all.” – Sedlacek, 2004

Non-Cognitive Variables

Positive Self-ConceptDemonstrates confidence, strength of character, determination, and independence

Realistic Self-AppraisalRecognizes and accepts any strengths and deficiencies, especially academic, and works at self-development; can reflect on previous behavior to make modifications

Successfully handling the system (coping with racism)Exhibits a realistic view of the system based on personal experience of racism; takes an assertive approach to dealing with existing wrongs, but is not hostile to society or a “cop-out”; understands their role in a multicultural world

Preference for long-term goalsAble to respond to deferred gratification, plans ahead and sets goals

Non-Cognitive Variables Availability of strong support person

Seeks and takes advantage of a strong support network or has someone to turn to in a crisis or for encouragement

Leadership experienceDemonstrates strong leadership in any area of his or her background

Community involvementParticipates and is involved in his or her community and identifies with a cultural, racial or geographical group

Knowledge acquired in a fieldAcquires knowledge or demonstrates learning in a sustained or culturally related way in any field

ACE Suspension/Appeal Process

Individual Interviews Professional Staff

Graduate Students

Common Questions, Rubric, Process

1 hour interviews, decision on the spot

Suspension Appeal Process

Introduction of NCV into appeal process

Through NCV process, we help students think more independently

Motivational Interviewing

- Cognitive Dissonance

- Ambivalence

Staff Training

Notice their non-verbal reactions as well as verbal responses.

Use person-centered approach; no judgment or criticism of the student for their situation.

Every student has different challenges and stressors.

Those lacking motivation just may not be ready for college at this time.

Student preparation for the interview will vary.• Many students are nervous, many are embarrassed.

• Some students get tearful, others are apathetic, others angry.

Incorporating Motivational Interviewing

Person-centered counseling style for addressing the common problem of ambivalence about change. A semester of failure can

create ambivalence.

Engaging – involving students in talking about issues, concerns & hopes, and to establish a trusting relationship.

Focusing – focusing conversation on habits or patterns that students want to change.

Evoking - eliciting their motivation for change by increasing students' sense of the importance of change, their confidence about change, and their readiness to change.

Planning - developing practical steps to implement the changes they desire.

Our Approach

Person-Centered counseling approaches and skills of motivational interviewing:

Open-ended questions Exploring ambivalence Active listening Express empathy Reflect content and feeling Immediacy Caring Confrontation Student-Centered

Assessing the Interview• Is the student taking the process seriously?

• Does the student identify specific ways they intend to, or have made changes to improve success?

• Does the student take responsibility for their lack of success and can they identify what contributed to their academic struggles?

• Does the student seem to have the academic skills and preparedness needed to advance academically?

• Is the student dealing with any extenuating circumstances? If so, what options/support do they have on campus/in comunity?

• Are mental health issues present that might need referral?

• What kind of support does the student have?

• Gauge the student’s level of motivation.

Appeal InterviewsWhat did you learn about yourself based

on your experience last semester, and how did that bring you here today?

What kinds of challenges or struggles have you faced?

What can you count as some successes from last semester? How might you

build upon these successes?

What types of involvement or connections have you made outside of

class?

How are you going to motivate yourself to be responsible for your academic

success?

What will you do differently or improve for

next semester?

What specific plans and goals will you have for next semester?

What will stay the same from first semester, and what may look

differently?

About 50 percent of university students typically leave before receiving a

degree. What makes you one of the 50 percent that will finish?

What support do you have to help you academically and personally?

On-campus? Off-campus? Resources?

What did you learn about yourself during this experience?

What motivates you to want to earn a college degree?

Decisions

You may recommend 3 courses of action based on your assessment of the student:

1. Scores below 15: Transition out of the university, Suspension

2. Scores 15-17: Referral to the Community College Connection Partnership Program

3. Scores 17-24: Grant the appeal and have the student placed on ACE Probation and continue with enrollment at St. Cloud State and the ACE program.

Numbers

Fall 2012

• 501 students

• 115 suspensions after first term (22%)

• 89 appeals granted

• 75 registered for third term (84%)

Fall 2013

• 400 students

• 90 suspensions after first term (22%)

• 67 appeal appointments

• 60 appeals granted

• 49 registered for third term (82%)

Fall 2014

• 276 students

• 68 suspensions after first term (25%)

• 56 appeal appointments

• 47 granted appeals

• 41 registered for third semester (87%)

Second Chance Success Program

Retake COLL 150 and/or COLL 110 (as needed)

Enroll in CEEP 101 (3 credits, liberal ed)

Regular meetings with our staff

Complete Mapworks Surveys

Attend 2 campus workshops/events of their choice

Alex

• Alex had an NCV admission score of 16/24

• She failed all first term courses, except one

• Suspension appeal interview score of 21/24

• Second semester, she: • met regularly with advisor, joined student organizations,

led service projects on campus, became an Orientation Leader

• Earned “A” in CEEP 101

• Ended with GPA of 3.5/4.0, returned for third semester

Mary

• Mary had a very high NCV score in admission (22/24)

• She withdrew from all courses, except one (earned A)• 4.0/4.0 GPA, but completion rate was 25%

• Appeal had a low NCV score (14), but staff was swayed by her story (extenuating circumstances)

• Spring stopped coming to probation meetings, didn’t submit coursework, stopped using support services

• Ended 2nd term with very low GPA, “D” in CEEP 101

• Not allowed to return for third semester

CEEP 101 and GPACorrelations

 Class

Grade

Spring Term

GPA

Grade Pearson Correlation 1 .858**

Sig. (2-tailed) 

.000

N 77 77

Spring Term GPA Pearson Correlation .858** 1

Sig. (2-tailed) .000 

N 77 78

**. Correlation is significant at the 0.01 level (2-tailed).

Findings• Grade in CEEP 101 probation course is a strong predictor of overall

term GPA

• Data shows that students who have high confidence in their abilities but low performance are less likely to persist than students who acknowledge their academic challenges. This is very visible in our students with ACT scores above 24. If an ACE student with an ACT score above a 24 was put on probation they were not retained into Fall.

• Those admitted to the program before the NCV admission process were over-represented on probation; those admitted with NCV scores were under represented on probation. Once on probation, the two populations had no statistically different outcomes from one another (used same common appeal process).

• Surprisingly, for all years (2012-2015) ACE students have been looked at there has been very little statistical predictor for outcomes based on:

Race/Ethnicity, Pell Eligibility (low SES), Parental Educational Attainment

• These numbers were found by performing Chi-square analysis on each above variable compared with students on probation. Each time analysis was performed no statistically significant differences were found. The only exception to this trend is that African American students have been statistically less likely to go on ACE probation than other students.

Findings

CEEP 101 helps us predict student success in terms of GPA and fall to fall retention.

NCVs help us predict student success in terms of GPA, fall to fall retention, and likelihood to become suspended.

NCVs and GPAACE 2012-2013 First Semester Admit vs Second Semester Admit

Factor Sig. (2-tailed) Mean Difference First Semester Admit Second Semester Admit

ACT_COMP 0.00 -0.86 18.89 19.75

Commitment to the Institution 0.02 0.25 6.27 6.02

MW FACTOR Self-Assessment: Analytical Skills 0.00 -0.33 4.44 4.76

MW FACTOR Self-Assessment: Self-Discipline 0.05 0.17 5.75 5.58

MW FACTOR Self-Assessment: Time Management 0.05 0.22 5.47 5.25

MW FACTOR Basic Academic Behaviors 0.00 0.35 6.00 5.65

MW FACTOR Advanced Academic Behaviors 0.01 0.25 5.22 4.98

MW FACTOR Academic Integration 0.02 0.23 5.74 5.51

MW FACTOR Social Integration 0.01 0.32 5.82 5.50

MW FACTOR Test Anxiety (Module) 0.03 -0.36 3.84 4.19

FALL_CCR 0.00 8.69 82.33 73.64

FALL_EARNED 0.00 1.27 11.34 10.06

FALL_GPA 0.00 0.41 2.50 2.09

HS_PCT 0.02 2.66 34.57 31.91

SECOND_FALL_GPA 0.01 0.28 2.43 2.15

SPRING_GPA 0.01 0.24 2.40 2.16

NCVs and RetentionIndependent Samples Test

t df Sig. (2-tailed) (p-value) Mean Difference

Self-ReportedCommunication

2.027065 74 0.046258 0.534495 Not RetainedRetained

5.0494.514

Self-ReportedAnalytical

2.487892 75 0.015071 0.654762 Not RetainedRetained

5.2264.571

HomesicknessSeparation

2.656584 21 0.014761 1.555556 Not RetainedRetained

4.52.94

Self-ReportedDistressed

3.098642 21 0.00544 1.914683 Not RetainedRetained

5.974.06

Test Anxiety 3.504127 75 0.000776 1.268519 Not RetainedRetained

5.03.73

NCVs and SuspensionNCV_2013 * Probation Crosstabulation

 Probation

Total.0 1.0

NCV_2013 .0 Count 199a 29b 228

Expected Count 204.7 23.3 228.0

1.0 Count 126a 8b 134

Expected Count 120.3 13.7 134.0

Total Count 325 37 362

Expected Count 325.0 37.0 362.0

 

 Value df

Asymp. Sig. (2-

sided)

Exact Sig. (2-

sided)

Pearson Chi-Square 4.189a 1 .041  

Likelihood Ratio 4.506 1 .034  

Fisher's Exact Test       .048

Linear-by-Linear Association 4.178 1 .041  

N of Valid Cases 362      

Recent Changes

• Requiring written responses and interviews

• As a comparison from term to term

• Provides opportunity for student to focus their thoughts

• Proxy for motivation

• Well-prepared vs. Skype example

Conclusions

NCVs assess soft skills that are fundamental to student success.

At SCSU: - High NCV scores predict high GPAs

- High NCV scores increase the likelihood of retaining the student

- High NCV scores decrease the likelihood of academic suspension

- High NCV scores increase likelihood of successful third semester GPA

NCVs can help us tailor our programs and allocation of time and resources to better serve students in need.

Thank you!

Holly SchuckAssistant Director

HMSchuck@stcloudstate.edu(320) 308-6296

Evans, N. J., Forney, D. S., Guido, F. M., Patton, L. D., & Renn, K. A. (2010). Student development in college: Theory, research, and practice. San Francisco, CA:

Jossey-Bass

Miller, W. R. & Rollnick, S. (2002). Motivational interviewing: Preparing people for change. New York, NY: Guilford Press

Sedlacek, W. E. (2004). Beyond the big test: Noncognitive assessment in higher education. San Francisco, CA: Jossey-Bass