The Impact of Non-Cognitive Variable
Assessment in Academic Probation:
A Case Study in Improving Retention
Holly Schuck, Assistant DirectorÁlvaro Plachejo, Coordinator
Christopher Clendenen, Graduate AssistantFirst Year & Transition Programs
St. Cloud State University
Agenda
Academic Collegiate Excellence (ACE) Program
Non-Cognitive Variables (NCV)
Applying NCVs
Probation & Suspension
Findings
Conclusions
Academic Collegiate Excellence ACE Program
Provisional acceptance program for traditional 1st year students
Fluctuated between 75 – 650 students Over-represented by males (57%) and students of color
(29%)
Admission Process
Evolution
Admissions: Recruitment Focus
Consistent Issues with Retention (F-S, 1-2)
First Year & Transition Programs: Retention Focus
ACE Requirements C- or above in 4 credits of transition coursework
COLL 150: Discovering the College Experience
COLL 110: Reading & Study Strategies
GPA of 1.75/4.0 or higher
ACE students who do not meet these standards are suspended and must appeal to return for 2nd semester
Standards Enforcement - SAP
• Students who do not meet the minimum 1.75 GPA requirements during first term are placed on academic warning for the University.
• This policy applies to all undergraduate students who have cumulative credits between 0-44 and is a completely separate academic issue from the ACE program.
• Students in the ACE Program may be ACE suspended and St. Cloud State University academic warning.
ACE Suspension Processes
Pre-2012 Begin NCVCurrent
2012-2013
Suspension Email
& “Yes”
Begin use of NCV in
admission and appeal
processes, via interview
Expand on NCV
assessment and
predictions of success/strugg
le
Crisis:
• Need to Improve Retention Rates for ACE (provisionally admitted)
students
• 2012 Level: Less than 60% persisted from first to third semester
Our solution?
Non-Cognitive Variables
Variables relating to adjustment, motivation, and perceptions
“The goal of using noncognitive variables is not to substitute this approach for the cognitive focus more
commonly employed in assessments,
but to add to the range of attributes that we consider in making the many judgments
required of us all.” – Sedlacek, 2004
Non-Cognitive Variables
Positive Self-ConceptDemonstrates confidence, strength of character, determination, and independence
Realistic Self-AppraisalRecognizes and accepts any strengths and deficiencies, especially academic, and works at self-development; can reflect on previous behavior to make modifications
Successfully handling the system (coping with racism)Exhibits a realistic view of the system based on personal experience of racism; takes an assertive approach to dealing with existing wrongs, but is not hostile to society or a “cop-out”; understands their role in a multicultural world
Preference for long-term goalsAble to respond to deferred gratification, plans ahead and sets goals
Non-Cognitive Variables Availability of strong support person
Seeks and takes advantage of a strong support network or has someone to turn to in a crisis or for encouragement
Leadership experienceDemonstrates strong leadership in any area of his or her background
Community involvementParticipates and is involved in his or her community and identifies with a cultural, racial or geographical group
Knowledge acquired in a fieldAcquires knowledge or demonstrates learning in a sustained or culturally related way in any field
ACE Suspension/Appeal Process
Individual Interviews Professional Staff
Graduate Students
Common Questions, Rubric, Process
1 hour interviews, decision on the spot
Suspension Appeal Process
Introduction of NCV into appeal process
Through NCV process, we help students think more independently
Motivational Interviewing
- Cognitive Dissonance
- Ambivalence
Staff Training
Notice their non-verbal reactions as well as verbal responses.
Use person-centered approach; no judgment or criticism of the student for their situation.
Every student has different challenges and stressors.
Those lacking motivation just may not be ready for college at this time.
Student preparation for the interview will vary.• Many students are nervous, many are embarrassed.
• Some students get tearful, others are apathetic, others angry.
Incorporating Motivational Interviewing
Person-centered counseling style for addressing the common problem of ambivalence about change. A semester of failure can
create ambivalence.
Engaging – involving students in talking about issues, concerns & hopes, and to establish a trusting relationship.
Focusing – focusing conversation on habits or patterns that students want to change.
Evoking - eliciting their motivation for change by increasing students' sense of the importance of change, their confidence about change, and their readiness to change.
Planning - developing practical steps to implement the changes they desire.
Our Approach
Person-Centered counseling approaches and skills of motivational interviewing:
Open-ended questions Exploring ambivalence Active listening Express empathy Reflect content and feeling Immediacy Caring Confrontation Student-Centered
Assessing the Interview• Is the student taking the process seriously?
• Does the student identify specific ways they intend to, or have made changes to improve success?
• Does the student take responsibility for their lack of success and can they identify what contributed to their academic struggles?
• Does the student seem to have the academic skills and preparedness needed to advance academically?
• Is the student dealing with any extenuating circumstances? If so, what options/support do they have on campus/in comunity?
• Are mental health issues present that might need referral?
• What kind of support does the student have?
• Gauge the student’s level of motivation.
Appeal InterviewsWhat did you learn about yourself based
on your experience last semester, and how did that bring you here today?
What kinds of challenges or struggles have you faced?
What can you count as some successes from last semester? How might you
build upon these successes?
What types of involvement or connections have you made outside of
class?
How are you going to motivate yourself to be responsible for your academic
success?
What will you do differently or improve for
next semester?
What specific plans and goals will you have for next semester?
What will stay the same from first semester, and what may look
differently?
About 50 percent of university students typically leave before receiving a
degree. What makes you one of the 50 percent that will finish?
What support do you have to help you academically and personally?
On-campus? Off-campus? Resources?
What did you learn about yourself during this experience?
What motivates you to want to earn a college degree?
Decisions
You may recommend 3 courses of action based on your assessment of the student:
1. Scores below 15: Transition out of the university, Suspension
2. Scores 15-17: Referral to the Community College Connection Partnership Program
3. Scores 17-24: Grant the appeal and have the student placed on ACE Probation and continue with enrollment at St. Cloud State and the ACE program.
Numbers
Fall 2012
• 501 students
• 115 suspensions after first term (22%)
• 89 appeals granted
• 75 registered for third term (84%)
Fall 2013
• 400 students
• 90 suspensions after first term (22%)
• 67 appeal appointments
• 60 appeals granted
• 49 registered for third term (82%)
Fall 2014
• 276 students
• 68 suspensions after first term (25%)
• 56 appeal appointments
• 47 granted appeals
• 41 registered for third semester (87%)
Second Chance Success Program
Retake COLL 150 and/or COLL 110 (as needed)
Enroll in CEEP 101 (3 credits, liberal ed)
Regular meetings with our staff
Complete Mapworks Surveys
Attend 2 campus workshops/events of their choice
Alex
• Alex had an NCV admission score of 16/24
• She failed all first term courses, except one
• Suspension appeal interview score of 21/24
• Second semester, she: • met regularly with advisor, joined student organizations,
led service projects on campus, became an Orientation Leader
• Earned “A” in CEEP 101
• Ended with GPA of 3.5/4.0, returned for third semester
Mary
• Mary had a very high NCV score in admission (22/24)
• She withdrew from all courses, except one (earned A)• 4.0/4.0 GPA, but completion rate was 25%
• Appeal had a low NCV score (14), but staff was swayed by her story (extenuating circumstances)
• Spring stopped coming to probation meetings, didn’t submit coursework, stopped using support services
• Ended 2nd term with very low GPA, “D” in CEEP 101
• Not allowed to return for third semester
CEEP 101 and GPACorrelations
Class
Grade
Spring Term
GPA
Grade Pearson Correlation 1 .858**
Sig. (2-tailed)
.000
N 77 77
Spring Term GPA Pearson Correlation .858** 1
Sig. (2-tailed) .000
N 77 78
**. Correlation is significant at the 0.01 level (2-tailed).
Findings• Grade in CEEP 101 probation course is a strong predictor of overall
term GPA
• Data shows that students who have high confidence in their abilities but low performance are less likely to persist than students who acknowledge their academic challenges. This is very visible in our students with ACT scores above 24. If an ACE student with an ACT score above a 24 was put on probation they were not retained into Fall.
• Those admitted to the program before the NCV admission process were over-represented on probation; those admitted with NCV scores were under represented on probation. Once on probation, the two populations had no statistically different outcomes from one another (used same common appeal process).
• Surprisingly, for all years (2012-2015) ACE students have been looked at there has been very little statistical predictor for outcomes based on:
Race/Ethnicity, Pell Eligibility (low SES), Parental Educational Attainment
• These numbers were found by performing Chi-square analysis on each above variable compared with students on probation. Each time analysis was performed no statistically significant differences were found. The only exception to this trend is that African American students have been statistically less likely to go on ACE probation than other students.
Findings
CEEP 101 helps us predict student success in terms of GPA and fall to fall retention.
NCVs help us predict student success in terms of GPA, fall to fall retention, and likelihood to become suspended.
NCVs and GPAACE 2012-2013 First Semester Admit vs Second Semester Admit
Factor Sig. (2-tailed) Mean Difference First Semester Admit Second Semester Admit
ACT_COMP 0.00 -0.86 18.89 19.75
Commitment to the Institution 0.02 0.25 6.27 6.02
MW FACTOR Self-Assessment: Analytical Skills 0.00 -0.33 4.44 4.76
MW FACTOR Self-Assessment: Self-Discipline 0.05 0.17 5.75 5.58
MW FACTOR Self-Assessment: Time Management 0.05 0.22 5.47 5.25
MW FACTOR Basic Academic Behaviors 0.00 0.35 6.00 5.65
MW FACTOR Advanced Academic Behaviors 0.01 0.25 5.22 4.98
MW FACTOR Academic Integration 0.02 0.23 5.74 5.51
MW FACTOR Social Integration 0.01 0.32 5.82 5.50
MW FACTOR Test Anxiety (Module) 0.03 -0.36 3.84 4.19
FALL_CCR 0.00 8.69 82.33 73.64
FALL_EARNED 0.00 1.27 11.34 10.06
FALL_GPA 0.00 0.41 2.50 2.09
HS_PCT 0.02 2.66 34.57 31.91
SECOND_FALL_GPA 0.01 0.28 2.43 2.15
SPRING_GPA 0.01 0.24 2.40 2.16
NCVs and RetentionIndependent Samples Test
t df Sig. (2-tailed) (p-value) Mean Difference
Self-ReportedCommunication
2.027065 74 0.046258 0.534495 Not RetainedRetained
5.0494.514
Self-ReportedAnalytical
2.487892 75 0.015071 0.654762 Not RetainedRetained
5.2264.571
HomesicknessSeparation
2.656584 21 0.014761 1.555556 Not RetainedRetained
4.52.94
Self-ReportedDistressed
3.098642 21 0.00544 1.914683 Not RetainedRetained
5.974.06
Test Anxiety 3.504127 75 0.000776 1.268519 Not RetainedRetained
5.03.73
NCVs and SuspensionNCV_2013 * Probation Crosstabulation
Probation
Total.0 1.0
NCV_2013 .0 Count 199a 29b 228
Expected Count 204.7 23.3 228.0
1.0 Count 126a 8b 134
Expected Count 120.3 13.7 134.0
Total Count 325 37 362
Expected Count 325.0 37.0 362.0
Value df
Asymp. Sig. (2-
sided)
Exact Sig. (2-
sided)
Pearson Chi-Square 4.189a 1 .041
Likelihood Ratio 4.506 1 .034
Fisher's Exact Test .048
Linear-by-Linear Association 4.178 1 .041
N of Valid Cases 362
Recent Changes
• Requiring written responses and interviews
• As a comparison from term to term
• Provides opportunity for student to focus their thoughts
• Proxy for motivation
• Well-prepared vs. Skype example
Conclusions
NCVs assess soft skills that are fundamental to student success.
At SCSU: - High NCV scores predict high GPAs
- High NCV scores increase the likelihood of retaining the student
- High NCV scores decrease the likelihood of academic suspension
- High NCV scores increase likelihood of successful third semester GPA
NCVs can help us tailor our programs and allocation of time and resources to better serve students in need.
Evans, N. J., Forney, D. S., Guido, F. M., Patton, L. D., & Renn, K. A. (2010). Student development in college: Theory, research, and practice. San Francisco, CA:
Jossey-Bass
Miller, W. R. & Rollnick, S. (2002). Motivational interviewing: Preparing people for change. New York, NY: Guilford Press
Sedlacek, W. E. (2004). Beyond the big test: Noncognitive assessment in higher education. San Francisco, CA: Jossey-Bass
Top Related