1

2014 Institutional Effectiveness & Student

Success Scorecard

Update on Institutional Effectiveness Process &

Report on Student Success Scorecard

Prepared by theOffice Of Institutional

ResearchMay 2014

Presented to the Board of Trustees by:Dr. Hannah Lawler, Dean, Institutional ResearchTeresita Rodriguez, Vice President, Enrollment Development

2

SMC Office of Institutional Research

Institutional Effectiveness DefinitionInstitutional Effectiveness is the systematic and continuous

process of measuring the extent to which a college achieves its mission, as expressed through the goals and strategic objectives developed in an educational master plan

Dashboards

Institutional Priorities

Innovative &

Responsive Academic

Supportive Learning

Stable Fiscal

Sustainable Physical

Supportive Collegial

3

SMC Office of Institutional Research

2011-2016 Institutional Effectiveness Cycle/Process

Step 1: Organize Data • Develop IE key indicators (KI)• Select KIs for Institutional Priorities

Dashboard and draft targets• Identify KIs needing further inquiry

Step 2: Dig into Data • Conduct follow-up studies

Step 3: Develop Action Plan• Identify areas for intervention and

develop strategies for improvement

Step 4: Act• Implement strategies

Step 5: Assess Action Plan• Assess effectiveness of implemented

strategies

Annually: Key Indicators will be updated with the most recent annual data and a report of the progress of the Institutional Effectiveness process will be reported to the Board of Trustees

4

SMC Office of Institutional Research

Update for Year 3 (2013-2014)

• Addition of 8 new key indicators

• Changes in methodology for 3 key indicators

• Elimination of 2 key indicators

• Findings from Student Equity Interviews

• Impact of Student Behaviors Study

5

SMC Office of Institutional Research

Institution-Set Standards v. Target Goals• Institution-set standards: standards reflecting

satisfaction performance of student learning and achievement.

• Target goals: a measurable outcome expressed either as a quantifiable value (for example, a target of 75%) or a trend (for example, year-over-year decrease), when achieved, will meaningfully move the needle on institutional effectiveness by 2015-2016

6

SMC Office of Institutional Research

Institutional Priorities - StandardsKI 2011 2012 2013 2014 Institution-

set std.

Persistence 73.2% 76.2% 78.1% 73.2% 71.8%

Transfer 49.5% 51.9% 47.0% 47.4% 47.0%

Basic Skills Improve

69.3% 71.5% 71.5% 71.9% 66.7%

Basic Skills Transition

35.9% 34.4% 36.5% 37.1% 33.5%

CTE Completion

46.5% 45.8% 45.9% 42.0% 43.8%

Equity – Completion

23.2% 25.2% 25.1% 23.3% <=25.7%

Equity - Transfer

24.6% 24.1% 24.4% 24.5% <=25.5%

7

SMC Office of Institutional Research

Key Indicator 20142015/16 Target Goal

In progress- needs

attention

In progress – on target

Target achieved

Persistence 73.2% 75% X    

Transfer Rate 47.4% Remain Steady X  

Basic Skills Improve 71.9% 73% X    

Basic Skills Transition 37.1% 39% X    

CTE Completion 42.0% 47% X    

Equity - Completion 23.3% Decrease XEquity - Transfer 24.5% Decrease X  

Institutional Priorities - Target Goals

8

SMC Office of Institutional Research

Semesters to Associate Degree Completion

2009-2010 2010-2011 2011-2012 2012-2013

7.34 7.79 7.78 8.16

5.65 5.71 5.49 5.77

11.5913.15 13.5

16.17

7.88 8.49 8.39 8.72

Total Full-time Part-time Mixed

9

SMC Office of Institutional Research

Key Findings

• SMC did not meet the institution-set standard for the CTE Completion indicator

• SMC exceeds the target on one of the indicators• Equity Gap – Completion Rate

• SMC meets the target on two of the indicators• Transfer Rate, Equity Gap – Transfer Rate

• SMC performs below the 2015-2016 target on four of the indicators…• CTE Completion, Persistence Rate, Basic Skills Improvement, Basic Skills

Transition

• …However, SMC demonstrates improvement on two of them• Basic Skills Improvement, Basic Skills Transition

10

SMC Office of Institutional Research

Basic Skills Strategies

• ENGL 20, ENGL 85• MATH 85, MATH 49 (coming Fall 2014)• Baltimore Model (ENGL 1 + non-credit support course)

Accelerated Courses

• Summer Jams• First Year Experience (FYE)Programs

• Prep 2 Test• Assessment App

Preparation for Assessment Test

11

SMC Office of Institutional Research

Student Equity Interviews

Interview with 17 African American and Latino students in Spring and Fall of 2013 (not generalizable to population)

Themes from interviews with students:

1. Interviewees feel welcome at SMC.

2. Proactive and early advising is critical for students’ progress towards their goals.

3. Interviewees entered college without the essential college success skills and struggled to adjust to college life. However, the Counseling 20 course helped teach skills and build confidence in many students.

4. Supportive friends and family members play an important role in guiding and motivating the interviewees in college.

12

SMC Office of Institutional Research

Student Equity Interviews (cont.)

5. Interviewees prefer to study alone in the library, typically in between or immediately after class

6. Interviewees value campus clubs and activities but are not actively involved in them.

7. Math is the biggest barrier for the interviewees.

8. Information becomes actionable for interviewees when reinforced by human contact.

9. The most successful interviewees have relationships with a faculty/staff member on campus.

10. According to the interviewees, good teachers are student-centered, promote student-to-student interactions, provide timely feedback, and connect students to resources

13

SMC Office of Institutional Research

Impact of Student Behaviors Study

Out

com

es

Stu

dent

Beh

avio

rs

Bac

kgro

und

Cha

ract

eris

ticsGender

Ethnicity

Age

Residence

Low Income

First gen

BA+ attainment of zip code

Ed goal

Unit load

Engl/math skill

Orientat/assess

Major

Welcome Center

# counseling visits

Special prog

Tutoring/SI

GPA

Probation

Persisted to term

Enrolled Engl/ESL/math

# yrs since HS

# course repeats

Persistence

Transfer

Basic skills course improvement

Basic skills transition to degree course

CTE completion

14

SMC Office of Institutional Research

Student Success Scorecard

Removed

• 3 metrics (voc and basic skills course success, basic skills progress)

• 800 page PDF

• Peer groups

• Self-assessment

Remains

• 4 metrics (completion, persistence, 30 units, CDCP) – with methodology changes

• Requirement to report to Board of Trustees

Added

• Web-based interface

• 2 metrics (CTE & remedial progress)

• Disaggregated data

• Measured against self

• Addition of student-counseling ratio

Built upon existing ARCC framework

15

SMC Office of Institutional Research

Student Success Scorecard Background

• Major limitation: excludes students without valid social security numbers (SSNs). • Santa Monica College (SMC) enrolls a large

proportion of students without valid SSNs• In fall 2013, 14% of the student population did not

report a valid SSN.

• Aligns with areas being measured by Institutional Effectiveness Dashboards

16

SMC Office of Institutional Research

SMC ScorecardCohort Year

Metric2003-2004

2004-2005

2005-2006

2006-2007

2007-2008

Statewide 2007-2008

Completion 62.9% 56.3% 53.3% 52.5% 47.7% 48.1%

Persistence 71.6% 66.6% 67.5% 68.7% 71.7% 70.5%

30 Units 71.0% 66.1% 67.0% 68.5% 67.5% 66.5%

Remedial Progress - Math 33.2% 29.9% 28.2% 29.2% 28.8% 30.6%

Remedial Progress - English

48.2% 42.5% 42.6% 43.7% 42.8% 43.7%

Remedial Progress - ESL 64.6% 60.7% 61.7% 62.1% 58.7% 27.1%

CTE 58.0% 52.6% 53.2% 51.7% 49.4% 53.9%

CDCP NA NA NA 7.2% 7.5% NA

17

SMC Office of Institutional Research

Key Findings

• Five-year decreasing trend for the completion, remedial progress, 30 units, and CTE rates.

• The college demonstrated improvement in the persistence and CDCP rates over the last five and two years, respectively.

• SMC performed better than the system-wide averages on three of the eight metrics (persistence, 30 units, remedial progress ESL)

18

Questions/Discussion