Overcoming Challenges with the SLO process

CHALLENGE - The Business Model

Problems with the “production process” in higher education

Learning is a SUBTLE Process.

Students vary ENORMOUSLY!!

Disciplines & professions require diverse methods of instruction.

The “production process” in higher education is

“More Intricate and complicated

than that in any industrial

enterprise…”

The University is not a business:Treating the University as a business has

damaged the quality of education and undermined its requests for funding.

Given the catastrophes that the business world has imposed on the global economy, I find it insane that the business model has been superimposed on education.

--Marc Epstein

ANTIDOTE Stay informed!

Increase ability to clearly articulate how education is different from business

Remember colleges are not “cookie cutters” of each other

Define what we do; otherwise, somebody will do it for us

CHALLENGE – Complex Mission

ANTIDOTE Define your local priorities

Have conversations about who you serve and why

Plan strategically

Pay attention to legislation

CHALLENGE –Tools Missing

ANTIDOTE Determine “real” tools needed

Identify tasks and path to completion

CHALLENGE #2 RESISTANCE TO CHANGECHALLENGE – Formalized

Assessment is

ANTIDOTEIdentify and chart layers

Identify and chart levels

Diagram connections

Keep it simple!

CHALLENGE - External Factors

ANTIDOTEIdentify external forces for your local college

Prepare counter evidence if needed

Choose to be collegial

Work cooperatively

CHALLENGE - Resistance to Change

Change Happen

s!!

The phases of transition through change

Focusing on the past and denying change

Assessing where they stand and

resisting change

Imagining and exploring ways to

the future

Commitment to work together for a

new future

The phases of transition through change

Focusing on the past and denying

change

Assessing where they stand and

resisting change

Imagining and exploring ways to

the future

Commitment to work together for a

new future

SLOs added to the

accreditation standards

2002

SLOs Proficiency

2012

ANTIDOTE Acknowledge and tolerate resistance

Encourage and support change

For those that are stuck in concrete..

Breathe & Move on!

CHALLENGE - FACULTY M.I.A.

ANTIDOTEIdentify reason for each MIA faculty

Match people with their skills

Make meetings etc. productive AND enjoyable

Publicize accomplishments of faculty work

Streamline faculty work

Schedule meetings at faculty friendly times

Plan for time to discuss our craft!

CHALLENGE - Budget

ANTIDOTESimplify tasks

Develop enrollment management plan (Assert faculty role)

Choose to find the positive

Find the light!

Talk more about students and learning

CHALLENGE - TIME

ANTIDOTERelook at the work of faculty

Use meetings for the “work”

Put minutia in emails

Rethink committee structure

Focus on the 10+1

And, the Current Biggest Challenge

Closing the Loop:

Providing Credible Evidence

Closing the Loop means going back to square one, or…

Assessing the effectiveness of changes to curriculum, pedagogy, resources, assessment methods, or learning

outcomes implemented in response to assessment

results.

GLOSSARY FOR TODAY

CLO Course Learning Outcome or Course Student Learning

Outcome

PLO Program Learning Outcome or Program Student Learning

Outcome

ILO Institutional Learning Outcome or Institutional Student

Learning Outcomes

CASE STUDY #1

Course Learning Outcomes & Closing the Loop

English 49 Success Rates

ENGL 049

58% 57%

65%71%

76%

Spring 2007 Spring 2008 Spring 2009 Spring 2010 Spring 2011

ENGL 049

60%

51%

61%

68%73%

Fall 2006 Fall 2007 Fall 2008 Fall 2009 Fall 2010

Note that English 49 Coordination data showed that a change in assessment during Fall 2008 from a final exam (71% on average) to portfolio (82% on average) showed an 11% increase in subsequent success rates of those students enrolled in English 101/105.

Success Rates of English Lab Students

Fall 2010 Spring 2011 Fall 10 - Spring 11Average

72%

79%75%

79%82% 81%

62% 63% 62%

English Tutoring 1-2 sessions English Tutoring 3 or more sessions No English Tutoring

Case Study #2

Program Learning Outcomes &

Closing the Loop

Trait 1     Trait 2     Trait 3     Trait 4    

Courses: Green = FoundationsBlue = First Year, no COEYellow = 2nd Sem, w/ COEBrown =Advanced / Final Course

TotalEnrollment

0-1 2 3 4% at

2 or <0-1 2 3 4

% at 2 or <

0-1 2 3 4% at 2 or <

0-1 2 3 4% at 2 or <

Total at 2 or

Higher

Below

Target

ART-121: 3-D Foundations 26 2 1 13 10 92% 3 2 7 14 88% 7 4 7 8 73% 4 3 8 11 85% 85%  

ART-137: Fund. of Drawing 23 2 1 7 13 91% 1 3 9 10 96% 8 5 4 6 65% 2 1 7 13 91% 86%ART-137: Fund. of Drawing 33 4 1 10 18 88% 1 1 15 16 97% 9 7 8 9 73% 3 4 11 15 91% 87%ART-127: Intro Painting 16 2 0 8 6 88% 1 2 8 5 94% 7 4 1 4 56% 2 1 7 6 88% 81% XART-127: Intro Painting 5 1 0 1 3 80% 0 0 3 2 100% 2 1 1 1 60% 1 0 1 3 80% 80% XART-147: Clay Sculpture 16 1 1 6 8 94% 1 1 5 9 94% 5 2 3 6 69% 2 1 4 9 88% 86%ART-154: Survey of Ceramics 10 2 1 2 5 80% 1 2 2 5 90% 4 1 1 4 60% 1 1 2 6 90% 80% XART-155A: Intro Ceramics 16 2 1 3 10 88% 1 1 6 8 94% 5 3 3 5 69% 1 2 4 9 94% 86%ART-155B: Intro Ceramics 10 1 1 4 4 90% 1 1 3 5 90% 5 2 1 2 50% 2 1 3 4 80% 78% XDART-120: Intro Digital Art 23 2 1 9 11 91% 1 2 6 14 96% 3 5 7 8 87% 2 1 14 6 91% 91%PHOT-120: Intro Photo 25 3 1 13 8 88% 1 1 11 12 96% 4 4 7 10 84% 1 1 13 10 96% 91%PHOT-120: Intro Photo 22 2 0 14 6 91% 0 1 9 12 100% 5 3 6 8 77% 1 1 11 9 95% 91%PHOT-120: Intro Photo 26 2 1 7 16 92% 2 1 13 10 92% 4 3 11 8 85% 0 1 9 16 100% 92%PHOT-120: Intro Photo 24 2 1 10 11 92% 1 1 8 14 96% 2 2 13 7 92% 2 1 15 6 92% 93%PHOT-120: Intro Photo 30 3 1 11 15 90% 1 2 17 10 97% 4 3 10 13 87% 3 1 12 14 90% 91%PHOT-150: Intro Dig Photo 17 2 1 7 7 88% 1 1 9 6 94% 2 3 8 4 88% 2 2 5 8 88% 90%ART-128: Intermediate Paint 12 1 0 6 5 92% 1 1 4 6 92% 3 2 4 3 75% 1 0 4 7 92% 88%ART-128: Intermediate Paint 12 1 1 4 6 92% 0 0 7 5 100% 4 3 3 2 67% 3 1 5 3 75% 83% XART-138: Creative Drawing 18 1 1 7 9 94% 1 1 7 9 94% 3 6 4 5 83% 2 1 9 6 89% 90%ART-180: Printmaking 13 1 0 3 9 92% 0 0 8 5 100% 3 2 5 3 77% 1 1 3 8 92% 90%ART-230: Figure Painting 18 0 0 7 11 100% 0 0 3 15 100% 2 1 9 6 89% 1 0 6 11 94% 96%ART-250: Ceramic Workshop 12 1 0 5 6 92% 0 0 0 12 100% 3 1 6 2 75% 1 0 4 7 92% 90%ART-255A: Inter Ceramics 2 0 0 0 2 100% 0 0 0 2 100% 1 0 0 1 50% 0 0 1 1 100% 88%ART-255B: Inter Ceramics 5 1 0 2 2 80% 0 0 1 4 100% 1 0 2 2 80% 0 1 2 2 100% 90%DART-140: Typography 27 2 0 13 12 93% 0 0 12 15 100% 4 2 12 9 85% 1 0 15 11 96% 94%PHOT-121: Basic Lab Photo 10 1 0 6 3 90% 0 0 2 8 100% 3 2 4 1 70% 0 1 7 2 100% 90%PHOT-122: Basic Lab Photo 11 1 0 3 7 91% 0 0 4 7 100% 2 1 3 5 82% 0 1 5 5 100% 93%PHOT-151: Int Digital Photo 7 0 0 3 4 100% 0 0 2 5 100% 1 0 4 2 86% 0 0 3 4 100% 96%ART-270: Pro.Practices 13 0 0 4 9 100% 0 0 1 12 100% 0 0 4 9 100% 0 0 2 11 100% 100%

TOTAL 482 43 15 188 236 91% 19 24 182 257 96% 106 72 151 153 78% 39 28 192 223 92% 89%

3 or < 424 3 or < 439 3 or < 304 3 or < 415

What does it all mean?

•Student scores across the program met and exceeded expectations, but Vocabulary trait score significantly lower.

• Scores increased with course level, but not by a statistically significant amount.

• Lowest scores clumped around 1st year courses without Conditions on Enrollment.

•Results did not tell us about student learning accumulated through course sequencing in the program

Program Response

•Discussed and analyzed how vocabulary is handled by each instructor, looking for effective practices

•Created formative evaluations covering vocabulary, including vocabulary quizzes and vocabulary sheets for specific courses and sections and made these available to all instructors

•Created department Primer on Critiques (Power Point and handout) that was made available to all instructors

•Developed list of program-wide essential vocabulary

•Revised CORs and syllabi to make essential vocabulary an explicit part of instruction in Foundations Courses: 2-D, 3-D Design, and Fundamentals of Drawing

•Reviewed COEs for courses above Foundations level and determined that some realignment of enforced sequencing through COEs was appropriate, pending review of current program degrees and development of AA-Ts

We re-assessed this PLO in Spring 2012.

The results showed improvement across all four

traitswith marked improvement on

the Vocabulary trait.

Case Study #3

Institutional Learning Outcomes &

Closing the Loop

One way…..

Data compilations

ILO Program Program

Communicate ideas clearly and concisely in written, oral, and other forms, using a variety of complementary media.

Acquire and critically evaluate information, solve complex problems, and make informed decisions.

Set goals and develop a plan to achieve those goals.

Another way

Follow the same pattern as you did for CLOs and PLOs

Create a new processes for ILOs

You want EVIDENCE??

Types of EVIDENCE??Updated CORsYearly budget requestProgram Review documentsComprehensive Assessment ReportClass schedulesCollege catalogMeeting minutesEmailsStudent work samplesAssessment samples

Ass

essm

ent

of

cour

ses

LOs

Examples from Monterey Peninsula College Documents

Executing the Plan

2010

2011

Strengthened connections between Program Reflectionsand Program ReviewInstituted time for Program Reflections at all Flex Day events

Equated Institutional SLOs with GEOs.

MPC

Strengthen connections between Program Reflections and Program Review

Adjust language in all resource allocation processes to reflect Program Reflections and/or student learning

And the new Challenge …….

Linking Assessment with Faculty Evaluations

Potential Benefit of Closing the Loop

Consciously Incompetent

Unconsciously incompetent

Consciously Incompetent

✖

Unconsciously Competent

Unconsciously incompetent

Consciously Incompetent

Consciously Competent

Unconsciously Competent

Unconsciously incompetent

Consciously Incompetent

REMEMBER!!

Differences among colleges is NATURAL, EXPECTED, and BENEFICIAL.

The very essence of education and a creative, productive, innovative populous is variance in ways of looking at issues and problems.

REMEMBER!!

Incorporate your dialog in already existing college forums.

OR

Create new forums for dialog as necessary

AND

Eliminate forums/meetings that are not productive

REMEMBER!!

Don’t let the desire to “do it right” make the process foreign to your campus.

Questions??

Thank You!