Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement

NAVIGATING THE PROCESS OF STUDENT LEARNING OUTCOMES: DEVELOPMENT, EVALUATION, AND IMPROVEMENT

Shannon M. Sexton, Julia M. Williams, & Timothy ChowRose-Hulman Institute of Technology

Introduction & Workshop Goals

RHIT/IRPA - NASPA 2009

Introduction Goals

1. Discuss the process of developing institute student learning outcomes as well as common difficulties.

2. Assist participants in developing their own learning outcomes3. Discuss evidence that can be used to evaluate student learning

outcomes achievement.4. Assist participants in identifying articles of evidence for their

learning outcomes.5. Discuss using an e-portfolio as a means of collecting and rating

evidence.6. Allow participants to rate sample evidence using a sample rating

rubric.7. Discuss how to use the rating results to close the loop and

improve student learning.

Group Activity #1


Define Teamwork Include characteristics of teamwork Include traits of those characteristics Time allotted 10 minutes

About Rose-Hulman Terre Haute, Indiana 1800+ undergraduate

students B.S. degrees in

engineering, science, and mathematics

80%+ engineering students

Accredited through ABET and North Central


CASO


Commission on the Assessment of Student Outcomes

Faculty committee with 1 representative from each academic department

Regular meetings throughout the academic year

Tasked with developing Institute-wide student learning outcomes and a way to measure them

Historic Timeline


1996 – 1998 1999 – 2000 2001 – 2006 2007 – present

Removing the Jargon


Domains = Categories of skills Learning Outcomes = Definition of the

skills students should have Performance Criteria = Characteristics of the learning outcome

Rubrics = Use of a scale to identify traits in a document

Domains and Performance Criteria

8

Domain Student Learning Outcome

Performance Criteria

Technical Skills Outcomes developed by each department.

Levels A – C indicate the level of performance achieved by each student.

Professional Skills

1. Leadership2. Teamwork3. Communication


World Citizenship 4. Cultural & Global Awareness

5. Ethics6. Service



Developing Outcomes


Importance of preliminary research Teams of faculty developers Developing in steps & gaining institute

buy-in Importance of measurability

Writing Outcomes – Activity #2


Revise your teamwork definition into a student learning outcome with rubrics

Time allotted 15-20 minutes

10 minutes to stretch your legs

Take a Break


Evidence of Outcomes


Choosing appropriate evidence New vs. Current Importance of Rubric

Collecting evidence How to physically collect & store evidence

Course Mapping Tracking where submissions are coming

from Accountability

Criterion Description and Rubric Teamwork B1: Demonstrate how you reached a

decision as a team.

Primary Traits: A passing submission for this criterion must:

1. Describe the team goal.2. Provide a description of a specific team decision and describe the following:

- the process of making the decision- how multiple team members contributed to the outcome- how team members’ ideas were critically evaluated

3. Show that the ultimate outcome was consistent with the team’s decision-making process.Potential documents: Documents appropriate for this criterion include (but are not limited to): Memo or reflective statement on team process from a lab group design team, debate team, service project.

Additional information:

13


Course Mapping14

Course mapping


Identifying Evidence – Activity #3 Where do you already have existing

assignments that could be used as articles of evidence for the Teamwork outcome? Group Brainstorming Activity

15


Domains and Performance Criteria

16

Performance CriteriaLevel C – associated with awareness, application, or evaluation.Level B – associated with application.Level A – associated with knowledge or comprehension.

A-Level Criteria

B-Level Criteria

C-Level Criteria


Collecting Evidence


Use of an E-portfolio system Faculty input Student input Assessment staff input

Rating Evidence


Rating preparation Administrative side of rating

Paying faculty volunteers, tech support, number of volunteers, etc.

Process of rating days

Portfolio Rating Methodology Evaluation rubric, A, B, C Inter-rater reliability: initial, subsequent Document rating: Pass/Fail/Exemplary Wholistic rating, NOT grading


Rating Documents – Activity #4


Rate 2-3 teamwork documents using the provided rubric.

15 - 20 minutes

Using Rating Results


Identifying constituents How to communicate results to institute What to do with the results

Example Result


2005 2006 20070%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

36%

63%71%

52%

62%

73%65%

74%83%

Historical Portfolio Rating Results

Teamwork Communication Ethics

Mee

ts S

tand

ard

Please contact us for more information after the conference.


Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement

Documents

Transcript of Navigating the Process of Student Learning Outcomes: Development, Evaluation, and Improvement