Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D....
-
Upload
harry-gilbert -
Category
Documents
-
view
216 -
download
1
Transcript of Learning Outcomes Made Easy Using the Best Tools Jeffrey D. Keith, Ph.D. J. Kelly Flanagan, Ph.D....
Learning Outcomes Made EasyUsing the Best Tools
Jeffrey D. Keith, Ph.D.
J. Kelly Flanagan, Ph.D.
Russell T. Osguthorpe, Ph.D.
Danny R. Olsen, Ph.D.
Tom Mallory
Jeff Fox, Ph.D.
Brigham Young University, Provo UT
2
11 Academic Colleges 56 Academic Departments 400 Degree Programs 1,600+ Faculty 8,000+ Courses
33,000+ Students
3
Carnegie Classification: Research University (High Research Activity)
Northwest Commission on Colleges and Universities
30 Specialized Accrediting Organizations
BRIGHAM YOUNG UNIVERSITY
4
U.S. Regional Accreditors
5
Assessment Charge
Publish expected learning outcomes for all of their courses and programs
Provide evidence that the expected learning outcomes are realized by students
Demonstrate how such data collection and analyses leads to continuous improvement of student learning, the curriculum and the university
Colleges and universities across the country are being asked to:
6
Why “Student Learning Outcomes”?
1. Ethics: Assessment should ultimately (first and always) be about student learning Student major selection Aligning institutional, department goals Student “take-aways”
2. Pressure: Stakeholders Accrediting entities Government agencies Employers
7
What are Student Learning Outcomes?1. Definition: Statements defining what students
should know, understand and/or be able to do as a result of their learning experiences in a degree program Cognitive (knowledge) Behavioral (skills) Affective (attitudes)
2. Assessment: What counts as evidence that a given level of competence has been achieved.
3. Clear and assessable statements of the “essential and enduring knowledge, abilities, and attitudes or dispositions” that enable a learner to apply what he or she has learned to situations encountered in the real world.
Direct & Indirect EvidenceDIRECT MEASURES Imbedded performance
assessment mapped to learning objectives
Course and homework assignments
Exams, quizzes Standardized tests Term papers, reports Research projects Capstone projects, senior
theses, exhibits or performances
Pass rates, scores on licensure, certification, area tests
Rubric scores
INDIRECT MEASURES Course evaluations Alumni questionnaire data Senior survey data National Survey of Student
Engagement data Employer surveys Graduate school placement
rates Department exit interviews Focus group interviews with
students, faculty, etc. Job placement Registration or course
enrollment information8
9
Evidences of Learning on Pathways to Graduation
10
University Goal AwarenessTo Publish Course Level Learning Outcomes by August 31, 2011
Law
Relig
ious
Ed
Man
agem
ent
Educa
tion
Engin
eerin
g
Fine
Arts
Socia
l Scie
nces
Physic
al S
ci
Life
Scie
nces
Human
ities
Nursin
g
Unive
rsity
0%
20%
40%
60%
80%
100%
22%
40% 44%60% 61% 62% 68% 71% 72% 73%
80%
63%
By College as of June 1
11
Published, Mapped Course Outcomes
October 2010
April 18 June 1 July 21 Aug 31 GOAL
0%
20%
40%
60%
80%
100%
9%22%
29% 32%
100%
0.0830000000000001
% Courses with Published Outcomes% Course Outcomes Mapped to Program Outcomes
2011
12
Obstacles & Challenges Culture of assessment disparity across campus. Quality of stated “Expected Learning Outcomes”. Obtaining faculty consensus within departments. Faculty engagement in the process as part of the
status quo without any extra compensation. Moving from a faculty-centric to a student centric
language of stated outcomes. Appropriate channels of evidence—direct, indirect. Demonstrating closing the assessment loop Applied Technology—cost, time, scalability, usability
Design Goals• Ease of use
Editing, viewing, and linking of program and course outcomes must be easy
• Leverage existing IT web services Security, identity, course information
• Data accessibility Existing university applications
Course catalog, registration, syllabus builder
14
Achieving Design Goals
• Ease of use Usability
• Leverage existing OIT web services University IT department adopted SOA SOA registry
• Data accessibility Adopted a service model
Common interfaces for interoperability Service registered in SOA registry
Learning Outcomes
Service
Learning Outcomes Web site
Exams
Syllabus Builder
Discussions
CourseContent
Program Assessment
Tool
CourseRegistration
CourseCatalog
Course Assessment Tool
CourseEvaluations
Challenges
• Course Learning Outcome Outliers Certificates Emphases GE Service Course
• Building a Learning Outcomes Assessment tool that meets disparate needs.
Faculty Involvementhttp://www.youtube.com/byuctl#p/u/0/-oy6ztc1kq0
Course Catalog
Course Evaluations
Program Assessment Tool
Just Beginning Design…
Course Content
Discussions
Exams
Syllabus Builder
26
LO Website 1
29