Direct vs Indirect Assessment of Student Learning: An Introduction
-
Upload
katelyn-hogan -
Category
Documents
-
view
31 -
download
1
description
Transcript of Direct vs Indirect Assessment of Student Learning: An Introduction
Direct vs Indirect Assessment of Student Learning: An Introduction
Dr. Sheila Handy, Chair Business Management and Co-Chair University Assessment Committee
Sponsored by University Assessment Committee
February 24, 2014
Assessment• “The most important purpose of assessment should be not
improvement or accountability but their common aim: Everyone wants students to get the best possible education. Everyone wants them to learn what’s most important. A college’s mission statement and goals are essentially promises that the college is making to its students, their families, employers, and society. Today’s world needs people with the attributes we promise. We need skilled writers, thinkers, problem-solvers and leaders. We need people who are prepared to act ethically, to help those in need, and to participate meaningfully in an increasingly diverse and global society. Imagine what the world would be like if every one of our graduates achieved the goals we promise them! We need people with those traits, and we need them now. Assessment is simply a vital tool to help us make sure we fulfill the crucial promises we make to our students and society. “
Linda Suskie, 2010
Setting the Stage
•Middle States Commission on Higher Education
• “…each program should employ direct measures of student learning…”
• “…each stated learning outcome should be aligned to a measure, a performance goal, and a finding. Measures can be used to evaluate more than a single learning outcome.”
Assessment Measures - Definitions
Direct Indirect
•Direct examination or observation of student knowledge, skills, attitudes or behaviors to provide evidence of learning outcomes.
•Perceived extent or value of learning experiences
Assessment Measures - Examples
Direct Indirect
•Exam▫Standardized▫Locally developed▫Embedded questions
•Juried review•Portfolios with rubrics
•Surveys▫Students▫Alumni
•Focus Groups•Student Records▫Class Attendance
Short Table Activity
• Introductions
•Envelopes▫Match Assessment Measure and Description▫Sort into Direct and Indirect Measures
Standardized Tests
•Licensing Exams▫Education - Praxis, PAPA▫Nursing - NCLEX▫Accounting - CPA Exam
•Major Field Test▫Developed by ETS
▫Administered in a proctored environment as paper and pencil or online
MFT subjects▫Biology▫Business▫Chemistry▫Computer Science▫Criminal Justice▫Literature in English
▫Math▫Physics▫Political Science▫Psychology▫Sociology
Standardized TestsAdvantages Disadvantages
Carefully developed Highly reliable Professionally scored Nationally normed
▫ If tests don’t match LOs of program, scores will be low
▫ MC questions that test facts – faculty may focus on higher order skills
▫ May be expensive▫ Students may not take
seriously
Locally Developed ExamAdvantages Disadvantages
• Can be tied-in to specific LOs• Can be administered as part of
regular semester testing• Can use publisher developed
questions
• Reliability and generalizability not known
• Time required to prepare questions
• Scoring may take a long time• Norms not available
Embedded QuestionsAdvantages Disadvantages
• Can be conducted as part of regular student testing
• Does not require collection of additional data
• Can be used for both grading and assessment
• Faculty may think they are being assessed
• Takes time to develop and grade
Embedded Questions
•One way to assess learning without faculty “angst”▫Add assessment questions as a quiz administered
at the beginning of the next course in the sequence
▫Effective if different faculty teach multiple sections of an introductory course
Juried Review
•Suitable for assessing ▫Performing arts projects▫Written assignments▫Oral presentations
Juried ReviewAdvantages Disadvantages
• Provides experience and practice for student performers
• Can be formative and summative – indicating “value added”
• Can also introduce students to the standards they will be expected to achieve
• Achieving consensus with regard to the evaluation rubric
• May not be sufficient to determine if students have achieved the LOs set by the faculty
Portfolios With RubricsAdvantages Disadvantages
• Provides students with evidence to submit to potential employers
• Tracks student work over time• Can assess higher levels of
learning• Allows students to reflect on
their learning
• Faculty must agree on assessment rubric
• Time consuming• Storage of evidence
Case Study
Your goal is to develop a plan to assess this student learning outcome for the Engineering Program for the upcoming academic year. This plan should include two direct measures.