K2 Floor Plan - Kukje Gallery · Installation View . Louise Bourgeois . 1-1=3951 , 70 . H=3951 DN
Sizing up Your Students, Part 3: Designing Effective Evaluations Marti Atkinson and Bill Doherty...
-
Upload
samantha-kane -
Category
Documents
-
view
219 -
download
2
Transcript of Sizing up Your Students, Part 3: Designing Effective Evaluations Marti Atkinson and Bill Doherty...
Sizing up Your Students, Part 3:
Designing Effective Evaluations
Marti Atkinson and Bill DohertyApril 10, 2008
For audio call Toll Free 1-888-886-3951
and use PIN/code 240228
Parts of this presentation were adapted from materials created by:Noelle C. Griffin PhD, Assistant Director, UCLA Center for the Study of Evaluation
Original material can be found at: http://www.lmu.edu/Page18044.aspx
Housekeeping
• Maximize your CCC Confer window.• Mute your phone (*6) if you have visitors or noise in your office.• Please note phone audio may be in presenter-only mode.• Ask questions over the phone when the presenter prompts. • Ask questions throughout presentation via the chat window.• Turn on or off Closed Captioning by clicking on the icon.• Save the presentation or chat by clicking on the icon.
AgendaBackground and Terms
Planning your Evaluation Strategy - Defining your Goals, Objectives & Outcomes
- Choosing your Measurements
- Online Quizzes and Exams - Performance based assessments
Analyzing data – rubrics
Background and TermsEnormous body of literature on this subject
Download list of resources at
http://nlight.com/ONE/evalsSurveys_resources.doc
Background and Terms
Some Basics
Assessments versus evaluations
Reliability - repeated measurements are same or very similar
Validity – you are measuring what you say you are
Background and Terms
Kinds of Measures
Quantitative: generates numeric data
Qualitative: generates descriptive data
Background and Terms
Collection Methods
Survey, Quizzes, and Exams
Interviews
Observations
Work Products
Background and Terms
Authentic – Performance & Behavioral
Assignments
Projects – individual and group
Behavioral observations– General Participation– Discussion contributions
Portfolio
Background and Terms
Triangulation of Data
Planning Your Evaluation Strategy
Define your course goals, learning objectives and outcomes
Determine what kind data and measurement tools will best indicate how well your Learning Objectives & Outcomes are being met?
Planning your Evaluation Strategy Goals, Objectives, Outcomes
Goals - Represent the overall mission
Objectives - Broader perspective on what generally students should know, value or do
Outcomes - What students should demonstrate/represent relative to objective
Why bother with Goals, Objectives, and Outcomes?
They provide the basis for: – Course content– Evaluating students– Communications
• Students• Other faculty and administration• Outside world
– Curriculum planning
ExampleGoal: The course will prepare students to become a
rocket scientist
Objective: Students will be able to understand the working components of a rocket and methods for a successful launch
Outcome: Students will be able to build and launch a rocket.
Learning Objectives and Outcomes Bloom’s Taxonomy of Learning
Knowledge of terminology (define, identify, describe)
Comprehension (discuss, explain, summarize)
Application (apply, utilize, interpret)
Analysis (analyze, compare, contrast)
Synthesis (integrate, organize, construct)
Evaluation (appraise, defend, judge)
Measurement Tool – Surveys, Quizzes & Exams
Types of Items
Quantitative (numbers) often closed ended (limited responses)
• True/False• Multiple choice• Ratings• Single answer
Qualitative (narrative & content) often open ended items
• Short answer
• Essay
Measurement Tool - Quizzes & Exams
Examples of Closed Ended Items
What is your gender?• Male – Female
What is your age ? (10-25) (26-40) (41-55) (55+)
Which college do you attend? EVC Butte Foothill De Anza
Who is your favorite artist of the 20th Century?• Picasso• Joan Miro• Marc Chagall• Salvador Dali
Measurement Tool - Quizzes & Exams
Examples of Open Ended Items
Describe the stylistic elements of Art Deco.
Why did Van Gogh cut off his ear?
What is art?
What is your life story until this point?
Measurement Tool - Quizzes & Exams
Watch for Bias in Constructing Items
How long ago did you stop using street drugs?• Just this week
• A month ago
• A year ago
• Over a year ago
Measurement Tool Performance Assessments
Portfolios
Capstone assignments
Performances/projects
Internships/field activities
Students evaluating students
Measurement Tool – Performance Assessments Potential Strengths
Direct evidence of student learning
Students motivated
Realistic context
Can be integrated into the grading process
Data collection can be relatively unobtrusive to students
Measurement Tool – Performance Assessments Potential Weaknesses
Additional development/coordination time
Faculty trust/buy in critical
Faculty time for scoring
Reliability/validity uncertain
Lack of norms/comparative data
Measurement Tool – Performance Assessments Portfolios
What is it? A collection of multiple targeted pieces of student work that provide evidence of a learner’s achievements along a continuum of learning.
Can be formative or summative (i.e., developmental vs. showcase)
Can be “online” or “old school”
Measurement Tool – Performance Assessments Portfolio Pitfalls
“Kitchen sink” approach
Lack of pre-planning
Retrospective re-creation
Request vs. requirement (i.e., built into program)
Under-estimating faculty time involved
Measurement Tool – Performance Assessments Capstone Assignment/Project
What is it? A culminating project or assignment that provides evidence of achievement of learning goals; generally integrative in nature.
Does not only have to be last year; can be spaced in terms of developmental milestones in the program providing longitudinal/comparative data
Measurement Tool – Performance Assessments Capstone Considerations
Does assignment capture the breadth of program learning expectations?
Does assignment link back to what is actually being taught in the courses?
Measurement Tool – Performance Assessments Data Collection Process
How will assignments be collected/saved for analysis?
Integrate into regular classroom process, otherwise compliance difficult
Will collection be online or hard copies?
Where will data be stored?
How will anonymity be maintained?
Measurement Tool – Performance Assessments Scoring Criteria
Rubrics: A set of criteria that identify expected dimensions of achievement regarding specific assignments, as well as levels of achievement along those dimensions
Checklists
Other approaches
Measurement Tool – Performance Assessments Strategies for Developing Rubrics
Professional org/ discipline standards
Review representative student work (failing though exemplary) and analyze what criteria led to those grades
Student feedback
Analysis of grading criteria already used by faculty to grade similar assignments
Ask faculty to grade sample work and provide criteria they used, looking for commonalities
Pilot/practice use
Measurement Tool – Performance Assessments Students
Students evaluating themselves
Students evaluating other students
– Need to train students in critiquing guidelines http://nlight.com/ONE/critiquingGuidelines.doc
– Use of rating scales useful
Evaluating Team Projects
References“Assessing for Learning” (2004). Peggy L. Maki, Stylus
Press
“Assessment Clear & Simple: A Practical Guide for Institutions, Departments, and General Education” (2004). Barbara E. Walvoord, Jossey-Bass.
“A Practical Guide to Alternative Assessment” (1992). Joan L. Herman, Pamela Aschbacher & Lynn Winters, ASCD.
“Scoring Rubrics in the Classroom” (2001). Judith Arter & Jay McTighe, Corwin Press.
Q & A
Ask now, or via email if you prefer:
Marti Atkinson, [email protected]
Bill Doherty, [email protected]
Upcoming/Archived Seminars
For upcoming desktop seminars and links to recently archived seminars, check the @ONE Web site at:
http://www.cccone.org/seminars/index.php
Thanks for Attending
For information on @ONE’s trainings go to:
http://www.cccone.org/index.php