How to Develop and/or Update A Simple and Clear Assessment Plan
description
Transcript of How to Develop and/or Update A Simple and Clear Assessment Plan
![Page 1: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/1.jpg)
How to Develop and/or Update A Simple and Clear
Assessment Plan
Office of Academic Program Assessment
Spring 2013
![Page 2: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/2.jpg)
Overview Introduction Definition of Basic Terms Differences Between Goals and Outcomes Criteria for Evaluating Goals and Outcomes Using Rubrics in PLOs The Next Step: Developing the Assessment
Plan
![Page 3: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/3.jpg)
Handouts1. PowerPoint slides2. WASC Rubric for Assessing the Quality of
Academic Program Learning Outcomes3. Examples of Goals and Outcomes4. Curriculum map (Sociology)5. Rubric (BA Nutrition and Food)6. VALUE Rubric on Critical Thinking
![Page 4: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/4.jpg)
Learning Goal“a high-level, very general statement of learning expected of graduates, aligned with the institution’s mission, vision, and values (more specific learning outcomes are derived from goals)”
(WASC Draft 2013 Handbook of Accreditation, p. 45)
![Page 5: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/5.jpg)
Learning Outcome“a concise statement of what the student should know or be able to do. Well-articulated learning outcomes describe how a student can demonstrate the desired outcome; verbs such as “understand” or “appreciate” are avoided in favor of observable actions, e.g., “identify,” “analyze.” Learning outcomes can be formulated for different levels of aggregation and analysis. Student learning outcomes are commonly abbreviated as SLOs, course learning outcomes as CLOs, program learning outcomes as PLOs, and institution-level outcomes as ILOs.”
(WASC Draft 2013 Handbook of Accreditation, p. 48)
![Page 6: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/6.jpg)
Assessment PlanAssessment—an ongoing, iterative process consisting of four basic steps: 1.defining learning outcomes2.choosing a method or approach and then using it to gather evidence of learning3.analyzing and interpreting the evidence4.using this information to improve student learning
(WASC Draft 2013 Handbook of Accreditation, p. 40)
![Page 7: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/7.jpg)
Qualities of Goals(adapted from Graves, 2000)
General but not vague Transparent to all stakeholders (no jargon) Realistic Simple—usually not multifaceted Something the program addresses Stated in terms of the learner
![Page 8: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/8.jpg)
Qualities of Outcomes(adapted from Graves, 2000)
More specific than goals Directly relate to goals
hierarchical relationship cause-effect relationship
Focus on what students learn and/or associated processes
More objectives than goals Stated in terms of the learner Transparent to all stakeholders
![Page 9: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/9.jpg)
More Qualities of Outcomes(WASC Rubric for Assessing the Quality of
Academic Program Learning Outcomes) Complete & well-organized Both discipline-specific and institution-wide
outcomes Clearly distinguish between levels (e.g.,
undergraduate versus graduate) Have explicit criteria statements developed
by faculty (e.g., rubrics)
![Page 10: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/10.jpg)
Example: BA General Communication
GOAL: Students will communicate effectively in a
variety of contextsSUB-GOALS:1. Demonstrate proficiency in oral
communication2. Demonstrate proficiency in written
communication
![Page 11: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/11.jpg)
Example: BA General Communication
Learning Outcomes for Sub-Goal 1(Demonstrate proficiency in oral communication)1.determine presentation needs in different situations2.correctly use visual aids3.make appropriate language choices4.use proper structure5.effectively deliver presentations
![Page 12: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/12.jpg)
Example: BA General Communication
Learning Outcomes for Sub-Goal 2 (Demonstrate proficiency in written communication)1.implement a variety of style sheets2.use thesis statements3.use appropriate organizational strategies4.apply transitions5.include appropriate evidentiary support material6.employ grammar conventions
![Page 13: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/13.jpg)
The Importance of Verbs(Mager, 1975, cited in Brown, 1995)
Multiple Interpretations to know to understand to really understand to appreciate to fully appreciate to grasp the significance
of to enjoy to believe to have faith in
Fewer Interpretations to write to recite to identify to sort to solve to construct to build to compare to contrast
![Page 14: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/14.jpg)
Criteria for Evaluating GoalsHave program goals been specified?1. Is each goal comprehensible? 2. Is each goal justified?
– represents institutional goals– represents what faculty and/or discipline feel to
be important3. Is each goal stated in terms of the learner? 4. Will all goals actually be addressed in the program? 5. Can all goals be achieved by students in the
program?
![Page 15: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/15.jpg)
Criteria for Evaluating OutcomesHave program learning outcomes (PLOs) been developed?1. Does the program specify clear PLOs?2. Are there distinct differences in the level of
specificity between program goals and program learning outcomes?
3. How will achieving program learning outcomes help to reach program goals?
4. Are PLOs achievable within the curriculum?
![Page 16: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/16.jpg)
Criteria for Evaluating Outcomes5. Have each of the following been considered in
developing PLOs?: – Do PLOs focus on the key knowledge, skills and
values students learn in the program? – Are the verbs in PLOs clear and unambiguous?
Do they describe the observable behaviors by which students will demonstrate learning?
– Are the criteria for performance on each PLO explicit and transparent? (Are there benchmarks?)
– How and where can you observe each PLO?– What expectations do you have for student
achievement for each PLO?
![Page 17: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/17.jpg)
Rubrics for PLO Specification Especially for skills-based outcomes Rubrics allow faculty to agree on explicit criteria
statements that describe observable behaviors differentiate levels of performance
• e.g. , Cultural and global awareness/sensitivity including demonstrated understanding, respect and
• support of multiple perspectives from other disciplines, societies, individuals, groups, and cultures
![Page 18: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/18.jpg)
Example Rubric
BA in Nutrition and FoodAwareness of the integration of the different concentrations in FACS and their importance to the relationships between humans and their diverse environments as individuals and groups as a whole.
![Page 19: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/19.jpg)
Rubric Development Time-consuming Look at existing rubrics
e.g., VALUE rubrics can be used ... without change with revisions as a heuristic
![Page 20: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/20.jpg)
Group Discussion Your program includes “critical thinking” as a
learning goal and has decided to assess it this year. You have not yet specified PLOs or criteria for this goal.
Look at the VALUE rubric1. How useful would this rubric be to your program
for establishing PLOs and explicit criteria?2. Choose one dimension. How would you need to
revise the rubric to represent how that dimension is realized in your program?
![Page 21: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/21.jpg)
Debrief
1. What did you learn about the process of specifying PLOs and criteria from doing this?
2. What did you learn from trying to adapt the rubric?
![Page 22: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/22.jpg)
The Next Step:Sustainable Assessment PlansGood Assessment Plans: Focus on the program (e.g., majors and
concentrations) rather than courses Have a limited number of goals and outcomes Are developed collaboratively Are ongoing Are manageable Use multiple methods of data collection Anticipate how results will be used and lead to
improvement
![Page 23: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/23.jpg)
Comprehensive Assessment Planning
Include both content area knowledge and skills/competencies
Multi-year plan with 1-2 outcomes assessed each year
Direct and indirect methods Capstone student projects (direct) Student exit surveys: self-reporting on all
outcomes
![Page 24: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/24.jpg)
Writing an effective assessment Plan
Outcome Assessed and/or Assessment Question
Method(s) to Collect EvidenceBrief description of what will be collected, how, and by whom. (Enter one method per row; add rows as needed.)
Method to Analyze/ Evaluate* Brief description of how the evidence will be analyzed or evaluated and by whom. (When applicable, include scoring criteria or rubric in an Appendix.)
Timeline & StatusList the semester/dates when the evidence will be collected and evaluated.
Lead Team membersList the name(s) of those who will oversee collecting, analyzing, reporting, and using results.
LO 1 Capstone Assignment (Direct)
Include sample size, scoring criteria, expected outcomes, etc.
LO 1 Exit Survey (indirect)
From UH-Manoa Assessment Template: http://http://manoa.hawaii.edu/assessment/resources/templates.htm
![Page 25: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/25.jpg)
The Next Step:Sustainable Assessment Plans
What should be assessed??
IPP definition of ProgramsWhat appears on the diplomaUndergraduate majors/concentrationsGraduate majors/concentrations
Do we have to assess all of these separately??
![Page 26: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/26.jpg)
The Next Step:Sustainable Assessment Plans
Issues• too many outcomes, not enough time• too many programs, not enough time
![Page 27: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/27.jpg)
An Example: Biological Sciences Undergraduate programs 1,449 students (excluding pre-major) BA Biological Sciences BS Biological Sciences (7 concentrations) Common curriculum for all concentrations: two LD
and two UD coursesHow can we assess so many programs??
Too Many Programs—Not Enough Time
![Page 28: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/28.jpg)
Learning Goals/OutcomesStudents will demonstrate the ability to
communicate in the Biological SciencesWritten CommunicationOral CommunicationInformation Literacy
Students will demonstrate the ability to apply the process of scienceCritical ThinkingQuantitative Reasoning
![Page 29: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/29.jpg)
Skills & Competencies
Learning Goal 1: Students will demonstrate the ability to communicate in the Biological Sciences
Learning Goal 2: Students will demonstrate the ability to apply the process of science
LO’s/Courses
Written Communication
Oral Communication
Information Literacy
Critical Thinking
Quantitative Reasoning
Bio 1 I I I I IBio 2 I,D I,D I,D I,D Bio 100 D D D D DBio 184 DBio 121 D Bio 139 D D DBio 160 D D D D DBio 188 M M M MElectives Various Various Various Various Various
BA & BS-General: Curriculum Map
![Page 30: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/30.jpg)
The problem
Shared courses: Bio 1, 2, 100, 184Other courses: problematic!Bio 188: Evolution
Required by 4 of 7 concentrationsWill capture 70% of students
![Page 31: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/31.jpg)
Five Year Assessment PlanL.O. / Year 1 2 3 4 5
Written Communication
Bio 188/capstone
Oral Communication
Capstone
Information Literacy
Bio 188/capstone
Critical Thinking Bio 188/capstone
Quantitative Reasoning
Bio 188/capstone
Content Knowledge
Bio 188/capstone
Indirect self-assessment
Bio 188/capstone
Bio 188/capstone
Bio 188/capstone
Bio 188/capstone
Bio 188/capstone
![Page 32: How to Develop and/or Update A Simple and Clear Assessment Plan](https://reader036.fdocuments.in/reader036/viewer/2022062521/56815dea550346895dcc11b6/html5/thumbnails/32.jpg)
For more information
• Please contact Office of Academic Program Assessment– Amy Liu ([email protected])– Julian Heather ([email protected])– Shannon Datwyler ([email protected])– Elizabeth Strasser ([email protected])