Post on 12-Feb-2016
description
Identification and Articulaton of Enduring Student Learning Outcomes
Karl A. SmithSTEM Education Center / Technological Leadership Institute /
Civil Engineering – University of Minnesota &Engineering Education – Purdue University
ksmith@umn.edu - http://www.ce.umn.edu/~smith
King Fahd University of Petroleum and Minerals
Design and Implementation of Cooperative Learning
August 19-21, 2013
2
Session 2 Layout• Reflection on Session 1• Enduring student learning outcomes (BIG ideas)• Taxonomies of learning outcomes• Connecting outcomes and assessment strategies
Five Minute University
• http://www.youtube.com/watch?v=kO8x8eoU3L4
Streveler and Smith 3
Back
war
d D
esig
n
Context
Content
Assessment
Pedagogy
C & A & PAlignment?
End
Start
Yes
No
Understanding by Design (Wiggins & McTighe, 2005)
Content-Assessment-Pedagogy (CAP) Design Process Flowchart
4
Streveler, Smith & Pilotte (2012)
3 Stages of Backward Design
Are the desired results, assessments, and learning activities ALIGNED?
Identify the Desired Results
Determine Acceptable Evidence
Plan Learning Experiences
What should learners know, understand, and be able to do?
Streveler and Smith 5
3 Stages of Backward Design
Are the desired results, assessments, and learning activities ALIGNED?
Identify the Desired Results
Determine Acceptable Evidence
Plan Learning Experiences
How will we know if the learners have achieved the desired results? What will be accepted as evidence of learners’ understanding and proficiency?
Streveler and Smith 6
3 Stages of Backward Design
Are the desired results, assessments, and learning activities ALIGNED?
Identify the Desired Results
Determine Acceptable Evidence
Plan Learning Experiences
What activities will equip learners with the needed knowledge and skills? What materials and resources will be useful?
Streveler and Smith 7
Elements of Content
• Knowledge-centered aspects of Content– Focuses on the information, topics, “stuff” of the
learning unit• Curricular priorities
• Learning-centered aspects of Content – Focuses on how the learner interacts with the
content
8Streveler and Smith
Designing Learning Environments Based on HPL (How People Learn)
Streveler and Smith 9
KNOWLEDGE-CENTERED What do you want learners to learn?
Streveler and Smith 10
To understand a topic or subject is to use knowledge in sophisticated, flexible ways. Knowledge and skill are necessary elements of understanding, but they are not synonymous with understanding. Matters of understanding require more: [Learners] need to make conscious sense and apt use of the knowledge they are learning and the principles underlying it.
-Understanding by Design
Wiggins and McTighe (1998)
Understanding Big Ideas
Streveler and Smith 11
Wiggins & McTighe Curricular Priorities
Good to be familiar with
Important to know
Enduring outcomes
12
Click icon to add picture
Streveler and Smith
Streveler and Smith 13
14
Understanding Misunderstanding
A Private Universe – 21 minute video available from www.learner.org
Also see Minds of our own (Annenberg/CPB Math and Science Collection – www.learner.org)
1. Can we believe our eyes? 2. Lessons from thin air3. Under construction
Exercise
Determine for your curriculum (re)design
Enduring outcomes
Important to know or understand
Good to be familiar with
Streveler and Smith 15
LEARNER-CENTEREDHow do people learn?
Streveler and Smith 16
Information Processing Model of Learning
• Input via attention – to short term memory – to long term memory – retention and retrieval
Streveler and Smith 17
Information Processing Model of Learning
• Key areas for instruction– Attention and processing power = cognitive load (bandwidth)
• LIMITED – need to be careful how one uses the learner’s bandwidth– Connected nature of memory
• Link to what is already known– PRIOR KNOWLEDGE
• Need for organization– Structural knowledge – ADVANCED ORGANIZERS
– Multiple inputs (both in timing and in modes of input) = number and strength of connections = richer learner that will be retained longer
• PRACTICE• MULTIPLE MODES OF INPUT (visual, audio, explanation to yourself and
others)
Streveler and Smith 18
EXERCISE
Streveler and Smith 19
What are your initial ideas about how you can help your learners achieve the enduring outcomes of the
course you are redesigning?
ASSESSMENTHow will you know learners have learned what you want them to learn?
Streveler and Smith 20
We cannot measure learning directly. Instead we must make inferences
from evidence. Sound inferences depend on
assessments that are aligned with content.
Objectives of Assessment What is the Purpose of the Assessment?
Key questions to ask yourself about assessment
1. What should I be assessing?2. What is the best assessment to use?3. How can I be sure I am consistently
interpreting the results of the assessments?
Streveler and Smith 22
1. What should I be assessing?
• Assessments should be aligned with your curricular priorities. – Enduring outcomes should ALWAYS be assessed.– Important to know outcomes are usually assessed.– Good to be familiar with information might not be
assessed.
Streveler and Smith 23
2. What is the best assessment to use? Assessments Aligned to Curricular Priorities
Adapted from: Wiggins, G. & McTighe, J. (1997). Understanding by Design. Alexandria, VA: ASCD
Traditional Quizzes and Tests• Selected Response
Closed-ended, convergent Well-structured Scored via “answer key”
Academic Prompts• Constructed Response
Open-ended, ill-structured, divergent Academic conditions (e.g. exams, drills) Requires analysis, evaluation, and/or synthesis Judgment-based scoring (e.g., rubrics)
Performance Tasks • Papers, projects, design tasks, etc.
Open-ended, complex, ill-structured, divergent Approximation of practice, specific audience Higher autonomy, more personalized Judgment-based scoring (e.g., rubrics)
Familiar w
ith
Importa
nt to kn
ow
Enduring
Understanding
3. How can I be sure I am consistently interpreting the results of the assessments?
• For close-ended and well-structured assessment prompts.– Construct an accurate grading key– Determine what level of performance is acceptable for
demonstrating “mastery” [i.e. “pass”]• For more open-ended prompts and for performance tasks
– Construct a matrix what will link key features with how those features will be demonstrated to show mastery. This is called a RUBRIC.
– Determine what level of performance is acceptable for demonstrating “mastery.”
Streveler and Smith 25
Some steps for creating a rubric
• Look at work you feel is of varying levels of quality.• What characteristics make some work “better” than
others?• What are markers within the work point to better
performance?
• Exercise: Work together to create a rubric for assessing– A memo– A weld
Streveler and Smith 26
LEARNING OBJECTIVES AND TAXONOMIES
Learning objectives are the bridge between what you want learners to learn and how you know they learned it.
27Streveler and Smith
Constructing Learning Objectives Learning Activities vs. Learning Objectives
Streveler and Smith 28
Constructing Learning Objectives Using Verb-Noun Format
Streveler and Smith 29
Mapping Learning Objectives to Curricular Priorities
Annotated Example: Ruth Wertz - High Level Weekly Planning (Annotated Example).docx
Streveler and Smith 30
• Bloom’s taxonomy of educational objectives: Cognitive Domain (Bloom & Krathwohl, 1956)
• A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives (Anderson & Krathwohl, 2001).
• Taxonomy of significant learning (Fink, 2003)• Evaluating the quality of learning: The SOLO
taxonomy (Biggs & Collis, 1982)• Facets of understanding (Wiggins & McTighe, 1998)
31
Taxonomies
Streveler and Smith
Revised Bloom’s Learning Taxonomy
Source: http://www.celt.iastate.edu/pdfs-docs/teaching/RevisedBloomsHandout.pdfStreveler and Smith 32
Revised Bloom’s Learning Taxonomy
Source: http://www.celt.iastate.edu/pdfs-docs/teaching/RevisedBloomsHandout.pdf
Streveler and Smith 33
Revised Bloom’s Learning Taxonomy
Source: http://www.celt.iastate.edu/pdfs-docs/teaching/RevisedBloomsHandout.pdf
Streveler and Smith 34
Mapping Learning ObjectivesRange of Low-Order & High-Order Objectives
Annotated Example: Ruth Wertz - Map of Weekly Learning Objectives (Annotated Example).docx
Streveler and Smith 35
Exercise
• For the course you are (re)designing:– Starting with your enduring outcomes – write
learning objectives for each of your 5 most important curricular priorities.
– Place those learning objectives in Bloom’s revised taxonomy.
Streveler and Smith 36
37
Session 3-4 Preview• Pedagogies of Engagement – Cooperative Learning and
Challenge Based Learning– Informal – Bookends on a Class Session– Formal Cooperative Learning
• Key Resource– Review Smith, Sheppard, Johnson & Johnson, “Pedagogies of
engagement.”
KINDS OF ASSESSMENT
Streveler and Smith 38
We cannot measure learning directly. Instead we must make inferences from evidence. Sound inferences depend on assessments that are aligned with content.
Objectives of Assessment What is the Purpose of the Assessment?
Formative Summative
Provide Quantitative Feedback (e.g.,
score or grade)
Evaluate Completion of Learning Objectives
Summarize Learning Achievement
Provide Qualitative Feedback
Evaluate Learning Progress
Improve/Redirect Learning Activities
Assessment ExamplesFormative and Summative Assessments
Formative Summative
Tests & Quizzes Self-test Quiz
Written Exam
Academic Prompts
Group Discussion Reflection Paper
Peer Review
Written ExamWritten Assignment
Practical Exam
Performance Tasks
Oral ExamReflection Paper
Practice lab practical with feedback
Design ProjectResearch Paper
Case StudyGraded lab practical
Observation & Dialogue
Oral ExamInformal Check for Understanding
Oral ExamPerformance Evaluation
Examples above are not exhaustive.
Glossary of important terms in summative assessment
• Criterion model of assessment– Determine a level of learning considered
acceptable and then measure every learner against that criterion.
• All could pass, none could pass.
• Normative model of assessment– Learners are measured against each other.
• “Grading on a curve.”
• Rubrics
Streveler and Smith 41
EXAMPLES OF FORMATIVE FEEDBACK
Streveler and Smith 42
Quick Thinks
• Reorder the steps• Paraphrase the idea• Correct the error• Support a statement• Select the response
Johnston, S. & Cooper,J. 1997. Quick thinks: Active- thinking in lecture classes and televised instruction. Cooperative learning and college teaching, 8(1), 2-7
Streveler and Smith 43
Minute Paper
• What was the most useful or meaningful thing you learned during this session?
• What question(s) remain uppermost in your mind as we end this session?
• What was the “muddiest” point in this session?
• Give an example or application
• Explain in your own words . . .
Angelo, T.A. & Cross, K.P. 1993. Classroom assessment techniques: A handbook for college teachers. San Francisco: Jossey Bass. Streveler and Smith 44
Session Summary (Minute Paper)
Reflect on the session
1. Most interesting, valuable, useful thing you learned.2. Things that helped you learn.3. Question, comments, suggestions.4. Pace: Too slow 1 . . . . 5 Too fast5. Relevance: Little 1 . . . 5 Lots6. Instructional Format: Ugh 1 . . . 5 Ah
Streveler and Smith 45
MOT 8221 – Spring 2009 – Session 1
Q4 – Pace: Too slow 1 . . . . 5 Too fast (3.3)Q5 – Relevance: Little 1 . . . 5 Lots (4.2)Q6 – Format: Ugh 1 . . . 5 Ah (4.4)
0
5
10
15
20
25
Q4 Q5 Q6
1
2
3
4
5
Streveler and Smith 46
Exercise
• For your 5 most important learning objectives:– What kind of assessments might you use to
measure learners’ attained the level of mastery of the learning objectives?
– Where can you insert formative assessment to check learners’ progress?
Streveler and Smith 47
49
Taxonomies of Types of LearningBloom’s taxonomy of educational objectives: Cognitive Domain
(Bloom & Krathwohl, 1956)
A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives (Anderson & Krathwohl, 2001).
Facets of understanding (Wiggins & McTighe, 1998)
Taxonomy of significant learning (Fink, 2003)
Evaluating the quality of learning: The SOLO taxonomy (Biggs & Collis, 1982)
50
The Six Major Levels of Bloom's Taxonomy of the Cognitive Domain(with representative behaviors and sample objectives)
Knowledge. Remembering information Define, identify, label, state, list, match Identify the standard peripheral components of a computer Write the equation for the Ideal Gas Law
Comprehension. Explaining the meaning of information Describe, generalize, paraphrase, summarize, estimate
In one sentence explain the main idea of a written passage Describe in prose what is shown in graph form
Application. Using abstractions in concrete situations Determine, chart, implement, prepare, solve, use, develop
Using principles of operant conditioning, train a rate to press a bar Derive a kinetic model from experimental data
Analysis. Breaking down a whole into component parts Points out, differentiate, distinguish, discriminate, compare
Identify supporting evidence to support the interpretation of a literary passage Analyze an oscillator circuit and determine the frequency of oscillation
Synthesis. Putting parts together to form a new and integrated whole Create, design, plan, organize, generate, write
Write a logically organized essay in favor of euthanasia Develop an individualized nutrition program for a diabetic patient
Evaluation. Making judgments about the merits of ideas, materials, or phenomena Appraise, critique, judge, weigh, evaluate, select
Assess the appropriateness of an author's conclusions based on the evidence given Select the best proposal for a proposed water treatment plant
51
Remember Understand Apply Analyze Evaluate Create
Factual Knowledge – The basic elements that students must know to be acquainted with a discipline or solve problems in it.a. Knowledge of terminologyb. Knowledge of specific details and elements
Conceptual Knowledge – The interrelationships among the basic elements within a larger structure that enable them to function together.a. Knowledge of classifications and categoriesb. Knowledge of principles and generalizationsc. Knowledge of theories, models, and structures
Procedural Knowledge – How to do something; methods of inquiry, and criteria for using skills, algorithms, techniques, and methods.a. Knowledge of subject-specific skills and algorithmsb. Knowledge of subject-specific techniques and methodsc. Knowledge of criteria for determining when to use appropriate procedures
Metacognitive Knowledge – Knowledge of cognition in general as well as awareness and knowledge of one’s own cognition.a. Strategic knowledgeb. Knowledge about cognitive tasks, including appropriate contextual and conditional knowledgec. Self-knowledge
The Cognitive Process Dimension
The Know
ledge Dim
ension
(Anderson & Krathwohl, 2001).
52http://www.uwsp.edu/education/lwilson/curric/newtaxonomy.htm
53http://www.celt.iastate.edu/pdfs-docs/teaching/RevisedBloomsHandout.pdf
54
Dee Fink – Creating Significant Learning Experiences A TAXONOMY OF SIGNIFICANT LEARNING
1. Foundational Knowledge • "Understand and remember" learning
For example: facts, terms, formulae, concepts, principles, etc. 2. Application
Thinking: critical, creative, practical (problem-solving, decision-making) Other skills
For example: communication, technology, foreign language Managing complex projects
3. Integration Making "connections" (i.e., finding similarities or interactions) . . .
Among: ideas, subjects, people 4. Human Dimensions
Learning about and changing one's SELF Understanding and interacting with OTHERS
5. Caring Identifying/changing one's feelings, interests, values
6. Learning How to Learn Becoming a better student Learning how to ask and answer questions Becoming a self-directed learner
Cognitive
Affective
Meta
SOLO Taxonomy • The Structure of Observed Learning Outcome (SOLO)
model consists of 5 levels of understanding– Pre-structural - The task is not attacked appropriately; the student
hasn’t really understood the point and uses too simple a way of going about it.
– Uni-structural - The student's response only focuses on one relevant aspect.
– Multi-structural - The student's response focuses on several relevant aspects but they are treated independently and additively. Assessment of this level is primarily quantitative.
– Relational - The different aspects have become integrated into a coherent whole. This level is what is normally meant by an adequate understanding of some topic.
– Extended abstract - The previous integrated whole may be conceptualised at a higher level of abstraction and generalised to a new topic or area.
http://en.wikipedia.org/wiki/Structure_of_Observed_Learning_Outcome
Teaching Teaching and Understanding Understanding
• Biggs SOLO taxonomy
• http://www.youtube.com/watch?v=iMZA80XpP6Y