Post on 30-Dec-2015
Agenda Peer Assessment Roundtables – Student
Learning
10:00 - 10:10 – Welcome and Explain Process
10:10 - 10:30 – Full Group: Coaching Assessment – SLOs
10:30 – 10:50 –Roundtable Discussion 1: Student Learning
Outcomes
10:50 – 11:00 - Full Group: Coaching Assessment - Curriculum
Mapping
10:50 – 11:10 – Roundtable Discussion 2: Curriculum Mapping
11:10 – 11:30 – Roundtable Discussion 3: Example Outcomes:
Findings and Changes
11:30 – 11:50 – Roundtable Discussion 4: Assessment Process
11:50 – 12:00 – Wrap Up
Peer Assessment Roundtables
Coaching Assessment
Avoiding Garbage In/Garbage Out:
Student Learning OutcomesCurriculum Mapping
Fundamental Assumption:
Assessing student learning at the program level starts with well written student learning outcomes.
Fundamental TIP:
There’s nowhere for a program to go if they
are starting with poorly written student learning
outcomes
What the student will√ Know√ Do
….at the end of the course or program
Assessing Fundamental
Structure of SLO
Students will be able to <action verb>, <action verb>, and <action verb>, <something>, <something else>, and <another thing>.
KNOWLEDGECOMPREHENSION
APPLICATIONANALYSIS
SYNTHESISEVALUATION
CiteCountDefineDraw
IdentifyList
NamePointQuoteRead
ReciteRecordRepeatSelectState
TabulateTell
TraceUnderline
AssociateClassifyCompareComputeContrast
DifferentiateDiscuss
DistinguishEstimateExplainExpress
ExtrapolateInterpolate
LocatePredictReportRestateReview
TellTranslate
ApplyCalculateClassify
DemonstrateDetermineDramatize
EmployExamineIllustrateInterpretLocate
OperateOrder
PracticeReport
RestructureScheduleSketchSolve
TranslateUse
Write
AnalyzeAppraiseCalculate
CategorizeClassifyCompareDebate
DiagramDifferentiateDistinguish
ExamineExperiment
InspectInventoryQuestionSeparateSu rize
Test
ArrangeAssemble
CollectComposeConstruct
CreateDesign
FormulateIntegrateManageOrganize
PlanPrepare
PrescribeProduceProposeSpecify
SynthesizeWrite
AppraiseAssessChoose
CompareCriticize
DetermineEstimateEvaluate
GradeJudge
MeasureRankRate
RecommendReviseScoreSelect
StandardizeTest
Validate
Lower level courseoutcomes
KNOWLEDGECOMPREHENSION
APPLICATIONANALYSIS
SYNTHESISEVALUATION
CiteCountDefineDraw
IdentifyList
NamePointQuoteRead
ReciteRecordRepeatSelectState
TabulateTell
TraceUnderline
AssociateClassifyCompareComputeContrast
DifferentiateDiscuss
DistinguishEstimateExplainExpress
ExtrapolateInterpolate
LocatePredictReportRestateReview
TellTranslate
ApplyCalculateClassify
DemonstrateDetermineDramatize
EmployExamineIllustrateInterpretLocate
OperateOrder
PracticeReport
RestructureScheduleSketchSolve
TranslateUse
Write
AnalyzeAppraiseCalculate
CategorizeClassifyCompareDebate
DiagramDifferentiateDistinguish
ExamineExperiment
InspectInventoryQuestionSeparate
SummarizeTest
ArrangeAssemble
CollectComposeConstruct
CreateDesign
FormulateIntegrateManageOrganize
PlanPrepare
PrescribeProduceProposeSpecify
SynthesizeWrite
AppraiseAssessChoose
CompareCriticize
DetermineEstimateEvaluate
GradeJudge
MeasureRankRate
RecommendReviseScoreSelect
StandardizeTest
Validate
AdvancedCourse / Program
outcomes
Components – Grading Criteria
•What are you looking for in student work to be able to tell if they “get it”
Example #1Gather factual information and apply it to a
given problem in a manner that is relevant, clear, comprehensive, and conscious of possible bias in the information selected
BETTER: Students will apply factual information to a problem COMPONENTS- Grading Criteria:
RelevanceClarity
Comprehensiveness Aware of Bias
Example #2Imagine and seek out a variety of possible goals,
assumptions, interpretations, or perspectives which can give alternative meanings or solutions to given situations or problems
BETTER: Students will provide alternative solutions to situations or problemsCOMPONENTS - Grading Criteria:
Variety of assumptions, perspectives, interpretations
Analysis of comparative advantage
1xx
X
X
X
7
1
2
3
4
5
6
Student Learning Outcomes
1xx
X
X
X
2xx
X
X
2xx
X
2xx
X
X
3xx
X
X
X
3xx 3xx
X
X
X
4xx
X
X
X
X
Capstone
X
X
1xx
K
K
K
7
1
2
3
4
5
6
Student Learning Outcomes
K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
1xx
S
K
K
2xx
A
A
2xx
S
2xx
A
A
K
3xx
A
K
A
3xx 3xx
A
K
A
4xx
S
A
K
S
Capstone
S
S
1xx
K
K
K
7
1
2
3
4
5
6
Student Learning Outcomes
K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
1xx
S
K
K
2xx
A
A
2xx
S
2xx
A
A
K
3xx
A
K
A
3xx 3xx
A
K
A
4xx
S
A
K
S
Capstone
S
S
1xx
K
K
K
7
1
2
3
4
5
6
Student Learning Outcomes
K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
1xx
S
K
K
2xx
A
A
2xx
S
2xx
A
A
K
3xx
A
K
A
3xx 3xx
A
K
A
4xx
S
A
K
S
Capstone
S
S
1xx
K
K
K
7
1
2
3
4
5
6
Student Learning Outcomes
K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
1xx
S
K
K
2xx
A
A
2xx
S
2xx
A
A
K
3xx
A
K
A
3xx 3xx
A
K
A
4xx
S
A
K
S
Capstone
S
S
1xx
K
K
K
7
1
2
3
4
5
6
Student Learning Outcomes
K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
1xx
S
K
K
2xx
A
A
2xx
S
2xx
A
A
K
3xx
A
K
A
3xx 3xx
A
K
A
4xx
S
A
K
S
Capstone
S
S
1xx
K
K
K
7
1
2
3
4
5
6
Student Learning Outcomes
K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
1xx
S
K
K
2xx
A
A
2xx
S
2xx
A
A
K
3xx
A
K
A
3xx 3xx
A
K
A
4xx
S
A
K
S
Capstone
S
S
1xx
K
K
K
7
1
2
3
4
5
6
Student Learning Outcomes
K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
1xx
S
K
K
2xx
A
A
2xx
S
2xx
A
A
K
3xx
A
K
A
3xx 3xx
A
K
A
4xx
S
A
K
S
Capstone
S
S
1xx
K
K
K
7
1
2
3
4
5
6
Student Learning Outcomes
K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
1xx
S
K
K
2xx
A
A
2xx
S
2xx
A
A
K
3xx
A
K
A
3xx 3xx
A
K
A
4xx
S
A
K
S
Capstone
S
S
1xx
K
K
K
7
1
2
3
4
5
6
Student Learning Outcomes
K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
1xx
S
K
K
K
2xx
A
A
2xx
A
2xx
A
A
K
K
3xx
A
K
K
K
3xx
A
A
3xx
S
S
A
S
S
A
4xx
S
A
S
S
Capstone
S
S
Build a curriculum
1xx
K
K
K
7
1
2
3
4
5
6
Student Learning Outcomes
K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
1xx
S
K
K
K
2xx
A
A
2xx
A
2xx
A
A
K
K
3xx
A
K
K
K
3xx
A
A
3xx
S
S
A
S
S
A
4xx
S
A
S
S
Capstone
S
S
Diagnose Learning
Analyze Curriculum
1xx
K
K
K
7
1
2
3
4
5
6
Student Learning Outcomes
K= Knowledge/Comprehension; A= Application / Analysis; S= Synthesis /Evaluation
1xx
S
K
K
K
2xx
A
A
2xx
A
2xx
A
A
K
K
3xx
A
K
K
K
3xx
A
A
3xx
S
S
A
S
S
A
4xx
S
A
S
S
Capstone
S
S
Diagnose Learning
Peer Assessment Roundtables
Coaching AssessmentAvoiding Garbage In/Garbage Out
:
1.Student Learning Outcomes2.Curriculum Mapping
3.Collecting Assessment Information4.Sharing and Acting on Results
Agenda Peer Assessment Roundtables – Student Learning
10:00 - 10:10 - Welcome and Explain Process
10:10 - 10:20 - Full Group Coaching - Collecting Assess.
Information
10:20 - 11:00 - Roundtable Discussion 1: Collecting Assess.
Information
11:00 - 11:05 - Full Group Coaching - Faculty
Motivation/Collaboration
11:05 - 11:15 - Roundtable Discussion 2: Faculty
Motivation/Collaboration
11:15 - 11:25 – Full Group Coaching - Sharing, Acting on Results
11:25 - 11:50 - Roundtable Discussion 3: Sharing, Acting on
Results
11:50 – 12:00 - Wrap Up
Peer Assessment Roundtables
Coaching AssessmentAvoiding Garbage In/Garbage Out
:
•Student Learning Outcomes 1=Goals•Curriculum Mapping
•Collecting Assessment 2=Information•Sharing and 3=Acting on Assessment Results
Peer Assessment Roundtables
Coaching AssessmentAvoiding Garbage In/Garbage Out
:
•Student Learning Outcomes 1=Goals•Curriculum Mapping
•Collecting Assessment 2=Information•Sharing and 3=Acting on Assessment Results
Direct and Indirect Measures
Direct:•Ratings by their field supervisors •Licensure or certification exams •Capstone experiences•Other work scored using a rubric •Portfolios •Scores on tests •Electronic class discussion threads •Feedback from computer-simulated tasks
Indirect:• Course grades • Retention and graduation rates • Admission rates into graduate programs• Post grad activity of alum• Placement rates of graduates • Alumni perceptions • Student feedback • Course student rating forms • Student, alumni, and employer satisfaction
Using Samples of Student Work for Assessment
Advantages: •Information is already available •No student motivation problems•No direct cost •Reflects what faculty actually teach
Using Samples of Student Work for Assessment
Disadvantages:
•Evidence not comparable across institutions •Information is in multiple parts and formats•Quite a bit of work•Common standards/rubrics/training needed
Performance Element
Exemplary(4)
Proficient(3)
Developing(2)
Emerging(1)
Not Present (0)
Score
I. Communication Define problem in your own words.
Identifies the main idea or problem with numerous supporting details and examples which are organized logically and coherently.
Identifies the main idea or problem with some supporting details and examples in an organized manner.
Identifies the main idea or problem with few details or examples in a somewhat organized manner.
Identifies the main idea or problem poorly with few or no details or states the main idea or problem verbatim from the text.
Does not identify the main idea or problem.
4 3 2 1 0
N/A
Comments:
II. Analysis Compare & contrast the available solutions.
Uses specific inductive or deductive reasoning to make inferences regarding premises; addresses implications and consequences; identifies facts and relevant information correctly.
Uses logical reasoning to make inferences regarding solutions; addresses implications and consequences; Identifies facts and relevant information correctly.
Uses superficial reasoning to make inferences regarding solutions; Shows some confusion regarding facts, opinions, and relevant, evidence, data, or information.
Makes unexplained, unsupported, or unreasonable inferences regarding solutions; makes multiple errors in distinguishing fact from fiction or in selecting relevant evidence.
Does not analyze multiple solutions.
4 3 2 1 0
N/A
Comments:
III. Problem Solving Select & defend your chosen solution.
Thoroughly identifies and addresses key aspects of the problem and insightfully uses facts and relevant evidence from analysis to support and defend potentially valid solutions.
Identifies and addresses key aspects of the problem and uses facts and relevant evidence from analysis to develop potentially valid conclusions or solutions.
Identifies and addresses some aspects of the problem; develops possible conclusions or solutions using some inappropriate opinions and irrelevant information from analysis.
Identifies and addresses only one aspect of the problem but develops untestable hypothesis; or develops invalid conclusions or solutions based on opinion or irrelevant information.
Does not select and defend a solution.
4 3 2 1 0
N/A
Comments:
Benefits of Rubrics
• Timely feedback• Detailed feedback• Critical thinking• Communication• Teaching skills• Level playing field
Faith and Christ-Centered Learning
How do you collect data on your student faith-growth learning outcomes?
Agenda Peer Assessment Roundtables – Student Learning
10:00 - 10:10 - Welcome and Explain Process
10:10 - 10:20 - Full Group Coaching - Collecting Assess.
Information
10:20 - 11:00 - Roundtable Discussion 1: Collecting Assess.
Information
11:00 - 11:05 - Full Group Coaching - Faculty
Motivation/Collaboration
11:05 - 11:15 - Roundtable Discussion 2: Faculty
Motivation/Collaboration
11:15 - 11:25 – Full Group Coaching - Sharing, Acting on Results
11:25 - 11:50 - Roundtable Discussion 3: Sharing, Acting on
Results
11:50 – 12:00 - Wrap Up
Peer Assessment Roundtables
Coaching AssessmentAvoiding Garbage In/Garbage Out
:
•Student Learning Outcomes 1=Goals•Curriculum Mapping
•Collecting Assessment 2=Information
• Faculty Motivation/Collaboration
•Sharing and 3=Acting on Assessment Results
Why Aren’t Grades Enough?
• Relativity• Attendance-Participation• Extra Credit• Consistency• Curriculum
Tangible Actions To Promote Faculty Participation In
Assessment
• Faculty Meetings• Syllabi• Performance Expectations• Scholarship
Agenda Peer Assessment Roundtables – Student Learning
10:00 - 10:10 - Welcome and Explain Process
10:10 - 10:20 - Full Group Coaching - Collecting Assess.
Information
10:20 - 11:00 - Roundtable Discussion 1: Collecting Assess.
Information
11:00 - 11:05 - Full Group Coaching - Faculty
Motivation/Collaboration
11:05 - 11:15 - Roundtable Discussion 2: Faculty
Motivation/Collaboration
11:15 - 11:25 – Full Group Coaching - Sharing, Acting on Results
11:25 - 11:50 - Roundtable Discussion 3: Sharing, Acting on
Results
11:50 – 12:00 - Wrap Up
Peer Assessment Roundtables
Coaching AssessmentAvoiding Garbage In/Garbage Out
:
•Student Learning Outcomes 1=Goals•Curriculum Mapping
•Collecting Assessment 2=Information•Sharing and 3=Acting on Assessment Results
Brief Guidelines On Sharing Assessment
Results • Use good teaching practices • Brevity• Most important information:
1. How you and your colleagues define a successful student 2. Whether you are satisfied with your evidence of student success 3. What you are doing about unsatisfactory results
• Visuals
Useful Reporting of Assessment Results
What will audiences for assessment results care most about?
•Matters they can do something about •Interesting and unanticipated findings •Meaningful differences
Venues for Sharing Assessment Results
• Web sites • Emails • Newsletters • Alumni magazines • Departmental memos • Press releases • Brochures • Presentations • Posters or banners
Using Assessment Results Effectively and Appropriately
• What’s most important?• Think about process• Stakeholders • Communication/Information • Watch interpretation
What if the results are bad?
• Do you have the right learning goals? • Do you have too many learning goals? • Take a hard look at your courses:
Content and requirements, Sequencing and prerequisites, Admissions criteria, Placement criteria, Advising, Tutoring, Teaching methods, Co-curricular activities
•Keep going
Agenda Peer Assessment Roundtables – Student Learning
10:00 - 10:10 - Welcome and Explain Process
10:10 - 10:20 - Full Group Coaching - Collecting Assess.
Information
10:20 - 11:00 - Roundtable Discussion 1: Collecting Assess.
Information
11:00 - 11:05 - Full Group Coaching - Faculty
Motivation/Collaboration
11:05 - 11:15 - Roundtable Discussion 2: Faculty
Motivation/Collaboration
11:15 - 11:25 – Full Group Coaching - Sharing, Acting on Results
11:25 - 11:50 - Roundtable Discussion 3: Sharing, Acting on
Results
11:50 – 12:00 - Wrap Up