WORKSHOP: ANALYTIC RUBRIC FOR TEACHING, … rubric increases consistency and cuts down on instructor...

Process Education Conference 2010 Preparing Today's Students to Solve Tomorrow's Problems

WORKSHOP: ANALYTIC RUBRIC FOR TEACHING, ASSESSING, AND GRADING QUANTITATIVE HOMEWORK PROBLEMS

Tuesday, June 29th, 2:00-3:00LOCATION: Room 4025

Facilitators:

Jim Morgan, Texas A & MBarbara Williams, University of IdahoSteve Beyerlein, University of Idaho

Abstract: The workshop presenters have developed a universal rubric for assessing quantitative (analytical) problem-solving homework. The rubric is intended for use by college-level students as well as for formative assessment and grading by instructors and tutors. In this workshop participants will become acquainted with features of the rubric, engage in hands-on scoring of student work from two different disciplines, perform real-time analysis of inter-rater reliability, and inventory tips for using the rubric to teach, assess, and evaluate problem solving. Findings to date include: (1) Students who use the rubric when doing their homework develop a more reflective view of their work that goes beyond just crunching numbers, (2) Faculty collaboration surrounding an analytical rubric increases consistency and cuts down on instructor lead time, and (3) It is necessary to calibrate users/raters to insure consistent results.

Learning Objectives:• Explain significance of each dimension in a universal rubric for analytical problem

solving.• Use the rubric to score student work from two quantitative disciplines.• Inventory and implement strategies for improving inter-rater reliability of scoring.• Collect ideas for improving the rubric and using it teach problem solving.

Resources:1. Rubric2. Student work samples

a. Ex 1, Ex 4b. Student 44.5, Student 44.7c. Team 1

3. Tool for Peer Assessment of Homework

28 PE conf 10 Tab 3 - Tuesday activities


Facilitation Plan:1. Survey audience about their discipline and their current use of rubrics in teaching and in evaluating work products. (5 min)2. Present the new rubric for analytical problem solving. (10 min)3. Individually score two work products from one discipline. (10 min)4. Working in teams of 2-3, develop consensus scores for the two work products. (10 min)5. Report and discuss consensus scores with large group. (10 min)6. Generate tips for calibrating raters (e.g. graders) to reduce inter-rater reliability. (5 min)7. Conduct 2nd round of scoring (two work products from another discipline). (15 min)8. Teams perform an SII of the rubric focusing on transfer across disciplines. (10 min)9. Teams report out their SII’s. (10 min)



Resource: Guidance Template / Formative Assessment Rubric / Grading Rubric for Homework Problem



Resource: Student Work Samples - Ex 1 and Ex 4



Resource: Student Work Samples: Student 44.5 and Student 44.7



Resource: Student Work Samples - Team 1



Resource: Tool for Peer Assessment of Homework Your Name: Assessee’s Name: Orientation:For this activity you will be exchanging your sample pretest with one of your classmates and performing an assessment on their work. Assessment is an important “way of being” in engineering because it is a catalyst for growth and learning. Giving and receiving assessment with your peers should add depth to your understanding of concepts in this assignment – helping you perform better on the actual pre-test. Learning Objectives:

1. Assess work on practice pretest, commenting on strengths and areas for improvement as well as personal insights.

2. Rank different dimensions for documenting engineering solutions. Targeted Skills:Assessing Performance – providing feedback for improving performanceBeing Non-Judgmental – responding with an assessment mindsetSeeking Assessment – analyzing past performance to improve future performanceGeneralizing Solutions – modify for broader applicability 1. Work in groups of two. 2. Exchange homework papers and identify all answers that differ from your answers,

or any answers/solutions about which you are uncertain. Initial your comments. 3. Identify one strength (S) and one area for improvement (I) for each of the following

dimensions of homework quality. Provide supporting details for your conclusions. Explain *why* each strength is valuable. Explain *how* to implement improvement.

Sketching/Diagramming

S:

I:



Identification of Assumptions (defining variables w/units and process constraints)

S:

I:

Labeling and Use of Governing Equations (including reasoning w/equations)

S:

I:

Explicit Solution Pathway (no gaps between problem definition and results)

S:

I:

Presentation of Compelling Solutions (well-identified, correct answers w/units)

S:

I: Validation of Results (independent methods to check results and unit consistency)



S:

I:

Reflection on Results (thoughts about implications of results and lessons learned)

S:

I:

4. What are two things you discovered about your own work while performing this homework assessment?

5. When you are finished with the above tasks, orally present your observations and comments to the person you assessed. Use your written notes.

6. As a team, compose responses to following questions.

a) What are the two most important questions youwould like to have resolved about this assignment?



b) What relative weight would you give to each of the followingdimensions in the rubric you are given for documenting engineeringproblem solving/calculations?

___ Sketching

___ Problem Definition

___ Assumptions

___ Governing Equations

___ Software Usage/Documentation

___ Solution

___ Validation

___ Reflection on Results


WORKSHOP: ANALYTIC RUBRIC FOR TEACHING, … rubric increases consistency and cuts down on instructor...

Documents

Transcript of WORKSHOP: ANALYTIC RUBRIC FOR TEACHING, … rubric increases consistency and cuts down on instructor...