Peer evaluation in TBL: A Peer evaluation in TBL: A student-developed student-developed approachapproach
Derek R. Lane, Ph.D.Associate DeanCollege of Communications and Information Studies H. Lester Reynolds Engineering Endowed Professor
June 1, 2007 9:45 - 10:45 Room C150
TBL Conference 2007TBL Conference 2007REFLECTIONSREFLECTIONS
People have problems working together TBL is distinctive and importantDee Fink
“Students don’t like doing peer evaluations.”Interesting examples of TBL Self/Peer Assessment
Cooperative Learning Skills Self-directed Learning Interpersonal Skills Rudolph Navari
TBL Conference 2007TBL Conference 2007REFLECTIONSREFLECTIONS
iPeer 2.0 can help facilitate TBL peer evaluationsSophie Spiridonoff / Jim Sibley (UBC)
Medical students have mixed opinions about satisfaction with peer evaluation Dean Parmlee, Dan DeStephen, Nicole Borges (Wright
State U.)
What Makes a Team Work?
“Individual commitment to a group effort: this is what makes a team work, a company work, a society work, and a civilization work.”
--Vince Lombardi,Former Green Bay Packers Coach
Driving Questions
RQ1What is the purpose of peer evaluation?
RQ2 How do we improve our methods of helping students assess professionalism?
Students support that which they help to create.
Session Overview
I. Purpose of Peer Evaluation: A TBL Retrospective
II. Characteristics of High Performing Teams
III. Importance of Helpful Feedback
IV. Strategies for Establishing Meaningful Formative (PROCESS) and Summative (OUTCOME) Assessment
I. Purpose of Peer Evaluation in TBL
A TBL Retrospective
TBL Retrospective
Michaelsen, L.K., & Black, R. H. (1994). Building Learning Teams: The Key To Harnessing the Power of Small Groups in Higher Education. Growth Partners.
Performance Evaluation
Three Essential Components of Grading System Individual Performance
Individual Accountability for Reading/Homework
Assess Higher-Level Thinking Skills Group Performance
Provides Incentives to Support Cohesiveness
Justification for Putting Effort into Group Work
Peer Evaluation
Peer Evaluation
Solves Important Motivational Problems
Provides Incentives for Equitable Group Participation
Removes students’ fear that they will have to choose between receiving a low grade on the group assignments and having to “carry” group work (when other group members fail to do their fair share)
Performance Evaluation
Three Essential Components of Grading System Individual Performance Group Performance Peer Evaluation
Give sufficient weight to each of the components
Must be clear to students that the instructor thinks each is important (minimum 10%)
Must be responsive to student concerns for fairness
and equity (reduce social loafing)
Necessary Components of Necessary Components of TBL TasksTBL Tasks
Heterogeneous composition of diverse interdependent work teamsClear, specific, and widely shared team goals which encourage group cohesionSufficiently difficult and meaningful team activities that do not allow one person to accomplish the task alone.Regular, descriptive, specific, relevant, timely and usable internal peer feedbackExternal comparisons which are emphasized through immediate and ongoing feedback about organizational performances relative to other teams.
II. Characteristics of High Performing Teams
Team Performance and Effectiveness
In a well-run team, the overall performance is superior to that of individual efforts “The whole is greater than a sum of its
parts”
In a poorly run team, the overall performance is worse than what would be obtained by linearly combining the contributions of many individuals
The Wisdom of Teams: Creating the High Performance Organization Katzenbach & Smith ©1999
Six team basics define the discipline required for team performance:
1. Small Number (5-7; generally less than 12) 2. Complementary Skills3. Common Purpose4. Common Set of Specific Performance Goals5. Commonly Agreed Upon Working Approach
(structure, norms)6. Mutual Accountability
Team Effectiveness
PerformanceImpact
WorkingGroup
RealTeam
High-Performance Team
Pseudo-Team
PotentialTeam
The Team Performance Curve
From: Katzenbach & Smith, The Wisdom of Teams:Creating the High-Performance Organization.Used by permission.
Building Team Performance
Establish urgency, demand performance standards and directionMembers should be selected for skill and skill potential--not personalityPay particular attention to first meetings and actionsSet clear rules of behaviorSet immediate performance-oriented tasks and goalsChallenge the group with fresh facts and informationSpend lots of time togetherExploit the power of positive feedback, recognition, and reward
Katzenbach & Smith, 1993
Eight Characteristics of High Performance Teams
A Clear, Challenging Goal A Results-Driven Structure (clear roles,
effective communication) Competent Team Members Unified Commitment Collaborative Climate Standards of Excellence External Support and Recognition Effective Principled Leadership
LaFasto, F., & Larson, C. (2001). When teams work best: 6,000 team members and leaders tell what it takes to succeed. Thousand Oaks, CA: Sage.
III. Importance of Helpful Feedback
Formative vs. Summative Peer Evaluation Formative (process) peer evaluation
procedures can potentially provide a results-driven structure and establish the criteria or standards of excellence to ensure mutual accountability.
Summative (outcome) peer evaluations provide an overall standardized assessment of the extent to which individuals contributed to team success. (Helping Behavior)
IV. Strategies for Meaningful Assessment
TEAM FORMATIO
NDear God, please don’t make me work with THEM!
Assign a number from 1 - 10
Preliminary Individual Peer Evaluation Criteria
Reflect on past group experiencesConsider + - experiences What are the four or five most important issues that contributed to the group success or failure?Develop a list of preliminary criteria you believe are important for group success / peer evaluation
Preliminary List of Evaluative Criteria
Instructions: Develop a list of evaluative criteria that you believe is important for students to use when working in TBL instructional teams in order to be successful.
Final TBL Evaluative Criteria
Instructions: 1) Make introductions and share
contact information. 2) After introductions have been made
and contact information shared, decide on a name for your team. The final decision must be unanimous and the name should reflect positively on the qualifications of the individuals in your group. Provide the decision on the line below.
Group # ___ Team Name: __________________________________
Final TBL Evaluative Criteria
Share individual preliminary criteria with all group members. Consolidate the preliminary list of criteria and develop a final group list of the criteria that the group is comfortable using for peer evaluations.
Use the directions on the next page to develop procedures that incorporate the group criteria that will allow efficient feedback.
Submit the final peer evaluation procedures and criteria as well as the Group # and Firm Name along with all of the individual criteria sheets at the beginning of the next class.
Team Peer Evaluation Procedures & Criteria
Instructions: Before proceeding, read the four-page article written by Michaelsen and Schultheiss’ (1988) Making Feedback Helpful.
Develop a system for providing performance feedback to the members of your team. The performance feedback system should include:
1. Provide a statement of the team goals and objectives that you intend to achieve. These goals should reflect an integration of individual team members’ goals for the course.
2. A description of how your team intends to collect the data on which the feedback will be based. Please include a copy of a specific peer grading form that clearly indicates how data will be collected.
3. A description of the feedback process you intend to use. Please specify:a. When the feedback will be given? Be specific.b. Who will give it? Be specific.
4. Assess the difficulties that you are likely to encounter in implementing your performance feedback system. How will the team overcome these difficulties?
5. A statement of how the system provides input into the summative helping behavior grade at the end of the class.
Characteristics of Helpful Feedback
descriptive, not evaluative, and is owned by the sender (not anonymous)specific, not general (focus on specific issues and behaviors)honest and sincereexpressed in terms relevant to the self-perceived needs of the receivertimely and in contextdesired by the receiverusable; concerned with behavior over which the receiver has control
Michaelsen, L.K., & Schultheiss, E. E. (1988). Making feedback helpful. Organizational Behavior Teaching Review 13(3), 109-113.
Team Peer Evaluation Procedures & CriteriaYour performance feedback system will be evaluated using the
following criteria:
1. Is the team collecting data they will need to support the achievement of course goals?
2. Will the procedures they intend to use support the achievement of their objective(s)?
3. Are the procedures they intend to use consistent with the seven characteristics of helpful feedback?
4. Are the procedures practical (i.e., can they be implemented effectively given the specific situation in which they will be used)?
5. Have they accurately anticipated the problems they are likely to encounter in implementing their performance feedback system and have they developed strategies for overcoming the difficulties?
Key Elements Critical to Cooperation
Positive Interdependence individuals succeed only when group succeeds
Individual Accountability learn together to perform better as an individual
Promotive Interaction help, assist, support, encourage, praise efforts
Social Skills (Professionalism) communication, leadership, decision-making,
trust-building, and conflict management skills
Group Processing
Code of Cooperation
EVERY member is responsible for the team's progress and success.Attend all sessions and be on time.Listen to and show respect for the contributions of other members; be an active listener.Criticize ideas, not persons.Resolve conflicts constructively.Pay attention - avoid disruptive behavior.Avoid disruptive side conversations.Only one person speaks at a time.Everyone participates -- no one dominates.Be succinct, avoid long anecdotes and examples.No rank in the room.Attend to your personal comfort needs at any time but minimize team disruption.HAVE FUN! Adapted from the Boeing
Commercial Airplane Group
IU School of Medicine - South BendTBL Self/Peer Assessment Criteria
Cooperative Learning Skills Attendance Listening Interaction / Participation Information Sharing Defer to Needs of the Group
Self-Directed Learning Preparation Depth of Knowledge Identifies Limits of Knowledge Problem Solving Confidence
Interpersonal Skills Gives instructive feedback Accepts instructive feedback Shows sensitivity and concerns for others Clarity in rationale/reasoning process
Jackson, Hamlett, & Navari
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Robert Herring, III
Winston-Salem State U.
TBL and the Business Strategy Game
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (LZW) decompressor
are needed to see this picture.
Levine, R. E., Kelly, A., Karakoc, T., & Haidet, P. (2007). Peer evaluation in a clinical clerkship: Student attitudes, experiences, and correlations with traditional assessments. Academic Psychiatry, 31(1), 19-24.
Who seeks formative feedback?
360 Year 3 Undergraduate Medical Students at University of Aberdeen, UK
Less than half the students (46%) collected their formative feedback.
Females (p=.004) and higher achievers (p=.020) were more likely to seek formative feedback.
Sinclair & Cleland (June, 2007)Journal of Medical Education
Civil Engineering Design Team Criteria
Attendance Punctuality Attitude Respect Preparedness Completed Readings Completed Homework Ready for Meetings Preparation Goal Oriented Flexibility Quality Work Completed the First Time Accountability Trustworthy and Reliable Gives Proper Notice if Can’t Attend a Meeting Individual Initiative Team commitment Leadership Willingness to accept alternative viewpoints
Little and Cardenas (2001)
Quality of Technical Work
Ability to Communicate
Ability to Provide Leadership
Commitment to Team and Project
Demonstrated Effectiveness
Helping Behavior
Summative (outcome) peer evaluations provide an overall standardized assessment of the extent to which individuals contributed to team success. (Helping Behavior)
Standardized base 10 n-1 X 10 (6-1)*10 = 50 pts to assign Mean of group scores
Percent multiplier 100 point distribution (peer*GRAT)
Successful Teams
employ self examination and assessmentchange basic operating assumptions when necessarymonitor their own interaction patterns and progressare willing to make changesare more attentive to group processesbegin their discussion by attempting to analyze the problem before trying to search for viable solutions
Session Summary
I. Purpose of Peer Evaluation: A TBL Retrospective
II. Characteristics of High Performing Teams
III. Importance of Helpful Feedback
IV. Strategies for Establishing Meaningful Formative (PROCESS) and Summative (OUTCOME) Assessment
Student-Developed Peer Evaluation Procedures & Criteria
Feedback—both formative (process) feedback and summative (helping behavior) feedback should be direct, clear, and ensure that group members are accountable to the rest of the team.
The value of the process feedback is that all of the members of the team have the opportunity to discuss the feedback and make performance improvements over the course of the semester.