The role of course evalsin faculty reviews:
One perspective
John PetraitisCAFE
March 25, 2011
Summative vs. Formative
• Summative: –Course evals as ends–1st page of IDEA report
• Formative/Developmental–Course evals as means–Remaining pages of IDEA
Logic of IDEA• One size does not fit all
–Allows faculty to identify key objectives–E.g., exploration of personal value, vs.
creativity vs. understanding principles• Advertise the warts
–Reliability and representativeness
Three Key Pieces
• 12-item Faculty Info Form (FIF)
• 36-item Student Reaction to Instruction Form (SRIF)
• Multi-page Report
FIF
• 12 learning objectives
FIF• Which of the 12
objectives do I chose?– Depends on course– Depends on instructor– Depends on section– Depends on dept
• Recipes for disaster– Choosing all of them– Choosing none of them
(by not filling out FIF)
SRIF• Two kinds of Q’s
– 36 questions linked to 12 learning objectives.
– Extraneous influences, e.g.,
• Discipline• Student motivation• Class size
Marriage of FIF &SRIF• Weighting of
students responses (on SRIF) for results/report
• M = 0• I = 1• E = 2
The Report (page 1)
• Reliability and Representativeness– Reliability
• Based on number (not percent) of respondents• Would adding a few more respondents potentially alter the results
dramatically?
– Representativeness• Based on percent (not number) of respondents• 65% response rate is considered representative
The Report (page 1)
• Reliability and Representativeness– Reliability
• Based on number (not percent) of respondents• Would adding a few more respondents potentially alter the results
dramatically?
– Representativeness• Based on percent (not number) of respondents• 65% response rate is considered representative
The Report (page 1)
• Reliability and Representativeness– Representativeness is key– If representative, reviewers could use page 1 as
summative and formative– If not representative, reviewer might use if it
helps the candidate
• Averages:– Section A: “I” and “E” objectives
• Adjustments to IDEA Ratings– Discipline (national comparisons)– “Extraneous Influences” on
student ratings are used to adjust scores or ‘level the playing field’
• SRIF Item #39: I really wanted to take this course regardless of who taught it
• SRIF Item #43: As a rule, I put forth more effort than other students on academic work
• Class size
The Report (page 1)
• Improving Teaching Effectiveness (food for thought)
The Report (p. 2-3)
IDEA and your review file• Do your own formative evaluation
– What can you learn from pages 2-3?– Show your interest in professional
development– Do it even if IDEA is not reliable or not
representative• Do your own summative evaluation
– Make it easy. Make it clear.
PSY A260
PSY A375
PSY A412
All Courses
0 10 20 30 40 50 60 70 80
Comparison of IDEA evaluations for J. Petraitis to National Averages in Psychology
National Average in PSY A. Progress on relevant objectives B. Overall Rating: Excellent teacherC. Overall Rating: Excellent course D. Average of B & C Summary Evaluation (Av. of A & D)
…instructor clearly communicates objectives…
…agreement betw course objectives & what's taught.
…communicates how students are evaluated.
…agreement betw announced and used eval methods.
…instructor gives periodic feedback.
…prepared for class.
…meets/dismisses class on time.
…assignments are helpful…
…exams represent course objectives/materials.
…instructor is helpful with problems/questions…
…instructor is available for consultation…
…teaching aids (e.g., A/V, materials) are helpful…
…questions/comments are respected…
…instructor creates enthusiasm/desire to learn.
…instructor speaks clearly…
…instructional materials are excellent.
…conceptual learning, not rote memorization.
…recommend instructor to other students.
…glad I took course; would recommend it…
…instructor uses class time efficiently.
…attendence is critical for success.
…friendly, relaxed atmosphere….
…easily follow instructor & take notes.
…instructor is sensitive to student problems.
…workload (1=much lighter, 3=same, 5=much more)
…hours/wk (1=0, 2<2, 3=3to4, 4=4to7, 5>7)
2.5 3 3.5 4 4.5 5
CAS-SSPsy DeptAll Petraitis
Top Related