Online Course Evaluations Course... · You are being asked to complete an online course evaluation...
Transcript of Online Course Evaluations Course... · You are being asked to complete an online course evaluation...
Electronic Course Evaluations
Presentation to Department Chairs January 31, 2013
Collaboration between USM Office of Academic Assessment and PCE • Susan King, Director of Academic Assessment • Dr. Monique LaRocque, Executive Director, PCE • Dr. Khusro Kidwai, Director of Online Teaching and Learning
How did the USM Electronic Course Evaluation Initiative begin? • Faculty requests for an online course evaluation option • Increasing number of online and blended courses
Agenda
• History of course evaluations at USM • Results from Summer 2012 Pilot • Comparison: Paper-based vs. Electronic • Next Steps
History of Course Evaluations
1980 USM course evaluations (Green Form) were designed by faculty
2003 CONHP adopted the SIR-II course evaluation
2006 SIR-II option available University wide
2008 Scanning software: mainframe-based --> PC-based
2012 Piloted electronic course evaluation
Issues with Current Paper-Based System
• Faculty are requesting a quicker turn around time, and additional breakdowns of
the data..…but the paper-based system depends upon the Opscan scanner equipment, which is outdated and archaic, scantools software is slow at processing reports.
• Faculty depend on their department personnel for the manual distribution,
collection, organization, and accurate completion of forms….errors on the header sheets creates errors in the output reports.
• Faculty are requesting an electronic option to use with all their courses, online
and face-to-face courses…for examining their own course improvement. • Faculty are not able to receive an overall compilation of their evaluation results
for peer review…if they teach in more than one department (the use of varied forms across campus, Green Form, SIR-II form cannot be aggregated).
Paper-Based Process & Annual Costs
Costs
45,000 Scantron forms (both SIR-II and Green) $18,000
Header sheets
$1,250
Scantools software and scanner maintenance $2,500
Update scanner equipment (5-6 years @$7,000)
$1,250
Total Personnel costs (prepare header sheets; distribute, collect, scan and process evaluation forms)
$20,000
Total Annual Costs $43,000
Electronic Course Evaluation Pilot
Summer 2012 (Faculty: 91 | Students: 1357)
Student Interface
Automated Email to Students
Dear [first] [last],
You are being asked to complete an online course evaluation for your Summer courses. By clicking on the link
below you will be taken to a list of courses for which you have evaluations to complete. Click on the title of the
course to access the evaluation. Each course evaluation should take no longer than 5 minutes to complete.
Click here to access the evaluation or login at https://usm.campuslabs.com/courseeval
Thanks for your participation. If you have questions about the course evaluation process, please contact the
office listed below.
Office of Academic Assessment
General phone: 780-4383
Email: [email protected]
Mobile Interface
Mobile Interface
Faculty Interface
Reporting
Outcomes of Summer ’12 Pilot
• Demonstrated infrastructure to support electronic evaluation system • Minor technical glitches/problems can be worked out • Faculty pleased to receive their reports immediately after final grades • Response rate: 46% (low, but double compared to UC online survey, summer 2011)
Benefits of Electronic Version
• Quick turn-around; several breakdown reports available for faculty • Departments can choose their own questions; updating instrument is
convenient. • Students can provide feedback over a 1-2 week period; surveys can
be completed outside of class on mobile devices.
Comparison
Paper-based Electronic
Annual Cost $43K+ $31K
Personnel 1 halftime + multiple AAs 1 halftime
Turn-around time 6‒14 weeks Immediate
Instruments * Few variants Multiple
Reporting Simple Custom-designed
Student Experience Bubble sheets + A4 pencils Digital; multiple devices
Submission Window One-time Two-week window
Response rate * 65-75% ?
Response Rates at Other Institutions
Semester Institution Paper-based Online: Year 1 Incentive Online: Year 1+
FA 04 MS State 88% 42% Bookstore Voucher 97%
FA 05 Harvard 89% 65% View grades early 96%
FA 06 U of Kent 82% 46% University event tickets 95%
Paper-based
Transition period
Electronic
Res
po
nse
Rat
e
Strategies for Improving Response Rates
Common strategies being used at campuses: • Frequent email reminders, and announcements posted on campus • Allowing students to view grades at an earlier date • Incentives: campus raffle, gift cards, bookstore vouchers, tickets
for campus events, coupons for food • Posting how the evaluation results are being used on campus • Students able to use a mobile device for completing their evals • More instructor involvement….emphasizing the importance of
student feedback for class improvement
Next Steps
Next Steps
Platform Instrument Development
Possible Model for an Electronic
Course Evaluation System
University-wide Items
Department Items Each department selects items, in addition to Green Form items
Using all or some items on the USM’s Green Evaluation Form
USM Course Evaluation Landscape
Green Form
University-wide Items (Green Form)
Department Items Department Items
Custom SIR II (proprietary)
{paper-based} {electronic}
Web-based App’s
Phase 1 – Department Chairs Phase 2 – Faculty
Two electronic options available: • Green Form only • Green Form with additional Dept. items
• Faculty will receive a link to the Phase 2 App
• Faculty will be able to create/choose/vote from a pool of items.
• The items are agreed upon by the Department
• Items are added to the electronic course evaluation
Next Steps
January
• Meetings with USM Department Chairs
• Department Chairs App distributed for discussion
February
• Departments make decisions about electronic form
• Faculty App distributed to Departments who want additional
items via electronic version
• Options for vendors explored and in-house options
March
• For those participating…Departments make decisions
about their items
• Vendors and in-house options selected
April
• Remaining Departmental responses collected to create a
draft electronic course evaluation format
• Begin planning the Summer ‘13 pilot
• If desired by Departments, further discussion about fall ‘13
course evaluation plans.
Discussion