STUDENT EVALUATION OF FACULTY: Policies and Processes Kay Turpin Alison Joseph Western Carolina...
21
STUDENT EVALUATION OF FACULTY: Policies and Processes Kay Turpin Alison Joseph Western Carolina University SAIR 2010 New Orleans, LA
-
Upload
hector-armstrong -
Category
Documents
-
view
217 -
download
4
Transcript of STUDENT EVALUATION OF FACULTY: Policies and Processes Kay Turpin Alison Joseph Western Carolina...
STUDENT EVALUATION
OF FACULTY:Policies and Processes
Kay TurpinAlison Joseph
Western Carolina University
SAIR 2010 New Orleans, LA
Please…turn off cells phones and other devices
if you must leave the session early – leave quietly
avoid side conversations
SAIR 2010 New Orleans, LA
This presentation is available at:http://www.wcu.edu/27729.asp
Location: Cullowhee, NC
Member of UNC System
Fall 2010 enrollment - 9407
1440 FTF7503 UGs1904 GRs
Western Carolina University
SAIR 2010 New Orleans, LA
History of Evaluations at WCUNo university-wide policy on administration,
instruments, courses covered, and use of resultsScanned, paper forms administered during class timeLong wait for resultsNo flexibility in types of reportsSecurity dismal – esp. commentsIR office:
Purdue University’s Cafeteria Evaluation System7-12 departmentsselect undergraduate coursesinefficient, time-consuming process
SAIR 2010 New Orleans, LA
Planning for University-Wide Evaluations2002 through 2006
Faculty Senate estab. ‘teaching evaluation committee’Provost mandated online evaluations (PACE initiative)Developed SAI instruments based on course typesDeveloped guidelines for results usageInitial discussion on admin. evals and accessing
resultsResearched available options for administering evals
and selected Academic Management System’s CoursEval
Faculty “own” the evaluation process!SAIR 2010 New Orleans, LA
Course Evaluation Pilot – Spring 2007
• 5 departments (Mkt/BLaw, Chem/Phys, CJ, PoliSci, Psych)
• 328 courses (5689 surveys)
• 45% response rate
• Problems: login & email issues inexperienced staff oversaw administration
• End Result: CoursEval online system worked fine BUT….
SAIR 2010 New Orleans, LA
Full Implementation – Fall 2007
late startno scheduling guidelineskey people not involvedminimal error checkinglack of advertising and
incentives
inappropriate default forms
login issues from Spring pilot not addressed
questions added, doubling length
44% response rate
Began Fall without established policies and processes in place for administering the survey
and interpreting results
These issues would directly affect acceptance of online evaluations
by both faculty and students!SAIR 2010 New Orleans, LA
Establishing Policies & Processes
Faculty Senate
scheduling guidelineslow-enrolled coursesstudent incentivesrefined SAI
instrumentsplanned validity study
Implementation Committee
advertising/incentivesfaculty trainingsummer parts-of-terminstrument defaultsstudent email policylogin issues
Dec. 2007 through Summer 2008
SAIR 2010 New Orleans, LA
Spring 2007: 45% response rate (Pilot)
Fall 2007: 44% response rate
Were these major or minor issues?
Spring 2008: 33% response rate (Drop of 11%!)
Was our effort worth it?
Fall 2008: 46% response rate
Implementation Review
SAIR 2010 New Orleans, LA
Lesson 1: Involve everyone with a stake in the evaluation procedures and results; everyone with experience in processing & administering evaluations
Lesson 2: Consider all issues before implementing
Lesson 3: Build a strong communication system between faculty and survey administrators
Lessons Learned
SAIR 2010 New Orleans, LA
Overview of Current Policies
all course sections, all termsopen period set by course lengthuniversity-wide incentives allowedlow-enrolled courses identifiedcrosslisted coursesperiodic response rates & student
remindersfaculty evaluation (not course content)results availabilityuse in faculty AFE/TPR processIR office responsible for administration
SAIR 2010 New Orleans, LA
IR Responsibilities
Set-up & administer each semester’s evaluations
“Help desk” for faculty and studentsPursue new technologies & procedures
to improve process, reporting, and response rates
Communicate to Faculty Senate any issues arising affecting senate policies and survey administration
EVERYTHING!
SAIR 2010 New Orleans, LA
Evaluation Set-up (each term) – Back EndSet evaluation schedule for all coursesRequest SAI instrument modifications
SAIR 2010 New Orleans, LA
Default instrument examples:
1) instructional method of ‘OA=online activity’ theinstrument “Online Form” is default
2) instructional method of ‘F=face-to-face’ & course type ‘L=lab’
the instrument “Lab Form” is default
TERM YR COURSE SECT CRSTYPE INSTMETH XLISTCODE FACULTY Preferred Survey Form DEFAULTINSTR
FALL 2010 ATTR221 01 LEC F Scifers, James 1. Standard Lecture Form
FALL 2010 ATTR221 75 LAB F Scifers, James 2. Lab Form
FALL 2010 ATTR323 01 LEC F Scifers, James 1. Standard Lecture Form
FALL 2010 ATTR323 30 LAB F Scifers, James 2. Lab Form
FALL 2010 EDM322 59 LEC OA 0Z Berry, Robert 3. Online Form
FALL 2010 EMC445 50 LEC OA Hubble, Michael 3. Online Form
SAIR 2010 New Orleans, LA
Evaluation Set-up (each term)Set evaluation schedule for all coursesRequest SAI instrument modificationsData extracts from Banner (withdrawals &
grades)Error checks in Access databaseUpload files to CoursEval website:
faculty recordsstudents recordscourse information (includes primary instructor
only)secondary faculty (not included in course info.)enrollmentSAIR 2010 New
Orleans, LA
Evaluation Set-up (each term) – Front EndAssign instrument to coursesEstablish open/close dates and times –
automatedSet-up group emails for students/faculty –
automatedSet release date for survey results -
automated
For the main evaluation period each term – total time for front end set-up = approx. 4-5 hoursSAIR 2010 New
Orleans, LA
Features of CoursEval
Course Designator – helps us match instruments to courses
Students and Faculty – Find all associated courses and surveys
Consolidation of peopleActive Directory loginAutomated emails, open/close, results releaseResponse rates – real time, easy to locate and
SAIR 2010 New Orleans, LA
Features We Would Like
Better reporting – more flexibilityAdvance notice of outages and problemsAbility to access individual responsesFaculty member attached to course cannot be
changedSingle sign-on with other systems
(Blackboard)Separate surveys for each instrument and
time period add to workload (84 surveys for Summer 2010)
SAIR 2010 New Orleans, LA
End ResultsConsistent open periodsNo lost class timeSingle administrator; increased efficiencyAll faculty/courses under same policies100% class coverageResults available quickly, better reports, survey
historyCosts: $40,000 vs $84,000+ annuallyLower response rate; more thoughtful commentsHigh level of securityEnvironmentally friendly
SAIR 2010 New Orleans, LA
Future Plans and Ongoing Issues
Improve response ratesImplement Blackboard plug-inImprove Reports Available
Improve process to change SAI instruments:Set-up Sharepoint site for department headsUse Banner and Blackboard to automate
changes
SAIR 2010 New Orleans, LA
WCU’s Course Evaluation website:http://www.wcu.edu/8356.asp“Guidelines & Procedures for Administration & Oversight of Student Assessment of Instruction”
Institutional Planning & Effectiveness: http://www.wcu.edu/12829.asp 828-227-7239Kay Turpin [email protected] 828-227-3041Alison Joseph [email protected] 828-227-3042
AMS website:http://www.academicmanagement.com/
