Post on 17-Jan-2016
How to Evaluate Online Teaching & Improve Faculty DevelopmentThomas J. Tobin, B. Jean Mandernach, & Ann H. TaylorOLC International Conference, Orlando - October 16, 2015
Our Live WebcastTweet your questions and comments to
@OLCTodayusing hashtags
#olc15#OnlineQuality
Our session evaluation:
goo.gl/v6kiaPImage © 2008 Anthony Quintano. Used under Creative Commons BY license from Flickr.com
The goal
Support Faculty
Improve Teaching
Enhance Student Learning
The challenge
Evaluation of Teaching
Faculty Development
1.Identify Desired Results
• How do successful online/blended educators look/act?• What are the skills you want them to have?• What competencies should they master?
2.Determine Acceptable Evidence of Learning
• How will you and your faculty know if they’ve mastered those skills and competencies?
3.Plan Learning Experiences
• How will you help prepare them for success?
Start at the End: Backward Design
6
Integration of services and supportTraining
Mentoring
EVALUATION
Coaching
7
self
students
peers
Admini-strators
Holistic decision-making
training
mentoring
evaluationdevelopment
coaching
ask look
Six Strategies for Evaluating Online Teaching
1. Know your context.2. Study the foundations of evaluation & analytics.3. Implement formative data & evaluation first.4. Automate summative methods next.5. Create a holistic campus plan for
implementation.6. Implement and cycle the analytics plan.
How do faculty know if they
are effective in the online classroom?
• Are students learning?
Student Learning
Indicators of Effectiveness
Online Learning Experience Impact Factors
Institutional Resources
Program & Curriculum
Course Content & Design
Teaching BehaviorsInstitutional Context
Student Characteristics
Technology
Faculty Impact
Teaching Behaviors
Course Content & Design
Six Strategies for Evaluating Online Teaching
1. Know your context.2. Study the foundations of evaluation & analytics.3. Implement formative data & evaluation first.4. Automate summative methods next.5. Create a holistic campus plan for
implementation.6. Implement and cycle the analytics plan.
1. Encourage student-faculty contact
2. Develop reciprocity and cooperation among students
3. Use active learning techniques4. Give prompt feedback5. Emphasize time-on-task6. Communicate high expectations7. Respect diverse talents and
ways of learning
Principles of Good Practice: Technology
Chickering & Gamson (1987)Chickering & Ehrman (1996)
1. Attend to unique challenges of distance learning2. Be familiar with unique learning needs3. Master course content, structure, organization4. Respond to student inquiries5. Provide detailed feedback6. Communicate effectively7. Promote a safe learning environment8. Monitor student progress9. Communicate course goals10.Provide evidence of teaching presence
Faculty Competencies for Online Teaching
Pedagogical
Technical
Administrative
Penn State Competencies for Online Teaching Success
Measurable Online Teaching Behaviors
• Do you log in at least every other day?• Have you posted a biography of at least a paragraph, in addition to contact info?
• Do you post announcements at least weekly?
• Do you answer students in two days or less?
• Do you participate in discussion forums?• Do you give detailed assignment feedback?
(Piña and Bohn, 2014)Image ©2010 Steve Harris. Used under Creative Commons BY license from Flickr.com
• Who has access to this information?• How is the information used to
improve faculty support and programming?
• Who is responsible for evaluation of online teaching? Who is responsible for faculty development?
What standards, competencies, and expectations exist for “good” online teaching at your institution?
Six Strategies for Evaluating Online Teaching
1. Know your context.2. Study the foundations of evaluation & analytics.3. Implement formative data & evaluation first.4. Automate summative methods next.5. Create a holistic campus plan for
implementation.6. Implement and cycle the analytics plan.
Formative Evaluation:The SCARF Loop
Image ©2014 Thomas J. Tobin. Used under Creative Commons BY-SA license.
• improve the educational experiences you provide for your students,
• identify the professional education you need to further develop your capacity to teach well,
• prepare for your performance review with your supervisor, and
• assess your readiness to apply for promotion and tenure.
Image ©2011 F Delventhal. Used under Creative Commons BY license from Flickr.com
Self-evaluation
Formative: Finding the Right Tool
Techniques for Students• Surveys• One-minute papers• Muddiest point• “Suggestion box”• Anonymous tip line
Techniques for Instructors• Self reviews
• Peer reviewsTools• Qualtrics • SurveyMonkey• Survey Gold• Google Template• Email• Your LMS• Administrative
assistant or TA
Six Strategies for Evaluating Online Teaching
1. Know your context.2. Study the foundations of evaluation & analytics.3. Implement formative data & evaluation first.4. Automate summative methods next.5. Create a holistic campus plan for
implementation.6. Implement and cycle the analytics plan.
Student Ratings of Teaching Effectiveness• SEEQ• SALG• SRTE
Data Analytics• LMSs and other technology-based teaching tools
Administrative Evaluations• Checklist for Online Interactive Learning (COIL)
• Quality Online Course Initiative (QOCI)
• Quality Matters (QM) Rubric
• Online Instructor Evaluation System (OIES): Park University
• Peer Review of Online Teaching: Penn State
Summative: Finding the Right Tool
• What can you do to promote formative evaluation?
• How can you utilize summative evaluation to offer more effective and/or targeted faculty development initiatives?
• What other campus groups, organizations or departments can you work with to integrate evaluation and faculty support services?
How can you use evaluation information (formative or summative) to enhance faculty support?
Six Strategies for Evaluating Online Teaching
1. Know your context.2. Study the foundations of evaluation & analytics.3. Implement formative data & evaluation first.4. Automate summative methods next.5. Create a holistic campus plan for
implementation.6. Implement and cycle the analytics plan.
Campus Policy
Create campus policies and procedures for using online data for formative and summative teaching evaluation.
Annual Review, Merit PayStudent ratingsSelf-ratingsTeaching scholarshipAdministrator ratings
Promotion and Tenure
Administrator ratingsTeaching portfolio
Program Decisions
Student ratingsExit and alumni ratingsEmployer ratings
Six Strategies for Evaluating Online Teaching
1. Know your context.2. Study the foundations of evaluation & analytics.3. Implement formative data & evaluation first.4. Automate summative methods next.5. Create a holistic campus plan for
implementation.6. Implement and cycle the analytics plan.
The Teaching-Learning Lifecycle
Integrate teaching evaluations and analytics to continuously gauge and foster effective online instruction (minus the wires).
Image ©2006 Glogger~commonswiki. Used under Creative Commons BY-SA license from Wikimedia.com
Faculty members want informal learning opportunities, flexible scheduling, short sessions, and one-on-one support.
There isn’t a “best time.” Faculty members want anytime, anywhere professional development.
Rely on your assessment of your own faculty needs when designing the “right” program and resources for your institution.
• What evaluations are currently done at your institution that can inform faculty development initiatives?
• What evaluations can you integrate to more effectively support faculty?
How can you integrate evaluation and faculty development to create more holistic support for faculty at your institution?
Questions and Comments
Thank You!
Let’s Continue the Discussion
Thomas J. Tobint-tobin@neiu.edu
B. Jean Mandernachjean.mandernach@gcu.edu
Ann H. Tayloranntaylor@psu.edu
@OLCToday #olc15 #OnlineQuality
Our Live WebcastTweet your questions and comments to
@OLCTodayusing hashtags
#olc15#OnlineQuality
Our session evaluation:
goo.gl/v6kiaPImage © 2008 Anthony Quintano. Used under Creative Commons BY license from Flickr.com