CA12 Assessing Online Courses

Howard UniversitySpring 2015

Workshop ObjectivesUpon completing this workshop, you will be able to do the following:

1. Explain how the assessment of online courses differs from that of “in-person” courses.

2. List the benefits of assessing online courses.3. Access an online course on Blackboard.4. Assess the design of an online course, using the

Quality Matters rubric.5. Assess course delivery in an online course.

Definitions• In-person (face-to-face in classroom)• Hybrid* (30-79% online with

reduced classroom time)• DL (80-100% online)• Online (Hybrid* and DL)

• How many of you have taught an online course?

• How many of you have taught with Blackboard?

*aka “Blended”

Why Assess Online Courses?

Reaccreditation

Better Student Learning

Better Teaching

How Does Assessment Differ for “In-Person” vs. Online Courses?

“In-Person”• Students

– Student evaluations– Student performance

• Peers– Review of course materials– Classroom observation

• Administrators+ Implementation of policies

Online• Students

– Student evaluations (online with additional questions)

– Student performance

• Peers– Review of course materials– Course site visit

• Administrators+ Implementation of policies,

including HU Distance Ed Policy

Why Should Peer Reviewers Visit the Coursesite?

• The syllabus alone cannot reveal enough about the course design and delivery.

• Poor course design or delivery can lead to high student attrition rates and low student achievement.

• Careful peer reviews of coursesites help faculty improve their online courses.

Who Should Visit the Course?

QM• A team of three reviewers• All experienced QM-

certified online instructors• One Master Reviewer• One subject-matter expert

(SME)• One external reviewer

HU• A team of three reviewers• CETLA DL-certified online

instructors• One CETLA-certified

Reviewer (if available)• At least one subject-matter

expert (for departmental reviews)

Note: Visitors should request background information from the instructor (see the QM Instructor’s Worksheet) prior to the visit.

How Can You Visit a Course?

• Ask the instructor to add you as a student.

• Ask the instructor to make all assignments and tests available—password-protected if necessary.

• Log into http://blackboard.howard.edu

• If you can’t log in, contact Howard’s Help Desk (helpdesk@howard.edu or 202-806-2020).

What Affects the Quality of Online Courses?

• Technology• Administrative

infrastructure• Faculty training• Student readiness

• Course content• Course design• Course delivery

Why Is Assessing Course Design So Important in an Online Course?

“When learners can’t find what they need or are confused about where to go and what to do, it is harder for them to learn. Being an online learner is challenging enough without these additional barriers. Plus, frustrated learners tend to either drop out or drive the instructor crazy….”Patti Shank

How Can You Assess Course Design?

Use the Quality Matters (QM) rubric!• Based on research, best practices, and instructional design

principles• Aligned with national standards (e.g., Sloan Consortium

Pillars)• Aligned with accreditation standards for distance-learning

(e.g., Middle States)• Consistent with HU Provost’s syllabus guidelines

What Is the QM Rubric?

Rubric StandardsAnnotated Rubric

How Do You Review a Course?

• Read the annotation.• Look for evidence in the course—the site as well

as the syllabus.• Decide whether the standard has been met

according to the “85% Rule.”• Write a recommendation that is sensitive,

balanced, constructive, specific, and measurable.

How Do You Rate a Course?

• Award all points for a standard if a course meets at least 85% of the requirements for the standard; do not give partial credit.

• If at least two reviewers agree that a course meets a standard, award all points.

• If at least two reviewers agree that a course does not meet a standard, do not award any points.

• A course must meet all 3-point standards.• An effective course will earn 84 of the 99 points (85%).

Now you try it!Log into the CA12 Practice Review Site.

Standard 1.1 “Instructions make clear how to get started and where to find various course components.”

1. Not met2. Met

Standard 5.2 “Learning activities provide opportunities for interaction that support active learning.”

1. Not met2. Met

Standard 8.1 “Course navigation facilitates ease of use.”

1. Not met2. Met

Why Is Assessing Course Delivery So Important in an Online Course?

Watch this video:http://www.youtube.com/watch?v=KgdzpIs4uAI&feature=player_embedded

How Can You Assess Course Delivery?

• Ask the instructor to generate two Blackboard course reports (click here for instructions):– “Overall Summary of User Activity” for all

students– “Overall Summary of User Activity” for the

instructor • Visit the coursesite.

What Should You Look for in the Report?

What to Assess1. Instructor Presence &

Availability2. Instructor Feedback3. Instructor Participation

What to Check• Calendar statistics

• Tool statistics Which?• Tool statistics Which?

© The Learning House, 2013

What Should You Look for in the Report?

What to Assess1. Student Retention

2. Student Participation

What to Check• Calendar statistics• Tool statistics Which?

• Tool statistics Which?

© The Learning House, 2013

What Should You Look for in the Site?

What to Assess1. Instructor Presence &

Availability2. Instructor Feedback

3. Student Retention

What to Check1. Announcements What else?2. Discussion Board What else?

3. Discussion Board What else?

Note: In your “student” role, you cannot access the instructor’s grade book, Performance Dashboard, or Early Warning System.

© The Learning House, 2013

What Should You Look for in the Site?

What to Assess*4. Instructor Participation

5. Communication of University and/or Course Policies

6. Pacing

What to Check4. Discussion Board What else?

5. Syllabus What else?

6. Assignments What else?

© The Learning House, 2013