QUALITY ASSURANCE OF ONLINE LEARNING · 12‐8‐11 CWU Quality Assurance of Online Learning Page 3...

QUALITY ASSURANCE OF ONLINE LEARNING

December 28, 2011

As Prepared by

Dr. Tracy L. Pellett Dr. Tom Henderson Dr. Chris Schedler

12‐8‐11 CWU Quality Assurance of Online Learning of 43

TABLE OF CONTENTS

EXECUTIVE SUMMARY …………………………………………………………………….. p. 3 I. STUDENT EVALUATIONS OF INSTRUCTION……………………………………

p. 4

A. SEOI STUDY 1 …………………………………………………………………………. p. 4 B. SEOI STUDY 2 …………………………………………………………………………. p. 6 II. DFW RATES …………………………………………………………………………………

p. 7

III. NOEL‐LEVITZ PRIORITY SURVEY for ONLINE LEARNERS ………………

p. 8

IV. COURSE REVIEW ………………………………………………………………………..

p. 10

TABLE OF APPENDICES

APPENDIX I – FORM A SEOI ………………………………………………. p. 11 APPENDIX II – FORM A2010 SEOI ………………………………………

p. 15

APPENDIX III – FORM W2010 SEOI …………………………………...

p. 18

APPENDIX IV – A MOCK‐UP OF THE NOEL‐LEVITZ PRIORITY SURVEY FOR ONLINE LEARNERS…………………………

p. 22

APPENDIX V ‐ NOEL‐LEVITZ PRIORITY SURVEY FOR ONLINE LEARNERS YEAR‐TO‐YEAR REPORT 2009 to 2010………………

p. 31

APPENDIX VI – GUIDELINE FOR ONLINE COURSE DEVELOPMENT …………………………………………………………….…

p. 35

APPENDIX VII – FACE‐TO‐FACE AND ONLINE COURSE OBJECTIVES AND REQUIREMENTS COMPARISON FALL 2011 ……………………………………………………………………………….

p. 39


CWU ONLINE LEARNING QUALITY ASSURANCE EXECUTIVE SUMMARY

The highest priority of CWU’s human, physical, and financial resources is the success of its educational programs. It is important that the university ensure institutional, accreditation, and/or state licensing standards of quality and best practice. Currently, online program quality is assessed in the following ways:

• Analysis of Student Evaluation of Instruction (SEOI) – online courses are compared to face-to-face courses to measure differences in perceived teaching skill, outcome obtainment, and course effectiveness. This is accomplished through quarterly comparison and review.

• DFW rates – online courses are compared to face-to-face courses to measure student success as based on the percentage of students receiving either a grade of D, F, or withdrawal (W). Results are fairly course specific, although generalizations can be made at college and university levels. This is accomplished through quarterly comparison and review.

• Noel Levitz: Priority Survey of Online Learners (PSOL) – national standardized survey administered to all online learning students to measure satisfaction with various aspects of system, support services and aspects of learning environment. This is accomplished through yearly administration and review.

• Course Review – Courses that have been developed through work-for-hire agreements have been reviewed

through a “best practice” rubric (Keinath and Blicker, 2003). Course review can occur prior to or after teaching. After an assessment is completed, faculty have the opportunity to work with the CWU online instructional design team until the course meets basic standards for all instructional areas.

SUMMARY

• CWU students rate instruction highly – for both online and face-to-face courses. Online students generally rate faculty lower than students completing face-to-face courses; however, the differences are very small.

• DFW rates are between 10% and 13% for both face-to-face and online courses during the regular academic year.

DFW rates tended to be generally lower for face-to-face courses as compared to online courses; however, differences are very small. DFW rates tend to be lower for face-to-face classes and higher for online courses in the summer as compared to the regular academic year.

• The Noel-Levitz PSOL reported that CWU students were satisfied for most elements related to online academic,

instructional, enrollment, and student services. There was also improvement in perceptions from 2009 to 2010. Although the number of course offerings was of greatest concern in 2009, tuition cost and investment was of greatest concern in terms of meeting student expectation in 2010.

• Course review has been used primarily for work-for-hire agreements. A revised rubric is being developed and will be piloted during the Spring 2012 quarter for greater departmental and programmatic use. Preliminary qualitative analysis of syllabi from corresponding face-to-face and online course sections revealed a parallel number and level of cognitive demand/skill requirements in relation to course outcomes. Face-to-face courses tended to have a significantly higher percentage of the total grade based on quizzes/exams than corresponding online sections. Online courses tend to have a higher percentage of the total grade based on discussion-based activities and higher writing requirements than their corresponding face-to-face sections.

Many of the assessment tools needed to measure the quality of online learning are in place. CWU has plans to add more and refine the current analysis as based on various demographic variables. The quality of online learning at CWU is, in general, good. The Office of Online Learning will continue to work with departments across campus to improve and refine the assessment and learning outcomes of its online programs.


FINDINGS

I. STUDENT EVALUATIONS OF INSTRUCTION (SEOIs)

The question of whether students evaluate courses taught online any differently than courses taught face-to-face (f2f) in a traditional classroom is important as to perception of teaching effectiveness. Results from three terms (fall 2010, winter 2011, and spring 2011) were analyzed. Two studies of SEOIs were conducted. In the first study, five similar questions between face-to-face and online courses using three different course evaluation forms (old version SEOI f2f, new version SEOI f2f, online SEOI) were analyzed. In the second study, two different course evaluation forms (new version SEOI f2f, online SEOI) were analyzed.

A. SEOI STUDY 1

Similar questions from three different SEOI course evaluation forms (old version f2f, new version f2f, online) were analyzed. All questions used a five-point scale where 5 was the highest or best response and 1 was the lowest response. More than 40,000 individual responses were compared. Table 1 identifies those questions that were somewhat similar and were compared across all forms. Charts 1-3 provide results across each quarter analyzed. Overall, CWU students rate instruction highly – for both online courses as well as face-to-face. However, students tended to rate face-to-face courses slightly higher than online, although the differences were slight and non-significant.

Table 1 – Similar Questions on Old Verson Form A, New Version Form A, Online Form

Case Old Version Form A (f2f) New Version Form A (f2f) and Online Form

A. 3. The instructor provided useful feedback on student progress.

2.g./h. instructor provided useful feedback on student progress?

B. 7. Extra help was available when needed. 1.f./g. if YES was the instructor available to provide help?

C. 15. Class time was used efficiently. 2.i./j. class sessions ( online activities) were well organized?

D. 16. Instructor was interested in whether students learned.

1.b. Instructor seemed genuinely interested in whether students learned?

E. 21. Evaluative and grading techniques (tests, papers, projects, etc.) were fair.

2.e./f. evaluation and grading techniques were clearly explained?


Chart 1. Course Evaluation Comparison Fall 2010*

Chart 2. Course Evaluation Comparison Winter 2011*

Chart 3. Course Evaluation Comparison Spring 2011*

*Form A is the “old” paper SEOI forms used for lecture courses that meet f2f Form A2010 is a new SEOI being field tested online for Biology, IET & ITAM courses that meet f2f Form W2010 is a new SEOI form used for 100% online courses

A B C D E

Form A 4.25 4.40 4.42 4.48 4.43

Form A2010 4.14 4.31 4.24 4.42 4.29

Form W2010 4.03 4.24 4.26 4.24 4.27

1

2

3

4

5

Fall 2010 ‐ Comparison of Averages for Similar SEOIs Questions

A B C D E

Form A 4.33 4.46 4.47 4.54 4.48

Form A2010 4.10 4.05 4.19 4.32 4.26

Form W2010 4.17 4.11 3.98 4.15 4.43

1

2

3

4

5

Winter 2011 ‐ Comparison of Averages for Similar SEOIs Questions

A B C D E

Form A 4.30 4.44 4.43 4.51 4.46

Form A2010 4.26 4.40 4.36 4.48 4.35

Form W2010 4.03 4.21 4.28 4.21 4.28

1

2

3

4

5

Spring 2011 ‐ Comparison of Averages for Similar SEOIs Questions


B. SEOI STUDY 2

Two different course evaluation forms (new version SEOI f2f, online SEOI) were analyzed. Both forms have 18 questions that are very similar. The comparable questions on both forms use a 5 point Likert scale. Strongly agree was assigned a value of 5, strongly disagree is assigned a value of 1. During the fall 2010 and winter 2011 terms, Biological Sciences (Biology) and the Information Technology and Administrative Management (ITAM) departments field tested new SEOI forms and administered them online to the courses that met face-to-face. During spring 2011, the Industrial Engineering Technology (IET) department also began field testing the forms. Table 2 reports the means, mean differences, and “effect sizes” for face-to-face courses vs. those taught online. Effect sizes (magnitude of mean differences) are larger when the difference in means is larger. An effect size around 0.2 is usually considered small; an effect size around 0.5 is medium, .8 is large.

Table 2 – Comparison of Online and Selected F2F Courses 1. STUDENT LEARNING ENVIRONMENT. To what extent do you agree or disagree that the...

The number of the online questions ison the left, the F2F number is on the right.

Online Mean

F2F Mean

Differ ‐ ence

Effect size

Online Mean

F2F Mean

Differ ‐ ence

Effect size

Online Mean

F2F Mean

Differ ‐ ence

Effect size

1.a./1.a.

instructor fostered a fair and respectful learning environment? 4.43 4.55 (0.12) 0.16 4.37 4.48 (0.11) 0.14 4.42 4.61 (0.19) 0.24

1.b./1.b.

instructor seemed genuinely concerned with whether students learned? 4.24 4.42 (0.18) 0.19 4.15 4.32 (0.17) 0.17 4.21 4.48 (0.27) 0.28

1.c./1.c.

standards of online behavior were clearly communicated and enforced? 4.40 4.46 (0.06) 0.08 4.34 4.43 (0.09) 0.11 4.40 4.52 (0.12) 0.15

1.g.If you sought help, did the instructor provide help? 4.24 4.31 (0.07) 0.08 4.11 4.28 (0.18) 0.18 4.21 4.40 (0.19) 0.21

2. TEACHING FOR STUDENT LEARNING. To what extent do you agree or disagree that the...2.a./2.a.

course objectives were clearly communicated? 4.45 4.43 0.02 0.02 4.35 4.40 (0.05) 0.06 4.45 4.50 (0.05) 0.07

2.b./2.b.

overall course content was presented in an understandable 4.36 4.26 0.10 0.11 4.25 4.22 0.04 0.04 4.38 4.37 0.01 0.01

2.e./2.d.

assignments and tests were connected to course content? 4.49 4.43 0.06 0.08 4.43 4.37 0.06 0.07 4.49 4.56 (0.07) 0.09

2.f./2.e.

evaluation and grading techniques were clearly explained? 4.27 4.29 (0.02) 0.02 4.16 4.20 (0.04) 0.04 4.28 4.35 (0.07) 0.08

2.g./2.f.

instructions for class activities were clearly communicated? 4.27 4.31 (0.04) 0.04 4.17 4.26 (0.09) 0.09 4.30 4.37 (0.07) 0.07

2.h./2.g.

instructor provided useful feedback on student work? 4.03 4.14 (0.10) 0.10 3.95 4.05 (0.10) 0.09 4.03 4.26 (0.23) 0.21

2.i./2.h.

instructor provided timely feedback on student progress? 3.99 4.17 (0.18) 0.17 3.98 4.10 (0.12) 0.10 3.93 4.24 (0.31) 0.27

2.j./2.i. online activities were well organized? 4.26 4.24 0.03 0.03 4.18 4.23 (0.05) 0.04 4.28 4.36 (0.08) 0.08

2.k./2.j.

work completed off‐line was useful in understanding course 4.27 4.27 0.00 0.00 4.19 4.19 0.00 0.00 4.32 4.33 (0.01) 0.02

2.l./2.k.

instructor encouraged students to connect course content to issues beyond the university classroom? 4.26 4.31 (0.05) 0.06 4.16 4.23 (0.07) 0.07 4.26 4.40 (0.14) 0.16

2.m./2.l.

course activities challenged students to think critically? 4.38 4.32 0.06 0.07 4.11 4.28 (0.17) 0.15 4.35 4.41 (0.06) 0.06

GENERAL INFORMATION. How would you compare this course with all other courses of similar credits at this level (i.e., 100, 200, 300, etc.) taken at CWU? 3.a./3.a.

amount of work OUTSIDE of online environment 3.41 3.52 (0.11) 0.13 3.46 3.69 (0.23) 0.27 3.46 3.67 (0.21) 0.28

3.b./3.b.

level of engagement/active learning while IN the online environment 3.29 3.47 (0.18) 0.20 3.33 3.51 (0.18) 0.19 3.33 3.59 (0.26) 0.29

3.c./ 3.c. intellectual challenge presented to you 3.45 3.56 (0.11) 0.12 3.41 3.69 (0.28) 0.29 3.41 3.68 (0.27) 0.24

FALL 2010 WINTER 2011 SPRING 2011


Overall, CWU students rate instruction highly – for both online courses as well as face-to-face. The learning environment, communication of course objectives, assignment and test connection to course content, and standards of behavior were rated higher than other questions/areas for both modalities while level of engagement, amount of work outside of class, and intellectual challenge were rated lower than other questions/areas. Face-to-face students tended to rate instructional elements higher than online students for 12 of 18 items, although the differences were slight. Effect sizes were small for all items. II. DFW RATES Student success is the highest priority of the university. Therefore, CWU works to provide its students with accessible, diverse, personalized, distinctive, and rigorous curricular offerings. This commitment extends to all students, irrespective of location and modality of instruction. Examining student success or lack thereof through grade analyses can allow differences in delivery modality to be examined in terms of outcomes attainment. Chart 4 compares all face-to-face and online courses in a term (e.g., underclass, upper-class, and graduate courses were all used to compute the average). Chart 5 compares DFW rates for comparable face-to-face and online courses (e.g., Sociology 101) in a given quarter.

CHART 4. OVERALL DFW RATES

CHART 5. DFW RATES FOR COMPARABLE COURSES

Fall'09 Win'10 Spr'10 Sum'10 Fall'10 Win'11 Spr'11 Sum'11

F2F 11% 11% 11% 5% 11% 10% 10% 4%

WW 12% 11% 13% 16% 12% 12% 13% 14%

0%

5%

10%

15%

20%

25%

CWU OVERALL DFW RATESFace‐to‐Face vs. 100% Online (WW)

Fall'09 Win'10 Spr'10 Sum'10 Fall'10 Win'11 Spr'11 Sum'11

F2F 10% 9% 7% 8% 8% 6% 6% 5%

WW 11% 10% 12% 21% 13% 12% 15% 12%

0%5%

10%15%20%25%

CWU DFW RATES FOR COMPARABLE COURSESFace‐to‐Face vs. 100% Online (WW)


Overall, CWU students are rather successful–for both online and face-to-face courses. DFW rates for online (WW) courses are very similar to face-to-face courses except for summer quarters. Online courses tend to have higher DFW rates in the summer while the inverse is true of courses taught face-to-face. When DWF rates for comparable courses are charted, online courses tend to have slightly higher DFW rates than courses taught face-to-face. This difference was most pronounced in the summer, 2010.

III. NOEL-LEVITZ PRIORITY SURVEY FOR ONLINE LEARNERS (PS0L)

The Noel-Levitz Priority Survey for Online Learners (PSOL) is designed to assess the satisfaction and priorities of students in distance learning and online programs. CWU administered the PSOL during the summer of 2009 and 2010. Likert questions on the survey use a scale of 1 to 7 where 1 is lowest and 7 is highest. Many of the first 36 questions ask both level of student “importance” and “satisfaction” and correspond to five areas of institutional focus (institutional perceptions of quality, academic services, instructional services, enrollment services, and student services). Appendix 4 provides a copy of the Noel-Levitz PSOL survey while Appendix 5 provides the complete 2009 and 2010 results.

In examining scale results, students tended to rate most areas similarly both years (see Table 3). However, several questions saw marked increase from 2009 to 2010 (see Table 4). Specifically, questions related to technical assistance and response, information availability, and course availability improved from 2009 to 2010. A question related to registration convenience saw a marked decrease from 2009 to 2010 (see Table 5). Table 6 reports those student responses that produced the largest performance gaps. A performance gap is calculated by subtracting the satisfaction score from the importance score. A larger performance gap indicates that the institution is not meeting student expectations; a smaller performance gap indicates that the institution is doing a relatively good job of meeting expectations. For 2009, the question with the largest performance gap related to the number of course offerings while a question regarding tuition investment had the largest gap in 2010. Performance gaps decreased for six of nine questions from 2009 to 2010. Table 3. NOEL-LEVITZ PSOL 2010 & 2009 INSTITUTIONAL SUMMARY

2010 Averages 2009 Averages 2010-2009 Satisfaction

Major Areas Measured Import Satisfaction “Gap” Import Satisfaction “Gap” Difference Institutional Perceptions 6.20 5.30 0.90 6.24 5.36 0.88 (0.06) Academic Services 6.20 5.39 0.81 6.22 5.26 0.96 0.13 Instructional Services 6.22 5.40 0.82 6.27 5.38 0.89 0.02 Enrollment Services 6.33 5.45 0.88 6.36 5.45 0.91 0.00 Student Services 6.11 5.32 0.79 6.14 5.22 0.92 0.10

Table 4. Noel-Levitz PSOL- Questions with a 0.2 or Greater Increase Satisfaction in 2010 than in 2009

2010 Averages 2009 Averages 2010 -2009 Satisfaction

Import Satisfaction Gap Import Satisfaction Gap Difference 10. This institution responds quickly when I request information.

6.44 5.45 0.99 6.42 5.25 1.17 0.20

14. I receive timely information on the availability of financial aid. 6.30 5.08 1.22 6.30 4.87 1.43 0.21

16. Appropriate technical assistance is readily available. 6.15 5.48 0.67 6.23 5.21 1.02 0.27

22. I am aware of whom to contact for questions about programs and services.

6.25 5.38 0.87 6.26 5.16 1.10 0.22


2010 Averages 2009 Averages 2010 -2009 Satisfaction

Import Satisfaction Gap Import Satisfaction Gap Difference Campus #5 31. The instructions for using the technology for my online courses are clear to me.

6.45 5.57 0.88 6.53 5.26 1.27 0.31

Campus #8 34. This university provides adequate orientation to new online learners.

5.99 4.90 1.09 6.07 4.66 1.41 0.24

Campus #10 36. Courses necessary to meet my degree objectives are offered online.

6.31 5.10 1.21 6.40 4.90 1.50 0.20

Table 5. Noel-Levitz PSOL - Questions with a 0.2 or Greater Decrease in Satisfaction from 2009 to 2010

2010 Averages 2009 Averages 2010 – 2009 Satisfaction

Import Satisfaction Gap Import Satisfaction Gap Difference 18. Registration for online courses is convenient. 6.43 5.99 0.44 6.50 6.20 0.30 0.21

Table 6. Noel-Levitz PSOL - Questions with Greatest Gap Between Importance and Satisfaction in 2009 & 2010

2010 2009 Import Satisfaction Gap Import Satisfaction Gap

6. Tuition paid is a worthwhile investment. 6.54 5.04 1.50 6.52 5.17 1.35

12. There are sufficient offerings within my program of study. 6.49 5.22 1.27 6.55 5.06 1.49

4. Faculty provide timely feedback about student progress. 6.44 5.27 1.17 6.50 5.23 1.27

9. Adequate financial aid is available. 6.29 4.86 1.43 6.23 4.85 1.38

14. I receive timely information on the availability of financial aid. 6.30 5.08 1.22 6.30 4.87 1.43

15. Channels are available for providing timely responses to student complaints. 6.06 4.82 1.24 6.02 4.81 1.21

Campus #4. 30. There is consistency in the quality of instruction from my online courses to my classroom-base courses. 6.34 5.01 1.33 6.36 4.99 1.37

Campus #6. 32. There are a variety of instructional approaches presented in my online courses. 6.02 4.90 1.12 6.17 4.98 1.19

Campus #10. 36. Courses necessary to meet my degree objectives are offered online. 6.31 5.10 1.21 6.40 4.90 1.50


IV. COURSE REVIEW Course objectives and requirements serve as the foundation for the entire process of instructional design. Specific and measureable objectives and accompanying course requirements provide the goals for learning activities and assessment techniques to gauge student progress. In addition, course activities, student engagement, evaluative practices, and faculty feedback and responsiveness all contribute to a positive and effective learning environment. Each of these areas is important to monitor as online enrollment and course development continue to increase. Seventy-one courses have been developed through work-for-hire agreements since the summer of 2008. Work-for-hire agreements have included professional development and assistance from the CWU Online Learning Team. In addition, these courses have been reviewed through a “best practice” rubric (Keinath and Blicker, 2003) (See Appendix 6). All courses needed to have scored highly in all areas of the rubric before payment of services was provided to the faculty developer. Although some online courses have been developed and evaluated through a “best practice” rubric, many more online courses (i.e., more than 200) have been developed with undetermined professional development and support and no evaluation. It would be proposed that this rubric or something akin to it be used regularly by departments in examining course design quality and faculty planning efficacy. A revised rubric is being developed by the CWU Online Learning Team and will be piloted during the winter and spring, 2012 quarters with at least one department (e.g., Information Technology and Administrative Management) for greater refining, dissemination, and departmental and programmatic analysis applicability.

The examination and comparison of course syllabi (i.e., objectives and requirements) between similar courses taught differently (face-to-face and online) can provide comparative information as to course outcome, focus, and the ways which outcomes will be achieved. Bloom’s taxonomy of the cognitive domain (i.e., knowledge, comprehension, application, analysis, synthesis, and evaluation) can be used to analyze the course objectives and the level of cognitive demand/skill requirements for students. Course requirements provide information as to the type of activities utilized to meet course objectives and their relative weight in grading.

The following summary provides a qualitative analysis of two randomly selected courses from each of CWU’s four colleges offered Fall 2011, comparing face-to-face and online sections of each course (n=8). Appendix seven presents similarities and differences between course objectives (number of objectives and level of cognitive demand/skill) and course requirements (with relative percentage of overall grade).

Summary: The number and cognitive demand/skill level of course objectives are foundational elements of teaching and student learning. In addition, course requirements and relative weight in final grade determination is also important. Following are some observations of the eight randomly selected courses examined. Caution should be exercised in generalizing the results beyond this data set because of the minimum number of courses analyzed and the total variability in instructors, content areas, and courses offered at CWU.

Course Objectives and Level:

• Analyzed courses presented between 4-7 objectives each although some courses did not itemize the objectives.

• Corresponding sections of the same course (face-to-face and online) tended to have an equal number of objectives.

• Corresponding sections tended to have parallel levels of cognitive demand/skill requirements.


Course Requirements:

• Both face-to-face and online courses demonstrated a variety of course requirement types (e.g., assignments, tests, attendance, participation). Where there was a discrepancy between face-to-face and online sections, the online course tended to have a greater variety of learning activities and focus on higher order thinking than the corresponding face-to-face course.

• Almost all face-to-face and online courses required some form of quizzes and/or exams. The percentage for quizzes/exams ranged from 15% to 80% of the total grade. Face-to-face courses tended to have a significantly higher percentage of the total grade based on quizzes/exams than their corresponding online sections.

• All online courses included discussion-based course requirements, while most face-to-face courses required some form of graded participation, as well. Online courses tended to have a significantly higher percentage of the total grade based on discussion-based activities than their corresponding face-to-face section.

• Demonstrating the shift to multi-modal learning, many of the face-to-face courses have a hybrid dimension, with 5% to 50% of learning activities (discussion boards, quizzes/exams, homework assignments) completed online.

• As might be expected, online courses tended to have significantly higher writing requirements than their corresponding face-to-face sections, particularly when taking into account required postings to discussion boards.

12‐1‐11 CWU Online Learning Quality Assurance of 43

APPENDIX I

STUDENT EVALAUTION OF INSTRUCTION FORM A

This form is administered in‐class and delivered to Testing Services for scanning. This form is designed specifically to evaluate instruction in “lecture” courses.


APPENDIX II

STUDENT EVALAUTION OF INSTRUCTION FORM A2010

This form was developed by the CWU SEOI Task Force to replace the Form A for lecture classes. For this study this form was administered online to courses that met face‐to‐face in classrooms.


CENTRAL WASHINGTON UNIVERSITY

STUDENT EVALAUTION OF INSTRUCTION Form A 2010 Lecture

Use a #2 pencil to make dark, solid marks Erase errors completely

Course: _______________________________ Instructor: __________________________________ Time of day: ___________________

1. STUDENT LEARNING ENVIRONMENT. To what extent do you agree or disagree that the... Strongly

agree Agree Neutral Disagree Strongly disagree

a. instructor fostered a fair and respectful learning environment? O O O O O b. instructor seemed genuinely concerned with whether students learned? O O O O O c. standards of classroom behavior were clearly communicated and enforced? O O O O O d. instructor met class at scheduled times unless otherwise arranged? O O O O O

Yes No e. Did you seek help from the instructor outside of class during the course? O O Strongly


f. If YES, was the instructor available to provide help? O O O O O g. Please provide additional comments on the areas addressed in Section 1 above.

2. TEACHING FOR STUDENT LEARNING. To what extent do you agree or disagree that the...

Strongly agree Agree Neutral Disagree

Strongly disagree

a. course objectives were clearly communicated? O O O O Ob. overall course content was presented in an understandable sequence?

O O O O O

c. instructor used a variety of methods, as needed, to make content clear?

O O O O O

d. assignments and tests were connected to course content? O O O O Oe. evaluation and grading techniques were clearly explained? O O O O Of. instructions for class activities were clearly communicated? O O O O O g. instructor provided useful feedback on student work? O O O O O h. instructor provided timely feedback on student progress? O O O O O i. class sessions were well organized? O O O O O j. out‐of‐class work was useful in understanding course content? O O O O O k. instructor encouraged students to connect course content to issues beyond

the university classroom? O O O O O

l. course activities challenged students to think critically? O O O O O

m. Please provide additional comments for the areas addressed in Section 2 above.


GENERAL INFORMATION 3. How would you compare this course with all other courses of similar credits at this level (i.e., 100, 200, 300, etc.) taken at CWU? Was the... Much more

than most courses?

More than most courses?

About average?

Less than most

courses?

Much less than most courses?

a. amount of work OUTSIDE of class O O O O O b. level of engagement/active learning IN class O O O O O c. intellectual challenge presented to you O O O O O

d. Please provide additional comments for the areas addressed in Section 3 above.

4. For this class, about how many hours outside of class did you spend in a typical 7‐day week studying, reading, conducting research, writing, doing homework or lab work, analyzing data, rehearsing, and other academic activities?

O 0 (no) hrs/wk O 1‐3 hrs/wk O 4‐6 hrs/wk O 7‐10 hrs/wk

O 11‐15 hrs/wk O 16‐20 hrs/wk O 21+ hrs/wk

5. Why did you take this course? Please mark all that apply. Major requirement Minor requirement Certificate requirement Fulfills General Education requirement Reputation of instructor Time of day General interest Other? ______________________________________________________

6. What is your class standing?

O First year (0 ‐ 44 credits) O Sophomore (45 ‐ 89 credits) O Junior (90 ‐ 134 credits)

O Senior (135 or more credits) O Graduate O Other (e.g. post‐baccalaureate) 7. What grade do you expect to earn in this class?

O A O B O C O D O F O Other (Pass/Fail, etc.)

8. Please provide any additional comments about the course or instructor (e.g., instructor’s teaching effectiveness, course materials, classroom facilities, etc.).


APPENDIX III

STUDENT EVALUATION OF INSTRUCTION FORM W 2010

This form was designed by the CWU SEOI Task Force to evaluate courses that meet 100% online


CENTRAL WASHINGTON UNIVERSITY

STUDENT EVALAUTION OF INSTRUCTION Form W 2010

OnlineInstruction

1. STUDENT LEARNING ENVIRONMENT To what extent do you agree or disagree that the... Strongly


a. instructor fostered a fair and respectful learning environment? O O O O O b. instructor seemed genuinely concerned with whether students learned? O O O O Oc. standards of online behavior were clearly communicated and enforced? O O O O Od. the organization and design of the online environment were conducive to

learning? O O O O O

e. instructor was actively engaged in the class? O O O O O

Yes No f. Did you seek help from the instructor during the course? O O Strongly


g. If YES, did the instructor provide help? O O O O O h. Please provide any additional comments on the areas addressed in Section 1 above.

2. TEACHING FOR STUDENT LEARNING To what extent do you agree or disagree that the...

Strongly agree Agree Neutral Disagree

Strongly disagree

a. course objectives were clearly communicated? O O O O O b. overall course content was presented in an understandable sequence?

O O O O O

c. instructor used a variety of methods, as needed, to make content clear?

O O O O O

d. the instructor used online technologies to facilitate interaction among students and with instructor?

O O O O O

e. assignments and tests were connected to course content? O O O O Of. evaluation and grading techniques were clearly explained? O O O O Og. instructions for class activities were clearly communicated? O O O O O h. instructor provided useful feedback on student work? O O O O O i. instructor provided timely feedback on student progress? O O O O O j. online activities were well organized? O O O O O k. work completed off‐line was useful in understanding course content?

O O O O O

l. instructor encouraged students to connect course content to issues beyond the university classroom?

O O O O O

m. course activities challenged students to think critically? O O O O O

n. Please provide any additional comments for the areas addressed in Section 2 above.


3. INFORMATION ON ONLINE CLASS TECHNOLOGY a. How many online courses have you taken before this course?

O None O 1 to 2

O 3 to 4 O More than 4 b. How strongly do you agree that the technologies used in this course were reliable? O Strongly agree O Agree O Neutral O Disagree O Strongly disagree

c. Which online technologies used in this class were most useful for your learning? GENERAL INFORMATION

4. How would you compare this course with all other courses of similar credits at this level (i.e., 100, 200, 300, etc.) taken at CWU? Was the...? Much more

than most courses?

More than most

courses?About

average?

Less than most

courses?

Much less than most courses?

a. amount of work OUTSIDE of online environment O O O O Ob. level of engagement/active learning while IN the online environment

O O O O O

c. intellectual challenge presented to you O O O O O

d. Please provide additional comments for the areas addressed in Section 4 above.

5. For this class, about how many hours outside of class did you spend in a typical 7‐day week studying, reading, conducting research, writing, doing homework or lab work, analyzing data, rehearsing, and other academic activities and participating online?

O (no) hrs/wk O 1‐3 hrs/wk O 4‐6 hrs/wk O 7‐10 hrs/wk

O 11‐15 hrs/wk O 16‐20 hrs/wk O 21+ hrs/wk

6. Why did you take this course? Please mark all that apply.

Major requirement Minor requirement Certificate requirement Fulfills General Education requirement Reputation of instructor Offered Online General interest Other? ______________________________________________________


7. What is your class standing? O First year (0 ‐ 44 credits) O Sophomore (45 ‐ 89 credits) O Junior (90 ‐ 134 credits)

O Senior (135 or more credits) O Graduate O Other (e.g. post‐baccalaureate) 8. What grade do you expect to earn in this class?

O A O B O C O D O F O Other (Pass/Fail, etc.)

9. Please provide any additional comments about the course or instructor (e.g., instructor’s teaching effectiveness, course materials, online technologies, etc.).


APPENDIX IV

MOCK‐UP OF CWU’s NOEL‐LEVITZ PRIORITY SURVEY FOR ONLINE LEARNERS


CENTRAL WASHINGTON UNIVERSITY MOCK‐UP OF CWU’s 2009/2010 NOEL‐LEVITZ PRIORITY SURVEY OF ONLINE LEARNERS (PSOL)

Each item below describes an expectation about your experiences with this program. On the left, tell us how important it is for your institution to meet this expectation. On the right, tell us how satisfied you are that your institution has met this expectation. Importance to me ...

1 ‐ not important at all 2 ‐ not very important 3 ‐ somewhat unimportant 4 ‐ neutral

5 ‐ somewhat important6 ‐ important 7 ‐ very important N/A ‐ does not apply

... My level of satisfaction

1 ‐ not satisfied at all 2 ‐ not very satisfied 3 ‐ somewhat dissatisfied 4 ‐ neutral

5 ‐somewhat satisfied6 ‐ satisfied 7 ‐ very satisfied N/A ‐ not available/not used

1 2 3 4 5 6 7 N/A 1 2 3 4 5 6 7 N/A

O O O O O O O O 1. This institution has a good reputation. O O O O O O O O

O O O O O O O O 2. My program advisor is accessible by telephone and e‐mail. O O O O O O O O

O O O O O O O O 3. Instructional materials are appropriate for program content. O O O O O O O O

O O O O O O O O 4. Faculty provide timely feedback about student progress. O O O O O O O O

O O O O O O O O 5. My program advisor helps me work toward career goals. O O O O O O O O

O O O O O O O O 6. Tuition paid is a worthwhile investment. O O O O O O O O

O O O O O O O O 7. Program requirements are clear and reasonable. O O O O O O O O

O O O O O O O O 8. Student‐to‐student collaborations are valuable to me. O O O O O O O O

O O O O O O O O 9. Adequate financial aid is available. O O O O O O O O

O O O O O O O O 10. This institution responds quickly when I request information. O O O O O O O O

O O O O O O O O 11. Student assignments are clearly defined in the syllabus. O O O O O O O O

O O O O O O O O 12. There are sufficient offerings within my program of study. O O O O O O O O

O O O O O O O O 13. The frequency of student and instructor interactions is adequate. O O O O O O O O

O O O O O O O O 14. I receive timely information on the availability of financial aid. O O O O O O O O

O O O O O O O O 15. Channels are available for providing timely responses to student complaints.

O O O O O O O O

O O O O O O O O 16. Appropriate technical assistance is readily available. O O O O O O O O

O O O O O O O O 17. Assessment and evaluation procedures are clear and reasonable. O O O O O O O O

O O O O O O O O 18. Registration for online courses is convenient. O O O O O O O O

O O O O O O O O 19. Online career services are available. O O O O O O O O

O O O O O O O O 20. The quality of online instruction is excellent. O O O O O O O O

O O O O O O O O 21. Adequate online library resources are provided. O O O O O O O O

O O O O O O O O 22. I am aware of whom to contact for questions about programs and services.

O O O O O O O O

O O O O O O O O 23. Billing and payment procedures are convenient for me. O O O O O O O O

O O O O O O O O 24. Tutoring services are readily available for online courses. O O O O O O O O

O O O O O O O O 25. Faculty are responsive to student needs. O O O O O O O O

O O O O O O O O 26. The bookstore provides timely service to students. O O O O O O O O


Importance to me ... CAMPUS DEFINED QUESTIONS 1 - 10 1 ‐ not important at all 2 ‐ not very important 3 ‐ somewhat unimportant 4 ‐ neutral


... My level of satisfaction

1 ‐ not satisfied at all 2 ‐ not very satisfied 3 ‐ somewhat dissatisfied 4 ‐ neutral

5 ‐somewhat satisfied6 ‐ satisfied 7 ‐ very satisfied N/A ‐ not available/not used

1 2 3 4 5 6 7 N/A 1 2 3 4 5 6 7 N/A

O O O O O O O O #1 27. I prefer the online version of my summer classes vs. the classroom

setting. O O O O O O O O

O O O O O O O O #2 28. The quality of student‐to‐student interaction in my online courses is as

high as my classroom‐based courses. O O O O O O O O

O O O O O O O O #3 29. The requirements to succeed in online courses are equivalent to those

required in my classroom‐based courses. O O O O O O O O

O O O O O O O O #4 30. There is consistency in the quality of instruction from my online courses to

my classroom‐base courses. O O O O O O O O

O O O O O O O O #5 31. The instructions for using the technology for my online courses are clear

to me. O O O O O O O O

O O O O O O O O #6 32. There are a variety of instructional approaches presented in my online

courses. O O O O O O O O

O O O O O O O O #7 33. The quality of student interaction with faculty is excellent. O O O O O O O O

O O O O O O O O #8 34. This university provides adequate orientation to new online learners. O O O O O O O O

O O O O O O O O #9 35. Gender equity and equal opportunity exist in the online classroom. O O O O O O O O

O O O O O O O O #10 36. Courses necessary to meet my degree objectives are offered online. O O O O O O O O

N‐L –tell us how important each of the following sources of information were to your decision to enroll in this program.

O O O O O O O O 37. Catalog and brochures (printed)

O O O O O O O O 38. Catalog (online)

O O O O O O O O 39. College representatives

O O O O O O O O 40. Web site

O O O O O O O O 41. Advertisements

O O O O O O O O 42. Recommendation from instructor or program advisor

O O O O O O O O 43. Contact with current students and / or recent graduates of the program

O O O O O O O O 44. Ability to transfer credits

O O O O O O O O 45. Cost

O O O O O O O O 46. Financial assistance available

O O O O O O O O 47. Future employment opportunities

O O O O O O O O 48. Reputation of institution


Each item below describes an expectation about your experiences with this program. On the left, tell us how important it is for your institution to meet this expectation. Importance to me ...

1 ‐ not important at all 2 ‐ not very important 3 ‐ somewhat unimportant 4 ‐ neutral


1 2 3 4 5 6 7 N/A

O O O O O O O O 49. Work scheduleO O O O O O O O 50. Flexible pacing for completing a programO O O O O O O O 51. ConvenienceO O O O O O O O 52. Distance from campusO O O O O O O O 53. Program requirementsO O O O O O O O 54. Recommendations from employer

Summary Questions Choose the one response that best applies to you for each of the questions below. 55. So far, how has your online experience met your expectations?

O Quite a bit worse than I expected O Worse than I expected O About what I expected O Better than I expected O Quite a bit better than I expected O Much better than I expected

56. Rate your overall satisfaction with your online experience thus far.

O Not satisfied at all O Not very satisfied O Somewhat dissatisfied O Neutral O Somewhat satisfied O Satisfied O Very satisfied

57. If you had to do it over, would you enroll in this program again? O Definitely not O Probably not O Maybe not O I don’t know O Maybe yes O Probably yes O Definitely yes


Demographic Questions Please select the response for each item that best describes you from the pull down lists provided. 58. Gender

O Female O Male

59. Age

O 18 and under O 19 to 24 O 25 to 34 O 35 to 44 O 45 to 54 O 55 to 64 O 65 and older

60. Ethnicity/Race

O African‐American O American Indian or Alaskan Native O Asian or Pacific Islander O Caucasian/White O Hispanic O Other race O Race‐ prefer not to respond

61. Current Enrollment Status

O Primarily online O Primarily on‐campus

62. Current Class Load O Full time O Part time

63. Class Level

O First year O Second year O Third year O Fourth year O Special student O Graduate/professional O Other class level


64. Educational Goal O Associate degree O Bachelor’s degree O Master’s degree O Doctorate or Professional degree O Certification (initial or renewal) O Self‐improvement/pleasure O Job‐related training O Other educational goal

65. Employment

O Full‐time O part‐time O Not employed

66. Current Residence O Own house O Rent room / apartment / house O Relative’s home O Residence hall O Other residence

67. Marital Status

O Single O Single with children O Married O Married with children O Marital – prefer not to respond

68. Current Plans O Complete online degree program O Complete degree on campus O Transfer credits O Complete this course

69. Current Online Enrollment O 1‐3 credits O 4‐6 credits O 7‐9 credits O 10‐12 credits O 13–15 credits O More than 15 credits


70. Previous Online Enrollment

O no classes 0 1‐3 classes O 4‐6 classes O 7‐9 classes O 10‐12 classes O 13‐15 classes O more than 15 classes

CWU “CAMPUS DEFINED” ITEMS 71. I am willing to pay an additional fee of _______ for future online classes

O $25 ‐ $50 O $50 ‐ $75 O $75 ‐ $100 O $0.00

72. My home CWU campus is O Ellensburg O CWU ‐ Lynnwood O CWU ‐ Des Moines O CWU ‐ Wenatchee O CWU ‐ Yakima O CWU ‐ Moses Lake


73. Campus‐Defined Majors/Codes (this question was a large “drop‐down” list) 1001 ‐ Accounting (BS) 1002 ‐ Anthropology (BS) 1003 ‐ Anthropology (Large Plan) (BA) 1004 ‐ Anthropology (Small Plan) (BA) 1005 ‐ Art (BA) 1006 ‐ Art (BFA) 1007 ‐ Art (MA) 1008 ‐ Asia/Pacific Studies (ASTPROG) 1009 ‐ Asia/Pacific Studies (PHILDEPT) 1010 ‐ Biology (MS) 1011 ‐ Biology (BA) 1012 ‐ Biology (BS) 1013 ‐ Biology Teaching (BS) 1014 ‐ Business Administration (MANADEPT) 1015 ‐ Business Administration (FINODEPT) 1016 ‐ Chemistry (MS) 1017 ‐ Chemistry (BA) 1018 ‐ Chemistry (BS) 1019 ‐ Chemistry Teaching (BA) 1020 ‐ Communication Studies (BA) 1021 ‐ Computer Science (BS) 1022 ‐ Construction Management (BS) 1023 ‐ Early Childhood Education (BAED) 1024 ‐ Earth Science Teaching (BS) 1025 ‐ Economics (BS) 1026 ‐ Electronics Engineer Tech (BS) 1027 ‐ Elementary Education (BAED) 1028 ‐ Engineering Technology (MS) 1029 ‐ English (MA) 1030 ‐ English Language & Literature (BA) 1031 ‐ English/Lang Arts Teaching (BA) 1032 ‐ Environmental Geol Sciences (BS) 1033 ‐ Environmental Studies (BS) 1034 ‐ Environmental Studies 1035 ‐ Environmental Studies (INTLPROG) 1036 ‐ Exercise Science (MS) 1037 ‐ Exercise Science (BS) 1038 ‐ Experimental Psychology (MS) 1039 ‐ Family & Consumer Sciences(BA) 1040 ‐ Family and Consumer Sciences (MS) 1041 ‐ Family and Consumer Studies (BA) 1042 ‐ Fashion Merchandising (FCSDEPT)(BS) 1043 ‐ Fashion Merchandising (ITAMDEPT)(BS) 1044 ‐ FCS: Career & Tech Ed Teach (BS) 1045 ‐ Film and Video Studies (BA) 1046 ‐ Fine Arts (MFA) 1047 ‐ Flight Technology (BS) 1048 ‐ Food Science and Nutrition (BS) 1049 ‐ Food Service Management (BAS) 1050 ‐ Foreign Lang Tching Broad Area (BA)

1051 ‐ Foreign Language (BA)1052 ‐ Foreign Language Broad Area (BA) 1053 ‐ Foreign Language Teaching (BA) 1054 ‐ General Science Teaching (BS) 1055 ‐ Geography (Large Plan) (BA) 1056 ‐ Geography (Small Plan) (BA) 1057 ‐ Geology (MS) 1058 ‐ Geology (BA) 1059 ‐ Geology (BS) 1060 ‐ Gerontology (Large Plan) (BS) 1061 ‐ Gerontology (Small Plan) (BS) 1062 ‐ Global Wine Studies (BS) 1063 ‐ Health and Physical Education (MS) 1064 ‐ History (MA) 1065 ‐ History (Large Plan) (BA) 1066 ‐ History (Small Plan) (BA) 1067 ‐ History Teaching Broad Area (BA) 1068 ‐ Home Ec & Family Studies (MS) 1069 ‐ Individual Studies (ISPROG)(BA) 1070 ‐ Individual Studies (BA) 1071 ‐ Individual Studies (BMUS) 1072 ‐ Individual Studies (ISPROG)(BMUS) 1073 ‐ Individual Studies (BS) 1074 ‐ Individual Studies (ISPROG)(BS) 1075 ‐ Individual Studies (MA) 1076 ‐ Individual Studies (ISPROG)(MA) 1077 ‐ Individual Studies (MED) 1078 ‐ Individual Studies (ISPROG)(MED) 1079 ‐ Individual Studies (ISPROG)(MS) 1080 ‐ Individual Studies (MS) 1081 ‐ Industrial Ed Broad Area (BS) 1082 ‐ Industrial Education (BAED) 1083 ‐ Industrial Education (BS) 1084 ‐ Industrial Supervision (BS) 1085 ‐ Industrial Technology (BAS) 1086 ‐ Industrial Technology (BS) 1087 ‐ Info Tech & Admin Mgt (BAS) 1088 ‐ Information Tech & Admin Mgt (BS) 1089 ‐ Instructional Leadership (MED) 1090 ‐ IS: Social Sciences (Large) (BS) 1091 ‐ IS: Social Sciences (Small) (BS) 1092 ‐ Journalism (BA) 1093 ‐ Keyboard or Guitar Perf (BMUS) 1094 ‐ Law & Just (No Specialization) (BA) 1095 ‐ Law & Just (W/ Specialization) (BA) 1096 ‐ Law & Justice (MS) 1097 ‐ Master Teacher (MED) 1098 ‐ Math Teaching Secondary (BA) 1099 ‐ Mathematics (BS) 1100 ‐ Mathematics (MAT) 1101 ‐ Mathematics BS (Small Plan) (BS) 1102 ‐ Mechanical Engineer Tech (BS) 1103 ‐ Music (MMUS) 1104 ‐ Music (BA)

1105 ‐ Music Education (BMUS) 1106 ‐ Nutrition (MS) 1107 ‐ Paramedic (BS) 1108 ‐ Percussion Performance (BMUS) 1109 ‐ Philosophy (Large No Special) (BA) 1110 ‐ Philosophy (Large Plan) (BA) 1111 ‐ Philosophy (Small No Special) (BA) 1112 ‐ Philosophy (Small Plan) (BA) 1113 ‐ Physical Ed and School Health (BS) 1114 ‐ Physics (BA) 1115 ‐ Physics (BS) 1116 ‐ Political Science (Large Plan) (BA) 1117 ‐ Political Science (Small Plan) (BA) 1118 ‐ Primate Behavior (ANTHDEPT) (MS)1119 ‐ Primate Behavior (PRIMPROG) (MS) 1120 ‐ Primate Behavior & Ecology (ANTHDEPT) (BS) 1121 ‐ Primate Behavior & Ecology (PRIMPROG) (BS) 1122 ‐ Professional Accountancy (MPA) 1123 ‐ Psy: Mental Health Counseling (MS) 1124 ‐ Psychology (Large Plan) (BA) 1125 ‐ Psychology (Small Plan) (BA) 1126 ‐ Public Health (BS) 1127 ‐ Public Policy (POSCDEPT) (BS) 1128 ‐ Public Policy (PUBLPROG) (BS) 1129 ‐ Public Relations (BA) 1130 ‐ Reading Specialist (MED) 1131 ‐ Recreation and Tourism (BS) 1132 ‐ Resource Management (ANTHDEPT) (MS) 1133 ‐ Resource Management (GEOGDEPT) (MS) 1134 ‐ Resource Management (REMPROG) (MS) 1135 ‐ Safety & Health Management (BS) 1136 ‐ Safety & Health Mgt (BAS) 1137 ‐ School Administration (MED) 1138 ‐ School Counseling (MED) 1139 ‐ School Health Education (BA) 1140 ‐ School Health Education (BA) 1141 ‐ School Psychology (MED) 1142 ‐ Social Services (BS) 1143 ‐ Sociology (BS) 1144 ‐ Sociology (Large Plan) (BA) 1145 ‐ Sociology (Small Plan) (BA) 1146 ‐ Spanish (BA) 1147 ‐ Spanish Broad Area (BA) 1148 ‐ Spanish Teaching (BA)


1149 ‐ Spanish Teaching Broad Area (BA) 1150 ‐ Special Ed P‐12 Broad Area (BAED) 1151 ‐ Special Education (MED) 1152 ‐ Special Education P‐12 (BAED) 1153 ‐ Special Education P‐3 (BAED) 1154 ‐ String Performance (BMUS) 1155 ‐ TESOL (MA) 1156 ‐ Theatre Arts ‐ General Studies (BA) 1157 ‐ Theatre Arts (BFA) 1158 ‐ Theatre Arts Teaching K‐12 (BA) 1159 ‐ Theatre Production (MA) 1160 ‐ Theory/Composition (BMUS) 1161 ‐ Undeclared 1162 ‐ Visual Art Teaching (BA) 1163 ‐ Vocal Performance (BMUS) 1164 ‐ Wind Performance (BMUS)


APPENDIX V

NOEL‐LEVITZ PRIORITY SURVEY FOR ONLINE LEARNERS YEAR‐TO‐YEAR REPORT 2009 to 2010


CENTRAL WASHINGTON UNIVERSITY NOEL‐LEVITZ PRIORITY SURVEY FOR ONLINE LEARNERS YEAR‐TO‐YEAR REPORT 2009 to 2010

Questions with a change in satisfaction of 0.2 or more are highlighted in gray

Spring 2010 Spring 2009 Satisfaction

Import Satisfaction/ SD

Gap Import Satisfaction/ SD

Gap Difference

1. This institution has a good reputation. 5.87 5.56 / 1.29 0.31 5.95 5.56 / 1.31 0.39 0.00 2. My program advisor is accessible by telephone and e‐mail. 6.35 5.84 / 1.46 0.51 6.38 5.71 / 1.39 0.67 0.13 3. Instructional materials are appropriate for program content. 6.45 5.66 / 1.29 0.79 6.44 5.57 / 1.32 0.87 0.09 4. Faculty provide timely feedback about student progress. 6.44 5.27 / 1.58 1.17 6.50 5.23 / 1.51 1.27 0.04 5. My program advisor helps me work toward career goals. 6.07 5.26 / 1.69 0.81 6.07 5.22 / 1.52 0.85 0.04 6. Tuition paid is a worthwhile investment. 6.54 5.04 / 1.65 1.50 6.52 5.17 / 1.63 1.35 ‐0.13 7. Program requirements are clear and reasonable. 6.47 5.53 / 1.37 0.94 6.47 5.46 / 1.38 1.01 0.07 8. Student‐to‐student collaborations are valuable to me. 4.96 5.00 / 1.53

‐0.04 5.05 5.14 / 1.43

‐0.09 ‐0.14

9. Adequate financial aid is available. 6.29 4.86 / 1.90 1.43 6.23 4.85 / 1.83 1.38 0.01 10. This institution responds quickly when I request information. 6.44 5.45 / 1.43 0.99 6.42 5.25 / 1.62 1.17 0.2011. Student assignments are clearly defined in the syllabus. 6.55 5.64 / 1.37 0.91 6.61 5.50 / 1.44 1.11 0.14 12. There are sufficient offerings within my program of study. 6.49 5.22 / 1.56 1.27 6.55 5.06 / 1.59 1.49 0.16 13. The frequency of student and instructor interactions is adequate. 6.09 5.43 / 1.41 0.66 6.19 5.51 / 1.34 0.68 ‐0.08 14. I receive timely information on the availability of financial aid. 6.30 5.08 / 1.68 1.22 6.30 4.87 / 1.71 1.43 0.2115. Channels are available for providing timely responses to student complaints. 6.06 4.82 / 1.69 1.24 6.02 4.81 / 1.72 1.21 0.01 16. Appropriate technical assistance is readily available. 6.15 5.48 / 1.41 0.67 6.23 5.21 / 1.46 1.02 0.2717. Assessment and evaluation procedures are clear and reasonable. 6.30 5.51 / 1.43 0.79 6.31 5.44 / 1.41 0.87 0.07 18. Registration for online courses is convenient. 6.43 5.99 / 1.34 0.44 6.50 6.20 / 1.06 0.30 ‐0.21 19. Online career services are available. 5.53 5.09 / 1.49 0.44 5.62 5.13 / 1.46 0.49 ‐0.04 20. The quality of online instruction is excellent. 6.40 5.11 / 1.65 1.29 6.42 5.04 / 1.74 1.38 0.07


Appendix II ‐ continued

Spring 2010 Spring 2010 Satisfaction Difference

Import

Satisfaction/ SD Gap Import

Satis faction/ SD Gap

21. Adequate online library resources are provided. 6.16 5.62 / 1.43 0.54 6.21 5.53 / 1.49 0.68 0.09 22. I am aware of whom to contact for questions about programs and services. 6.25 5.38 / 1.52 0.87 6.26 5.16 / 1.63 1.10 0.22 23. Billing and payment procedures are convenient for me. 6.29 5.75 / 1.44 0.54 6.38 5.72 / 1.42 0.66 0.03 24. Tutoring services are readily available for online courses. 5.59 4.60 / 1.70 0.99 5.51 4.42 / 1.80 1.09 0.18 25. Faculty are responsive to student needs. 6.54 5.58 / 1.40 0.96 6.65 5.59 / 1.44 1.06 ‐0.01 26. The bookstore provides timely service to students. 6.20 5.76 / 1.37 0.44 6.34 5.68 / 1.48 0.66 0.08 #1 27. I prefer the online version of my summer classes vs. the classroom setting. 5.58 4.92 / 2.02 0.66 5.60 4.98 / 1.96 0.62 ‐0.06 #2 28. The quality of student‐to‐student interaction in my online courses is as high as my classroom‐based courses. 5.24 4.59 / 1.94 0.65 5.34 4.66 / 1.90 0.68 ‐0.07 #3 29. The requirements to succeed in online courses are equivalent to those required in my classroom‐based courses. 6.17 5.11 / 1.85 1.06 6.22 5.31 / 1.75 0.91 ‐0.20 #4 30. There is consistency in the quality of instruction from my online courses to my classroom‐base courses.

6.34 5.01 / 1.86 1.33 6.36 4.99 / 1.87 1.37 0.02 #5 31. The instructions for using the technology for my online courses are clear to me. 6.45 5.57 / 1.59 0.88 6.53 5.26 / 1.71 1.27 0.31 #6 32. There are a variety of instructional approaches presented in my online courses. 6.02 4.90 / 1.74 1.12 6.17 4.98 / 1.79 1.19 ‐0.08 #7 33. The quality of student interaction with faculty is excellent. 6.26 5.32 / 1.59 0.94 6.40 5.26 / 1.62 1.14 0.06 #8 34. This university provides adequate orientation to new online learners. 5.99 4.90 / 1.83 1.09 6.07 4.66 / 1.94 1.41 0.24 #9 35. Gender equity and equal opportunity exist in the online classroom. 6.17 6.30 / 1.09

‐0.13 6.24 6.23 / 1.16 0.01 0.07

#10 36. Courses necessary to meet my degree objectives are offered online. 6.31 5.10 / 1.81 1.21 6.40 4.90 / 1.77 1.50 0.20

Appendix II 2009 to 2010 – continued


Appendix II ‐ continued

2010 2009

Importance Importance 37. Source of information: Catalog and brochures (printed) 4.60 4.77 38. Source of information: Catalog (online) 5.85 6.21 39. Source of information: College representatives 4.53 4.71 40. Source of information: Web site 5.61 5.98 41. Source of information: Advertisements 3.56 3.82 42. Source of information: Recommendation from instructor or program advisor 5.27 5.64 43. Source of information: Contact with current students and / or recent graduates of the program 4.82 5.19 44. Factor to enroll: Ability to transfer credits 5.75 5.95 45. Factor to enroll: Cost 5.65 6.07 46. Factor to enroll: Financial assistance available 5.55 5.91 47. Factor to enroll: Future employment opportunities 5.80 5.88 48. Factor to enroll: Reputation of institution 5.47 5.84 49. Factor to enroll: Work schedule 5.70 6.26 50. Factor to enroll: Flexible pacing for completing a program 5.99 6.30 51. Factor to enroll: Convenience 6.30 6.53 52. Factor to enroll: Distance from campus 5.50 5.71 53. Factor to enroll: Program requirements 6.07 6.35 54. Factor to enroll: Recommendations from employer 4.08 4.28


APPENDIX VI

GUIDELINES FOR ONLINE COURSE DEVELOPMENT


Guidelines for Online Course Development (Adapted from Keinath and Blicker (2003)

This checklist is designed to serve a number of functions: • Provide guidance to individual instructors and departments as they develop online courses or modify face-to-face courses for

an online environment • Serve as a tool to groups evaluating the readiness and appropriateness of a course for online delivery • Serve as one possible assessment tool for online courses • Provide targets for course developers as they continually enhance their course offerings

Designing an online course In designing an online course, instructors must remember that technology is a tool, not a solution. The design of a good online course rests on the same foundation as the design for a face-to-face course: clear and coherent learning outcomes.

Are the learning outcomes for the course: Yes No Comments

a. Clearly stated

b. Assessable

c. Relevant to the course content

d. Supported by course activities and assignments

Is information about the course (course requirements, course activities, assignments, grading/evaluation, dates, policies, instructor information, etc.):

Yes No Comments

a. Easily located

b. Sufficient

Does the course design: Yes No Comments

a. Maintain consistency in design

b. Allow for clear progression through course requirements by the use of “modules” or some other form of course organizational pattern

c. Allow for practice of skills or application of theories

Does the technology and presentation used for the course: Yes No Comments

a. Allow for easy navigation through the course

b. Provide sufficient student support and direction

c. Provide access to students with disabilities


Evaluation and Assessment in online courses Once a course is designed, good online courses need a method to determine if students are learning what is intended. Evaluation and assessment need to be based on the stated and clear student learning outcomes, so that students know what is expected of them, and faculty can determine if students are learning what the course is designed to teach.

Evaluation and assessment in the course: Yes No Comments

a. Is linked to student learning outcomes

b. Incorporates a variety of evaluation and assessment tools

c. Takes into account varied student learning styles

d. Shows a clear relationship between student performance and grades in the course

e. Is both formative and summative

f. Requires demonstration of direct student learning

g. Allows for sufficient and timely feedback to the student

h. Provides the instructor with sufficient information for course improvement

Teaching and Learning in online courses The technology available to online instructors and students plays an important role in the online classroom. Technology needs to be used and designed to aid in teaching and learning, not to get in the way of teaching and learning.

Technology enhances teaching and learning in the course by:

Yes No Comments

a. Using available tools to support teaching and learning, such as e-mail, discussion, chat, assessments, surveys, quizzes, links, online materials, etc.

b. Having a clear relationship to the student learning outcomes

c. Providing links to students to test software and download necessary plug-ins (if applicable)

The graphics and other media tools used in the course:

a. Are relevant to the learning outcomes

b. Are accessible on dial-up accounts


Technology enhances teaching and learning in the course by:

Yes No Comments

c. Are viewable using a variety of browsers

d. Are compatible with multiple platforms

Recommendations:


APPENDIX VII

FACE‐TO‐FACE AND ONLINE COURSE OBJECTIVES AND REQUIREMENTS COMPARISON FALL 2011


College/Course Face-to-Face Online CAH: Course A Objectives Five Objectives:

1 Comprehension 3 Analysis 1 Evaluation

Five Objectives: 1 Comprehension 2 Analysis 2 Application

Course Requirements Attendance/Participation (5%) Exams (40%) Reflection Papers (35%) Cultural Event Paper (20%)

Discussion/Activities (20%) Quizzes and Exams (30%) Film Review (10%) Cultural Comm Paper (20%) Cultural Exp/Interview (15%) Case Study (5%)

Analysis/Observations Points are awarded for attendance in class every day. Attendance/Participation is less than 5% of the final grade. 40% of the final grade is based on exams. The course requires at least 15 pages of writing.

The online course requires more application of concepts to the analysis of a film, case study, and cultural events or interviews. There are more varied activities that move beyond the textbook. 20% of the grade is based on participation (online discussion and activities). 30% of the grade is based on exams/quizzes. The course requires at least 20 pages of writing, plus weekly written discussion board responses.

CAH: Course B Objectives Objectives not itemized but address

these cognitive levels: Knowledge Comprehension Application Analysis Synthesis

Six Objectives: 1 Comprehension 3 Application 1 Analysis 1 Synthesis

Course Requirements 1 Proposal/ 3 Reports (70%) Oral Presentation (10%) Exams (15%) Discussion Board (5%)

1 Proposal/ 2 Reports (35%) Resume (10%) Exams (30%) Participation: Discussion, Group Work, Peer Edit (20%) Research Bibliography (5%)

Analysis/Observations Information was not provided in the syllabus on the number of written pages required, but considering this is a writing course and the number of assigned reports, the written work must be substantial. The course has a hybrid dimension with participation (5%) in online discussion board.

Information was not provided in the syllabus on the number of written pages required, but considering this is a writing course and the number of assigned reports, the written work must be substantial. Participation is 20% of the grade and based on discussion board


College/Course Face-to-Face Online The F2F course includes an oral presentation.

posts, group exercises, and peer responses, so there is a high level of collaboration. The online course has a higher percentage of the grade (30%) based on exams.

COTS: Course A Objectives Five Objectives (both courses

standardized): 3 Comprehension 1 Application 1 Evaluation

Five Objectives (standardized for both courses): 3 Comprehension 1 Application 1 Evaluation

Course Requirements Discussion Board (30%) Paper (30%) Exams (40%)

Discussion Board (20%) Group Activities (20%) Quizzes and Exams (60%)

Analysis/Observations The course has a hybrid dimension with weekly participation (3 days/week) in online discussion board accounting for 30% of the grade. The course requires at least 15 pages of writing, including 5-7 page paper and 1 page/week for 8 discussion posts.

The online course has a higher percentage of the grade (60%) based on exams. The online course includes small group activities. The course requires no formal paper, but does include at least 10 pages of writing (1 page/week) in discussion board posts.

COTS: Course B Objectives Objectives not itemized but address

these cognitive levels (both courses standardized): Knowledge Comprehension Application

Objectives not itemized but address these cognitive levels (both courses standardized): Knowledge Comprehension Application

Course Requirements Participation Class/Online (10%) Exams Online (40%) Interview/Project (40%) Quizzes Class/Online (10%)

Modules Discussion (15%) Exams (40%) Interview/Project (45%)

Analysis/Observations The course is at least 20% hybrid with one class meeting replaced by online activities (quizzes, exams, discussion board). Information was not provided in the syllabus on the number of written pages required for the project or the discussion board postings.

Both courses are taught by the same instructor and therefore are almost identical. The only major difference is that the online course does not include quizzes.


College/Course Face-to-Face Online CEPS: Course A Objectives Four -

1 Knowledge 1 Comprehension 2 Application

Four - 1 Knowledge 1 Comprehension 2 Application

Course Requirements Attendance & Participation (17%) Exams and Quizzes (41%) Assignments (17%) Discussion Questions (8%) Writing Project/Assignment (17%)

Reading Units (chapter quiz, reaction paper, discussion) (44%) Modules (reaction papers, summary, exam, practice exercises) (14%) Discussion Board (14%) Writing Project/Assignment (28%)

Analysis/Observations Almost 20% of final grade is determined by attendance and class participation, although it is unclear as to how points are derived from such participation. 33% of the final grade is composed of assignments, quizzes, and discussions through online LMS.

Both courses are taught by the same instructor, although participation within the online course is measured less subjectively than in the face-to-face course. The testing (quiz/exam) element is more predominant in the face-to-face course.

CEPS: Course B Objectives Seven -

7 Synthesis Seven – 7 Synthesis

Course Requirements Journal Entries (48%) Weekly Assignments (30%) Posted Opinions/Replies (22%)

Journal Entries (48%) Weekly Assignments (30%) Posted Opinions/Replies (22%)

Analysis/Observations The course has heavy hybrid dimension with some online replacement (50%) of class time. All assignment grading derived through online LMS.

Both courses are taught by the same instructor, and they do not seem to differ for all practical purposes.

CB: Course A Objectives Four –

2 Comprehension 2 Application

Four – 2 Application 1 Analysis 1 Evaluation

Course Requirements Quizzes (80%) Book/Workbook Assignments (10%) Homework Assignments (10%)

Quizzes (30%) Discussion Forum (15%) Article Summaries (25%) Research Paper (25%) Topic Presentation (5%)

Analysis/Observations Objectives and grading system are primarily focused on comprehension as most of grade is derived from quizzes. Homework is completed

Varied course requirements and grading practices focused on higher order thinking skills. Writing assignments make-up


College/Course Face-to-Face Online through online LMS. 70% of grading as compared to

10% for face-to face course. CB: Course B Objectives Seven –

1 Knowledge 3 Comprehension 2 Application 1 Analysis

Four – 1 Knowledge 2 Comprehension 1 Application 1 Analysis

Course Requirements Short Essay Assignment (15%) Attendance/Participation (5%) Examinations (80%)

Discussion Forums (25%) Mini Analysis Projects (20%) Quizzes (25%) Examinations (30%)

Analysis/Observations Course is centered around seminars that are discussion, activity, and information based. Multiple choice tests are primary means for assessing student knowledge and comprehension.

Although more than half of the course requirements are test based, these include short response essays. Other requirements are writing-centric and focused on using research and information literacy skills.

QUALITY ASSURANCE OF ONLINE LEARNING · 12‐8‐11 CWU Quality Assurance of Online Learning Page 3...

Documents

Transcript of QUALITY ASSURANCE OF ONLINE LEARNING · 12‐8‐11 CWU Quality Assurance of Online Learning Page 3...