Post on 23-Dec-2015
Stephen Jones, David Monson, & Scott HowellBrigham Young University
Educause, October 4, 2002
Atlanta, Georgia
Copyright Statement
Copyright Scott L. Howell, Stephen M. Jones and David W. Monson, 2002. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.
Three Presentation Objectives/Questions
1. Stephen Jones - Introduction
2. Scott Howell What happens pedagogically to teaching and learning when a large-scale hybrid course redesign comes to campus? (Pew)
3. David Monson What literally happens to teaching and learning when a CMS system comes to campus?
4. Stephen Jones What are faculty and students saying about the impact of the CMS upon their teaching and learning?
Stephen Jones
Off CampusOff Campus
On CampusOn Campus
Independent LargeStudy Course Redesign
Off CampusOff Campus
On CampusOn Campus
Stephen Jones
Strategy—leverage technology-enhanced learning for campus, improve GE courses, experiment with cost/space savings
Strategy—”extend the blessings of learning,” anywhere, anytime350 Courses, 120 web-based25K enrollments
(Semester Online)
Independent learning model Inexpensive, scalable Content is of high quality Award winning Faculty can’t edit Doesn’t fit well on campus
Hybrid learning model Expensive Media rich, interactive Award winning Faculty can’t edit Doesn’t fit well off campus Long production cycles
Independent LargeStudy Course Redesign
Off CampusOff Campus
On CampusOn Campus
Stephen Jones
(Semester Online)
Faculty-developedFaculty-developedusing Blackboardusing Blackboard
Off CampusOff Campus
On CampusOn Campus
Stephen Jones
Independent LargeStudy Course Redesign
(Semester Online)
Hybrid model Inexpensive Faculty controlled Developed through iteration Leverages existing content Emphasizes student learning activities
over expensive media development
Faculty-developedFaculty-developedusing Blackboardusing Blackboard
Off CampusOff Campus
On CampusOn Campus
Stephen Jones
ResultFaculty gain confidence in teaching with technology, develop and maintain their own courses, and leverage the power of the CMS for improved learning.
Independent LargeStudy Course Redesign
Faculty-developedFaculty-developedusing Blackboardusing Blackboard
(Semester Online)
Stephen Jones
Independent LargeStudy Course Redesign
(Semester Online)
Off CampusOff Campus
On CampusOn Campus
The content faculty create can be further The content faculty create can be further developed for on and off campus use.developed for on and off campus use.
Faculty-developedFaculty-developedusing Blackboardusing Blackboard
Scott Howell
“The move toward a hybrid education has been quieter than much-hyped efforts to create completely virtual programs. Graham B. Spanier, president of Pennsylvania State University, calls the convergence of online and resident instruction "the single-greatest unrecognized trend in higher education today," and he touted it as part of the vision for his university in a speech last year.”
“'Hybrid' Teaching Seeks to End the Divide Between Traditional and Online Instruction By blending approaches, colleges hope to save money and meet students' needs”
Jeffery R. YoungChronicle of Higher Education, 3/22/02
“Within five years, you'll see a very significant number of classes that are available in a hybrid fashion," says John R. Bourne, a professor of electrical and computer engineering at Franklin W. Olin College of Engineering who is editor of the Journal of Asynchronous Learning Networks. "I would guess that somewhere in the 80- to 90-percent range of classes could sometime become hybrid." And he says he expects to see more students choose to take online courses even if they live on campus.”
“"A strong case is beginning to be made on the basis of research evidence that many students learn better online than face-to-face, and therefore a mixture is the best way," says Mr. Dede. "What proportion that mixture should be would vary from course to course.”
The Pew Program in Course Resign
A $6 million award program conducted by the Center for Academic Transformation at Rensselaer Polytechnic Institute with support from the Pew Charitable Trusts.
Encourages colleges and universities to redesign their instructional approaches using technology to achieve cost savings as well as quality enhancements.
Focuses on large-enrollment, introductory courses, which have the potential of impacting significant student numbers and generating substantial cost savings.
The Pew Program in Course Resign
BYU Awarded $200,000 in Round 3 (2001) to REDESIGN English 115: Freshman English Composition Course
Show PEW Video (5 min 45 seconds)
Instructor Quality andCurriculum Varied Narrowly
Before Redesign After Redesign 42 Classroom Meetings
170 Classrooms(7140 classroom hrs/yr)
2 Faculty/Student Conferences/Yr(15 Minutes)
154 Graduate Instructors/Yr(Classroom Size = 20)
0% Cost Savings
Instructor Quality and Curriculum Varied Widely
Instructor Time Significantly Higher
10 Classroom Meetings
45 Classrooms (1360 classroom hrs/yr – 74% Reduction)
120 Graduate Instructors/Yr(Classroom Size = 25)
40% Cost Savings
Instructor Time Significantly Lower
4 Faculty/Student Conferences/Yr(30 Minutes – 400% Increase)
English 115 (per year)
25% reduction in instructor timeCategory OnlineTraditional
Preparation 101.9 min/week117.56 min/week
Instruction 38.5 min/week132.82 min/week
Grading 146.25 min/week164.87 min/week
Office Hours 0 min/week147.69 min/week
Email 110.38 min/week17.31 min/week
Conferencing 75.63 min/week73.08 min/week
Development 27.38 min/week39.49 min/week
Total 500 min/week692.82 min/week
Two recent findings: First Finding
Two recent findings: Second Finding
“Based on our initial pilot we are encouraged that overall paper quality is higher in the online versus the traditional version of the course. We are also encouraged that there was a significantly higher rating for the introductions and conclusions and focus and organizations. Because these are global writing skills that were a focus in the online modules we developed, we feel the online portions of the course helped improve student writing.”
David Monson
Course Management System Study
Conducted in the summer of 1999
Research team from the McKay School of Education’s Instructional Psychology and Technology Department
Email survey of 270 randomly selected faculty
12 In-depth Interviews
Usability testing of selected commercial products
Purpose of the Study
Identify a suite of online course management tools that:
• collectively meet the instructional needs of faculty
at Brigham Young University,
• address the administrative needs that support
online instruction, and
• are compatible with present and future university
technology standards for online instruction.
Study Goals
Determine present and future needs of BYU faculty for
online course management through a campus-wide
survey and in-depth interviews of faculty.
Use data about faculty needs to identify an ideal feature
set for a suite of tools and support for online course
management.
Identify what is being done at BYU to provide faculty with
online course management support.
Study Goals
Identify what is being done at top universities to provide
faculty with online course management support.
Evaluate commercial online course management products
using the feature set as criteria.
Identify the top products and conduct usability testing with
faculty.
Recommend to the university possible courses of action
for providing faculty with online course management tools
and support.
Functionality Ranked by Current Use
Communication Syllabi Announcements Instructional resources in the classroom Feedback to students Faculty-to-student collaboration Learning resources from other sources Research resources
Findings
Surveys and interviews showed the primary concern of faculty:
• lack of time to use the tools.
• lack of funds that might be used to pay students to do to the work for them, saving faculty time.
• university training and support of the tools provided.
• ease of use.
Usability Tests
Usability tests with BYU faculty from each college and school on-campus were conducted on the top three commercial products:
• Blackboard CourseInfo
• WBTSystems TopClass
• Educational Technologies WebCT
Results: all faculty ranked CourseInfo first for product
ease of use.
Blackboard Adoption and Support
50
450
600
920
716
1409
-190
10
210
410
610
810
1010
1210
1410
Winter '00 Fall '00 Winter '01 Fall '01 Winter '02 Fall '02
Co
urs
es
• Hardware-Software• Support• Marketing
23.9% of sections are using Bb (2,003 of 8,365)
43.4% of enrollments are also Bb enrollments (82,820 of 190,767)
81% of students are enrolled in a Bb course (29,563 of 36,492)
65% of faculty are using Bb (1,626 of 2,492)
Blackboard Adoption and Support
Stephen Jones
Growth of Hybrid Courses
80% of BYU students took one or more courses during the last year that were a mix of F2F and online.
19
36
22
11
6 7
20
29
12
4
12
22
0
5
10
15
20
25
30
35
40
0 1 2 3 4 4+
Per
cent
of R
espo
nden
ts
Freshman Seniors
Faculty User Types
716 courses created, Winter 2002
228 (32%) were empty course “shells”
488 had active users 72 “Power” Users 14.8%
180 Moderate Users 36.9%
236 Light Users 48.4%
(Winter 2002)
BlackboardTM Survey, Winter 2002
• Faculty• 100 most active faculty users
• 60 responses• Students
• All students in those classes • 1697 responses
BlackboardTM Feature Use Data
Announcements 78.3% Course Documents 76.0% Course Information 61.7% Assignments 49.6% Staff Information 45.5% External Links 30.3% Quizzes Online 24.0% Discussion Board 19.3% Groups 2.3% Virtual Classroom 0.6%
(Winter 2002)
Students Faculty
Improve access to course materials 83% 85%
Improve class communication (one and two way) 70% 83%
Provide more flexibility for instructor and students 63% 76%
Instructor just trying out BlackboardTM 26% 27%
Academic dept. encourages instructor to use it 20% 20%
Student demand 16% 4%
Why is Blackboard being used? (Check all that apply)
Student / Faculty Comparison
Yes 78.0%
No 17.3%
Undecided 4.7%
Overall, would you prefer instructors use BlackboardTM in their courses?
BlackboardTM Student Survey
BlackboardTM Instructor Survey
“Many students like BlackboardTM courses. They enjoy working at their own pace, any time, day or night. They like not having to attend class. But many miss the face to face interaction and discussion and the discipline required of a traditional class environment. “ – Audiology & Speech Pathology Professor
How do you think BlackboardTM has been received by your students?
(Winter 2002)
BlackboardTM Student Survey
Yes 57%
No 41%
Unanswered 2%
Do you think BlackboardTM is changing the way instructors teach their courses?
(Winter 2002)
“It creates a more encompassing learning atmosphere. The teachers can refer the students to much more material and in a much more interactive environment. Blackboard is awesome.”
“Materials are more readily available. Communication is easier. Quizzes online save valuable class time. More time is spent learning in class and not being tested.”
“In a positive manner, it allows for more discussion among classmates. We can share ideas, and then learn from the ideas of others. On a negative side, some teachers have put so much on the BlackboardTM, that it is assumed that we should find all of the answers.”
BlackboardTM Student Survey
BlackboardTM Instructor Survey
87% indicated a change in their practice
13% indicated no change
What is the most significant teaching practice you have (or will) change now that this tool is available?
(Winter 2002)
BlackboardTM Instructor Survey
Post materials online - 40%
Student comments
“I can access grades, quizzes, tests and many other teaching materials online that were otherwise unavailable to me before.”
“I can do it at any computer on campus. I don’t have to carry around tons of papers, and I know I won’t lose it.”
What is the most significant teaching practice you have (or will) change now that this tool is available?
(Winter 2002)
BlackboardTM Instructor Survey
Increase faculty-to-student and student-to-student collaboration and communication - 32%
Student comments “I can contact other students in the class much
easier through Blackboard. Group projects are much smoother when we have the use of Blackboard.”
“Using Blackboard has increased my team member skills.”
Faculty comment “I communicate with my students more frequently
now via Blackboard than I had in the past.”
What is the most significant teaching practice you have (or will) change now that this tool is available?
(Winter 2002)
BlackboardTM Instructor Survey
Provide quizzes and homework online - 10% Use to increase pre-class preparation - 5%
What is the most significant teaching practice you have (or will) change now that this tool is available?
(Winter 2002)
1. Group problem-based learning activities with online collaboration. 2. Organizing out-of-class time. 3. Preparing students for class discussion--they post reactions to reading before coming to class.” – Political Science Professor
I am able to encourage students to learn from each other and to share at a deeper levels. My goal with this course has always been to encourage the use of critical thinking skills. Using BlackboardTM has allowed me to facilitate this type of interaction on a regular basis. Also, I am able to adjust course content to meet the current needs of the student interns. -- Family Science Professor
What is the most significant teaching practice you have (or will) change now that this tool is available?
BlackboardTM Instructor Survey
Are you meeting less often in class and more often online?
StudentsFaculty
Yes 26% 18%
No 74% 82%
Student / Faculty Comparison
BlackboardTM Student Survey
“Instead of three times a week, we meet once. But this is great, I have no complaints. I'm learning just as much if not more. I can go at my own speed.”
“My Org. Behavior class has all of our assignments and quizzes on BlackboardTM , with optional classroom lectures six times throughout the semester. I liked it a lot, I could complete my quizzes and things on my own time.”
Student comments
(Winter 2002)
BlackboardTM Instructor Survey
“We'll be in class two days a week instead of three. One day a week is an 'e-day.‘” – Political Science Professor
“Spring Term 2002 I will use the virtual classroom in three 1/2 hour periods during the week and allot additional time for electronic communication with students.” – Zoology Professor
“Yes, I meet more often online. Monday we meet in person, Wednesday in the virtual classroom and Friday my students post to the Discussion Board. – English Instructor
(Winter 2002)
Faculty comments
BlackboardTM Student Survey
Yes 13%
Not changed 78%
Decreased 7%
No response 2%
(Winter 2002)
Overall, is your workload as a student changed by an instructor’s use of BlackboardTM ?
BlackboardTM Student Survey
“I believe that they think it's easier for us to get more done so all of my teachers who use it pile more work into it.”
“I do more in my on-line class then my friend does in the same class, because there are always tests to make sure that we understand the material and are staying caught up.”
“I believe that with BlackboardTM , I am able to better manage my homework situation; therefore, I have been able to decrease my workload.”
“It is easier when I can use BlackboardTM. It saves my time a lot!”
(Winter 2002)
Student comments
StudentsFaculty
Positive comments 68% 78%
Negative comments 2% 12%
Neutral 1% 0%
No response 29% 10%
Can you think of at least one example of BlackboardTM use that has helped your learning / teaching experience?
(Winter 2002)Student / Faculty Comparison
BlackboardTM Student Survey
“I like being able to read other's responses to what we discussed in class.”
“It allows me to access information when I need it. Timing is very important... If I'm working on a project, but am behind a little I can access the information at my own pace.”
“It has made communicating with teachers, TAs, and classmates MUCH easier. Specifically when we are doing group projects BlackboardTM allows us to have a common meeting place.”
“It allows me to take quizzes at my convenience.”
Student comments
(Winter 2002)
BlackboardTM Instructor Survey
“I think I'm doing a better job of focusing the students' learning experience and delivering a specific and meaningful learning outcome.” – Organizational Leadership Professor
Faculty comments
(Winter 2002)
BlackboardTM Student Survey
58% responded with at least one comment
42% did not respond
Can you think of at least one example of BlackboardTM use that has frustrated your learning experience?
(Winter 2002)
BlackboardTM Student Survey
30.8% - technical problems
5% - Student dislike for online instruction
4.4% - Professors not updating courses enough to be helpful
2.9% - Teachers post assignments right before class
2.8% - Hard to get computer access
Top-ranking categories of negative responses
(Winter 2002)
BlackboardTM Student Survey
“I prefer more interpersonal communication as opposed to the mass communication style of BlackboardTM when it comes to my education.”
“Sometimes instructors expect that if they post something on BlackboardTM right before class we will have time to look at it. It seems to encourage professors to procrastinate.”
“…sometimes navigation of the site isn't that easy. Many professors put things in different locations.”
“It's only frustrating when I can't get into it. Like, when…the system is down for a while...”
Can you think of at least one example of BlackboardTM use that has frustrated your learning experience?
(Winter 2002)
BlackboardTM Instructor Survey
“Has provided ease in communicating with class through mass emails, tracking grades on-line, ect.” – Family Science Professor
“The last test… proved clearly that those who used the online materials did very well. Those who didn't had test scores MUCH lower.” – Italian Instructor
“It makes it so that I don't have to be personally available quite as often, which is definitely valuable when you are teaching 200 students in one course.” – Psychology Instructor
Has this tool helped create any efficiencies in managing your courses?
(Winter 2002)
BlackboardTM Instructor Survey
“I, and my TA's, use the Communication link to contact students who have unusually high absences. We encourage them to come and see us. In a large class (166 students) it is way too easy to get lost and lose interest…” - Political Science Professor
“Hybrid is the way to go - part online for efficiency, part classroom for personal touches.” - Counseling & Special Ed Professor
General comments and observations about your BlackboardTM experience including the tool itself.
(Winter 2002)
Seven Principles of Good Practice
Good Practice Encourages Contacts Between Students and
Faculty Develops Reciprocity and Cooperation Among
Students Uses Active Learning Techniques Gives Prompt Feedback Emphasizes Time on Task Communicates High Expectations Respects Diverse Talents and Ways of Learning
Arthur Chickering and Stephen EhrmannAAHE Bulletin, October 1996
“My ASLP teacher has used the discussion board a lot, and I really think that helps in learning. I find it really beneficial to be able to see what other students think and what they are getting out of our assignments.”
Freshman
Contact, Cooperation, Active Learning, Feedback, Diverse Ways
Time on Task“It allowed me to listen in my religion class, and not get caught up in taking notes, . . .”
Freshman
“. . .we could go at our own pace and rate of understanding rather than having to keep up with the class (either being a faster or slower learner than the rest of the class.)
Senior
“. . .at least no time is wasted in class trying to figure out who has what grades.”
Senior
“The more contact I make with them through technology, the more my students seem to believe that I am readily accessible to help them…” – English Instructor
Virtual can = Available
Contact
“Helped me to become more technically literate.”
Freshman
Diverse Ways
“If I can’t remember if we have a homework assignment. No Worries. I don’t stress about it and cry, because I know that I can always go to Blackboard and look up what my assignment is. It has saved me a lot of useless tears.”
Freshman
Reduced Stress
Conclusions, Observations
The ability to extend the classroom using a virtual environment can create a seamless, connected experience for students that Helps organize their out of class time
Connects them to each other and the professor
Puts them in control, offers choice
Greatly improves their access to information
Conclusions, Observations
Teaching a class using a CMS causes faculty to Think reflectively about their practice Align content, assessment, and learning activities Think about the type and quality of student-to-
student interaction needed outside of class Use assessment more strategically and make
grades (and rubrics) more public
Video
Next Steps, Future Opportunities
Move from Uploading documents to creating and publishing
learning objects Online content (“Putting my course online”) to
communities of practice Individual, opportunistic to institutional, strategic
course development Tool training to pedagogical dialogue, improvement “Slide, slide, quiz” to adaptive assessment, intelligent
tutoring Move from hybrid / blended/ e- / online learning to
improving learning Focus away from the technology and toward the goal of
improvement