Analysis of the Blended-Learning Model: Global Perspectives Pilot Study

Meridian Joint School District No. 2

Meridian, Idaho

By. Dr. Eian Harm, Research Coordinator

Tobey Jossis, M.Ed., Virtual School House Coordinator

Lori Gash, M.Ed., Social Studies Coordinator

1

Introduction

Purpose

As part of an initiative undertaken by Meridian Joint School District No. 2 (MJSD) to create an

online learning program, the district has created the Virtual School House. The Virtual School House

provides students with a valuable alternative to the traditional classroom instructional model. Within

the Virtual School House a pilot study was created aimed at general education students in the 9th grade.

As part of this program a yearlong pilot study to analyze both a completely online version as well as a

blended-learning model of a Global Perspectives (Social Studies) class was created. Students were

randomly chosen and asked if they wanted to participate in the online or blended version of the course.

The blended-model classroom as currently created within MJSD is one where students earn 70% of their

grade through online instruction within Meridian’s Virtual School House. The remaining 30% of the

student’s grade is the result of in-class instruction and projects performed within a face-to-face

classroom. This study seeks to analyze the effectiveness of the blended-learning pilot, gather results to

improve the online and/or blended-learning program, and to glean some sense of the benefits of online

education models (online v. blended-models) for various types of students throughout MJSD. The

MJSD’s Virtual School House has the goal of continually improving online education within the district as

well as to spread knowledge of online education for others to avoid future roadblocks. The two parts of

the study consisted of analyses within the following aspects of MJSD’s Global Perspectives course and

Virtual School House:

An initial pilot study of a completely online version of the Global Perspectives class. This

class was held at Centennial High School. The teacher of the class taught two (2) completely

online versions of the class as well as six (6) traditional in-class models.

A subsequent blended-learning model pilot study (2/3 online, 1/3 in-class application)

during spring semester of the 2012/2013 school year.

2

Evaluation Questions

1. Does the Effectiveness Index (EI) Model developed by Mashaw (2012) provide an effective

method to assess online and blended-learning instruction?

2. Is the EI survey that was created by Meridian School District a valid and reliable tool for

future use within the Virtual School House?

3. How can the EI Model and survey help to improve the blended-learning model?

Literature Review

Much of the current literature concerning online or “virtual” education is focused upon students

at the university and college level. An attempt to quantify the effectiveness of these online courses has

begun recently with several efforts to validate measurement techniques and develop models for

evaluation and improvement (Artino & McCoach, 2008; Mashaw, 2012). A framework created and

tested by Mashaw (2012) sought to develop a simple and accurate model for measuring the

effectiveness of online courses. This model, called the Effectiveness Index (EI) brings together much of

the literature on both teacher and course effectiveness. The author has incorporated the following

concepts to develop a framework with which to evaluate online learning programs. This framework

focuses upon:

1) Student learning experiences.

2) Content of the course.

3) The impact of technology.

4) The ability for students to interact, participate, and receive feedback.

5) The teacher or mentor’s impact.

6) Hindrances or discouragements encountered.

3

This framework can be used not only to collect data on performance within the Virtual School

House, but also be used as a way to systematically improve it as well. Additionally, the EI framework can

be used as a model to assess the effectiveness of other computer and technology-based initiatives

within the district.

The Impact of Online Learning

Studies have shown the link between participation and other social aspects of learning with their

impact on student satisfaction and effectiveness of a class (Alavi & Dufner, 2005; Richardson & Swan,

2003). This knowledge matches well with the currently accepted social constructivist theory of learning

(Driscoll, 1994). This theory emphasizes the necessity of full psychological, physiological, and emotional

student engagement in the learning process. Social-constructivist learning is therefore impacted by the

methods and the environment in which the learning occurs (Driscoll, 1994; Mashaw, 2012). Since online

learning, by nature, involves modifying the typical school environment and decreasing the social impact

of learning, much research as to its implementation and effectiveness is therefore necessary. As

Schaber, Wilcox, Whiteside, Marsh, & Brooks (2010) state “the biggest challenge [is] to design an online

learning experience that [will] have the same positive outcomes as the classroom experience” (p. 5).

This, of course, needs to be accomplished without replicating much of what doesn’t work in the

traditional classroom. The success of internet-based education will depend on teachers and educational

leaders transforming the student learning experience through the use of technology. Simply placing

information online will not lead to student success—the use of dynamic and creative methods for

students to interact with that same material is what is needed (Dixson, 2010).

Creating effective online courses

As stated previously, it is the transformation of teaching and learning, enabled through the use

of technology that will ultimately lead to effective and successful online and blended learning programs.

Simply recreating traditional methods will not increase student learning solely by the fact that the

4

information is now “online”. Prior research has begun to shed light on some of the aspects of online

and blended education that have been found to be most successful- however further research into

these new methodologies and how to impart them to teachers is needed. This study, in part, is seeking

to find and measure the factors that are most closely associated with successful online learning and

determine their impact on student outcomes. Many of these factors are the same as those associated

with good traditional classroom teaching as well-- the titles have merely been changed to highlight their

application to technology. What is intriguing and of interest to this study is the impact of the technology

on our 21st century students. A study by Mashaw (2012) highlights the technological factors that most

influence learning. These were found to be the design and usability, the degree of interaction enabled,

and user control (self-pacing) enabled through its use. In fact, the research by Mashaw has been

corroborated by preliminary survey analyses by MJSD that has shown a strong correlation between the

technology and student perception of both the course as a whole and its effectiveness compared to a

traditional class. This fact alone leads one to see the importance of choosing proper technology,

allowing students to work at their own pace within it, and assuring its “user friendliness”. Improvement

within this category is an area that is unique to technology-based instruction.

Another of the most impactful aspects of student perception and engagement with online

learning is that of participation and feedback both between the teacher and other students (Dixson,

2010; Lloyd, Dean, and Cooper, 2009; Mashaw, 2012; Schaber et al., 2010). Much of the recent

cognitive psychology research has pointed to the importance of interpersonal connections helping to

increase motivation and learning (Driscoll, 2010; Seigler, 1998). It is essential then, to find methods

which increase this connection and feedback within the online environment. The development of active

learning strategies through collaboration, group discussion, and student to student interaction have

been shown throughout the literature as effective and essential methods to promote success (Dixon,

2010; Gayton and McEwen, 2007). Interestingly enough, studies by Gayton & McEwen (2007) and

5

Young (2006) have shown that too much instructor presence in collaborative and discussion

environments could possibly decrease student participation. Other research, however, shows that

instructors need to be present in a “one-on-one” manner with individual students in order to prevent

student feelings of isolation (Dixson, 2010; Garrison, Anderson, and Archer, 2000; Lewis and Abdul-

Hamid, 2006). In light of this information, instructors should set up environments that foster interaction

between students while providing consistent and meaningful instructor feedback at the individual level.

Much of the literature concerning online education indicates that the technology used (variety

of presentation tools, user friendliness, ability to work at one’s own pace, unique learning methods

enabled) and interpersonal connections created within the course have a strong effect on student

perception and effectiveness of online learning. It may be that these factors build the “foundation” of

student perception upon which the entire learning experience is built- students who feel connected and

interested in the learning process are more open to the learning experience and will therefore succeed

at higher levels. MJSD’s Virtual School House will therefore continually work to find ways to improve its

online program with an emphasis upon these two factors, highlighting its impact on the overall learning

experience. Since the implementation of online learning is new to the MJSD, research and analysis of

these methods is imperative. It is for this reason that we seek to study the effectiveness of our online

programs and those within the Virtual School House. This study will examine the overall effectiveness of

the blended-learning course, be used as a framework to modify and improve Meridian’s online courses,

as well as collect data to analyze the use of online learning models at the secondary level.

Methodology

Data Collection and Analysis

In order to test the validity of the survey and quantify the effectiveness of the completely online

course, a survey was developed based on the Effectiveness Index Model (EI) created by Mashaw (2012).

The validity and reliability-tested survey was given to 34 students (n=34) who participated in the

6

completely online version of a global perspectives class during the fall semester (2012-2013) in MJSD.

The survey was presented in “hard-copy” format and later entered into the Excel program for analysis

after students completed the class.

Survey reliability and validity. Preliminary analyses of the survey sent to students participating

in the Global Perspectives completely online pilot study showed strong reliability with regard to each

category of the EI assessment framework (factors). These values were determined through a

Cronbach’s Alpha analysis. Values of factor reliability (alpha values) for the Effectiveness Index model

are shown in the table below.

Table 1. Cronbach’s Alpha Values for Effectiveness Index (EI) Factors

Factor Number of Questions Alpha Value (α)

Learning Experiences 5 .907

Contents of the Course 4 .819

Technology Used 5 .885

Participation/Communication/Feedback 3 .742

The Instructor 5 .808

Discouragements 5 .859

The high alpha values for each factor associated with the Effectiveness Index (EI) indicate that

this survey tool is a valid and reliable method for measuring how students perceive the various aspects

of the online course. All values exceeded the typically accepted value of 0.70 as a baseline for reliability

(Garson, 2012).

As a result of the preliminary analysis for the Effectiveness Index (EI) assessment of the Global

Perspectives fully online course, it can be assumed that the survey tool used from Mashaw (2012) is

valid and reliable. This method of obtaining data related to student perception among various aspects

7

of the online course held high internal reliability (.742 to .907) among the six constructs of the model.

This survey tool can be confidently used in future analyses of online instruction within MJSD.

Survey data analysis. Survey data was collected via the K12 Insight internet-based survey

program. The modified EI survey was entered into the program and a link embedded into the online

portion of the global perspectives course. This occurred at two different times during the blended-

learning course spring semester of the 2012-2013 school year. First, the survey was sent at the mid-

point of the semester (after students had acquired adequate knowledge of the blended-learning

process, then again at the culmination of the school year. The survey was opened on the two weeks

prior to the end of the school year and closed on the final day of school. The data was imported into the

SPSS statistical program, reverse coded items were corrected, and Cronbach's Alpha values found to

ensure reliability above .70. Since only the blended-learning classes took the survey due to it pertaining

to only online learning, no comparisons were made between blended and traditional classrooms using

the survey data. Instead descriptive statistics were run based upon questions within each aspect of the

EI framework (see appendix A and B). Mean values were found as well as standard deviations for each

individual category. Also a composite EI value was found through the following equation:

Effectiveness Index (EI) = (5 X Learning experience)+ (Engagement with Content)+ (Technology) +

(Participation) + (Instructor Inspiration) – (Discouragements)

The idea behind this equation is that the overall learning experience should be equal to all other

portions of the course combined (Mashaw, 2012). This, in effect, makes the learning experience

(relevance, course content, depth of learning) the most influential portion of what is measured under

the EI context. Next, descriptive statistics were found for each of the individual questions used on the

survey. This enabled a more detailed picture of what students were highlighting in particular for positive

or negative notions within the course. These values were entered into tables and analyzed.

8

Grade data analysis. Grade data analysis was performed by downloading total course grades

via the Power School program. Grades were collected from all of the participating teacher’s Global

Perspectives courses taught at three points throughout the semester. For this pilot study a single

teacher was used to make the grade comparison to reduce the variation that would arise between

different teachers. In essence this was to see if given the same teacher, blended learning was more or

less effective when compared to traditional classroom models. The grade data was segregated based

upon the blended and traditional classes and an initial t-test was performed to look for overall

differences between means. Once this was performed, 9th grade students were isolated and means

were compared via t-tests between the two groups (blended and traditional). Finally, an ANOVA analysis

was performed looking at differences between grade levels in order to determine the impact of this

variable on overall course grades. It was found that the grade level did indeed significantly affect group

means. It was therefore determined that only 9th grade students would be used to make analyses of

mean grades between the blended-model and traditional class groups, which were a majority of the

students.

Results

Correlations and EI value influence on student perception

Through analysis of the Global Perspectives blended-learning student survey data it is evident

that all of the categories of the EI index play a direct role in how students rate the class overall. This is

shown through the strong positive and significant correlations that each factor has with the overall

evaluation for the course. This shows that all of the factors from the EI model play a direct role in

student perception of the online learning experience. It should be noted that there was a strong

correlation between the technology used (variety of tools, self-pacing, display, ease of use) and the

student perception of the online class (r=0.74). Very high correlations were found between the overall

learning experience with both the instructor’s performance (r=0.70) and the course content (r=0.66). As

9

would be expected, negative correlations were found between the discouragements encountered

throughout the course and both the overall experience and the comparison between blended and

traditional classrooms. The correlation with discouragements was small in nature indicating that the

other pieces of the EI framework were more relevant to the overall experience when compared to the

“negative” aspects.

Table 2. Pearson Correlation Coefficients for Effectiveness of Blended Learning Course with EI Category

Perceived Experience within Class vs. EI category

Learning Experience Engagement w/

Course Content Technology Participation Instructor Discouragements Total EI

0.72* 0.66* 0.74* 0.71* 0.70* -0.35* 0.77*

Comparison of Blended Classroom with Traditional Classroom vs. EI Category

Learning Experience Engagement w/

Course Content Technology Participation Instructor Discouragements Total EI

0.64* 0.49* 0.44* 0.69* 0.57* -0.25 0.64*

Blended learning pilot study

Grade Comparison and Analysis. A grade comparison and analysis was made once during the

fall semester and twice throughout the spring 2012-2013 semesters. This was performed in order to

gain information regarding student outcomes between the two models of the course. This comparison

was obtained from both students within the fully online, blended-learning course, as well as the

traditional course. Table 3, table 4 and figure 1 below show the grade comparison over time for the two

classroom models.

Table 3. Fall Semester 2012/2013- Fully online model vs. traditional classroom grade comparison (as mean class grade point average and standard deviation)

Fully Online Model (n=37) Traditional Classroom

(n=51)

Grade Mean Scores & Standard Deviation

3.22(.33) 3.38(.55)

* mean group grades for fall semester were not significantly different

10

Table 4. Semester 2- Blended learning model (n=46) vs. traditional classroom (n=76) grade comparison (grades as Grade Point score)

Blended Model Traditional Model Effect Size (Difference Between Groups)

Date

Cohen's d Relative Size of effect

1/21/2013 2.02(1.32) 3.20(1.24) -0.95 Large

4/3/2013 1.67(1.41) 3.17(1.25) -1.20 Large

5/31/2013 2.11(1.34) 3.12(1.15) -0.88 Large

* all findings showed a significant difference between blended and traditional models at p <.05 level.

Figure 1. Grade Comparison of blended model vs. traditional model through semester.

These data show that there were significant differences between group grades throughout the

course of the year with the blended learning model having overall lower mean scores. The values and

effect sizes fluctuated throughout the course of the year starting at -.95 (blended classroom having a

lower mean than the traditional group) after the first month of the course. The effect size grew to 1.20

1.50

1.70

1.90

2.10

2.30

2.50

2.70

2.90

3.10

3.30

3.50

Gra

de

Po

int

Ave

rage

Date

Grade Comparison of Blended and Traditional Classrooms throughout Semester 2012-2013

Blended Course Grade Average

Traditional Course Grade Average

11

at approximately the midpoint of the semester. As modifications to the course were made in April (see

Appendix C), the effect size and difference decreased through the end of the semester to -.88 with the

blended model (M=2.11, SD=1.34) having lower overall means than the traditional classroom (M=3.12,

SD=1.15). The graph above (figure 1) shows the slightly steeper slope of the line for the blended

learning group indicating that these students were increasing in grade points more rapidly than the

traditional course. This is following the interventions implemented within the course such as decreased

assignment load and the ability for students to turn in late work. Though this opportunity was given to

students (make up work), they still ended the semester with a mean grade point average that was

significantly lower and large in effect size when compared to the traditional classroom. The semester-

end difference was however, the smallest difference between groups throughout the entire semester.

Student Survey Data. Student survey data was segregated into the five values determined by

the EI framework. Additionally, an overall EI value was found for comparison within particular years and

for comparison across multiple years. Values found from the survey across the course of the semester

showed that the overall EI value decreased slightly from the initial student survey on March 18th, 2013

to the final survey on May 27th, 2013 (see figure 2 below). Values that showed more negative student

perceptions (when compared to the initial survey) were found for the categories of learning experience,

students’ engagement within the course, and their interaction with the technology. Increases,

indicating more positive student attitudes were found for the participation and communication within

the course as well as the instructors’ performance and inspiration categories of the EI framework.

Additionally, there were less reported discouragements on the final survey.

12

Table 5. Composite Effectiveness Index (EI) values compiled from student survey (based on 1-6 Likert Scale; 3.5 is neutral or average value).

EI Factor Mean and Standard

Deviation as of 3/18/13

Mean and Standard

Deviation as of 5/27/13

Total Effectiveness Index EI 108.58 (44.90) 91.48 (16.77)

Learning Experience 3.02 (1.07) 2.90 (0.87)

Course Engagement 3.29 (0.97) 3.20 (1.15)

Technology 3.07 (1.26) 2.90 (1.03)

Participation/Communication

/ Feedback

2.94 (1.33) 3.01 (1.29)

Instructor Performance

Inspiration Average

3.48 (1.09) 3.65 (1.18)

Discouragements* 4.44 (0.90) 4.39 (0.65)

* Lower discouragements value indicates more positive perception by students.

Figure 2. Student survey data comparison for EI category (average is 3.5)

2.00

2.50

3.00

3.50

4.00

4.50

5.00

Me

an S

core

EI Category

Student Survey Data Comparison for Effectiveness Index Category

3/18/13

5/27/13

13

Individual question analysis. Analysis of individual questions within each of the larger EI

framework variables can be seen in Appendix B. This was performed in order to assess the nuance

behind the EI framework values and to provide a focus on targeted improvement of the course based

upon these factors.

Conclusions

Based on observations, quantitative survey, and grade data within Centennial High School’s

Blended-Learning Global Perspectives course some patterns or themes arose that can and have been

used to modify the course for a future pilot study analysis. These themes are associated with the

technology and the variety of tools needed to maintain student interest, time allocated to assignments,

the notion of students working more deeply within highly relevant topics, collaboration & participation

with and among students, and finally training students to work effectively in an online and blended-

learning environment. It must be taken into consideration that all of these results are on the basis of a

blended-learning model enacted by a single set of teachers within a single school. It also must be

understood that these results are showing the inevitable “learning curve” that must be part of any new

endeavor. This research was performed in order to obtain a more clear understanding of where issues

will arise in the transition to a blended-learning model, as well as to be used for future improvement in

these types of courses.

Students in a blended learning environment

Online education has been shown, in many cases, to be more rigorous than face-to-face

instruction (Dixson, 2010). Since online courses are inherently different than the traditional face-to-face

model, students new to this type of teaching and learning will certainly need some formal introductory

training or even a class in itself designed around online teaching and learning. In fact, literature on

online education speaks to a need for students “to understand how an online course works” and for

14

them to be “comfortable with the independent, fast-paced format” prevalent in online instruction

(Dixson, 2010; Maki and Maki, 2007; Mandernach, Donnelli, & Dailey-Hebert, 2006). It seems from this

research that creating a successful model of blended-learning is not quite as simple as popular opinion

might suggest. The information gleaned from this pilot study seems to correspond with the more

rigorous research that suggests that students and teachers alike will require training and induction into

this method of learning. Interviews with the participating teachers (online and in-class) indicate that

students had to be offered direct guidance and instruction on how to promptly and properly complete

and submit assignments. During the spring semester there were in essence three teachers for the

course: one solely grading the online work, one observing the online classroom, and another working in

the face-to-face environment. The participating teachers stated that the in-class instructor needed to

be the online instructor as well. This enabled the in-class teacher to be directly involved with both the

students’ in-class portion of the course as well as what was accomplished online. This avoided the

perception that these were two separate courses. Several reasons for this necessity were highlighted.

First, this format enabled the teacher the ability to highlight and model successful online practices that

were certainly needed with freshman (9th grade) students. Evidence for this is also seen in the notion

that the previous semester’s fully online (non-blended) learning course had students who had outcomes

statistically equal to those of the traditional classroom (see table 3). In this format the typical (certified)

teacher was present during the entire class period and was grading the online portion of the work. That

being said, the in-class teacher explained how he felt that low grades during the blended-learning

portion were an in-class time management issue as evidenced by the facts that several students earned

“A” and “B” grades and that those who routinely worked on assignments at home were performing at

higher levels. What must be taken into consideration is the expectation, stated upon the onset of the

class, that some degree of work must be performed outside of school hours. Much of the previous

research indicates the need for students to be self-motivated and self-paced learners in the “fast-paced

15

and minimally directed learning environment” associated with online and blended learning

(Mandernach, et al., 2006). Our current findings seem to match this thought and indicate that a quality

teacher must be present in order to keep students on-task throughout the course. These findings may

be isolated to this age group (14-15 year old students), though further research would be necessary to

verify this claim. This concept may be isolated to students new to online and blended-learning or

possibly due to developmental stages associated with freshman students. In either case however,

currently our students need support in order to be successful in the online environment. With these

concepts in mind, an important question arises: if all students are expected to participate in some

aspect of online learning, what techniques or skills must we teach these students in order for them to be

successful in the fully independent online or blended-learning environment?

As Global Perspectives students are typically high school freshman (9th grade) it is highly likely

that course is an introduction to online learning. It seems that it is imperative that time is given for

teachers to model, and students to learn effective online techniques (Mandernach et al., 2006). This

process should create students who become increasingly independent in the online and virtual world

and develop the College and Career readiness skills so often talked about. This is not a process that will

occur without a targeted introduction to this media and a commitment to its successful implementation

by educational leaders, teachers, students, and parents alike.

Additionally, the blended-learning teachers stated that motivation was a key factor in student

success. The teacher stated that they were able to motivate students to a greater degree when the

online instructor was also the lead teacher in the in-class portion as well. The statement that the

teacher could stay “on top” of students to ensure assignments were completed and submitted

summarizes it best. It was hypothesized by the participating teacher that this was the reason there

were no differences between mean grade point averages during the completely online course during the

16

first semester. This motivation and instructor inspiration is seen as one of the main aspects of

successful online instruction and a major portion of the Effectiveness Index used to evaluate this course

(Mashaw, 2012). Instructor motivation and inspiration values were found to have a mean of 3.65 (1.29)

and were rated as the highest score given on both the mid-term and end-of-year survey. The significant

correlation between instructor inspiration when paired with the overall online experience (r=.70)

certainly highlights the importance of focusing on this criterion.

Depth of knowledge. Much evidence gathered from the students spoke to an issue regarding

the workload within the online course as seen by the initial number of assignments (total of 90).

Modifications to the amount of individual assignments were made at the mid-point of the course and

will be incorporated into the following year’s curriculum. For the following year’s pilot study a full 30%

of the assignments will be removed so that teachers may work to increase students’ depth of

knowledge, as opposed to simply working through high numbers of assignments. Through our survey,

students stated that on average they were working slightly harder in this course than they typically do in

other courses they participate in (mean of 4.26 out of 6, with 1 being much less than and 6 being much

more than their typical workload). Low values on the EI were given for all questions regarding workload

and pace of the course. Appendix D shows a “word cloud” created based upon students’ responses to

the question: “if there was one thing that I could change about this course what would it be”. The larger

words indicate those most often stated. It is obvious from this visual where students’ complaints were

focused- high numbers of assignments. Upon observation the students, even in the face of decreasing

grades and missed assignments, were still working on the project set before them in a calm, respectful,

and obedient manner. It is this willingness to participate even in the face of difficulty that leads this

researcher to believe that modifications to the workload are certainly part of the solution.

17

Utilizing the technology to create a depth of knowledge. Previous research has shown that the

online environment needs to transform learning for it to be successful. This concept points toward the

need for online educators to find new methods which inspire collaboration and participation among

both students and teacher alike. Dixson (2010) goes as far as to say that “collaborative [and] interactive

activities seem to be a necessary component of effective online instruction” (p. 2). For this reason it

seems imperative that in a blended-learning environment, students are engaged in collaborative

activities centered on relevant materials and information to the greatest extent possible. While in the

face-to-face portion of the class, the use of pedagogy such as problem-based learning scenarios (PBLs),

simulations, and Socratic Seminars are just some of the examples with which teachers can make the

topics “come to life”. It seems from the evidence regarding the Learning Experience framework

(M=2.90, SD=.87) that this relevance is extremely important. In fact, the notion that the Effectiveness

Index rates the Learning Experience equal to all other framework areas combined shows that this is the

most important area to focus upon. The lowest values within this category were found for questions

regarding students being inspired by the content to learn more on their own (M=2.61, SD=1.27). It

seems imperative that blended learning teachers attempt to create relevance and connection of the

material to students’ lives through multiple avenues. These could include use of the technology to

interact with students from other schools, states, or even countries, the integration of “real-time”

current events, the incorporation of local information to make connections to the content of the course,

or simply allowing students to have discussions in order to be exposed to other points of view. It is also

hypothesized that using internet-based methods to make these collaborative connections is something

that the 21st Century blended-learning classroom must embrace. Examples of this can be seen in

activities such as webinars, email communications, online posting, blogs, discussions, etc. Throughout

observation of this class, students were either taking an online exam or creating individual PowerPoint

presentations. Students wore headphones for a majority of the class period and the silence within the

18

room was something that was noted by the researcher several times as the observation was conducted.

In fact one student responded that the “room is always silent” and that they would hope to have “more

discussions in the classroom” as they “do not talk to anyone and don’t have the chance to.” This

participation and communication aspect of the blended-learning class is something that seems to be

essential to making the course an effective method of teaching and learning for our students (Alavi &

Dufner, 2005; Driscoll, 1994; Richardson & Swan, 2003).

The technology. The findings of this research coincide with previous studies highlighting the

importance of well working, easily manipulated, and engaging technology with its impact on the online

or blended-learning experience. Within this study there were strong and significant correlations

between the facets of technology and the students’ experience within the course (r=.74). The overall

value for this technology aspect was rated below average by students with a focus being on the lack of

ability to work at their own pace. This “pace” factor is written in much of the research literature as a

major benefit of online instruction enabling the differentiation and individualization of the material for

varied learners. Methods of allowing students to work at an individual pace while still being held to

complete work and stay on task must be formulated in order to improve this aspect of the online and

blended experience. Especially in the adolescent aged students, it will be imperative to develop

successful techniques that still maintain structure within the course, but allow at least the perception of

flexibility and control over pace.

Finally, issues associated with technology were highlighted. This evidence arose from both the

survey data as well as the classroom observations. Upon students entering the classroom they

obediently gathered their computers and began the login process. In many of the cases, it was five

minutes before students could begin working- and in one case that was noticed, it took over ten minutes

before the student was able to begin work. The idea of the teacher (or a student assistant) turning on

19

computers ahead of time is something that was discussed and the teacher attempted with the second

class period. The login times were substantially reduced as a result, though some comments were heard

about the internet not working- all issues that should be considered in the face of this emerging

technology and teaching methodology. Previous research shows that the level of difficulty that students

have with the technology side of the class (login, internet connection, properly working links) has a

direct and significant impact on the overall perception of the course (Mashaw, 2012). As mentioned

above, this matched our research in which we found strong and significant correlations to overall

effectiveness in this area. With this knowledge a continual focus should be made to reduce these

difficulties as they directly impact the overall effectiveness of the blended-learning classroom.

Implications for Joint District No. 2. Modifications to the Blended-Learning Global Perspectives

Blended-Learning class are planned for the 2013-2014 school year based upon these findings. MJSD is

going to continue with the pilot study of this blended learning model for another school year. With this

further pilot study several changes will be made and tested for improvement. First, modifications will

be made during the current semester to reduce workload, attempt to build a more collaborative and

interactive atmosphere, and develop relevance through the use of the technology. Analysis of student

perception and student achievement during the subsequent pilot study will be compiled and reported

using the same EI survey. Lastly, as part of the pilot for the next year (2013-2014) a second teacher (two

additional classes) will be included in the study. This will be an attempt to determine if there are large

degrees of variability associated with individual teachers or if similar trends are found with other

teachers as well. The ultimate goal is develop successful techniques for use within MJSD’s Virtual

School House and to see if this model is one that can be implemented district-wide. Through this

analysis we will be attempting to determine teacher characteristics that lead to more successful

students in online and blended learning environments and to develop the appropriate tools, pedagogy,

and course requirements to suit this format of education. This study will also try to determine how

20

blended learning can be suitable for a variety of school settings and cultures. The larger question

resulting from this study is whether it is possible to create a blended-learning model that is deemed

equal to or more effective for students when compared to that of a traditional classroom. In the move

toward 21st Century teaching and learning the blended-learning model is spoken of as a highly effective

model. Using the evidence and research from this and other studies MJSD is attempting to develop a

model that shows evidence of this success and can be used to create 21st Century models to create

College and Career Ready students.

21

Work Cited

Alavi, M. & Dufner, D. (2005). Technology-mediated collaborative learning: A research perspective. In

S.R. Hiltz & R. Goldman (Eds.), Learning together online: Research on asynchronous learning

networks. Mahwah, NJ: Lawrence Erlbaum, 191-213.

Artino, A.R. & McCoach, B.D. (2008). Development and Initial Validation of the Online Learning Value

and Self-Efficacy Scale. Journal of Educational Computing Research, Vol. 38(3), pp. 279-303.

Dixon, M.D. (2010). Creating effective student engagement in online courses: What do students find

engaging? Journal of the Scholarship of Teaching and Learning, Vol. 10(2), pp. 1-13.

Driscoll, M.P. (2000). Psychology of Learning for Instruction. 2nd Ed. Needham Heights, MA: Allyn &

Bacon.

Garrison, D.R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment:

Computer conferencing in higher education. The Internet and Higher Education, 2, pp. 87-105.

Garson, G.D. (2012). Testing Statistical Assumptions. Statistical Associates Publishing. North Carolina

School of Public and International Affairs.

Gayton, J. and McEwen, B.C. (2007). Effective online instructional and assessment strategies. The

American Journal of Distance Education, Vol. 21(3), pp. 117-132.

Lewis, C.C. and Abdul-Hamid, H. (2006). Implementing effective online teaching practices: Voices of

exemplary faculty. Innovative Higher Education, Vol. 31(2), pp. 83-98.

Lloyd, J.M., Dean, L.A., & Cooper, D.L. (2009). Students’ technology use and its effects on peer

relationships, academic involvement, and healthy lifestyles. National Association of Student

Personnel Journal (NASPA), Vol. 46(2), pp. 695-709.

Mashaw, B. (2012). A Model for Measuring Effectiveness of an Online Course. Decision Sciences

Journal of Innovative Education, 10(2), pp. 189-221.

Richardson, J.C., & Swan, K. (2003). Examining social presence in online courses in relation to students’

perceived learning and satisfaction. Journal of Asynchronous Learning Networks, Vol. 7(1).

Schaber, P., Wilcox, K., Whiteside, A., Marsh, L., & Brooks, D. (2010, July/August). Designing Learning

Environments to Foster Affective Learning: Comparison of Classroom to Blended Learning.

International Journal for the Scholarship of Teaching and Learning, 4(2), 1-18.

Siegler, R.S. (1998). Children’s Thinking, 3rd Edition. Upper Saddle River, NJ: Prentice-Hall, Inc.

Young, S. (2006). Student views of effective online teaching in higher education. The American Journal

of Distance Education, 20(2), pp. 65-77.

22

Appendix A- Effectiveness Index (EI) Survey modified from Mashaw (2012)

Online Course Evaluation in the Virtual School House

Please give feedback about one of the on-line course that you took. The objective is to find out how to

improve our online and blended-learning courses and whether or not online courses are effective.

Circle which class you are in: online // in-classroom

I. Learning Experiences

1 not

really

2 maybe 3

somewhat

4

moderate

yes

5 yes 6 Strong

agreement

1. The course clearly helped

me to comprehend and

understand the world in which

I live.

2. The course changed, or

influenced my way of thinking,

or attitude, about the world.

3. What I learned in this

course will have tremendous

value in my future career,

personal development, or life.

4. In practice or in my future

career, I can apply with

confidence what I learned

from this course.

23

5. Now, if I have to work out

an issue related to the topics

of this course, I am able to

handle it on my own.

II. The Course and Teacher:

1 Not

really

2 Maybe 3

Somewhat

4

Moderate

yes

5 Yes 6 Strong

Agreement

1. The Course Objectives were

communicated clearly so that

I could understand the

learning goals.

2. Each topic was distinct, yet

related to the overall objective

of the course.

3. The course content & topics

inspired me learn more.

4. The course challenged and

inspired me to find my own

answer to problems.

24

III. Technology used:

1 Not

really

2 Maybe 3

Somewhat

4

Moderate

yes

5 Yes 6 Strong

Agreement

1. The webpage/interface was

designed to be attractive,

pleasing and easy to use.

2. The links between sites

were designed well & and

were easy to navigate.

3. The variety of presentation

tools kept my interest and

helped me learn.

4. I liked the website software

used, it made the learning

experience easy.

5. During the course, I could

go at my own pace.

25

IV. Participation/Communication/Feedback:

1 Not

really

2 Maybe 3

Somewhat

4

Moderate

yes

5 Yes 6 Strong

Agreement

1. It was easy to communicate

with the online teachers as

well as the group.

2. Questions and comments

were answered promptly, and

the discussion were lively.

3. Surprisingly, I was

encouraged to get involved in

group discussion, and

participate more than

average.

V. The instructor's performance and inspiration:

1 Not

really

2 Maybe 3

Somewhat

4

Moderate

yes

5 Yes 6 Strong

Agreement

1. The explanations &

presentation by the instructor

were impressive, inspiring and

effective.

2. The assignments were

related to the course, well

designed, reasonable, yet

challenging.

26

3. Throughout the course, I

was motivated to find my own

answers to challenges.

4. Throughout the course,

feedback was provided so that

I was aware of my learning

progress & grade.

5. The instructor was available

and provided appropriate

assistance when asked to do

so.

VI. Discouragements, if any:

1 Not

really

2 Maybe 3

Somewhat

4

Moderate

yes

5 Yes 6 Strong

Agreement

1. I often had problems with

the technical side of the

webpage and the course.

2. The presentations/lectures

or explanations were static,

boring and discouraging.

3. The course was not

designed to be flexible as

expected.

4. Assignments were

unreasonable, very time

consuming and almost

impossible to finish.

27

5. The course was designed

like a self learning course,

without much guidance.

VII. Overall Rating and effectiveness of the entire course:

1 Lowest 2 3 4 5 6 Highest

1. When considering the

learning experiences, my

overall rating for this course

is:(1:lowest, 6 Highest)

2. My rating for the learning

effectiveness, compared to

physical classroom (1: not

effective,6:much more than a

classroom)

28

General Information:

F D C B A

My final grade in this course

(expecting, or was)

I never read for

enjoyment

Sometimes All the

time

How much do you enjoy

reading on your own time?

Please contact

harm.eian@meridianschools.org if you have any questions regarding this survey.

29

Appendix B- Survey analysis by individual Effectiveness Index factor questions

Learning Experience and Content

Mean Std. Deviation

This course clearly helped me to comprehend and

understand the world in which I live.

3.22 1.085

This course changed or influenced my way of

thinking or attitude about the world.

3.48 1.563

What I learned in this course will have tremendous

value in my future career, personal development, or

life.

2.52 1.377

In my education or future career I can apply with

confidence what I learned from this course.

2.43 1.237

Now if I have to work out an issue related to the

topics of this course, I am able to handle it on my

own.

2.83 1.114

Course Content Engagment

Mean Std. Deviation

The Course Objectives were communicated clearly I could understand the learning goals.

3.52 1.47

Each topic was distinct yet related to the overall objective of the course. 4.22 1.41

The course content and topics inspired me learn more. 2.61 1.27

This course challenged or inspired me to find my own answers to problems.

3.48 1.62

30

Technology Used

Mean Std. Deviation

The webpage interface was designed to be attractive, pleasing, and easy to use.

3.04 1.26

It was easy to navigate through the online material and lessons.

3.65 1.61

The variety of presentation tools kept my interest. 2.65 1.23 I liked the website software used it made the learning experience easy.

3.00 1.38

During the course I could go at my own pace. 2.13 1.55

Participation/Communication/Feedback

Mean Std. Deviation

It was easy to communicate with the online teachers as well as my classmates.

3.52 1.81

Questions and comments were answered promptly and the discussions were informative and engaging.

3.22 1.48

I was encouraged to get involved in discussions and participate more than I normal would.

2.30 1.26

Instructor’s Performance and Inspiration

Mean Std. Deviation

The personal interaction with my instructor(s) was inspiring and effective.

3.00 1.41

The assignments were related to the course, well designed, reasonable, yet challenging.

3.70 1.26

Throughout the course I was motivated to find my own answers to challenges.

3.43 1.73

Throughout the course feedback was provided so I was aware of my learning progress and grade.

3.83 1.59

An instructor was available and provided appropriate assistance when asked to do so.

4.48 1.59

31

I felt valued by my instructors and my participation was important.

3.48 1.44

Student Comparison to Traditional Classroom

Mean Std. Deviation

When considering my learning experience in this online or blended learning course, my overall rating would be:

3.30 1.30

When comparing how much I learned in this course to a physical classroom, I feel I am learning: (1=much less; 6=much more)

2.48 1.28

Compared to previous courses my effort in this course was (is): (1=much less; 6=much more)

4.26 1.45

Discouragements (higher values indicate more

negative perception).

Mean Std. Deviation

I often had problems with the technical side of the webpage and the course.

4.35 1.30

The presentations lectures or explanations were static boring, and discouraging. 4.17 1.47

This course was NOT flexible I could not work at my own speed or interest.

4.52 1.59

Assignments were unreasonable very time consuming and almost impossible to finish

4.39 1.12

This course was designed like a self learning course without guidance or explanation.

4.52 1.44

32

Appendix B- Course modifications of the 2013-2014 blended learning pilot study based on this

research

1. Desks will be positioned so that teacher can see screens at all times.

2. Computers set-up and powered-up prior to start of class.

3. Classroom teachers clearly articulates the expectations of each day. These expectations should

be in alignment with established pacing guide. Can be accessed through the “Latest News”

block within the Virtual School House.

4. Classroom teacher should remind students of upcoming tests and remind them that they are

allowed one page of notes to use on the test. Notes to be complete on day prior.

5. During test days, teacher will remind students of testing protocol and refer incidents of cheating

to office or follow school policy. Cheating includes looking up answers online while taking the

test.

6. Teacher must view all student tests prior to submission to assure students complete and submit

online tests on the day of the test.

7. Classroom teacher takes an active role in monitoring and helping students. This includes but is

not limited to:

a. Frequent walks through the room to make sure students are working on the current

day’s assignment.

b. Helping students with various questions/problems/issues as they arise.

c. Monitoring tests to make sure students are actually taking their test and that they are

using appropriate pacing.

Other additions for Fall 2013 Pilot

33

Teacher will be trained to use the VSH and expect to grade all online assignments.

The teacher will “blend” their teaching with the online curriculum to enhance the

learning and motivate learners to have a better perspective of current global events.

34

35

Appendix D- Word Cloud based on survey question “If you could take this course over again, what

would you change?”

36

37