Examining the Multimedia Redundancy Effect among Education Majors:

Assessing Pedagogical Usability of Instructional Videos

Sang Chan Weber State University

Jane StricklandIdaho State University

AACE,

New Orleans, LA

October 29, 20141

•Many studies (e.g., Craig, Gholson, & Driscoll, 2002; Kalyuga,

Chandler, & Sweller, 1999, 2000; Mayer, Heiser, & Lonn, 2001, etc.) compared redundant groups (graphics, text, and narration) with non-redundant groups (graphics and narration).

• Results: Non-redundant group significantly outperformed the redundant group.

2

Introduction

•Used a short treatment time

•Measured only short-term learning

• Rarely used a topic of a formal course

• Rarely discussed pedagogical usability

3

Introduction

• To further examine the redundancy effect by comparing posttest scores, delayed posttest scores, and usability perceptions of education majors enrolled in an online technology class.

• Redundant (R): Narration, graphics, & text• Non-Redundant (NR): Narration & graphics

4

Purpose

1. Is there a significant difference in posttest scores in learning Microsoft Access 2013 between R and NR?

2. Is there a significant difference in delayed posttest scores?

3. Is there a significant difference in pedagogical usability perceptions?

5

Research Questions

• Cognitive load theory

6

Theoretical Framework

• The study used convenience sampling.

• 42 students participated in the study.

• Female, traditional students (majority)

• No students with disabilities

• Native English speakers7

Participants

• T1: Narration, graphics

• T2: Narration, graphics, and text

Group Treatment Survey Posttest Delayed Posttest

T1 X1 O2 O6 O10

T2 X2 O4 O8 O12

8

Research Design

•Designed and developed the videos

•Developed a posttest and delayed posttest

•Adapted the pedagogical usability survey

• Piloted with a small group of four

9

Materials

•Obtained the informed consents•Assigned students randomly •Administered treatments

• Completed surveys (αR = .83, αNR = .88)

• Took posttest • Took delayed posttest (6 weeks later)

10

Procedure

• Little’s MCAR, χ2 (252) = 263.08, p = .303• Test scores• Violated normality & equal variances for

the independent-t test• Outliers in test scores

• Survey responses: • Met the independent-t test assumptions

11

Data Screening

• Posttest and delayed posttest • Mann-Whitney U (adjusted α= .025)• Survey• Independent t-test (α = .05)

Note: We found similar results for parametric and non-parametric tests with and without outliers.

12

Data Analysis

•No significant difference between the two

treatments on posttest scores, U = 164.50,

p = .489

• Small estimated effect size, r = Z/ = .11

13

Result: 1

•No significant difference between the two

treatments on delayed posttest scores, U

= 118, p = .538

• Small estimated effect size, r =.11

14

Result: 2

•No significant difference between the two

treatments on survey responses, t(40)

= .60, p = .549

• Small estimated effect size, d = .18

15

Result: 3

•No significant differences in posttest and delayed posttest•Not consistent with previous studies

(i.e., Craig, Gholson, & Driscoll, 2002; Mayer, Heiser, & Lonn, 2001)• Consistent with McNeill, Doolittle,

and Hicks (2009) and Wu (2011)

16

Discussions

•Useful text: Short and easy-to-read with slow narration• “The text was so helpful in understanding the

content. I have never used access [sic] before & [sic] I think it can be helpful.” • “. . . The visuals were great and i appreciate

having the subtitles to keep me on track.” • “Seeing you perform the task but also having the

words on the bottom helped reinforce the idea.” 17

Discussions

• Read text to reinforce narration.• Eight words, on average, per line• Processed the elements in different orders.• Anecdotally, text was useful.

18

Posttest Delayed Posttest

M Mnd M Mnd

Redundant 80.44 87.50 71.20 74

Non-redundant 80.33 84 60.22 67.50

Discussions

• Insignificant results on posttest and delayed posttest may result from:• Small sample size • Low power•High data variability• Low motivation• Prior preparations before tests

19

Discussions

•Useful aspects of the videos:• Review what was learned: “… Reviewing

after each topic helped reinforce the concepts.” • Clear content and logical sequence: “It is

useful that the objectives built on one another. I also thought it was helpful to see what the end product would look like.” • Multimedia elements: “I really liked the

reading at the bottom of screen so I could follow along with the speaker.”

20

Discussions

• Present the text slightly faster (1 or 2 seconds) than narration (120 words/minute) •Keep the text available on the screen with

graphics although the narration was silent•Keep the text short (8 words) and easy to

read (large font)

21

Implications

• Investigate such use of text • Explore ways to use text as a supporting

element •Use a large sample size•Use a more reliable instrument to reduce

data variability • Consider motivation: Low motivation

may result in high data variability. 22

Recommendations

Thank You!

Q & A

23

• Primary analysis: 6 subscales (75.63%)• Low reliability for some subscales• Low correlations among items• Conceptual overlapping•Unstable factor• Revised survey suggested 5 subscales.

24

Revising Survey

Factor Loading Communality1 2 3 4 5

Q20: The time allotted for completing this tutorial was adequate for me. .79         1

Q8: The difficulty level in this tutorial was appropriate for me. .75         1

Q1: Dividing this content into sections assisted me in learning the concepts. .65         1

Q2: The objectives informed me of what was expected for me to learn. .65         1

Q3: The overview of what I would be learning in this module assisted me. .56         1

Q16: The sequencing (order) of the materials helped me to learn this content.

.53         1

25

Exploratory Factor Analysis

Factor Loading Communality1 2 3 4 5

Q24: I understand how to create a Query after completing this tutorial.

 .96       1

Q23: I understand how to create a Table after completing this tutorial.

 .94       1

Q22: I will be able to create a database after completing this tutorial.

.88        1

Q17: I learned this material more quickly using this computer-based tutorial than I would in a traditional (face-to-face) format.

  .95      1

Q12: It was more useful for me to learn the content in this format than in an instructor-led, traditional classroom.

   .78     1

Q21: The tutorial was an adequate alternative to having an instructor lead me in learning this material

   .74     126

Exploratory Factor Analysis

Factor Loading Communality1 2 3 4 5

Q13: I was interested in the topic of this tutorial.

.81 1

Q18: I felt motivated to learn the content from this tutorial.

.71 1

Q19: I would like to learn other technologies using this type of computer-based tutorial.

.65 1

Q7: This tutorial was designed for my needs as a future educator.

.65 1

Q9: The images in this tutorial helped me to learn the content.

.87 1

Q10: The audio narration in this tutorial helped me to learn the content.

.86 1

Q11: The animations in this tutorial helped me to learn the content.

.49 1

27

Exploratory Factor Analysis

• Factor 1: Design Considerations (α = .84)• Factor 2: Learning Confidence (α = .96)• Factor 3: Learning Format (α = .86)• Factor 4: Learning Motivation (α = .87)• Factor 5: Multimedia Elements (α = .78)•Overall reliability α = .89• Total variance = 75%.

28

Revised Survey