Reactions to Gamified TrainingMichael B. Armstrong & Richard N. Landers, Ph.D.

Department of Psychology

AbstractThis study examined the impact of gamification on anticipated reactions to workplace training, providing a preliminary evaluation of gamified training design. The moderating effects of experience with games and technology and attitudes towards the use of games and technology in training were explored as well. By exploring the effects of gamified training design on anticipated reactions, evidence can be gathered to support future research regarding the effects of gamified training on workplace learning, behavioral change, and organizational results through a full-scale gamified training system.

Literature ReviewGamification is the use of game elements in non-game contexts in order to motivate users to behave a certain way (Deterding, Khaled, Nacke, & Dixon, 2011). Examples include:• Gamifying exercise: Award points to people for losing weight and

display points on a scoreboard so people can see where they stand in relation to others in a friendly competition.

• Gamifying school: Utilize a fantasy narrative to exchange taking tests for slaying dragons and exchange projects for epic quests.

• Gamifying work: Distribute badges or ribbons for completing different training modules. Race to collect them all!

This study explored how effective gamification could be in a potential workplace training scenario. • Training design effectiveness can be evaluated by measuring

reactions to training, which impacts learning, behavioral change, and organizational outcomes (Kirkpatrick, 1996).

• More experience with a technology or system (e.g. video game) means people have to expend fewer cognitive resources using it than someone new to that system who is distracted by learning the system (Landers & Callan, 2012; Sweller, 1988).

• Individual differences (e.g. attitudes) among people differentiate how each person will react (Kraiger, 2008).

Method and Hypotheses• 150 participants • Participants received two different vignettes: The control vignette described

a typical training scenario via lecture and Microsoft PowerPoint. The experimental vignette described the same training content but via an interactive video game instead of the lecture and presentation.

• Participants completed surveys measuring reactions to the training scenarios, as well as experience with technology/games and their attitudes towards technology/games in the workplace.

Hypothesis 1. Gamified training design will affect anticipated reactions to training. Hypothesis 2. Experience with games and technology will moderate the relationship between training design and anticipated reactions. That is, the relationship between training design and training reactions will be stronger when trainees have more experience with games and technology.Hypothesis 3. Trainee attitudes toward the value of games and technology used for training will moderate the relationship between training design and anticipated reactions. That is, the relationship between training design and training reactions will be stronger when trainees have more positive attitudes toward games and technology.

Conclusions

Figure 1. The present model.

Figure 3. Hypothesis 3: Condition x Attitude Interaction.

• Hypothesis 1 was not supported. Overall, anticipated gamified training did not significantly affect reactions compared to anticipated traditional training.

• Hypotheses 2 and 3 were supported. With higher ratings of experience with games and attitudes towards using games in work, reactions increased from anticipated traditional training to anticipated gamified training.

• This study is limited in its generalizability to actual work training designs. The results reached in this study are based on possible training scenarios instead of actual training sessions. Future research should measure reactions and learning outcomes based on gamified training.

• This study found that gamification in work training deserves further study. For trainees high in experience with games and high in attitudes towards the use of games in work, gamified training can produce higher reaction ratings, and possibly learning, behavioral, and organizational outcomes.

ReferencesDeterding, S., Khaled, R., Nacke, L. E., & Dixon,

D. (2011, May 7-12, 2011). Gamification: Toward a definition. Paper presented at the 2011 Annual Conference on Human Factors in Computing Systems (CHI '11), Vancouver, BC, Canada.

Kirkpatrick, D. (1996). Great Ideas Revisited. Techniques for Evaluating Training Programs. Revisiting Kirkpatrick's Four-Level Model. Training and Development, 50(1), 54-59.

Kraiger, K. (2008). Transforming our models of learning and development: Web-based instruction as enabler of third-generation instruction. Industrial and Organizational Psychology, 1(4), 454-467. doi: 10.1111/j.1754-9434.2008.00086.x

Landers, R. N., & Callan, R. C. (2012). Training evaluation in virtual worlds: Development of a model. Journal of Virtual Worlds Research, 5(3), 1-20.

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12(2), 257-285.

Analyses and ResultsHypothesis 1. To test this hypothesis, we regressed training reactions onto a dummy-coded indicator of condition. Training design condition did not significantly affect reactions (b = .105, SE = .117, p = .370, R2 = .081).

Hypotheses 2 & 3. To test these hypotheses, we conducted two separate hierarchical multiple regressions. In Step 1, we regressed reactions onto condition and moderator variable. In Step 2, we added in the interaction of condition and moderator. Results can be found in Tables 1 and 2 and are illustrated in Figures 2 & 3.

Table 2. Regression Results of Hypothesis 3.

Traditional Gamification0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Low ExperienceHigh Experience

Reacti

on

s t

o T

rain

ing

Traditional Gamification0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Low AttitudeHigh Attitude

Reacti

on

s t

o T

rain

ing

Figure 2. Hypothesis 2: Condition x Experience Interaction.

Table 1. Regression Results of Hypothesis 2.