Running head: EFFECTS OF ENGAGEMENT ON ATTENTIONAL … · Title: The Effects of Engagement and...
Transcript of Running head: EFFECTS OF ENGAGEMENT ON ATTENTIONAL … · Title: The Effects of Engagement and...
Running head: EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
1
Title: The Effects of Engagement and Gamification on Attentional Bias Modification
Author: Veronique Servant
Date: June 2015
Institution name/journal where submitted: McGill University
The use of this database indicates agreement to the terms and conditions
Academia is a database that promotes the free exchange of ideas and scholarly work, setting a
platform on which to foment and improve student discourse
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
2
Abstract
Attentional bias modification training has been shown to be a reliable method of reducing
rejection bias. Recently, the effects of gamification and engagement have been explored in an
attempt to create engaging versions of mental health treatments that can be provided over the
internet. In this study, a dynamic version of the matrix attentional training task was created to
make it more engaging and test for more powerful effects of attentional training. Results showed
that the dynamic condition was more engaging than the static condition, but only relative to the
perceived ease of the task. Attentional training produced signficant results, however, only in the
static faces condition through improving acceptance bias. Future research is needed to explore
additional ways to make the task more engaging and to improve the effects of attentional
training.
Keywords: acceptance bias, attentional bias, attentional bias modification training,
engagement, gamification, rejection bias, visual search
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
3
A great deal of research has been conducted into finding effective treatment for individuals
with anxiety disorders and related vulnerabilities. One of the psychological constructs
underlying anxiety disorders is attentional bias. This is a bias individuals show in how they
attend to certain stimuli; for example, individuals who have high anxiety often demonstrate an
attentional bias toward threatening stimuli (Zvielli, Bernstein, & Koster, 2014), and individuals
with relatively low self-esteem demonstrate an attentional bias toward rejecting stimuli
(Dandeneau & Baldwin, 2004). Attentional bias has been measured across numerous studies
using modified versions of the visual attention dot probe paradigm developed by MacLeod,
Mathews, & Tata (1986). This visual probe task (VPT) task involves participants making a
neutral response (pressing a button) to a neutral stimulus (visual probe). MacLeod, Rutherford,
Campbell, Ebsworthy, & Holker (2002) conducted a study measuring attentional bias using a
VPT which involved displaying pairs of words (one emotionally negative, one neutral), after
which a probe would appear behind one of the words. Participants then had to identify the probe
(either a single pixel or two adjacent pixels). Participants’ reaction times were measured and
used to calculate their attentional bias based on the reasoning that reaction times would be
shorter when participants’ attention was drawn to the stimulus behind which the probe appeared.
An attentional bias to automatically attend to negative stimuli is thought to exacerbate the
negative feelings contributing to low self-esteem and/or high anxiety. This reasoning led to the
development of attentional bias modification, a process of training attention in such a way as to
remove the attentional bias and thereby eliminate its negative effects. MacLeod et al. (2002)
developed a modified attentional training paradigm to modify the attentional bias toward
negative stimuli exhibited by anxious individuals. The training task involved a VPT which
displayed pairs of words (one emotionally negative, one emotionally neutral) after which a probe
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
4
would appear behind one of the words. By consistently presenting the probe after the
emotionally neutral word, a pattern becomes evident whereby attention is automatically directed
toward future neutral stimuli where the probe is expected to appear. This task was successful in
training participants to display an attentional bias away from, instead of towards, negative
information. MacLeod & Clarke (2015) also conducted a study involving attentional bias
modification training with individuals suffering from high anxiety, and the attentional training
was successful in reducing attention to threat and anxiety symptomatology.
Research on attentional bias modification training has predominantly involved modified
dot probe paradigms, but alternative methods have also proven successful in modifying
attentional bias. For example, Dandeneau, Baldwin, Baccus, Sakellaropoulo, & Pruessner
(2007) conducted an attention training task where participants were instructed to perform a
visual search for a specific image (a smiling face amongst frowning faces in the experimental
condition, or a 5-petalled flower amongst 7-petalled flowers in the control condition). The
experimental condition served to modify attentional bias, as participants had to inhibit attention
to the rejecting faces in order to find the accepting face. Individuals with low self-esteem
showed significant improvements in inhibiting attention to rejecting faces after completing the
attentional training, whereas individuals in the control condition did not show such
improvements (Dandeneau, Baldwin, Baccus, Sakellaropoulo, & Pruessner, 2007).
Despite the demonstrated effects of attentional training, the procedure – particularly the
modified dot probe task – has often been described by participants as being repetitive, boring, or
frustrating (Beard, Weisberg, & Primack, 2012). These negative feelings could lead participants
to become less involved in the task and persist less in the training, which would make it less
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
5
useful as an intervention. It is therefore important to ensure that participants remain engaged and
participate in the full extent of the attentional training.
Engagement
A great deal of research examining the educational effects and influences of engagement
has focused on the process of learning and the role of student engagement. An early model of
student engagement by Finn (1989, 1993) included affective (e.g. sense of belonging) and
behavioral (e.g. involvement in school) components. Research has since examined multiple
subtypes of engagement, including cognitive engagement, behavioral engagement, and emotional
or psychological engagement. Cognitive engagement has been found to be associated with
processes such as intrinsic motivation, adaptive learning strategies involving task-mastery goals,
and self-regulation (Meece, Blumenfield, & Hoyle, 1988; Appleton, Christenson, Kim, &
Reschly, 2006). Fredricks, Blumenfield, & Paris (2004) define behavioral engagement as
relating to participation and involvement in school activities, differing from emotional
engagement, which draws on the valence of social connections in the environment (such as
relations with teachers, peers, and the school). In a study examining 1st-grade children, Hughes
& Kwok (2007) found that increased classroom activity resulted in increased behavioral
engagement and was associated with higher reading achievement. Goodenow (1993) conducted
a study examining psychological engagement amongst adolescent students and found that
psychological engagement was associated with positive characteristics such as greater class
participation, increased class attendance, and greater persistence in challenging activities.
Student engagement has been recognized as an important contributor to the process of
learning. Prensky (2005) compares children of even a few decades ago to children today,
illustrating a profound difference in attitudes towards learning at school. Children today are
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
6
exposed to a myriad of devices that provide entertainment on a daily basis, causing them to
develop the expectation and the need to be highly engaged by their surroundings. In the typical
school context involving a teacher giving a lecture, children have difficulty paying attention and
getting into the process of learning if the material is not inherently interesting and presented in an
engaging fashion (Prensky, 2005). Educators are now faced with the challenge of modifying
their teaching styles to incorporate “the same combination of desirable goals, interesting choices,
immediate and useful feedback, and opportunities […] that engage kids in their favorite complex
computer games” (Prensky, 2005). To address this change in learning style, research has
examined engagement through active learning. Students engaging in active learning processes,
such as the use of clickers, display higher levels of class involvement, which in turn results in
better preparation, increased attention, and better recall of class material (Caldwell, 2007). A
study by Freeman et al. (2007) also examined the effects of engaging students through active
learning (responding to daily multiple-choice questions with clickers or cards) in a course on
introduction to biology. They found that students in the engaging course design had lower
failure rates and scored higher on both an identical midterm and the final exam compared to
students who had taken the course with the same professor in the past (Freeman et al., 2007).
Gamification
The importance of engagement and the consequential need for processes such as
gamification stem from the increased presence of media and technologies such as video games
and smart phones in everyday life. In recent years, a great deal of research has been conducted
regarding the process of gamification, a method of transforming services by adding game
components and elements in non-game contexts (Deterding, Dixon, Khaled, & Nacke, 2011).
Goasduff & Pettey (2011) indicated that gamification is used as a means to achieve increased
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
7
levels of engagement, influence positive behaviors, and stimulate innovation. They identified
four key means by which gamification can increase engagement: (a) accelerated feedback cycles;
(b) clear goals and rules of play; (c) a compelling narrative; and (d) challenging but achievable
tasks. Gartner predicted that by 2015, over 50 percent of organizations managing innovation
processes will undergo the gamification transformation (Goasduff & Pettey, 2011). In an
analysis of gamification processes, Muntean (2011) illustrates how engagement can be increased
through the application of game mechanics and dynamics to learning tasks and processes.
Therapeutic Applications
Gamification and engagement offer promising and necessary methods to provide mental
health support via contemporary methods such as the internet. A study conducted by the Kraiser
Family Foundation (Rideout & Roberts, 2010) found that children and teenagers, ages 8-18,
spend an average of 7 hours and 38 minutes using entertainment media devices (such as
computers and smart phones) per day, which equates to over 53 hours per week. On a similar
note, another study found that 37% of teens had smartphones in 2013, a sharp increase from the
23% of teens who had smartphones in 2011 (Madden, Lenhart, Duggan, Cortesi, & Grasser,
2013). This surge in the presence of smartphones is also apparent in the adult population:
Duggan (2013) found that 91% of American adults own cell phones, where 60% of adults use
their phone to access the internet and 50% of adults use their phone to download apps.
While such devices are used primarily for entertainment and social networking purposes,
researchers are looking into ways to use these devices to confront a major obstacle in the mental
health industry: unmet need for treatment. A recent epidemiological study examined the
prevalence of mental health disorders among American adolescents and compared the rates of
mental health services sought, and the results were striking: Less than one in five adolescents
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
8
suffering from disorders involving anxiety, eating, or substance use received services
(Merikangas et al., 2011). In a separate study examining a sample of individuals in the United
States with major depression, 61.7% reported unmet need for treatment, 46.0% of whom listed
financial concern as the reason for not seeking treatment (Mojtabai, 2009). Converting mental
health services and treatments into forms applicable to the internet and mobile devices would
provide easy and affordable access to many individuals with unmet need for treatment.
Gamification has been one of the processes by which researchers have sought to create
online and mobile versions of mental health treatments. For example, Dennis & O’Toole (2013)
created a mobile gamified version of the attentional bias modification training task based on the
dot-probe paradigm. They designed a game on the iPod Touch where players, through tracing
certain characters on the screen with their finger, receive either attentional bias training or
placebo training. Their study involved 78 individuals high in trait anxiety, and results showed
that long training conditions (45 minutes of gameplay) but not short training conditions (25
minutes of gameplay) showed significant reductions in cognitive processes implicated in
attentional bias to threat (Dennis & O’Toole, 2013). A separate study by Enock, Hofmann, &
McNally (2014) created a mobile version of the dot-probe task and sought to determine whether
it was possible to administer attentional bias modification via smartphones. Their study
succeeded in demonstrating that it was feasible to conduct attentional bias modification via
smartphones.
Toward a Gamified Version of the Matrix Visual Search Task
The present study aims to analyze the effects of delivering via internet a gamified version
of the matrix attentional bias training task presented by Dandeneau, Baldwin, Baccus,
Sakellaropoulo, & Pruessner (2007) in order to determine whether increased engagement
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
9
strengthens the effects of the attentional training. We created a novel modified version of the
attentional training matrix by adding more game-like features and characteristics, such as
alternating matrix arrays, image flips, and level rotations. The goal of gamifying the attentional
matrix was to increase the engaging properties of the task. We hypothesized that the gamified
version would be more engaging than the standard version, based on participants’ perception of
how engaged they felt during the task. This was measured by a novel self-report questionnaire
of various items related to engagement. We expected to find this effect for both types of stimuli
of the task; specifically, that the gamified version of the matrix would be more engaging in both
the experimental stimuli and control stimuli conditions. We also hypothesized that the gamified
version, through being more engaging, would lead to more powerful effects of attentional bias
modification. This was determined by measuring the changes in attentional bias modification
and comparing the outcomes of each version of the task. Specifically, we expected that the
gamified version of the experimental stimuli matrix would show greater effects of attentional
bias modification than the standard version of the experimental stimuli matrix. No differences
were expected in either the gamified or standard version of the control stimuli matrix, as the
control stimuli have no effects in training attentional bias.
Methods
Participants
Participants were 78 undergraduate students from McGill University. 61 students
participated in the study to receive course credit, and 17 additional students not registered in the
subject pool were also recruited offering financial compensation of $10 CDN per participant who
completed the study. Of the 78 students who participated, 9 were male, 68 were female, and 1
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
10
preferred not to disclose their gender. The mean age of the participants was 20.65 years (range =
18-25).
Participants were randomly assigned to one of four matrix conditions. The final
breakdown included 20 assigned to the static flowers condition (19 female, 1 preferred not to
disclose); 15 to the dynamic flowers condition (3 male, 12 female); 19 to the static faces
condition (2 male, 17 female); and 24 to the dynamic faces condition (4 male, 20 female).
Measures
A number of questionnaires were implemented to measure various personality
characteristics. The study was broken up into 5 sections: (a) pre-training questionnaire1; (b) pre-
training visual probe task; (c) attentional bias training matrix; (d) post-training visual probe task;
and (e) post-training questionnaire1,2.
Engagement scale. The engagement scale was a novel measure created for this study to
measure participants’ perceived levels of engagement during the matrix attentional training task.
Questions were written based on the components of being immersed in a task, such as a sense of
time being distorted and a balance between perceived efficacy and task challenge
(Csikszentmihalyi, 1997), as well as the characteristics which make a game engaging, such as
clear rules and goals in addition to a sense of enjoyment (Prensky, 2001). The scale consisted of
10 items, each ranked on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly
agree). Participants were asked to think back to when they were performing the matrix task, and
to indicate how much they agreed or disagreed with each item. Items included statements such
as “I found it easy to do” and “I felt I was doing poorly at it” for measuring perceived efficacy,
1 No significant effects of any of the measures included in the questionnaires were found; as such, they will not be
further discussed. 2 Anagrams were used as a threat manipulation, but no significant effects of the task were found, so they will not be
further discussed.
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
11
and statements such as “I had fun doing it”, “it was boring” (reverse-scored item), and “I felt
completely involved in it” for measuring how immersed participants felt in the task.
Visual Probe Task
The visual probe task (VPT) was based on the well-established method of examining
attentional biases in anxious individuals (Bradley, Mogg, Falla, & Hamilton, 1998). The stimuli
included 32 images of faces which were procured from 16 different individuals. For each
individual, two images of different facial expressions were obtained: one neutral pose and one of
either a smiling (acceptance) or a frowning (rejection) pose. The two images presented on the
screen were of the same person and included either one of each expression or two copies of the
neutral expression. This could happen in one of five arrangements: frowning-neutral, neutral-
frowning, neutral-neutral, neutral-smiling, or smiling-neutral. This resulted in 8 acceptance-
neutral pairs, 8 rejection-neutral pairs, and 16 neutral-neutral pairs. All pictures were in color
and were presented against a white background.
The VPT consisted of 8 practice trials and 80 experimental trials that were presented in
random order for each participant. For each trial, the fixation symbol (the plus sign, +) was
presented in the center of the screen for 500 ms. Following the fixation period, a pair of faces
was displayed for 500 ms, after which a directional probe (an arrow pointing either up or down)
appeared behind either the picture on the left or the picture on the right. The probe remained on
the screen until the participant responded by pressing a key on the keyboard to indicate the
appropriate orientation of the probe (up arrow key for a probe pointing up; down arrow key for a
probe pointing down). Throughout the task, each of the 16 emotional-neutral pairs were
presented four times, differing by the location of the emotional face (left or right) and by the
orientation of the probe (up or down). Each neutral-neutral pair was presented only once. Each
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
12
probe orientation replaced an equal number of emotional and neutral pictures on each side of the
screen. Figure 1 illustrates the sequence of images for one trial.
As participants completed the study at their own convenience, it was assumed that screen
size and resolution would vary between participants. For this reason, prior to commencing the
task, participants were instructed to modify the size of a line in the middle of the screen by either
shortening or lengthening it so that it was approximately 4 inches long. This served to modify
the size of the display so the image sizes would conform to the indented parameters based on the
design of the VPT conducted by Dandeneau et al. (2007). Images were 45 x 70 mm with a
distance of 115 mm between their centers. Following this, participants were shown a screen with
instructions explaining that they were to fixate on the plus sign (+) and then indicate, as quickly
as possible, the orientation of the probe. Lastly, participants were instructed to set themselves up
such that their eyes were approximately 60 cm from the screen. Once ready, participants then
went through the 8 practice trials. Following this, a short instruction page was given to inform
participants that faces would be shown following the fixation sign, but that participants should
do their best to ignore the faces and to focus on correctly identifying the orientation of the probe.
Attentional Bias Training Matrix
The attentional bias training matrix (ABTM) was based on previous research examining
the attentional training effects of the matrix task (Dandeneau & Badwin, 2004).
Levels and Steps. The ABTM contained 12 different levels. Each level had a predefined
sequence of steps, which included code defining how the images would be displayed in the
matrix. The code included the following variables:
Pattern. This indicated which pre-defined array pattern would be used (e.g. 1 row by 3
columns).
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
13
Smiles. This indicated the number of smiles to be populated within the matrix.
Smiles_reset. This indicated the number of smiles to be clicked in order to advance to
the next step.
Reset_time. This indicated the amount of time, in seconds, after which the matrix would
advance to the next step if the participant had not yet responded (i.e. a smile had not yet
been clicked).
Flip_chance. This indicated the percent chance that any image in the pattern would
perform a rotating flip action.
Cycle_time. This indicated the amount of time, in seconds, between chances for images
to flip.
Flip_smile. This indicated the percent chance that any image, after flipping, would
change into a smiling face.
Each level lasted 30 seconds, during which its sequence of steps would be repeated until the time
limit was reached. After reaching the time limit, the matrix would automatically switch to and
load the following level. Levels were presented in continuous blocks of 4 with a break of 10
seconds in between each block.
Matrix stimuli: faces and flowers. The experimental and control conditions were based
on the type of stimuli displayed in the matrix: faces or flowers. Images of faces were used in the
experimental condition. The stimuli consisted of 144 images of faces procured from 48 different
individuals. For each individual, two images of different facial expressions were obtained: a
smiling (acceptance) pose and a frowning (rejection) pose. The task was to search for and click
on a smiling face presented among frowning faces.
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
14
Images of flowers were used in the control condition. The stimuli consisted of 144 images
of flowers. Flowers consisted of either 5 petals or 7 petals, and differed based on how many
petals were colored in. The task was to search for and click on the 5-petaled flower presented
among other 7-petaled flowers.
Matrix design: static and dynamic. Two different types of matrices were employed:
static and dynamic. The static design was a replication of the matrix visual search task in
Dandeneau et al. (2007). In the static condition, each level comprised only one type of step. The
pattern was set to 4 rows by 4 columns. Of the 16 images, 15 were non-target images
(neutral/frowning face or 7-petaled flower), and only 1 was a target image (smiling face or 5-
petaled flower). The step would be completed only when the participant successfully located
and clicked on the target. Participants would repeat the step for as long as they clicked on the
target image, until the level duration was over and the next level was loaded.
In the dynamic condition, levels 1 and 12 consisted of one type of step containing a 4 x 4
matrix (see Figure 2), whereas levels 2 through 11 consisted of multiple steps, each with unique
sequences of different array arrangements ranging from 1 x 2 to 5 x 5 (see Figures 3 and 4).
Each level had different features regarding flips, cycle time, and reset time, with most steps
automatically advancing after 5 seconds. The goal of the dynamic matrix was to make the task
more engaging by adding flips to images and through automatically advancing to the next step
after the participant had not made a response after 5 seconds.
Procedure
Participants were given the URL of the online study and instructed to complete the study at
their convenience, with the requirement of a computer with a working keyboard and mouse. The
URL directed participants to the pre-training questionnaire. After completing the pre-training
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
15
questionnaire, participants were directed to the pre-training VPT. Once having completed the
VPT, participants were randomly assigned to one of 4 ABTM conditions. Following completion
of the ABTM, participants were directed to the post-training VPT. After completing the second
VPT, participants were directed to the post-training questionnaire. At the end of the second
questionnaire, participants were thanked for their time and redirected to a debriefing page.
Results
Data analysis
Calculation of attentional bias scores. The VPT was used to measure the participant’s
attentional bias – specifically, acceptance bias and rejection bias. Only the trials in which the
participant correctly identified the probe would contribute to the bias, thus all trials in which the
participant made an error were discarded (2.96% of all data). A maximum cut-off RT was
calculated for each participant by calculating 2 standard deviations above their mean RT, and
any RT above the cut-off was replaced by the cut-off instead. In addition, any reaction time
lower than 200 ms was removed. This was done to remove the effect of any outliers in which the
participant had an extraordinarily short or long reaction time. One participant was excluded
from the study due to having numerous trials with RTs above 4000 ms.
Rejection scores were calculated for each VPT by subtracting the mean score of valid
rejection trials (where the probe was behind the rejecting face) from the mean score of invalid
rejection trials (where the probe was behind the non-rejecting face). A positive rejection bias
score indicates an attentional bias toward rejecting faces, whereas a negative rejection bias score
indicates a bias of disengaging (looking away) from rejecting faces. A change in rejection bias
was calculated by subtracting the rejection bias score of the second VPT from that of the first.
Similarly, acceptance scores were calculated by subtracting the mean score of valid acceptance
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
16
trials from the mean score of invalid acceptance trials. A positive acceptance bias score indicates
an attentional bias toward accepting faces, whereas a negative acceptance bias score indicates a
bias of disengaging from accepting faces. A change in acceptance bias was calculated by
subtracting the acceptance bias score of the second VPT from that of the first.
Engagement was measured by a novel scale created to encompass various aspects of what
makes a task engaging. Preliminary analyses suggested that the scale was not measuring a single
construct. A factor analysis was performed on the 10-item scale, which yielded two separate
factors, one loading on two of the questions, and another loading on the remaining 8 questions.
Two engagement scores were computed based on these two factors. An ANOVA yielded a
significant difference in the 2-item engagement factor between the static (M = 2.62, SD = 0.90)
and dynamic (M = 3.26, SD = 0.91) conditions of the matrix, F(1,74) = 8.344, p = .005 (see
Figure 5). No significant differences were found in the 8-item engagement factor between the
static (M= 2.89, SD = 0.75) and dynamic (M = 3.06, SD = 0.63) conditions, F(1,74) = 1.148, p =
.287. These results illustrate that the dynamic condition was found to be more engaging than the
static condition only in relation to the two-item subscale of the engagement score.
It is important to note that the two questions in this subscale were related to the difficulty
of the task as perceived by the participant (“I found it easy to do” and “I felt I was doing poorly
at it”), whereas the remaining 8 questions measured other concepts such as the amount of fun
and involvement experienced during the task. Thus the dynamic condition was more engaging
than the static condition, but only in the sense that the dynamic condition was perceived as easier
than the static condition.
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
17
Matrix Scores
Two matrix-performance scores were calculated: the total number of hits (targets clicked)
and the total number of errors (non-targets clicked). Two participants were excluded from the
study for having a total number of errors that exceeded 100. An ANOVA was performed to
compare the differences in number of hits, and a significant difference was found between the
static (M = 88.08, SD = 26.69) and dynamic (M = 199.78, SD = 27.22) conditions, F(1,74) =
8.822, p = .004. A second ANOVA yielded a significant difference in the number of errors
between the static (M = 5.52, SD = 6.48) and dynamic (M = 9.63, SD = 8.06) conditions, F(1,74)
= 7.408, p = .008. Thus participants in the dynamic condition were performing a greater number
of visual searches overall, as illustrated by a higher number of both hits and errors compared to
the static condition. Correlations were performed to examine the aforementioned differences. A
positive correlation was found between the 2-item engagement factor and hit score, r(76) =
0.347, p = .002. This supports participants’ self-reports of level of difficulty of the task, as the
participants who found it easier were attaining higher scores.
Changes in Acceptance Bias
An ANOVA was performed to compare changes in rejection bias, and contrary to
expectations, no significant differences were found between the faces and flowers groups,
F(1,74) = 0.71, p = .401. A second ANOVA was conducted to examine changes in acceptance
bias, yielding a significant difference between the static (M = 15.62) and dynamic (M = -5.56)
conditions of the matrix, F(1,74) = 5.26, p = .025 (see Figure 6). The same ANOVA yielded an
interaction between matrix design and stimuli that fell just short of significance, F(1,74) = 3.94,
p = .051. Independent samples t-tests were conducted to compare the change in acceptance bias
for flowers and faces in the dynamic and static matrix designs. There was a significant
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
18
difference in acceptance bias change between the dynamic face (M = -12.15, SD = 36.81) and
static face (M = 24.28, SD = 40.41) conditions, t(41) = 3.097, p = .004. There was no significant
difference in the flowers conditions, for which acceptance change scores were very close to zero,
t(33) = 0.217, p = .830. These results indicate that the flowers condition yielded no change in
attentional bias, whereas the static faces condition did yield a significant change in attentional
bias, specifically in acceptance bias. To more directly assess change in bias, paired samples t-
tests were conducted to compare the acceptance bias values preceding and following the matrix
task. There was a significant change in acceptance bias in the static face condition, t(18) = -
2.636, p = .017. No significant changes in acceptance bias were found in the other conditions.
Altogether, the results illustrate that only the static faces condition showed significant
improvement in acceptance bias.
Correlations were performed to further analyze the significant improvement in acceptance
bias observed in the static faces condition. A positive correlation was found between the 2-item
engagement factor and change in acceptance bias, r(17) = 0.595, p = .007. Thus, in the static
faces condition, the easier the participants perceived the task, the greater the change in their
acceptance bias. This correlation was nonsignificant in the other three conditions.
Discussion
The present study aimed to examine the effects of a gamified version of the matrix
attentional bias modification task (Dandeneau, Baldwin, Baccus, Sakellaropoulo, & Pruessner,
2007). It was hypothesized that the gamified version would be more engaging than the standard
version, and that the gamified version, through greater engagement, would yield more powerful
effects of attentional bias modification. The gamified version was found to be significantly more
engaging, but only relative to self-reported perceptions regarding the ease of the task. In
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
19
addition, changes in attentional bias were only observed in the standard face condition, which
contradicted expectations of engagement improving effects of the attentional bias training.
Matrix Score and Engagement
Participants in the dynamic conditions of the matrix were found to have significantly
higher scores in the matrix than those in the static conditions. This is presumably due to the fact
that in the static condition, each trial had only one level which displayed a ratio of 1 smile to 16
frowns, whereas in the dynamic condition, trials had different levels with ratios of smiles to
frowns varying from 1:1 to 1:15 with an average ratio of 1:4.89. Thus participants in the
dynamic condition, having fewer stimuli to search, could more quickly and more easily locate
the target stimulus and advance to the next step. Higher hit scores correlated positively with
both overall engagement, the 2-item engagement factor, and the 8-item engagement factor, and
higher error scores correlated negatively with the 8-item engagement factor. These results show
that the better the participants did on the task (i.e. getting higher hits and less errors), the more
they perceived the task as both easier and as being more involving and fun.
How Engagement Affects Change in Acceptance Bias
We found that, overall, the dynamic condition was perceived as easier than the static
condition. We also found that the static face condition was the only condition in which
participants showed improvements in acceptance bias. This would suggest that the easiness
factor of engagement may actually serve to hinder attentional training. However, within the
static face condition, the 2-item engagement factor correlated positively with change in
acceptance bias, and acceptance bias correlated positively with the number of errors. This
suggests that the effects of the visual search task do not rely solely on feelings of success, as it
was not sufficient for participants in the static faces condition to simply score more hits without
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
20
any errors. Instead, a sense of accomplishment in the face of challenge appears to be one of the
important mechanisms underlying the effects of the training task. This would also explain why
the dynamic face condition did not experience any change in acceptance bias; despite the fact
that participants in the task perceived the task as easy, the task itself was of a lower level of
difficulty, and thus there was no element of challenge to overcome. However, as there was no
effect of attentional training in the dynamic flowers condition, overcoming a challenging task
and gaining a sense of accomplishment cannot be the only relevant factor underlying the effects
of attentional training. It is clear that both a challenging nature and emotional stimuli are critical
features in order for the visual search task to be effective in modifying attentional bias.
The dynamic version of the matrix was not found to be engaging in the manner assessed by
the 8-item engagement factor. The dynamic characteristics of the matrix were created to involve
the participant more through the use of moving images and alternating patterns of stimuli arrays.
A lack of self-reported engagement suggests that increased attention alone is not sufficient to
engage participants in the task. It may prove effective to alter the characteristics and features of
the dynamic matrix so as to increase the various subtypes of engagement (for example, adding a
social component, such as working with a team, to increase emotional engagement). It is also
possible that our engagement scale was not a valid measure of engagement. The scale was
untested prior to this study, and thus had no evidence of being an accurate measure of task
engagement. Additional research is needed to explore alternative methods of increasing the level
of engagement of the matrix task in addition to improved methods to measure task engagement.
Challenging Critiques of Mechanisms Underlying Attentional Bias Training
The mechanisms underlying the effects of attentional training are unclear; however, results
from attentional training studies continue to provide insight into potential explanations. For
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
21
example, Dandeneau et al. (2007) sought to examine the mechanisms behind the matrix
attentional training task and to determine whether the effects of the attentional training might be
desensitization to negative stimuli as opposed to modification of attentional bias. Subjects were
randomly assigned to three different conditions: (a) attentional bias modification; (b) exposure
(in which the subjects were shown the same stimuli, but not required to perform any action); and
(c) control. The researchers hypothesized that if the effects of the attentional training were due
to desensitization, the attentional bias modification and exposure conditions would show similar
results. Their results showed that individuals with low self-esteem in the attentional training
condition showed a decrease in attentional bias to rejection information, whereas individuals
with low self-esteem in the exposure condition displayed an increase in this bias. They
concluded that the effects of the attentional training task is not due to desensitization, and an
active engagement is required to produce the modification of attentional bias.
The design of our study, using two faces conditions and two flowers conditions, allow us to
address certain questions that have been raised concerning attentional bias modification. For
example, the matrix task has been criticized for the use of flower stimuli as a control group, with
the assumption that the use of facial stimuli is what accounts for the change in attentional bias in
the dot probe task. Our findings disprove this assumption, as participants in the dynamic face
condition did not experience improvements in acceptance bias change (and in fact showed
tendencies toward reduced acceptance bias). This provides evidence against the belief that the
flowers serve as a poor control group, as it is not simply the facial characteristics of the stimuli
that account for the change in attentional bias. On a similar note, one might criticize the
mechanism underlying the attentional training task, suggesting that the change in attentional bias
could be an effect of reinforcement; the more targets successfully clicked, the more the search
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
22
for that specific target is reinforced. Again, the fact that participants in the dynamic face
condition did not experience a positive change in acceptance bias disproves this theory. Thus
reinforcement alone is not a mechanism underlying attentional bias modification.
Limitations and Conclusions
A few limitations should be considered when interpreting the results of this study. First, the
study was conducted entirely online, thus participants completed the study at their own
convenience. As such, there was no way to observe whether participants completed the study in
one sitting or whether participants were multi-tasking and performing other tasks while
completing the study. Distractions or prolonged periods of breaks in the midst of the study may
have potentially affected participants’ results. A second potential limitation involved the
duration of the attentional training task. Participants completed 12 levels of 30 seconds each,
with two 10-second breaks in between, for a total of six minutes of attentional training. The
length of the attentional training session mirrored the training in the study by Dandeneau et al.
(2007) and was kept relatively short in order to prevent participants from growing bored of the
repetitive task; however, this length is still much shorter than the typical attentional training dot-
probe paradigm. Additional research is needed to explore the effects of longer training sessions
in order to determine whether the length of the training makes a difference in terms of producing
stronger effects of attentional training.
The gamified version of the matrix visual search task was engaging in the sense that it was
easy, but it did not appear to succeed in engaging participants in terms of a general sense of task
involvement and enjoyment. The gamified version also contradicted our expectations in terms of
the effects it would have in bolstering the effects of the attentional bias training. Not only did
the gamified version fail to enhance the attentional bias training, but it also suggested tendencies
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
23
towards reductions in acceptance bias. However, the results did serve to analyze the underlying
mechanisms of attentional bias training. Perceiving the task as challenging and attaining a sense
of accomplishment appear to be important factors that drive the effects of attentional training.
Future gamification attempts might look to combine these two factors with features adapted to
support the various subtypes of engagement in order to produce a truly engaging and effective
task.
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
24
References
Appleton, J. J., Christenson, S. L., Kim, D., & Reschly, A. L. (2006). Measuring cognitive and
psychological engagement: Validation of the student engagement instrument. Journal of
School Psychology, 4(5), 427-445. doi:10.1016/j.jsp.2006.04.002
Beard, C., Weisberg, R. B., & Primack, J. (2012). Socially anxious primary care patients’
attitudes toward cognitive bias modification (CBM): A qualitative study. Behavioral and
Cognitive Psychotherapy, 40(5), 618-633. doi:10.1017/S1352465811000671
Bradley, B. P., Mogg, K., Falla, S. J., & Hamilton, L. R. (1998). Attentional bias for threatening
facial expressions in anxiety: Manipulation of stimulus duration. Cognition and Emotion,
12(6), 737-753. doi:10.1080/026999398379411
Caldwell, J. E. (2007). Clickers in the large classroom: Current research and best-practice tips.
CBE-Life Sciences Education, 6(1), 9-20. doi:10.1187/cbe.06–12–0205
Csikszentmihalyi, M. (1997). Flow and the psychology of discovery and invention. Harper
Perennial. New York. Retrieved from http://www.bioenterprise.ca/docs/creativity-by-
mihaly-csikszentmihalyi.pdf
Dandeneau, S. D., & Badwin, M. W. (2004). The inhibition of socially rejecting information
among people with high versus low self-esteem: The role of attentional bias and the
effects of bias reduction training. Journal of Social and Clinical Psychology, 23(4), 584-
602. Retrieved from
http://www.psych.mcgill.ca/perpg/fac/baldwin/dandeneaubaldwin.pdf
Dandeneau, S. D., Baldwin, M. W., Baccus, J. R., Sakellaropoulo, M., & Pruessner, J. C. (2007).
Cutting stress off at the pass: Reducing vigilance and responsiveness to social threat by
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
25
manipulating attention. Journal of Personality and Social Psychology, 93(4), 651-666.
doi:10.1037/0022-3514.93.4.651
Dennis, T. A., & O'Toole, L. J. (2014). Mental health on the go: Effects of a gamified attention-
bias modification mobile application in trait-anxious individuals. Clinical Psychological
Science, 2167702614522228. doi:10.1177/2167702614522228
Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to
gamefulness: Defining "gamification". In Proceedings of the 15th International
Academic MindTrek Conference: Envisioning Future Media Environments (pp. 9-15).
doi:10.1145/2181037.2181040
Duggan, M. (2013). Cell phone activities 2013. Pew research internet project. Retrieved from
http://www.pewinternet.org/files/old-
media//Files/Reports/2013/PIP_Cell%20Phone%20Activities%20May%202013.pdf
Enock, P. M., Hofmann, S. G., & McNally, R. J. (2014). Attention bias modification training via
smartphone to reduce social anxiety: A randomized, controlled multi-session experiment.
Cognitive Therapy and Research, 38(2), 200-216. doi:10.1007/s10608-014-9606-z
Finn, J. D. (1989). Withdrawing from school. Review of Educational Research, 59(2), 117-142.
doi:10.3102/00346543059002117
Finn, J. D. (1993). School engagement & students at risk. Washington, DC: National Center for
Education Statistics. Retrieved from http://files.eric.ed.gov/fulltext/ED362322.pdf
Fredricks, J. A., Blumenfield, P. C., & Paris, A. H. (2004). School engagement: Potential of the
concept, state of the evidence. Review of Educational Research, 74(1), 59-109.
doi:10.3102/00346543074001059
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
26
Freeman, S., O'Connor, E., Parks, J. W., Cunningham, M., Hurley, D., Haak, D., . . . Wenderoth,
M. P. (2007). Prescribed active learning increases performance in introductory biology.
CBE Life Sciences Education, 6(2), 132-139. doi:10.1187/cbe.06-09-0194
Goasduff, L., & Pettey, C. (2011). Gartner says by 2015, more than 50 percent of organizations
that manage innovation processes will gamify those processes. Gartner Press Release.
Retrieved from http://www.gartner.com/newsroom/id/1629214
Hughes, J., & Kwok, O.-m. (2007). Influence of student-teacher and parent-teacher relationships
on lower achieving readers' engagement and achievement in primary grades. Journal of
Educational Psychology, 99(1), 39-51. doi:10.1037/0022-0663.99.1.39
MacLeod, C., & Clarke, P. J. (2015). The attentional bias modification approach to anxiety
intervention. Clinical Psychological Science, 3(1), 58-78.
doi:10.1177/2167702614560749
MacLeod, C., Mathews, A., & Tata, P. (1986). Attentional bias in emotional disorders. Journal
of Abnormal Psychology, 95(1), 15-20. Retrieved from
http://psycnet.apa.org/journals/abn/95/1/15.pdf
MacLeod, C., Rutherford, E., Campbell, L., Ebsworthy, G., & Holker, L. (2002). Selective
attention and emotion vulnerability: Assessing the causal basis of their association
through the experimental manipulation of attentional bias. Journal of Abnormal
Psychology, 111(1), 107-123. doi:10.1037//0021-843X.111.1.107
Madden, M., Lenhart, A., Duggan, M., Cortesi, S., & Grasser, U. (2013). Teens and technology
2013. Retrieved from http://www.pewinternet.org/files/old-
media//Files/Reports/2013/PIP_TeensandTechnology2013.pdf
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
27
Meece, J. L., Blumenfield, P. C., & Hoyle, R. H. (1988). Students' goal orientations and
cognitive engagement in classsroom activities. Journal of Educational Psychology, 80(4),
514-523. doi:10.1037/0022-0663.80.4.514
Merikangas, K. R., He, J.-p., Burstein, M., Swendsen, J., Avenevoli, S., Case, B., . . . Olfson, M.
(2011). Service utilization for lifetime mental disorders in U.S. adolescents: Results of
the National Comorbidity Survey-Adolescent Supplement (NCS-A). Journal of the
American Academy of Child & Adolescent Psychiatry, 50(1), 32-45.
doi:10.1016/j.jaac.2010.10.006
Mojtabai, R. (2009). Unmet need for treatment of major depression the United States.
Psychiatric Services, 60(3), 297-305. doi:10.1176/ps.2009.60.3.297
Muntean, C. I. (2011). Raising engagement in e-learning through gamification. In The 6th
International Conference on Virtual Learning ICVL (pp. 323-329). Retrieved from
http://icvl.eu/2011/disc/icvl/documente/pdf/met/ICVL_ModelsAndMethodologies_paper
42.pdf
Prensky, M. (2001). Fun, play, and games: What makes games engaging. In Digital Game-Based
Learning. McGraw-Hill. Retrieved from
http://www.autzones.com/din6000/textes/semaine13/Prensky(2001).pdf
Prensky, M. (2005). "Engage me or enrage me": What today's learners demand. Educause
review, 40(5), 60-65. Retrieved from
https://web.elcosd.org/sandbox/users/dnoll/weblog/5e07f/attachments/536f0/Engage%20
Me%20or%20Enrage%20Me.pdf
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
28
Rideout, V. J., & Roberts, D. F. (2010). Generation M2: Media in the lives of 8- to 18-year-olds.
Menlo Park, California: Henry J. Kaiser Family Foundation. Retrieved from
https://kaiserfamilyfoundation.files.wordpress.com/2013/04/8010.pdf
Zvielli, A., Bernstein, A., & Koster, E. H. (2014). Dynamics of attentional bias to threat in
anxious adults: Bias towards and/or away? PLoS ONE, 9(8), e104025.
doi:10.1371/journal.pone.0104025
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
29
Figure 1. An illustration of the three steps in one trial of the visual probe task. In the first step,
participants must focus their attention on the cross in the center of the page. In the second step,
two stimuli appear very briefly on each side of the screen. In the final step, a directional probe
(up or down arrow) appears behind either the left or right stimulus.
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
30
Figure 2. Examples of a static face matrix (left) and a static flower matrix (right).
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
31
Figure 3. Examples of two different types of steps in a dynamic flower matrix.
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
32
Figure 4. Examples of two different types of steps in a dynamic face matrix.
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
33
Figure 5. Mean difference values in self-reported engagement between the four matrix
conditions. A significant difference was found in the level of engagement reported by
participants from the dynamic design compared to participants from the static design, the latter
finding the task as significantly less engaging. The error bars attached to each column in the
figure represent standard errors.
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
34
Figure 6. Mean changes in acceptance bias in each of the matrix conditions. A significant
difference was found between the stimuli, where faces yielded significant changes while flowers
did not. A significant design by stimuli interaction was also found, where static faces yielded a
significant increase in acceptance bias, while dynamic faces yielded a slight decrease in
acceptance bias. The error bars attached to each column in the figure represent standard errors.
EFFECTS OF ENGAGEMENT ON ATTENTIONAL BIAS MODIFICATION
35
Statement of Contribution
For this project I was responsible for numerous tasks. I conducted extensive research of
the literature with input and suggestions from my supervisor. I was responsible for the design of
the experiment, which involved creating and ordering the questionnaires (using FluidSurveys),
designing the matrix levels, and creating the list of anagrams (which involved creating a survey
and recruiting volunteers to test my anagrams for level of difficulty). I also managed the
implementation and running of the experiment, which involved assigning credit to participants
who participated for course credit, recruiting and distributing financial compensation to external
participants, and collecting regular backups of the data. With help from my supervisor, I
conducted multiple analyses of the data using SPSS software. Finally, I was responsible for the
writing of my thesis, with feedback from my supervisor.
There were a number of aspects of the study provided for me to which I did not
contribute. All aspects of the visual probe task were provided to me by my supervisor, which
included all images used throughout the task as well as the coding for the web application.
Various aspects of the matrix task were also provided to me by my supervisor, including all
images used throughout the task as well as the coding for the web application (involving the help
of an external programmer). The methods of storage of data for both tasks were also provided by
my supervisor.