229

Cognitive Modeling Of Video-Game Player User-ExperienceCorey J. Bohil; Frank A. Biocca

Michigan State Universitybohi/@msu.edu

Abstract. This paper argues for the use of cognitive modeling to gain a detailed and dynamic look into userexperience during game play. Applying cognitive models to game play data can help researchersunderstand a player's attentional focus, memory status, learning state, and decision strategies (among otherthings) as these cognitive processes occurred throughout game play. This is a stark contrast to the commonapproach of trying to assess the long-term impact of games on cognitive functioning after game play hasended. We describe what cognitive models are, what they can be used for and how game researchers couldbenefit by adopting these methods. We also provide details of a single model - based on decision fieldtheory - that has been successfUlly applied to data sets from memory, perception, and decision makingexperiments, and has recently found application in real world scenarios. We examine possibilities forapplying this model to game-play data.

1. INTRODUCTION

A major goal of video game research is tounderstand and influence what a player is thinkingduring game play, and perhaps to effect long termchanges in the game player. One major theme inthis research includes game impact on personalitytraits and emotional states - most famously, effectson player aggressiveness. This type of research istypically carried out via survey methods. Playersanswer a battery of questions before the gamebegins to assess their personal traits and currentemotional state. After game play has completed,the player answers more questions and anychanges are attributed to the effects of theintervening game play period. Another prominenttheme is the effect of games on some aspect ofcognition (e.g., spatial skills [5]). Although thissecond research theme is concerned with cognitiveeffects of games, it is often carried out by pretestingthe participant using a standard laboratory task(e.g., a speeded search task) to assess the trait ofinterest (e.g., visual acuity), followed by a gameplay period, and then posttesting using the samestandard laboratory task again to see ifperformance has changed.

These approaches take a bird's eye view of thecognitive phenomena underlying game play. Theytreat the mind like an impenetrable black box,observing or manipulating inputs to the cognitivesystem, and observing the concomitant outcomes.Although these research endeavors are valuable,they take an indirect route to understandingcognition during play. In both approaches outlinedabove, game play effects are measured after thefact. In the case of questionnaire methods, the datais subjective - participants give some indication of

the game's effects through their answers to variousquestions. And although the second researchapproach aims at understanding something aboutcognition as it pertains to games, it too focuses oneffects and measures indirectly by assessingchanges after the fact.

What is needed is an approach that allows one totrack cognitive effects of games during the gameplay session. As the player progresses through thegame, seeking to accomplish various goals, makingdecisions, all manner of cognitive phenomena cometo bear. Learning is required (e.g., what strategiesworked before?). Attention has to be allocated.Memories of previous outcomes have to beaccessed. Decisions have to be made. Theoutcome of these cognitive events translate into theplayer's observable performance in the game, theirlevel of enjoyment or accomplishment, theirlearning. Understanding these dynamic events asthey unfold throughout the course of play, ratherthan trying to infer something about themSUbjectively or indirectly after the game is over,would be of great value to the designer of gamesconcerned with changing behavior, communicatingmessages, or just maximizing engagement and fun.In attempting to maximize game efficacy, thedesigner would likely benefit from knowing what aplayer is looking at when making decisions, whatdimensions are most salient, which dimensions areroutinely ignored, and when options are confusable.

In recent years, there have been efforts to peer intothe inner workings of the mind during the game playevents that cause them. Brain imaging techniques(e.g., fMRI) have been used to associate brain

. activity known to occur during aggressive thoughtwith violent game content [8]. Other studies have

https://ntrs.nasa.gov/search.jsp?R=20100012847 2018-06-27T23:21:36+00:00Z

230

tracked psychophysiological events (e.g., EEG) toinfer mental states during play [1]. This approach tostudying player cognition during game play is awelcome addition to the field. This research givesus valuable data synched in time to game events,and we can learn a lot by trying to interpret it.However, at this point in time, both imaging andpsychophysiolical data are difficult and expensive toobtain, the measures are still relatively crude, andfindings are often difficult to interpret. The degreeto which one can relate the observed bodily statesto aspects of cognition such as attention orexecutive function is an issue of lively (sometimeswithering) debate [7].

Luckily, there is another approach to understandingcognitive phenomena at our disposal - cognitivemodeling. For the past several decades, cognitivepsychologists have devised and tested scores ofdetailed mathematical models that offer preciseaccounts of the cognitive underpinnings ofbehavior, and demonstrated their links to theoreticalstructures like memory and attention. Given themature state of this field, as well as its widespreadrepresentation throughout academia, it is surprisingto find that it has very little representation in thegame studies literature (although some applicationsare noted below). The objective of this paper is toprovide a basic understanding of what cognitivemodels can provide researchers, and to advocatetheir use in studying video games.

2. COGNITIVE MODELING

What, exactly, is a "cognitive model"? A cognitivemodel is a mathematical interpretation (Le.,specification) of the set of principles embodied in atheory of cognition. Cognitive models make specificassumptions about the information represented inthe cognitive system (e.g., words and theirmeanings), along with the processes acting on thisinformation to produce observable cognitivebehavior (e.g., classifying an object). Moreconcretely, a model receives inputs like a person inan experiment (e.g., size of objects on a screen),performs mental operations (e.g., like comparingperceived stimulus information to information storedin memory), and outputs a response (e.g., emits aclassification of the object).

Models such as these are valuable for severalreasons. First, they require a researcher to movepast the initial stages of theorizing - oftencharacterized by vague verbal descriptions ofmental entities and their interactions - to taking adetailed, specific stance on these quantities andrelationships. Doing this affords the research

community a better opportunity to evaluate andcriticize a theory's quality. Second, making detailedquantitative statements in a cognitive model allowsa researcher to make precise, testable predictions.A third benefit is that simulating model behavior ona computer can lead to unexpected observationsand insights that the researcher might not otherwisehave reached. It is widely agreed in the modelingcommunity that this is an important benefit ofmodeling.

There currently exists a wide array of cognitivemodels that have been vetted over the years bymany experiments and data sets. These modelselucidate a range of topics. Many models aredesigned to capture steady-state performance incognitive tasks like recognition memory,discrimination ability, attention allocation, to name afew. These models are intended to account forspecific, circumscribed aspects of cognition such asrecognition, categorization, attention, etc. Anotherclass of models - known as connectionist models(also called neural net or parallel distributedprocessing models) - mimic fundamental aspects ofbrain anatomy (Le., populations of singleprocessing units or artificial neurons communicatingactivation levels back and forth) and capturelearning over the course of many training trials. Athird class of models - known as cognitivearchitectures (e.g, ACT-R, EPIC, Soar) - attempt tocapture several aspects of cognition in a singleunified framework (e.g., attentional processes,memory, visual search tendencies), reflecting thefact that all these processes come into playsimultaneously in the human cognitive system.Cognitive architectures have found wide applicationin human-computer interaction research and haveeven made their way into game research to someextent [4]. EXisting applications of cognitivemodeling in game research tends to take on acomputer-science flavor. These models arevaluable tools for making the game respond to theplayer in interesting ways or to create "smarter"non-player characters [3]. Our aim in this paper isto encourage much more widespread adoption ofthese techniques for gaining general understandingof the cognitive capacities invoked during videogame play.

3. APPLYING COGNITIVE MODELS TO GAMESTUDIES

The value of models for game research lies in thefact that models require inputs and produceoutputs. In between they offer precise statementsabout attention, learning, decision strategies andbiases, and so on. In doing so, a model often tells

231

the researcher why performance looks as it does.Although a model can't tell the designer exactly howto craft a game environment that teaches orentertains, discovering that current inputs placeunrealistic demands on attention might offerguidance by narrowing the range of necessarymodifications to gain desired results. An importantdetail, of course, is how one goes about applyingthese models.

Within a cognitive model lie parameters that capturethe modeled quantities (e.g., attention weights,learning rate, response biases). These values areindicators of the mental underpinnings ofobservable behavior. In order to make inferencesabout cognition, these models are often "fit" to a setof data. The computer takes the output of themodel (Le., predicted responses to events),compares it to player data (Le., actual responses)and adjusts the internal parameter values (Le.,changes assumptions about attention, etc.) until thepredicted responses are as close as possible to thedata. The reSUlting adjusted parameter valuesindicate things like how confusable the stimuli wereor which stimulus dimensions garnered the mostattention. These parameter values can be used tomake predictions for the player in later gamesessions or scenarios.

It is also important to verify that what the model tellsus is correct. In order to do this, researchers oftenattempt to fit a model to data using fixed parametervalues gleaned from prior knOWledge of theresearch participant. Achieving a good model fit(Le., a good prediction of player performance) bysetting parameter values a priori is a powerfuldemonstration that one understands the player'scognitive processes during play.

One way to obtain fixed parameter values for apriori prediction is to fit the model (by adjusting freeparameters) to one data set, and then use the best­fitting parameter values to see if the modelaccounts for additional data sets (without re­adjusting the parameter values). Another way todemonstrate our understanding is to set the modelparameters based on something else we alreadyknow about the player.

For example, one could take advantage of the kindsof data acquired through the survey methodsdescribed above. One recent (non-game) studyused results from a survey designed to assesswhether a person has an "action" orientation(tendency to accept risks to expedite achieving agoal) or a "state" orientation (tendency to be moredeliberative in order to avoid risks). Scores on thisquestionnaire were converted into parameter values

in a cognitive model and used to predict responseprobabilities and response time distributions in asports-related task [6]. Such an approach groundsmodel parameters in knowledge about theparticipant even before experimental manipulationbegins, and can still enable the model to makeinteresting predictions about behavior.

Another possibility would be for the researcher toset model parameters to reflect instructions given tothe player (either before the game or inside thegame). Instructing a player to pay attention only toRED enemies, for example, should be reflected in amodel's attention weight parameters (assuming themodel has them) and consequently in the model'spredicted response probabilities (and hopefully leadto a good model fit). An important long-term goal ofmodeling is to find parameter values that can leadto valid predictions across several experimentalconditions without the need to adjust parameters toaccount for each data set.

One challenge to applying cognitive models to datafrom video games is that events of interest must beoperationally defined. For example, someagreement might need to be reached about whatconstitutes "fighting or fleeing" in a game scenario.Another example would be determining whatqualitifies as a response option. Depending on thequestion under study, it may be wise to compareperformance only in situations with a constantnumber of response options. Such apples-to­apples comparisions might be necessary whentrying to determine response probabilities orresponse time distributions.

4. DECISION FIELD THEORY

Among the many aspects of cognition that can bemodeled and examined in games, perhaps the mostnatural starting point is to look at decision making.One popular class of models that illuminatesdecision making is known as "sequential sampling"models. Sequential sampling models simulate theaccumulation of information (Le. sampling) overtime in support of each choice alternative, leadingto the eventual selection of one option over others.Decisions are triggered by internal choicethresholds - the first accumulation process to reachthreshold wins, and the corresponding choice ismade. Figure 1 depicts this sampling process forthree choice options.

232

1 Decision Trial

Table 1: Hypothetical dimension values for weaponchoices

Also, each player is likely to display somedifference in preference for the choice dimensions.For example, the player's decision might be moststrongly influenced by the strength dimension 70%

of the time. Range might be the most influentialdimension 20% of the time, and ammo only 10% ofthe time. These values are model parameters. Themodel uses these values on each trial, along withother parameters representing initial biases,memory from trial to trial, and similarity betweenoptions, to produce a decision.

Figure 1 displays a characteristic example. Thefigure shows the (simulated) stochasticaccumulation over time of evidence (to the cognitivesystem) in favor each of the three options. The firstoption to reach an internal decision threshold "wins"the race, and supplies the response. As the figureshows, not only is a choice determined from thisprocess, but also the time to reach threshold. Fromtrial to trial, responses and termination times willvary, and over trials the model will provide responseprobabilities and response time distributions thatcan be compared to a player's data. By adjustingthe internal model parameters in order to fit theobserved responses, the model tells a tale aboutthe player's attention focus, memory, biases, andthe confusability of the response alternatives.

This decision-field theoretic model could potentiallyanswer many interesting questions. For example,how does action orientation predict game play?How does decision strategy change as a result oflearning throughout the game? Which dimensionsreceive the most attention, and which the least?

5. DISCUSSION

We've argued in this paper that cognitive modelingprovides a detailed and dynamic view into cognition- at the individual player level - as it unfolds duringvideo game play. Currently, this powerful approachis seldom utilized in game research. This is ashame, since cognitive modeling is a mature field,and there are many useful models available thathave been affirmed by decades of research incarefully controlled experiments.

Models can offer clues into the inputs required toproduce the outputs desired. If a game is to haveeducational value, (or for communication or evenjust for fun), then variables that influence modelbehavior should be manipulated to moderate playerbehavior. Currently, most game design is guidedby heuristics, prior experience, and flashes ofinsight.

Many of the cognitive models in existence today areready for extension to new areas. In fact, the fieldof cognitive psychology is increasingly marked byattempts to extend the reach of cognitive theories toreal-world scenarios. The application of these tools

nme

-Option2

-Option3

Decision Threshold

Strength Range AmmoOption 1 1.0 50 0.6

Option 2 0.5 150 0.8

Option 3 0.7 100 0.8

Figure 1: Information accumulation for three choiceoptions

In this section, we describe a sequential samplingmodel based on Decision Field Theory [2).Variants of this model have been successfullyapplied to a wide range of phenomena, includingdecision making, perception, and memory, amongothers. The model has mostly been applied to datafrom standard laboratory tasks, but has recentlybeen used to explain decision making in a sportsjudgment task [6).

On a given experimental trial (in the context ofgames, an operationally defined recurring event),the model assumes that each set of choice optionscan be characterized by values along salientdimensions. For example, when trying to choosethe best weapon for a fight, the player mightconsider three weapons along dimensions such asstrength, range, and ammunition supply. Eachweapon has its own set of values on thesedimensions, and the player makes someassessment of these values. Table 1 illustratessome hypothetical values.

233

is especially timely considering the recent explosionof research into serious games (games designed tocommunicate and educate players). Designers ofsuch games would likely benefit from a tool that canhelp foster a deeper understanding of what playersfocus on and are affected by during game play.

Finally, cognitive modeling dovetails well with theimaging and psychophysiological researchmentioned above. The relatively recent emergenceof the field of cognitive neuroscience attests to this.Cognitive models have become so powerful thatcompetition between theories is often difficult toassess on the basis of behavioral data alone.Neuroscience data is now routinely used to placebiological plausibility constraints on computationalmodels. In turn, cognitive modeling imparts a deeplevel of meaning to neuroscience results. Modelshelp neuroscientists understand the cognitiveimplications of their data.

In conclusion, cognitive modeling presents apowerful method for understanding what a player isthinking about while playing a video game.Research papers that describe cognitive modelsoften report their model derivations in detail so thatinterested readers can adopt these methods. Ourhope is that we've been able to convince readers ofthe allure of cognitive modeling for their own game'research.

6. ACKNOWLEDGMENTS

We thank Bradly Alicea, Nick Bowman, AndyBoyan, Allison Eden, & Mayur Mudigonda for theircomments on an earlier draft of this article. Thiswork was funded by an AT&T endowment to thesecond author.

REFERENCES

1. Baumgartner, T., Valko, L., Esslen, M., &Jancke, L. "Neural correlate of spatial presencein an arousing and noninteractive virtual reality:An EEG and psychophysiology study."Cyberpsychology & Behavior 9, 1 (2006), 30-45.

2. Busemeyer, J. R, & Townsend, J. T. "Decisionfield theory: a dynamic-cognitive approach todecision making in an uncertain environment."Psychological Review, 100 (1993),432-459

3. Funge, J. D. AI for Computer Games andAnimation: A cognitive Modeling Approach. AKPeters Ltd., Wellesley, MA, USA, 1999

4. Gray, W. D. (Ed.) Integrated Models of CognitiveSystems. Oxford University Press, New York,2007

5. Kaveri, S., & Greenfield, P. M. "Effect of videogame practice on spatial skills in girls and boys."Journal ofApplied Developmental Psychology15,1 (1994),13-32.

6. Raab, M., & Johnson, J. G. "Individualdifferences of action-orientation for risk-taking insports." Research Quarterly for Exercise andSport, 75,3 (2004), 326-336.

7. Uttal, W. R The New Phrenology: The Limits ofLocalizing Cognitive Processes in the Brain.The MIT Press, MA, USA, 2003

8. Weber, R, Ritterfeld, U., & Mathiak, K. "Doesplaying violent video games induce aggression?Empirical evidence of a functional magneticresonance imaging study." Media Psychology8,1 (2006), 39-60.