Cognitive Intersections: Meeting Narrative, Semiotics, and Neuroscience in Video Game Characters

Cody MejeurDepartment of EnglishMichigan State Universitymejeurco@msu.edu@cmejeur

A Question… Or Two.What is the relationship between the cognitive and

semiotics?

Addition: What is the relationship between the cognitive, semiotics, and narrative?

Focus: Video Game Characters

Semiotics and NarrativeSigns in games (including characters)

come togetherPlay between them is narrativeQuestion of scale?

Analysis of narrative (particularly close reading/analysis) requires focus on particular signs and groups of signs

Extract out to larger structuresAlways scripted and emergent qualities

Helpful Comparison: Pyramid of Popular Formula

Borrowing…

The Problem of StructureDoes structure actually match experience?What do structures miss or exclude?Always building yet another model?

What would narrative look like without structure?

Narrative as less of a structure, system, taxonomy, etc., more of a force or cognitive process? Less what it is, more what it does?

Can we design structures that are more fluid, dynamic, adaptable, experiential?

A Thought Experiment

Striving for Dynamism

Semiotics and CognitionCognitive Semiotics and Cognitive

LinguisticsFocus on neuroscience of individual

letters, sounds, wordsExpanding to compound words, clauses,

phrases, sentences.How does player use/interact with it?

Communication?Constructing a Narrative?

Do signs accumulate? Change over time according to

scripted and emergent relations

Character Sign

Signs and PlayersAll signs in games, especially characters,

have some relationship to the playerSigns are never fully stable

Always moving, changingLimits to change, sign must cohere

Force relations within and between signs

Speaking of Player Perception… Narrative and CognitionSigns coming together in narrativeDifficulty of studying narrative via neuroscience

Assumption that activity or interface=narrative?Challenging the Narrative/Play Distinction

Theory? Jenkins, Ryan, Walsh, PrinceDegrees of Narrativity,

Embedded/Emergent/Experiential NarrativeSocial Scientific Methods?

Interview playersForums researchProblem of sample sizes-is this inevitable?

Designing a fMRI experiment Test distinction/relationship

between narrative and play Problem: Controls, Isolating

“Narrative” Choice: Use First Person games

(shooters, exploration, etc.)Avoid character entanglements Mirror/Mimic Player

POV/ConsciousnessQuestion of interfaceAlexander Galloway–what

assumptions are built into the interface?

What are the narrative and cognitive implications of the interface?

Screening QuestionnaireControl for experience, predisposition toward or

against narrativefMRI Play

MRI compatible controllerEye-tracking, video recordingPlay segments of three games: Halo, Half-Life 2,

Bioshock. Similar play, different narrative configurations.

Post Scanner Questionnaire and Oral InterviewModified Game Experience Questionnaire,

Eindhoven University of Technology

Steps to Experiment

Final Suggestions/QuestionsBringing together Semiotics, Narrative, and

Cognition (inc. Cognitive Neuroscience) can help us understand video game characters and narrative meaning-making in games

The reverse is also true: games can change our concepts and theoriesChallenge us to rethink narrative and our

reliance on structureTo what extent is this possible?

What are the benefits and limitations of neuroscience?

What does the study of game narrative look like when it moves past model-making?