Nicolas Van Labeke

National Biomedical Research Unit in HearingLearning Sciences Research Institute – University of Nottingham

www.lsri.nottingham.ac.uk/nvl/

http://hearing.nihr.ac.uk/

Principled design of game-based

auditory learning environments:

Examples of ADT for Tinnitus

The Context

• Aim: Combine auditory perceptual learning, technology-enhanced learning and game theories to create auditory learning games suitable for use outside the lab

– Design games that are intrinsically motivating

– Wide range of age- and disability-related factors of people with hearing loss favours a casual game approach

– Principles of auditory learning suggest the use of intrinsic integration (Habgood 2007)

– training for learning testing of learning

Intrinsic Integration

• How do we design effective learning games?

• Learning material

• Motivational factors: challenge, fantasy, curiosity, control, cooperation, competition, recognition (Malone & Lepper 1987)

• Core mechanics– Rules of the game – Essential interactions required to create gaming

experience– Explain individual preferences to games– Define different game genre & flow experience

Challenge People are best motivated when they are working toward personally meaningful goals whose attainment requires activity at a continuously optimal (intermediate) level of difficulty.

Set personally meaningful goals.Make attainment of goals probable but uncertain.Give en-route performance feedback.Relate goals to learners' self esteem.

Curiosity Something in the physical environment attracts the learner's attention or there is an optimal level of discrepancy between present knowledge or skills and what these could be if the learner engaged in some activity.

Stimulate sensory curiosity by making abrupt changes that will be perceived by the senses.Stimulate cognitive curiosity by making a person wonder about something (i.e., stimulate the learner's interest).

Control People have a basic tendency to want to control what happens to them.

Make clear the cause-and-effect relationships between what students are doing and things that happen in real life.Enable the learners to believe that their work will lead to powerful effects.Allow learners to freely choose what they want to learn and how they will learn it.

Fantasy Learners use mental images of things and situations that are not actually present to stimulate their behaviour.

Make a game out of learning.Help learners imagine themselves using the learned information in real- life settings.Make the fantasies intrinsic rather than extrinsic.

Competition Learners feel satisfaction by comparing their performance favourably to that of others.

Competition is more important for some people than for others.People who lose at competition often suffer more than the winners profit.Competition sometimes reduces the urge to be helpful to other learners.

Cooperation Learners feel satisfaction by helping others achieve their goals.

Cooperation occurs naturally as well as artificially.Cooperation is more important for some people than for others.Cooperation is a useful real-life skill.Cooperation requires and develops interpersonal skills.

Recognition Learners feel satisfaction when others recognize and appreciate their accomplishments.

Recognition requires that the process or product or some other result of the learning activity be visible.Recognition differs from competition in that it does not involve a comparison with the performance of someone else.

Vockell, E. (2004). Educational Psychology: a practical approach. Online Workbook, last retrieved September 2009, http://education.calumet.purdue.edu/Vockell/EdPsyBook/.

Intrinsic Integration (Habgood 2007)

• Motivational effects of challenge, control, etc. are realized through the core mechanics

• Integrate the learning material with the core mechanics and NOT the fantasy

“Deliver learning material through the parts of the game that are the most fun to play, riding on the back of the flow experience produced by the game, and not interrupting or diminishing its impact”

“Embody the learning material within the structure of the gaming world and the player’s interactions with it, providing an external representation of the learning content that is explored through the core mechanics of the gameplay”

Design-based Research (DBR)

• to address theoretical questions about the nature of learning in context

• to consider approaches to the study of learning phenomena in the real world rather than the laboratory

• to go beyond narrow measures of learning

• to derive research findings from formative evaluation

adapted from Y. Mor 2008

A practical case: ADT for Tinnitus

• Collaboration with NBRUH tinnitus group

• Evolutionary framework for testing different motivators (fantasy, control, challenge) and gameplays

• Evaluating different game-based paradigms– STARv2 (baseline)

• Active listening, trial-based tasks

– Auditory Submarine• Interactive listening (sonar analogy)• system-induced gameplay (steering controlled pace)

– Auditory Treasure Hunter• Interactive listening (metal detector analogy)• listener-induced gameplay (listener's selection, activation, and

execution of ADT)

• Impact of visual representation in gameplay and task performance– Distractor vs. scaffolding– Effectiveness of auditory learning games

Auditory Submarine: Intrinsic

Integration

s s i dt

Fq

Time

df

Auditory Submarine: Intrinsic

IntegrationSx

SyGt

s s i dt

Fq

Time

df

Vs

Si• Mapping core mechanics &

learning materials

• Adaptive model

• Design Decisions– impact on learning – impact on gameplay

• Iterative design

• Incremental prototyping

Improving the Gameplay

• Adaptive methods

• Introducing levels and difficulty

– Challenge, competition, recognition

• Side-scrolling game approach?

Auditory Treasure Hunter:

introducing control

• Still on the drawing board

• Analogy of metal detector for mapping ADT

• Ground filled with different objects of different values (e.g. golden and silver nuggets, metal trash, etc.).

• Trade-off between difficult/valuable and easy/cheap

• External constraints (e.g. time limit, charges in metal detector)

• Levels and decision-making by “treasure map” exploration & clearing

Study overview

• Cross-over design– Comparison of treatments

(training phase 1)– Comparison of game experience

(training phases 1-3)

• Evaluation based on tinnitus-related outcomes measures AND desirability & flow experience

• Iterative Design process– Prototype development at

NBRUH (SDG)– Usability testing with

NBRUH/LSRI members (mostly normal hearing)

– Piloting and in-depth testing with participants to previous Tinnitus Study

STARv2

AuditorySubmarine

AuditoryTreasure Hunter

Technical considerations

• Aims: deployment on personal computer desktop, laptop, (potentially) mobile/UMPC

• Silverlight– Microsoft competitor to Adobe Flash– Subset of high-level framework .Net (i.e. shared libraries &

common API)– Browser and off-browser execution modes– Off-line/online data management

• BUT …– Audio manipulation supported with basic APIs– Lack of robust and reusable libraries

• requirements for ADT are ground-breaking!• Current prototypes based on a proof-of-concept synthesizer!

– Accumulation of layers on top of (generic) audio management

• IT IS A PROTOTYPE-BASED APPROACH!

• TECHNOLOGY IS IMPROVING FAST

Thanks

• NBRUH Software Dev. Group

– Mike Sharples

– Stuart Whitehead

– Martin Morrison

• NBRUH Tinnitus Group

– Deb Hall

– Derek Hoare

– Victoria Kowalkovski