Evaluating User Experience And

User Engagement for Design

Jennefer Hart PhD Year One

Supervisors: Professor Alistair Sutcliffe & Dr Antonella De Angeli

Manchester Business School July 2011

Research Contribution

  Extend (and Verify) User Experience Theories

  Explore the Nature of Interactivity

  Develop and Test New Research Methodologies

  Inform Design Practice

  Improve Product Design

Research Aims & Objectives

Extend the Cognitive Models within

User Experience

Develop Research Methodologies

Explore Prolonged Patterns of

Interaction

Inform Design

Guidelines

➤  To Investigate Key Constructs (e.g., Interaction, Immersion, Flow, Presence, Aesthetics, Emotion & Individual Dispositions).

➤  To Capture User Experiences (as they occur).

➤  To Capture UX Patterns (over a prolonged time frame)

➤  To Inform Design Guidelines (by determining what interactive features elicit positive UX)

Research Questions RQ1: What are the key constructs that contribute

to user engagement and user experience?

RQ2: What design protocols or features lead to positive user experiences?

RQ3: What are the patterns of interaction over prolonged use?

Applied Method

X-Section Survey

METHODS

Diary Study

Retrospect Interview

Research Plan & Methodology

Quasi-Experiment

STUDY 1

Controlled Experiment

STUDY 2

Longitudinal Study

STUDY 3

Adapted Method

Observation

Survey

Interview

METHODS

Test Method

Research Plan (1)

Study One: Quasi Experiment   Compare 3 different levels of interactive protocols

  Aesthetics & Usability

  Interaction & Engagement (Flow & Presence)

  Individual Aesthetic Disposition

*[Karapanos, Martens & Hassenzahl, 2009]

RSQ 1: What are the key constructs that contribute to user experience?

  Develop Research Methodology   Survey Instrument

  Observation – Critical Incidents

  Interval Mood Scales (iScale*)

  Video - Facial Expression

The National Gallery – Menu-based Interaction

The Google Art Project – Virtual Interaction

The Louvre – Interactive Animated Guide

Research Plan (2) Study Two: Controlled Experiment

  Compare 2-3 different levels of interactive protocols   Bespoke Technologies   Same Content V Different Interactive Features (e.g. video, presence,

virtual, etc)

  Research Methodology (from Study One)   Tested Survey Instrument   Observation – Critical Incidents   Interval Mood Scales (iScale)   Video – Facial Expression

  Focus on Key Design Features   Interactive Guide (Presence)   Virtual Environment (Flow & Engagement)

RSQ 2: What design ‘features’ lead to positive user experiences?

Research Plan (3)

Study Three: Longitudinal Study   Compare 2-3 different levels of interactive protocols

  Existing technologies (within same domain)

  Prolonged Time Frame   Capturing patterns of prolonged user experiences

  Adapted Research Methodology   Diary Based Study

  Experience Probes

  Adapted Mood Scales (iScale)

  Two Phased Survey

  Staged Interviews

  Selected Observation

RSQ 3: What are the patterns of interaction over prolonged use?

Research Technologies: Personal Fitness Tools

Classic Text Based Style Video Based Interaction Style

Active (Haptic) Interactive Style Interactive Customization Facility

any questions?

Thank-you

  Hassenzahl, M. (2003) The thing and I: understanding the relationship between user and product. In M.Blythe, C. Overbeeke, A. F. Monk, & P. C. Wright (Eds.), Funology: From Usability to Enjoyment, Dordrecht: Kluwer Academic Publishers, pp. 31-42.

  Csikszentmihalyi, M. (2002) Flow: The Classic Work on How to Achieve Happiness, London: Rider.   Hartmann, Sutcliffe, & De Angeli, (2008) Towards a Theory of User Judgement of Aesthetics and User Interface

Quality. ACM Transactions on Computer-Human Interaction, 15(4), 15-30.   Hassenzahl, M. (2003) The thing and I: understanding the relationship between user and product. In M.Blythe, C.

Overbeeke, A. F. Monk, & P. C. Wright (Eds.), Funology: From Usability to Enjoyment, Dordrecht: Kluwer Academic Publishers, pp. 31-42.

  Hassenzahl, M. (2004) The Interplay of Beauty, Goodness, and Usability in Interactive Products, Dordrecht, Human Computer Interaction (19) 4, 319-349.

  Jordan, P. W. (2002) Designing Pleasurable Products: An Introduction to the New Human Factors, CRC Press.   Karapanos, E, Martens, J, B., & Hassenzahl, M. (2009) Reconstructing Experiences through Sketching, ACM TOCHI.   Law, E. L., Roto, V., Hassenzahl, M., Vermeeren, A. P. O., & Kort, J. (2009) Understanding, scoping and defining user

experience: A survey approach, Proceedings: CHI 2009, Botson MA, pp. 710-728.   Leong, T. W., Vetere, F. & Howard, S. (2005) The Serendipity Shuffle Proceedings of the 19th conference of the computer-

interaction special group (CHISIG), Citizens online: considerations for today and the future, OZCHI 2005, Canberra, Australia.

  Nielson, J. (1993) Usability Engineering, Morgan Kaufmann, San Francisco.   Norman, D. A. (2004) Emotional Design: Why we love (or hate) everyday things, New York: Basic Books.   Shen, K. N., & Khalifa, M. (2008) Exploring Multidimensional Conceptualizations of Social Presence in the Context of

Online Communities, Intl. Journal of Human-Computer Interaction, 24(7), 722-748.   Sutcliffe, (2010), Designing for User Engagement: Aesthetic and Attractive User Interfaces. Edited by Carroll, J.M,

Synthesis Lectures on Human-Centred informatics, Morgan & Claypool Publishers, Chapter 1.   Tractinsky, N., Katz, A. S. & Ikar, D. (2000) What is beautiful is usable, Interacting with Computers, 13 (2), 127-145.   Tractinscky and Zmiri, (2006) Exploring Attributes of Skins as Potential Antecedents of Emotion in HCI, In: Fishwick,

P, (ed.) Aesthetic Computing, MIT Press. Cambridge.   Wright, P., & McCarthy, J. (2009) Experience-Centered Design: Designers, Users, and Communities in Dialogue,

Synthesis Lecturers on Human-centered Informatics, Morgan & Claypool Publishers.

References