I did that! Measuring Users’ Experience of Agency in their own ActionsAUTHORS: DAVID COYLE, JAMES MOORE, PER OLA KRISTENSSON, PAUL C. FLETCHER, AND ALAN F. BLACKWELL

PRESENTER: MARY YOUNG

Agenda

Relevance in HCI

Background

Experimental Design

Conclusion

Relevance in HCI

“Users strongly desire the sense that they are in charge of the system and that the system responds to their actions.”

- Designing the User Interface: Strategies for Effective Human-Computer Interaction

Shneiderman’s Seventh Golden Rule of Interface Design:

Support an internal locus of control

Agency in interface design

Examples of Users’ Experience of Agency

Views of Agency in HCI

Media Agency Intelligent Agent

Design Agency

Background

Definition of Agency“A person’s innate sense of being in control of their actionsand through this control of being responsible for, or having ownership of, the consequences of these actions.”

A Little Bit of NeurologyTwo theoretical positions:•Agency is based on the sensory comparison of predicted versus

actual consequences

• Agency is a reconstructive inference based on the assumption that one’s intention has cased an external event

Humans do not always perceive agency correctly• Ouija boards

• Gambling

• Neurological disorders

Measuring Agency

Voluntary actions cause changes in our perception of time

This is termed intentional binding

Measuring AgencyHigher temporal binding indicates a greater sense of agency

Two methods:◦ Libet Clock

◦ Robust measurement

◦ However, user must look at a clock

◦ Interval Estimation◦ Less robust

◦ No clock

Libet Clock

Calculating AgencyAction measurements

Baseline error Report time when button was pressed; no beep

Active error Report time when button was pressed; beep

Outcome measurements

Baseline error Report time when beep occurred; no button pushed

Active error Report time when the beep occurred; button was pushed

Calculating Agency

Error = actual time – perceived time

Action binding = action error[active] – action error[baseline]

Outcome binding = outcome error[baseline] – outcome error[active]

Total binding = action binding + outcome binding

Again: greater total binding values indicate greater sense of agency.

Experiment

Goals of Experiments•Develop methods to implicitly measure agency in an HCI context

•Determine how perception of agency is affected by different input modalities

•Determine how perception of agency is affect by varying degrees of computer assistance

•Target demographic: everyone

What’s it like to be a button?

Experiment 1: Changing Input Modalities•Independent variable: input modality• Skinput vs MAC keyboard

•Dependent variable: length of binding• Libet clock

•Design: Within-subjects

•Participants• 21 right-handed individuals ages 20-40

•Recruited via email to a University bulletin board

Experiment 1: ProcedureTrials:

40 trials of:• Action baseline error• Action active error• Outcome baseline error• Outcome active errorPer input modality or 320 total per participant.

Randomization: modality alternated, random order of measurements, but measured in blocks

Experiment 1: Results

Modality Actionbinding

Outcomebinding

Total binding

Button 6.81ms(45.6ms)

36.11ms(45.46ms)

42.92ms(67.43ms)

Skin-based 29.66ms(42.84ms)

79.82ms(91.23ms)

109.47ms(74.54ms)

Significance testing:• Bonferroni corrected paired sample t-test• H0: No difference in total binding for input modality• t(18)=4.05, p<0.01• Conclusion: Reject H0

Experiment 1: Analysis and Discussion•No significant difference in baseline conditions

•Greater sense of agency for skin-based input than keyboard

•Furthermore: input modality matters in a quantifiable way•Possible applications: stylus vs touch

Experiment 2: Computer Assistance•Independent variable: level of computer assistance•No, mild, medium, and high assistance

•Dependent variable: difference in interval estimation

•Design: Within-subjects

•Training: 12 practice trials

•Participants•27 right-handed individuals ages 20-40•Recruited via email to a University bulletin board

Experiment 2: ProcedureTrials:

12 trials of:• None (α = 0)• Mild (α = 3)• Medium (α = 6)• High (α = 9)

Per time interval (150ms, 400ms, 700ms) or 144 trials per participant

Randomization: assistance levels blocked, interval was random

Experiment 2: Results

NoAssistance

MildAssistance

MediumAssistance

HighAssistance

-16.78ms(70.70ms)

-16.32ms(82.03ms)

9.93ms(85.92ms)

4.53ms(79.54ms)

• Bonferroni corrected paired sample t-tests• H0: No difference in estimation errors• No significant difference between no and mild

assistance, p=0.97• No significant difference between medium and

high assistance, p=0.679• Significant difference between mild and medium

assistance level, p < .01

Experiment 2: Analysis and Discussion•Participants were aware of all four assistance levels

•Threshold for computer assistance, before which assistance doesn’t hinder experience

•Mapping of assistance variable to agency

•Future applications:•More complex systems such as machine learning or AI

• Level of agency when a computer hinders rather than helps

Final Discussion and Conclusion• Paper provides two empirical methods for measuring agency

• Future interfaces can be evaluated based on user’s perception of agency

• Other factors to be researched:•Quality of feedback•Uncertainty•Differences between demographics

ReferencesShneiderman, B., & Plaisant, C. (2004). Designing the user interface: Strategies for effective

human-computer interaction. (4 ed.). Pearson.

Harrison, C., Tan, D., & Morris, D. (2010, April). Skinput: Approximating the body as an input surface. 28th Annual SIGCHI conference on human factors in computing systems , New York, NY.

Image creditsSlide 4:

Auto correct: http://www.freemake.com/blog/12-funny-autocorrect-mistakes/Auto complete: http://www.esarcasm.com/13557/google-autocomplete/Vim: http://www.celsius1414.com/tag/vim/Photoshop: http://pe-images.s3.amazonaws.com/photo-effects/photo-in-photo/new/photoshop-colorize.jpg

Slide 5:Auto correct: http://www.freemake.com/blog/12-funny-autocorrect-mistakes/Auto complete: http://www.esarcasm.com/13557/google-autocomplete/Vim: http://www.celsius1414.com/tag/vim/Photoshop: http://pe-images.s3.amazonaws.com/photo-effects/photo-in-photo/new/photoshop-colorize.jpg

Slide 10: Libet clock: http://www.informationphilosopher.com/solutions/scientists/libet/Libet_Clock.gif