UX-Research 6b Results-Evaluation EyeTracking · 2019. 12. 9. · UX-Research Results Evaluation ....

UX-ResearchResults Evaluation

Eyetracking

KP Ludwig John

UX Researchresults evaluation Eye Tracking


Eyetracking

KP Ludwig John

Eyetracking results evaluation

https://www.youtube.com/watch?v=YsCokWHGLXcErgoneers Eyetracker 2013 - Ronaldo


Eyetracking

KP Ludwig John

Eyetracking

What you’ve got so far

results evaluation

Recordings of • Fixations and Saccades • its order and duration • mapped onto the test object

Next steps

by all of your tested users (separate)

ReviewCumulate and VisualizeUnderstand and communicate


Eyetracking

KP Ludwig John


AnalysisTools

ReviewCumulate and Visualize


Eyetracking

KP Ludwig John

Eyetracking

Heatmap

results evaluation

source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 118

Based on either

• number of fixations or

• fixation length

recommendedone long fixation should count for more than a short one

Shows how looking is distributed over the stimulus.

Can be created with one or many users

http://www.nngroup.com/reports/how-to-conduct-eyetracking-studies


Eyetracking

KP Ludwig John

Eyetracking

Heatmap


- Display Colors can be chosen

Software Defaultred = area most looked at („hot“)green = area least looked at („cold“) blanc = no fixations

n/n group

red = area most looked at („hot“)blue = area least looked at („cold“)

gray = no fixation at all

analysis tools



Eyetracking

KP Ludwig John

Eyetracking

Heatmap

Problems

• no indication of the order people looked at items

• difficult with dynamic elements

that appear or move:drop down, pop up, animation

• probably misleading

Qualities• easy to read

• very popular


analysis tools

!



Eyetracking

KP Ludwig John

Eyetracking

source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 135/137

Gaze differs from page to pageeven referring to the same test object

Influenced by

• InterestSubpages may attract different interests

• Experiencegained on previous pages of the same site

Screenshots from kiehls.com (sells cosmetics)

analysis tools

Heatmap

User Recall


http://kiehls.com


Eyetracking

KP Ludwig John

Eyetracking

User RecallScreenshots from kiehls.com (sells cosmetics)

Subpage/product pageUser interest: gathering information about the specific product

No fixations to Menuat the top of page!!


Heatmap may indicate, users did not look at the navigation. But they did!Just not on this page.

analysis tools

Experience with site: Familiar with navigation and categories of the site from previous visits to Home and other product pages

http://kiehls.com



Eyetracking

KP Ludwig John

Eyetracking

User Recall


HomeUser interest: • general orientation • search for categories Experience with site: • none

Many fixations to Menuat the top of page!

User want get ageneral idea of the site.

Once they’ve decided forone of the categories, interest (and gaze) changes.

analysis tools

Screenshots from kiehls.com


http://kiehls.com


Eyetracking

KP Ludwig John

Eyetracking

User Recall


Screenshots from kiehls.com (sells cosmetics)

HomeUser interest: • general orientation • search for categories

Experience with site: • none

Subpage / product pageUser interest: • gathering information about the specific product

Experience with site: • Familiar with navigation and categories …

No fixations to Menu doesn’t necessarily mean its bad designed or placed! Just no interest at the given context and moment.

analysis tools


http://kiehls.com


Eyetracking

KP Ludwig John

Eyetracking

Gaze Replays

analysis tools

Watch the test session video again

You see a dot, or user’s eye, moving around

if recorded, you hear ambient noise or users comments

Reliable Gaze Mapping during Screen Changes Scroll Down, Close Ad window


Eyetracking

KP Ludwig John

Eyetracking

Gaze Replay

source: IMS.UX Team „WeScan“ 2017

analysis tools

ExampleEvaluation of Print Layout

TaskFind out, how to get in contactwith the company

Comparisonof two test persons

!


Eyetracking

KP Ludwig John

Eyetracking

Gaze Replay

source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies PDF page 116f

Pros

Cons

• you can slow down the replaytherefor seeing everything the user looked at

• Order of Fixations is visible

• Relation of gaze and dynamic elements of the item(drop down menu etc.) is visible

• Rather time consuming to watch even at real speed(best results by watching at 1/3 speed)The eye moves so quickly you often do have to slow and replay segments to get an accurate idea of what happened.

analysis tools



Eyetracking

KP Ludwig John

Eyetracking

Gaze Plots

source: UX-Lab Test 2016 „Visibility of Posters in natural context“

results evaluation

show the location, order, and time spent looking at locations on the stimulus

primary function: to reveal the time sequence of looking or where we look and when we look there.

https://www.tobiipro.com/learn-and-support/learn/steps-in-an-eye-tracking-study/interpret/working-with-heat-maps-and-gaze-plots/

Time spent looking … is shown by the diameter of the fixation circles. The longer the look, the larger the circle.

https://www.tobiipro.com/learn-and-support/learn/steps-in-an-eye-tracking-study/interpret/working-with-heat-maps-and-gaze-plots/


Eyetracking

KP Ludwig John

Eyetracking

Gaze Plots


results evaluation

Eye gaze of one user on one page

Blue dots show:

• where the user’s fixation were

• numbers in the dots depict order of fixations

• size of the dot relates to duration of fixation

• time slider may be used todepict parts of the session(first 10 sec. for instance)


Eyetracking

KP Ludwig John

source: IMS.UX 2019 Team „Foreign and Confused“

Gaze Plots


Eyetracking

KP Ludwig John

Eyetracking


Problems• dots cover each other

• dots cover the area the user is interested inpossible to adjust transparency of the dots

• time consuming to analyze

Qualities• to distinguish between users

with different behaviorheatmap can be misleading in this case

analysis tools

!• difficult with dynamic elements drop down, pop up; elements that appear and disappear

source: IMS.UX 2019 Team „Foreign and Confused“

Gaze Plots



Eyetracking

KP Ludwig John

Eyetracking

Gaze Plots


• difficult with dynamic elements drop down, pop up; elements that appear and disappear

analysis tools

!



Eyetracking

KP Ludwig John

EyetrackingAreas of Interest

analysis toolshttps://www.tobiipro.com/learn-and-support/learn/steps-in-an-eye-tracking-study/data/digging-into-areas-of-interest/

https://www.tobiipro.com/learn-and-support/learn/steps-in-an-eye-tracking-study/data/digging-into-areas-of-interest/


Eyetracking

KP Ludwig John

Eyetracking

source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies pdf p. 138

• draw boxes around elements on pages • name them • have the system compare

how long and how often these items were looked at

analysis tools

What is it?Areas of Interest



Eyetracking

KP Ludwig John

Eyetracking analysis toolsAreas of Interest https://www.tobiipro.com/learn-and-support/learn/steps-in-an-eye-tracking-study/data/digging-into-areas-of-interest/



Eyetracking

KP Ludwig John

Eyetracking

What you getspreadsheet data displaying the total number of users who looked at each AOI and for how long.

difficult with dynamic elements (drop down, pop up; moving images)

analysis toolsAreas of Interest https://www.tobiipro.com/learn-and-support/learn/steps-in-an-eye-tracking-study/data/digging-into-areas-of-interest/



Eyetracking

KP Ludwig John

Eyetracking

Analyzing Animation

source: www.nngroup.com/reports/how-to-conduct-eyetracking-studies pdf p. 139

Possible:

Technical possible to create Heatmaps,but they show the static picture of one part of the video (instead of real video situation at each moment of tracking)

Scene changes make (automatic) mapping impossible

Areas of Interest in dynamic content

analysis tools

Static data visualization(Heatmap, gaze plot) with changing (dynamic) test items very difficult

on top of that:Websites (interactive applications) asynchronous user behavior whereas in Video scenes have predetermined duration



Eyetracking

KP Ludwig John

Eyetracking

source: https://www.youtube.com/watch?v=DExiqRSWSeM

Areas of Interest in dynamic content

analysis tools


Eyetracking

KP Ludwig John

Eyetracking

Areas of Interest in dynamic content / moving environmentInfrared ID Markers to define AOI in natural user environment

analysis tools


Eyetracking

KP Ludwig John


Infrared ID Markers to define AOI in natural user environmentAreas of Interest in dynamic content / moving environment

source: UX-Lab Test 2016 „Evaluation of machine interface in production process“


Eyetracking

KP Ludwig John


Areas of Interest in dynamic contentInfrared ID Markers to define AOI in natural user environment


Eyetracking

KP Ludwig John


Infrared ID Markers … would indicate the Posters of Interest … and therefor influence test situation

Areas of Interest in dynamic content / moving environment



Eyetracking

KP Ludwig John

Eyetracking

Solution: Gaze Replay combined with human coding and note taking and manual measurement (if needed) of observation time for each item of interest


Areas of Interest in dynamic content / moving environment

analysis tools


Eyetracking

KP Ludwig John


some moremetrics


Eyetracking

KP Ludwig John

Eyetracking

Time to first Fixation (TTFF)

source: iMotion Eye Tracking The complete Pocket guide pdf p. 15

more metrics

amount of time it takes a respondent to look at a specific AOI from stimulus onset.

Easy to compare and cumulate


Eyetracking

KP Ludwig John

Eyetracking

Time spent


Amount of time that respondents spent looking at an AOI

Prolonged visual attentionpoints at a high level of interestShorter times indicate that other areas on screen or environment are more attracting

more metrics


Eyetracking

KP Ludwig John

Eyetracking

Respondent Count


How many of respondentsactually guided their gazetowards a specific AOI

Higher account showsAOI is widely attended toLow counts shows that little attention is paid to it

more metrics


Eyetracking

KP Ludwig John

Eyetracking

Respondent Count


How many of respondentsactually guided their gazetowards a specific AOI

Higher account showsAOI is widely attended to

more metrics


Eyetracking

KP Ludwig John


Interpretation

Understand



Eyetracking

KP Ludwig John

Eyetracking

Eye motion is tightly linked to visual attention.

You just can‘t move your eyes without moving attention.


InterpretationSeeing / Perceiving

While eye tracking can tell us what people look at and what they see, it can’t tell us what people perceive

You can however certainly shift attention without moving your eyes.


Eyetracking

KP Ludwig John

Eyetracking InterpretationSeeing / Perceiving

You open the fridge in search for some milk.While it is right in front of you, you can‘t seem to find it. You keep looking until you close the door empty-handed.


Eyetracking

KP Ludwig John

EyetrackingSeeing / Perceiving

Even though you saw the container (and this is what eye tracking data would tell us), you were probably not paying sufficient attention to actually perceive that it was right in front of your eyes.


Eyetracking

KP Ludwig John

EyetrackingSeeing / Perceiving

During a search task one can easily fixate briefly on the search object and miss its presence, especially if the object has an unexpected shape or size …


Eyetracking

KP Ludwig John

Eyetracking

source: https://www.tobiipro.com/learn-and-support/learn/steps-in-an-eye-tracking-study/interpret/what-does-eye-tracking-data-tell-us/

Interpretation

Sometimes Fixations do not necessarily translate into a conscious cognitive process.

Follow up the eye tracking test with an interview to assess the participant’s motivations or expectations.

Seeing / Perceiving

This may happen because our expectation of what the object (or scene) should look like modulates our visual attention and interferes with the object detection.


Eyetracking

KP Ludwig John

Eyetracking

• which visual elements attract immediate attention • which visual elements attract above-average attention • if some visual elements are being ignored • in which order the visual elements are noticed • how the visual material compares to other material

Reveals


Interpretation

However, tracking gaze positions alone doesn‘t tell us anything particular about the cognitive processes and the emotional states that drive eye movements.

Combine Eyetracking with other data / insightsTherefor


Eyetracking

KP Ludwig John

iMotions Biometric Research Platform

Eyetracking

source: iMotion Eye Tracking The complete Pocket guide pdf p. 20ff

Combine Eyetracking with other data / insights

Interpretation


Eyetracking

KP Ludwig John

iMotions

Eyetracking

source: iMotion Eye Tracking The complete Pocket guide pdf p. 20ff

Physiological data generated with biometric sensors:

• facial expression analysis (webcam-based)a: human observationA confused expression while scanning the items in the fridge followed by a sad expression would have told us how you were feeling when you realized that the milk was missing.

• b: softwareWhile facial expressions can measure the presence of an emotion (valence), they can’t measure the intensity of that emotion (arousal) )

• Galvanic Skin Response (GSR, electrocardiography) (quantifies level of emotional arousal and stress levels based on the changes in skin conductance)

• EEG (electroencephalography) (Messung Hirnaktivität)measures the electrical activity of the brain at the scalpprovides information about the levels of engagement (arousal), motivation, frustration, cognitive workload.


Interpretation


Eyetracking

KP Ludwig John

Eyetracking

source: iMotion Eye Tracking The complete Pocket guide pdf p. 22/32

• EMG (Electromyographic sensors )to monitor muscular responses to any type of stimulus; associated with emotional expressions (facial EMG) or consciously controlled hand/finger movements.

• Electroencephalography (ECG, records heart rate (EKG)) & Photoplethysmography (PGP) optical technique to detect changes in blood circulationobtain insights into physical state, anxiety and stress levels (arousal)


Interpretation

iMotions Biometric Research Platform


Eyetracking

KP Ludwig John

Eyetracking Interpretation

Easier / Low cost but very helpful:

Post Test InterviewsIA6-2018 „LandsApp“


Eyetracking

KP Ludwig John

Eyetracking InterpretationPost Test Interviewscollecting personal feelings, thoughts and attitudes

Do it immediately after the Eyetracking test!people forget, make up stories

IA6-2018 „LandsApp“


Eyetracking

KP Ludwig John


Synthesize Findings

Understand


and communicate


Eyetracking

KP Ludwig John

Eyetracking synthesize findings

Results … Eyetracking Data One list per test

source: UX-Lab Test 2013 „Usability of MMIs in Cars“


Eyetracking

KP Ludwig John


source: UX-Lab Test 2013 „Usability of MMIs in Cars“

Results … same datatransformed into visual representation


Eyetracking

KP Ludwig John

Eyetracking

• Test goals • Criteria of success • Measurements

Synthesize your findings

• Test persons (number, representing target group, interested in topic)

• Equipment / Situation(accuracy of measurement, natural user situation)

• Criteria of success / Measurements(assumptions correct?)

• Test scenario / tasks / prototype(Did scenario and prototype meet the test goals? Have the tasks influenced the measured results in one way or the other?)

Based on your

Include a critical evaluation of your test conditions

synthesize findings


Eyetracking

KP Ludwig John

Eyetracking

Always take notes while testingpossibly think of backup recording (video) solution(system may crash, files not saved correctly, operating errors by test team)

Notes will also help to focus your analysis on specific situations and issues

Recommendations

synthesize findings


Eyetracking

KP Ludwig John

Eyetracking

Heatmaps is a good starting pointbut do not tell the whole story!

Use Gaze plots and replays to get a detailed understanding

Search for patterns in user behavior instead of pure aggregation (heatmap)

Recommendations


synthesize findings



Eyetracking

KP Ludwig John

PracticePrepare your report

• Test scenario • Critical evaluation of test conditions • Measurements and results • Interpretation / Findings • Integrate visuals and video

to communicate your results

Summarize your test results into one document:

Questions?


Eyetracking

KP Ludwig John

PracticeFor procedures to summarize your findings by Thinking aloud / Note taking:

Refer to the lecture Questionnaire Analysis section qualitative analysis


Eyetracking

KP Ludwig John


Eyetracking

KP Ludwig John

Practice

hs-augsburg.de/homes/john

Schedule and Scripts:

UX-Research 6b Results-Evaluation EyeTracking · 2019. 12. 9. · UX-Research Results Evaluation ....

Documents

Transcript of UX-Research 6b Results-Evaluation EyeTracking · 2019. 12. 9. · UX-Research Results Evaluation ....