Whitepaper Eye Tracking - Virtual Dimension Center · 2018-09-24 ·...

Whitepaper

Eye Tracking

© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)

Prof. Dr.-Ing. Dipl.-Kfm. Christoph Runde, Franziska JöckelVirtual Dimension Center (VDC) FellbachAuberlenstr. 1370736 Fellbachwww.vdc-fellbach.de

Eye Tracking

Market Analysis MissionMeasurement Methods

Application FieldsConstruction Types SummaryIntroduction

Structure

1. Introduction

2. Market analysis

3. Construction types

4. Measurement methods

5. Application fields

24.09.2018 2


6. Summary



Structure

1. Introduction

2. Market analysis




Introduction

24.09.2018 3


6. Summary



� Eye tracking refers to the visual movements of a person, consisting mainly of fixations (points that are focused), saccades (fast eye movements) and regressions.

� Eye tracking systems consist of

(1) recording devices/ techniques

Introduction to the field of technology

Introduction

24.09.2018 4

(2) the algorithms for processing the raw measurement data and

(3) the utilization in the form of representation methods of the processed information.



� The purpose of eye tracking is usually to determine where a person is looking

� In order to calculate the intersection of the human visual axis with relevant objects, the following three parameters must be recorded:

o Viewing direction

Eye position in the spatial coordinate system


Introduction

24.09.2018 5

o Eye position in the spatial coordinate system

o Coordinates of all relevant objects in the spatial coordinate system

� Eye tracking technologies deal with capturing either the first or the first two types of information

� If the viewing objects are on a screen, tracking systems also involve the third type of information; otherwise it requires a 3D space model



Anatomy of the human eye with several basic terms


Introduction

24.09.2018 6

Anatomy of the

human eye

Image: anatomyclass01.us



Structure

1. Introduction

2. Market analysis




Market Analysis

24.09.2018 7


6. Summary



Market figures, market development

� Dynamically growing market

� Market volume 2018: approximately $ 400 million

� By 2023, the market volume will reach $ 1.43 billion

Market Analysis

24.09.2018 8

� Assistive communication applications had the largest eye-tracking market share in 2016

� North America as the largest regional market

� External systems as the most important form of eye tracking systems

Image: Eye Tracking Market, 2018

Global market development eye

tracking 2016-2023



Market participants

� Among the most important eye tracking companies rank:

o Tobii AB,

o Eyetracking Inc.,

o Seeing machines Ltd.,

o Smarteye AB,

o LC Technologies Inc.,

o Eyetech Digital Systems Inc.,

o Ergoneers GmbH,

o PRS IN VIVO,

Market Analysis

24.09.2018 9

o Sensomotoric Instruments (SMI) GmbH [now part of Apple],

� Detailed overview of companies with assignedtechnologies and application markets on the next slide

o PRS IN VIVO,

o SR Research Ltd.



Market participants

Zu den wichtigsten Eye-Tracking Unternehmen zählen:

� Tobii AB,

� Eyetracking Inc.,

� Seeing machines Ltd.,

� Smarteye AB,

� Sensomotoric Instruments (SMI) GmbH

� LC Technologies Inc.,

� Eyetech Digital Systems Inc.,

� Ergoneers GmbH,

� PRS IN VIVO,

SR Research Ltd.

��

�� !��! �"��

#� �� !� �� "��

#�� "

��#��" �� !��#� ��$��%��

#��"��& #�'( � #� ��!�� "� #� ��)�

%�� #�'( � *

Market Analysis

24.09.2018 10

� Sensomotoric Instruments (SMI) GmbH [nun Teil von Apple],

Weitere:

� Eye Sight

� Sightcorp

� Cogisen

� Irisbond

� Tobii Pro

� Pupil labs

� SR Research Ltd.

� GazeHawk

� Eyezag

� Eyeware

� Lumen research

� Arrington research

� Gazepoint

�� #�'( � ��! �"��

�� +�� #, +��"�� - �� "��.� ��#&��

�� #�'( � �� &�� /� .� ��" ��#&��

+ &�� 0 �� #�'( � �� "��!��

� 1�$ (� #�'( � .� ��

��1 � �� "��%�� "��.� ��

#�� "�� !�� ''��

��( � #�'( � �� "�� %��

#�� " �� "��"�� ! ��

+�#��-��2 #�'( ��#�!�� "� + ��" �� " !��

��" �� ! �"�� !�� "��#� ��

0 �� " ��

�� " �� *

� 1&�� 3�� "��)!��&��$ � ��%��& ��

T

a

b

e

l

l

e

:

N

e

u

m

a

n

n

,

V

.

,

2

0

1

8



Structure

1. Introduction

2. Market analysis


3.1 Tower-Mounted Eye Tracking

3.2 Remote Eye Tracking

Construction Types

24.09.2018 11

3.2 Remote Eye Tracking

3.3 Mobile systems



6. Summary



Tower-Mounted Eye Tracking

� Head of the test person is fixed in the device

� Three techniques ca be distinguished:

o Pan tilt systems: mechanically movable components enable the camera (optics) to track the test person´s head movements

o Tilting mirror systems: camera and optics remain fixed Image: Fa. SR Research

EyeLink 1000

tower mounted

eye tracker

Construction Types

24.09.2018 12

o Tilting mirror systems: camera and optics remain fixed in space, servo-driven mirrors allow tracking of the eye during head movements

o Fixed camera systems: these systems do without any mechanically moving components and achieve freedom of movement by means of image processing methods

� Advantage: Delivers more accurate results in higher image frequency, disadvantage: intrusive, immobile



Remote Eye Tracking

� Enables non-contact measurements, eliminating the need for mechanical components such as transmission cables or chin restraints

� The test person can move freely within a certain range of motion after successful calibration

� Compensation of head movements: a person can fix

Remote

Eyetracker

(integrated in

screen)

Tobii

Image: Tobii

Construction Types

24.09.2018 13

� Compensation of head movements: a person can fix a place and perform head movements without losing the fixation location

� To register eye movements on a computer screen, the components of the device can either be installed directly in a screen or mounted below or next to it

� Advantage: not intrusive, disadvantage: immobile



Mobile systems

� Eye tracking solutions are installed in glasses frames or similar and are connected to a portable computer

� One camera system is pointed on the environment (world cam), one on the pupil (eye cam)

Mobile eye

tracker

Eye squere

Image: eve spare

Construction Types

24.09.2018 14

cam)

� An algorithm projects one into the other

� These systems make it possible to largely observe people in natural interaction with products and media

� Possible applications outside the laboratory

� Advantage: mobile, disadvantage: intrusive



Structure

1. Introduction

2. Market analysis



4.1 Eye attached tracking

Measurement Methods

24.09.2018 15

4.1 Eye attached tracking

4.2 Electric potential measurement

4.3 Optical methods

� Oculography

� Infrared or LASER reflections

� 3D Tracking


6. Summary



Eye attached tracking

The position and orientation of an object connected with the eye (e.g. a specific contact lens) can be measured:� Reflection of mirrored contact lenses is recorded by a

camera� Electromagnetic tracking by means of a search coil:

Mirror

reflection

systems

Image: Holqvist, K; Andersson 2017

Measurement Methods

24.09.2018 16

� Electromagnetic tracking by means of a search coil: contact lenses provided with coils are exposed to a magnetic field � eye movement can be calculated from induced tension

� Photoelectric tracking: photoelectric sensors conduct electricity as a function of the incident light and thus of the eye movement

� Video-based tracking of artificial markers (black dots on the sclera)

Search

Coil

Photo-

electric

limbus

tracker

Image: Duchowski, A. T., 2017

Image: Gilroy, H., 2018



Electric potential measurement

� Measuring the electrical potential by positioning electrodes around the eye

� Electrooculograms (electro-oculography - EOG) measure the electrical voltage between the retina (negative pole) and the cornea (positive pole)

� Imec, a research center for nanoelectronics and

Example of an EOG

system


Measurement Methods

24.09.2018 17

� Imec, a research center for nanoelectronics and digital technologies, has introduced a fast, wireless eye tracking system (ITF Belgium 2018) based on EOG

o Five dry contact electrodes on glasses

o Ergonomic design of everyday glasses

o Sampling rates of 256 per secondImage: imec

Fast, wireless eye

tracking

ergonomically

designed



Optical detection - Oculography

1. Oculography� Optical detection of eye properties during rotation and translation

� Measurands of oculographyo Pupil shapeo Eyelid angle and cornea curvature

Example pupil

forms


Measurement Methods

24.09.2018 18

o Eyelid angle and cornea curvatureo Position of the limbuso Retinao Position face and eyes

� Two fundamentally different methods can be distinguished regarding the evaluation of these measured values:o Regression analysis (appearance-based approach)o Parametric-geometric fitting

Retina

characteristics

around the visual

nerve

Example of a

bright pupil

Image: N.N., 2018





� Regression analysis (appearance-based approach): The captured 2D image is compared to and merged with previously acquired images that have already been logically assigned to a meaning

� Parametric geometric fitting – today used for eyes and face

Camera images of

eyes for evaluation

Image: Eyeware

Measurement Methods

24.09.2018

� Parametric geometric fitting – today used for eyes and faceo Eyes: geometric image recognition, in which image features are mathematically detected � fitting of a known geometric primitive

o Face: Face-model-based approach determines facial features from the 2D image

The feature-based

evaluation„Iris-

Fitting“

Optical determination

of facial characteristics

from a 2D image

19

Image: Eyeware

Image: tobii




Advantages of oculography:

+ Mostly non-intrusive procedure (exception: retina tracking)

+ Falling prices of digital camera systems

+ Retina tracking is very accurate

Disadvantages of oculography:

Measurement Methods

24.09.2018 20

Disadvantages of oculography:

- Retina tracking is intrusive

- Regression analysis, parametric-geometric fitting and face-model-based approach are less accurate/ reliable

- Dependency on lighting conditions

- If only one camera is used, 3D spatial information is missing; oculography is unable to determine the focal point

- Stereoscopic camera systems limit the working space



Optical detection – Infrared or LASER reflection

2. Infrared or LASER (light amplification by stimulated emission of radiation) reflection

� For the absolute direction of view, several eye features must be measured, so that the head movement can be distinguished from the eye rotation

� Measuring principles of infrared or LASER reflectionThe four Purkinje-

Measurement Methods

24.09.2018 21

� Measuring principles of infrared or LASER reflectiono Pupil center-cornea-reflection (PCCR): measurement of

corneal reflection (also called Purkinje reflections/ image) relative to the pupil center

o Dual Purkinje Imaging (DPI) eye trackers: also measure the fourth Purkinje image � differentiate between translational and rotational eye movement; DPI-Eye trackers are very precise, but the viewer's head has to be fixed

o The regression analysis or the parametric-geometric fitting can be used to determine the pupil center

The four Purkinje-

reflections

The user of this DPI

eye tracker fixes her

head by biting in a

fixed dental imprint





Optical detection – Infrared or LASER reflection

Advantages of infrared/ LASER reflection:

+ PCCR systems are not intrusive (exception: mobile PCCR glasses)

+ If multiple cameras are used, the estimation of the head position in absolute 3D space is also possible

+ If (non-visible) infrared light is used, the entire setup is more independent of the

Measurement Methods

24.09.2018 22

+ If (non-visible) infrared light is used, the entire setup is more independent of the given lighting situation

Disadvantages of infrared/ LASER reflection:

- DPI systems are intrusive

- Natural sunlight interferes with the use of outdoor PCCR and DPI trackers

- Limited range (about 1m)

- If several cameras are used to generate 3D information, the working space is additionally reduced



Optical recording – 3D Tracking

3. 3D Tracking

Measurement of the absolute head position in 3D space

� Predominant technological approaches when using separate tracking systems:o Inertial measuring systems: determine position and angle

changes by integrating the acquired accelerations. The

Shutter glasses

(Volfoni ) which

enables stereoscopic

vision, combined

with an eye tracker

system of SMI and

an external optical

tracking system of

A.R.T.

Image: itersnews, 2018

Measurement Methods

24.09.2018 23

changes by integrating the acquired accelerations. The incorporation of additional sensors (for example magnetic field sensors) gives the designation nine-degree of inertial measuring system (9DOF-IMU). 9DOF IMUs are cost effective, enable fast measurements, but do not handle absolute tracking.

o Optical tracking: Infrared cameras detect active (IR LEDs) or passive (IR reflectors) IR light spots arranged in a specific spatial structure. The computer can calculate the position and orientation of the observed object from the pattern of the IR light spots, which is detected by at least two cameras.

Head Mounted

Display (HMD)

"Vive" with

assigned tracking

cameras (black

cubes) and input

devices/ controller

Image: HTC




� Using 3D depth cameras

oCamera systems that - unlike normal analog or digital cameras that display grayscale or color values of objects - allow the representation of distances of an entire scene.

o 3D camera and outside-in-tracking: regarding the

Microsoft Kinect

system with

explanation of the

integrated sensors

Image: Microsoft

Measurement Methods

24.09.2018 24

o 3D camera and outside-in-tracking: regarding the outside-in-tracking, the tracker (here: the 3D camera) is located in the reference, i.e. in the fixed spatial environment, and "looks" at the moving, tracked object (such as a flystick or the user's head)

Image: Microsoft




Measurement of head geometry

3D cameras can generate a 3D image of their environment. Eyeware uses this approach to create a user-specific 3D model of the head. Thereby the precise head geometry is known.

Eyeware 3D head

Measurement Methods

24.09.2018 25

Measurement of the position of the eyes

Using a laser, an eye tracker scans circularly over the iris, which reproduces the torsion rotation of the iris by comparing the striation on successive recordings. These trackers can thus determine the 3D position of the eye in the head coordinate system. Commercial tracking systems currently do not use this technology.

Eyeware 3D head

model

Image: Eyeware




Advantages of 3D Tracking

+ Separate tracking systems can achieve long ranges (10-20m) and, when connected in parallel, a large working space (e.g. 10m x 50m)

+ 3D cameras are inexpensive

+ 3D cameras are not intrusive

+ 3D cameras provide 3D head position and orientation information

Measurement Methods

24.09.2018 26

+ 3D cameras provide 3D head position and orientation information

+ 3D cameras can create a 3D image of the head � more precise localization of the eyes

Disadvantages of 3D Tracking

− The LASER detection according to Irie is intrusive and costly

− Separate tracking systems are usually costly. 9DOF IMUs are inexpensive, but can not be used as a stand-alone system, as they do not allow absolute tracking.

− Separate tracking systems are almost always intrusive. Markerless motion capturing tracks primarily the trunk, arms, hands, legs and feet. The accuracy of the detection of the head would be insufficient to derive a viewing direction.

.



Structure

1. Introduction

2. Market analysis




Application Fields

24.09.2018 27


� Usability

� Market research

� Film

� Psychology

� Ergonomics

6. Summary

� Medicine

� Educational sciences

� Security

� Virtual Reality



Usability

� Used by User Experience (UX) and Human-Computer Interaction (HCI) researchers

� Exploring usability in human-computer interactions �Explaining human behavior based on objective data

� Use in the field of software ergonomics: usability tests

Examination of

the usability of a

railway station

display

Image: Kuerzel, E., 2018

Application Fields

24.09.2018 28

� Use in the field of software ergonomics: usability tests analyze the perception of test persons during the execution of tasks with software or on a website

� Use in the ergonomic, UX and HCI-compliant design of all machine operations (e.g. kiosk systems) and cockpit systems of any vehicle (e.g. cars)



Market research

� A method to objectively measure the attention of consumers and their spontaneous reaction to advertising messages � effective design of communication

� Research aspects include the design of sales packaging, retail stores, point-of-sale promotions,

Market research

with eye tracking:

supermarket test

Image: Kuerzel, E., 2018

Application Fields

24.09.2018 29

packaging, retail stores, point-of-sale promotions, websites, newspapers and magazines and product design



Film

� Motion control of the extremities (also gestures) of virtual avatars in animated films by actors using Motion Capturing

� The technique is also used for capturing facial expressions

� Meanwhile, the capturing repertoire includes eye

Pascal Langdale as Caleb Smith in Faster

than Night photo by Vanessa Shaver, 3D

model by Dionisios Mousses/SIRT Centre

Image: Gilroy, H., 2018

Application Fields

24.09.2018 30

� Meanwhile, the capturing repertoire includes eye tracking



Psychology

� Applications in psychology and neuroscience to understand how and why eye movements arise and how we visually capture information

� Examination aspects are image and movement perception

Eye tracking/

cognitive

neuroscience

Image: Crane, H.D., 1978

Application Fields

24.09.2018 31



Ergonomics

� Eye tracking is used in ergonomics in order to evaluate performance, compare novices with experts, develop effective training protocols and cognitive strategies

� The same applies to the ergonomic, intuitive and efficient design of work systems in terms of

Image: Usability.gov

Heat Maps

represent where

and how long

the attention is

directed


Application Fields

24.09.2018 32

efficient design of work systems in terms of usability

Image: TU

Chemnitz

Image: TU

Chemnitz

Eye tracking

examinations in

a usability lab




Medicine

� Eye lasers: eye tracking measures the movements of the eye and adjusts the laser. Possibly the laser will be switched off until the eye has regained the desired position.

� Increasing importance in psychiatry (more reliable and earlier diagnoses, such as autism, ADHD)

High speed

tracking

Image: Irie, K., 2002

Application Fields

24.09.2018 33

and earlier diagnoses, such as autism, ADHD)

� In competitive sports, there is great interest in finding out how to train sport-specific eye movement strategies

� Optimization/ support of the work of a doctor: checking the concentration (on the right place), control of functions with the eyes

Image: Irie, K., 2002



Educational sciences

� Eye tracking is used in educational science to study learning processes

� In addition, classrooms and laboratories are equipped with this technology in order to educate the next generation in the use of eye tracking in a wide variety of areas

Tobii Pro eye

tracking

reveals

student

attention in

classroom

Image:

tobiipro.com

Application Fields

24.09.2018 34

wide variety of areas



Security

� Driving assistance systems

o By using eye tracking technologies, the driver's condition can be monitored and supported based on the opening degree of the eye, the head posture, including movements, and the rate of blinks

Application Fields

24.09.2018 35

o Thereby microsleep, inattention and loss of control can be prevented

� Smart home, device and data security, publicevents � capturing persons, behavioral patterns



Virtual Reality

� HMD integrated eye tracking was immediately an issue with the advent of low-cost HMDs

� Autostereoscopic display systems based on parallax barrier technology are aligned with the viewer's position relative to the display. Therefore the position of the viewer´s eyes must be known.

Arrington

Research

Eyeframe

Image: mellottsvrpage.com

Application Fields

24.09.2018 36

the position of the viewer´s eyes must be known.

� Development of the technology "FoveatedRendering": The main burden of the GPU is to be concentrated on the areas of a 3D scene that the user focuses on; whereas marginal areas should be allocated less GPU capacity � prerequisite: eye tracking

� Main areas of application: usability, design studies, market research

SMI for

Oculus Rift at

Carrefour

FOVE:

HMD with

integrated eye

tracking

Image: FOVE

Image: FH Kufstein



Structure

1. Introduction

2. Market analysis




Summary

24.09.2018 37


6. Summary



Summary

Application fields

• Usability

• Market research

Eye tracking-Measurement

methods

• Eye attached tracking

• Electric potential

Eye tracking-Construction types

• Tower-Mounted Eye Tracking

Summary

24.09.2018 38

• Market research

• Film

• Psychology

• Ergonomics

• Medicine

• Educational sciences

• Security

• Virtual Reality

• Electric potential measurement

• Optical methods

• Oculography

• Infrared or LASER reflection

• 3D tracking

Tracking

• Remote Eye Tracking

• Mobile systems



Summary

� There is a variety of different types and methods of eye trackers today. These are very different in terms of usage context, measurement, precision and price.

� The contact lens method and the electrical potential measurement no longer have any practical significance on the market today. DPI trackers are of very close medical importance.

� Today, eye tracking is done using optical techniques: oculography and pupil

Summary

24.09.2018 39

� Today, eye tracking is done using optical techniques: oculography and pupil center-cornea-reflection (PCCR) are the dominating procedures. The use of 3D cameras is not yet a product on the market.

� For HCI examinations, external systems are often used because they are not intrusive and the workspace with the interaction space in front of the machine interface is restricted anyway.

� High-precision, space-stable installation systems, such as for retina detection, are used primarily in medical applications.

� There is not the "best" eye tracking system (dependency on the application).



Sources

� Bergstrom, Jennifer Romano; Schal, Andrew:

Eye Tracking in User Experience Design.

Morgan Kaufmann, 2014, ISBN 978-0-12-

416709-4.

� Crane, H.D.; Stelle, C.M.: Generation-V dual-

Purkinje-image eyetracker. Applied Optics,

17(5), 1978, S. 527-537

� Duchowski, Andrew T.: Eye Tracking

Methodology. Theory and Practice. Third

Edition. Spring-er-Verlag, London, 2017

Online unter: http://itersnews.com/?p=24045;

abgerufen am 31.3.2018

� Kuerzel, Eric: How Eye Tracking is Driving the

Next Generation of AR and VR; online unter:

https://vrscout.com/news/eye-tracking-driving-

next-generation-ar-vr/; abgerufen am

31.3.2018

� Lobe, Adrian: Schau mir in die Augen, Kunde.

In: TagesWoche am 31.01.2015; online unter

https://tageswoche.ch/allgemein/schau-mir-in-

die-augen-kunde/; abgerufen am 31.3.2018

� Irie, K.; et al.: LASER-based eye-tracking

system. Behavior Research Methods, 34(4),

2002, S. 561-572

24.09.2018 40

� Eye Tracking Market 2018 Leading Growth

Drivers, Emerging Audience, Segments, Sales,

Prof-its & Analysis [aus:

http://www.crossroadstoday.com/story/37763

460/eye-tracking-market-2018-leading-

growth-drivers-emerging-audience-segments-

sales-profits-analysis]

� Gilroy, H.: Facial Capture. Online unter:

https://praxistheatre.com/author/fasterthannig

ht/ page/2/; abgerufen am 31.3.2018

� Holmqvist, K.; Andersson, R.: Eye Tracking. A

comprehensive guide to methods, paradigms,

and measures. Second edition. Lund Eye

Tracking Research Institute AB, Lund, 2017

� itersnews: Displays. 3D eye tracking smart

glass gives immersive 3D gaming experience.

die-augen-kunde/; abgerufen am 31.3.2018

� abgerufen am 31.3.2018

� tobii: Was ist Ihr Anwendungsfeld?; online

unter:

https://www.tobiipro.com/de/anwendungsfeld

er/ ; abgerufen am 31.3.2018

� tobii: Year-End Report January – December

2017; online unter:

https://www.tobii.com/group/

investors/calendar/year-end-report-q4-2017/;

abgerufen am 31.3.2018

� Nielsen, Jakob; Pernice, Kara: Eyetracking

Web Usability. New Riders Publishing,

Thousand Oaks, CA, USA 2009, ISBN 978-0-

321-49836-6.

VDC and Visual Sweden members regarding this topic

24.09.2018 41

Thank you very much for your interest! You are interested in this topic and you are looking for contact persons/ implementation partners? Please contact us.

© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)

Virtual Dimension Center (VDC) FellbachAuberlenstraße 1370736 Fellbachwww.vdc-fellbach.de

Whitepaper Eye Tracking - Virtual Dimension Center · 2018-09-24 ·...

Documents

Transcript of Whitepaper Eye Tracking - Virtual Dimension Center · 2018-09-24 ·...