1 Ceng 701 - Tracking Gökhan Tekkaya Gürkan Vural Can Eroğul METU, 2008.

11

Ceng 701 - TrackingCeng 701 - Tracking

Gökhan TekkayaGökhan Tekkaya

Gürkan VuralGürkan Vural

Can EroğulCan Eroğul

METU, 2008METU, 2008

22

OutlineOutline

– OverviewOverview– Tracking DevicesTracking Devices

ElectromagneticElectromagnetic

MechanicalMechanical

AcousticAcoustic

OpticalOptical

InertialInertial

CombinationsCombinations

– Eye TrackingEye Tracking– Finger/Hand TrackingFinger/Hand Tracking

33

Tracking - OverviewTracking - Overview

Purpose:Purpose:– The primary purpose of tracking is to The primary purpose of tracking is to

update the visual display based on the update the visual display based on the viewers head position and orientation.viewers head position and orientation.

– Instead of tracking the viewer's eyes Instead of tracking the viewer's eyes directly, we track the position and directly, we track the position and orientation of the user's head. From this orientation of the user's head. From this we determine the position and we determine the position and orientation of the two eyes. orientation of the two eyes.

44


We may also be tracking the user's We may also be tracking the user's hand(s), fingers, legs or other interface hand(s), fingers, legs or other interface devices.devices.

Want tracking to be as 'invisible' as Want tracking to be as 'invisible' as possible to the user.possible to the user.

Want the user to be able to move freely Want the user to be able to move freely with few encumberanceswith few encumberances

55


Want to be able to have multiple 'guests' Want to be able to have multiple 'guests' nearbynearby

Want to track as many objects as necessaryWant to track as many objects as necessary

Want to have minimal delay between Want to have minimal delay between movement of an object and the detection of movement of an object and the detection of the objects new position / orientation (< 50 the objects new position / orientation (< 50 msec total)msec total)

Want tracking to be accurate (1mm and 1 Want tracking to be accurate (1mm and 1 degree)degree)

66

OutlineOutline




AcousticAcoustic

OpticalOptical

InertialInertial



77

Tracking Devices - ElectromagneticTracking Devices - Electromagnetic

large transmitter and one or more small large transmitter and one or more small sensors.sensors.

transmitter emits an electromagnetic field.transmitter emits an electromagnetic field.

sensors report the strength of that field at sensors report the strength of that field at their location to a computertheir location to a computer

sensors can be sensors can be polledpolled specifically by the specifically by the computer or transmit computer or transmit continuouslycontinuously..

88


uses:uses:– projection based systems, HMDs projection based systems, HMDs

advantages are:advantages are:– large tracking volume (approx 10 feet / 3 meters) large tracking volume (approx 10 feet / 3 meters) – no line-of-sight restriction no line-of-sight restriction – sensors are small and light sensors are small and light – technology has been around for a while technology has been around for a while

disadvantages are:disadvantages are:– affected by metal in the nearby area affected by metal in the nearby area – latency can be high (0.1 seconds) latency can be high (0.1 seconds) – accuracy is low in large volumes accuracy is low in large volumes – somewhat expensive somewhat expensive

99


Polhemus:Polhemus:– Motion Tracking:Motion Tracking:

LIBERTY LATUSLIBERTY LATUS

LIBERTYLIBERTY

PATRIOTPATRIOT

FASTRAKFASTRAK

MINUTEMANMINUTEMAN

– Scanning:Scanning:FastSCANFastSCAN

– Eye Tracking:Eye Tracking:VisionTrakVisionTrak

1010


1111

Tracking Devices - MechanicalTracking Devices - Mechanical

rigid structures with rigid structures with multiple jointsmultiple jointsone end is fixed, the other one end is fixed, the other is the object being is the object being trackedtrackedcould be tracking users could be tracking users head, or their handhead, or their hand

1212


physically measure the rotation about joints in physically measure the rotation about joints in the armature to compute position and the armature to compute position and orientationorientation

structure is counter-weighted - movements structure is counter-weighted - movements are slow and smoothare slow and smooth

Knowing the length of each joint and the Knowing the length of each joint and the rotation at each joint, location and orientation rotation at each joint, location and orientation of the end point is easy to compute.of the end point is easy to compute.

1313


uses:uses:– BOOMs, Phantom BOOMs, Phantom

advantages are:advantages are:– low latency low latency – high accuracy high accuracy – sensors are small and light sensors are small and light – technology has been around for a while technology has been around for a while

disadvantages are:disadvantages are:– small volume small volume – only track one object at a time only track one object at a time

1414

Tracking Devices - AcousticTracking Devices - Acoustic

small transmitter and one medium sized small transmitter and one medium sized sensorsensoreach transmitter emits ultrasonic pulses each transmitter emits ultrasonic pulses which are received by microphones on the which are received by microphones on the sensor (usually arranged in a triangle)sensor (usually arranged in a triangle)as the pulses will reach the different as the pulses will reach the different microphones at slightly different times, the microphones at slightly different times, the position and orientation of the transmitter position and orientation of the transmitter can be determinedcan be determined

1515


Transmit acoustic pulseTransmit acoustic pulseListen for arrival (or echo)Listen for arrival (or echo)Inside-out or outside-inInside-out or outside-in– Outside-inOutside-in

Measure positionMeasure positionSound wave field, maybeSound wave field, maybeMultipath interferenceMultipath interference

– Inside-outInside-outMeasure position and orientationMeasure position and orientationMust wait for receipt of signalMust wait for receipt of signal

1616


advantages are:advantages are:– Small and lightweightSmall and lightweight– Not magneticNot magnetic– Relatively little infrastructureRelatively little infrastructuredisadvantages are:disadvantages are:– Line-of-sight requirementLine-of-sight requirement– Acoustic interference (e.g. jingle)Acoustic interference (e.g. jingle)– Ambient temperature concernsAmbient temperature concerns– PC wave ambiguityPC wave ambiguity– PC error accumulationPC error accumulation

1717


Logitech:Logitech:– 6 degrees of freedom6 degrees of freedom– 3D and 2D modes3D and 2D modes– Ultrasonic TechnologyUltrasonic Technology– High level of ruggednessHigh level of ruggedness– PC and Unix compatibilityPC and Unix compatibility– 250 dpi resolution/ 3D mode250 dpi resolution/ 3D mode– 400 dpi resolution/ 2D mode 400 dpi resolution/ 2D mode

1818

Tracking Devices - OpticalTracking Devices - Optical

LEDs or reflective LEDs or reflective materials are placed materials are placed on the object to be on the object to be trackedtracked

video cameras at video cameras at fixed locations fixed locations capture the scenecapture the scene

image processing image processing techniques are used techniques are used to locate the objectto locate the object

1919


Inside looking outInside looking out– beacons/landmarks are fixed in the environmentbeacons/landmarks are fixed in the environment– user-mounted sensorsuser-mounted sensors– accurate orientationaccurate orientation

Outside looking inOutside looking in– sensors are fixed in the environmentsensors are fixed in the environment– user-mounted beacons/landmarksuser-mounted beacons/landmarks– orientation can be a problem as distance to target growsorientation can be a problem as distance to target grows

Hybrid or cascadedHybrid or cascaded– head tracked inside-out from roomhead tracked inside-out from room– hands tracked outside in from headhands tracked outside in from head

2020


A. R. T. system from A. R. T. system from Germany:Germany:

2121

Tracking Devices - InertialTracking Devices - Inertial

Sense rate, integrate onceSense rate, integrate once

Sense acceleration, integrate twiceSense acceleration, integrate twice

2222


self-containedself-contained

gyroscopes / accelerometers usedgyroscopes / accelerometers used

knowing where the object was and its knowing where the object was and its change in position / orientation the device change in position / orientation the device and 'know' where it now isand 'know' where it now is

tend to work for limited periods of time tend to work for limited periods of time then drift.then drift.

2323


advantages are:advantages are:– Truly sourcelessTruly sourceless– Very very fastVery very fast– RobustRobust

disadvantages are:disadvantages are:– Bias, scale, and alignment errorsBias, scale, and alignment errors– Bias integration results in driftBias integration results in drift– gravity vector kills yougravity vector kills you

2424

Tracking Devices - CombinationsTracking Devices - Combinations

Intersense uses a combination on Acoustic and Intersense uses a combination on Acoustic and Inertial. Inertial can deal with fast movements Inertial. Inertial can deal with fast movements and acoustic keeps the inertial from driftingand acoustic keeps the inertial from drifting

2525

OutlineOutline




AcousticAcoustic

OpticalOptical

InertialInertial



2626

Eye TrackingEye Tracking

Eye tracking is the Eye tracking is the process of process of measuring either measuring either the point of gaze the point of gaze "where we are "where we are looking" or the looking" or the motion of an eye motion of an eye relative to the head.relative to the head.

2727

Eye TrackerEye Tracker

Eye tracker is a device for measuring eye Eye tracker is a device for measuring eye positions and eye movementspositions and eye movements

Eye movements are typically divided into Eye movements are typically divided into two episodes:two episodes:– fixation: the eye gaze pauses in a certain fixation: the eye gaze pauses in a certain

positionposition– saccades: the eye moves to another positionsaccades: the eye moves to another position

The resulting series of fixations and The resulting series of fixations and saccades is called a scanpathsaccades is called a scanpath

2828

Scanpath of EyeScanpath of Eye

An example of fixations An example of fixations and saccades over textand saccades over text

The typical pattern of eye The typical pattern of eye movements during movements during readingreading

Never move smoothly Never move smoothly over still textover still text

Most information is made Most information is made available during a fixationavailable during a fixation

2929

Application AreasApplication Areasof Eye Trackersof Eye Trackers

Psychology (notably psycholinguistics, the Psychology (notably psycholinguistics, the visual world paradigm)visual world paradigm)

Cognitive scienceCognitive science

Marketing researchMarketing research

Medical research (neurological diagnosis)Medical research (neurological diagnosis)

Human-computer interactionHuman-computer interaction

Vehicle SimulatorsVehicle Simulators

Communication systems for disabledCommunication systems for disabled

......

3030

Types of Eye Trackers - ITypes of Eye Trackers - I

Optical - Using video Optical - Using video imagesimages– the eye position is the eye position is

extracted from video extracted from video imagesimages

– a camera focuses on a camera focuses on one or both eyes and one or both eyes and records their records their movement as the movement as the viewer looks at some viewer looks at some kind of stimuluskind of stimulus

3131

Optical Eye Trackers - IOptical Eye Trackers - I

modern eye-modern eye-trackers usetrackers use– contrast to locate contrast to locate

the center of the the center of the pupilpupil

– infrared and near-infrared and near-infrared non-infrared non-collimated light to collimated light to create a corneal create a corneal reflection (CR).reflection (CR).

3232

Optical Eye Trackers - IIOptical Eye Trackers - II

two general techniques based on the two general techniques based on the location of the illumination sourcelocation of the illumination source– Bright PupilBright Pupil

creates greater iris/pupil contrast allowing for more creates greater iris/pupil contrast allowing for more robust eye tracking with all iris pigmentationrobust eye tracking with all iris pigmentation

allows for tracking in lighting conditions ranging from allows for tracking in lighting conditions ranging from total darkness to very brighttotal darkness to very bright

– Dark PupilDark Pupilnot effective for tracking outdoors as extraneous IR not effective for tracking outdoors as extraneous IR sources interfere with monitoringsources interfere with monitoring

3333

Optical Eye Trackers - IIIOptical Eye Trackers - III

use a sampling rate of at least 30 Hzuse a sampling rate of at least 30 Hz

although 50/60 Hz is most commonalthough 50/60 Hz is most common

modern video-based eye trackers run at modern video-based eye trackers run at 240, 350 or even 1000/1250 Hz240, 350 or even 1000/1250 Hz– needed in order to capture the detail of the very needed in order to capture the detail of the very

rapid eye movements during reading, or during rapid eye movements during reading, or during studies of neurologystudies of neurology

3434

Types of Eye Trackers - IITypes of Eye Trackers - II

Search coilsSearch coils– known as magnetic known as magnetic

eye trackerseye trackers– uses coils that are uses coils that are

embedded into a embedded into a tightly-fitting contact tightly-fitting contact lens or a rubber ring lens or a rubber ring that adheres to eyethat adheres to eye

3535

Magnetic Eye TrackersMagnetic Eye Trackers

Alternating magnetic field with horizontal Alternating magnetic field with horizontal and vertical components is used to induce and vertical components is used to induce electric currents in these coilselectric currents in these coils

These currents depend on the eye position These currents depend on the eye position – on the angle by which the eye is rotatedon the angle by which the eye is rotated

– thus provide a measure of the eye positionthus provide a measure of the eye position

3636

Types of Eye Trackers - IIITypes of Eye Trackers - III

ElectrooculogramElectrooculogram– pairs of electrodes pairs of electrodes

are placed either are placed either above and below the above and below the eye or to the left and eye or to the left and right of the eyeright of the eye

– a potential difference a potential difference occurs between the occurs between the electrodeselectrodes

3737

ElectrooculogramElectrooculogram

If the eye is moved from the center position If the eye is moved from the center position towards one electrode, this electrode "sees" towards one electrode, this electrode "sees" the positive side of the retina and the opposite the positive side of the retina and the opposite electrode "sees" the negative side of the retinaelectrode "sees" the negative side of the retina

A potential difference occurs between the A potential difference occurs between the electrodeselectrodes

Assuming that the resting potential is constant, Assuming that the resting potential is constant, the recorded potential is a measure for the eye the recorded potential is a measure for the eye positionposition

3838

Setups of Eye TrackersSetups of Eye Trackers

Setups vary greatlySetups vary greatly– some are head-some are head-

mountedmounted– some require the head some require the head

to be stable (for to be stable (for example, with a chin example, with a chin rest)rest)

– some function some function remotely and remotely and automatically track the automatically track the head during motionhead during motion

3939

Sample Devices - eyebox2Sample Devices - eyebox2

attached to store attached to store displays or billboards displays or billboards to count how many to count how many times viewed by times viewed by passersbypassersby

uses video imagesuses video images

infrared leds create infrared leds create red eyesred eyes

from up to 10 metersfrom up to 10 meters

price $1500price $1500

4040

Sample Devices - Lexus LS460Sample Devices - Lexus LS460

the vehicle with the the vehicle with the capacity to assess in capacity to assess in real-time the visual real-time the visual behavior of the driverbehavior of the driver

automobiles with the automobiles with the ability to monitor ability to monitor drowsiness, drowsiness, inattention, and inattention, and cognitive engagement cognitive engagement

4141

OutlineOutline




AcousticAcoustic

OpticalOptical

InertialInertial



1 Ceng 701 - Tracking Gökhan Tekkaya Gürkan Vural Can Eroğul METU, 2008.

Documents

Transcript of 1 Ceng 701 - Tracking Gökhan Tekkaya Gürkan Vural Can Eroğul METU, 2008.