Eye tracking presentation - UTKweb.eecs.utk.edu/~arahimpo/eyetracking.pdf · Eye tracking system...
Transcript of Eye tracking presentation - UTKweb.eecs.utk.edu/~arahimpo/eyetracking.pdf · Eye tracking system...
Real-timeEyeGazeTracking
Alireza RahimpourFall2010
1
Eyetrackingsystem:Nowadayseyegazetrackinghaswiderangeofapplicationsinhumancomputerinteraction.Oneoftheseapplicationsisusingtrajectoryofeyegazeinsteadoffootorhandfordisabledpeopletoexecutesomecommands.
OtherApplications:ü Automaticmonitoringofdriversforaccidentavoidanceü Commercialusesü Psychologicaluses
2
DifferenttypesofEyetrackers
1-UsingEOGsignal:• Systemsbasedonelectrooculographyuseelectrodesplacedonthe
facetodetectthemovementsoftheeyesIntrusive
2-Usingactiveinfraredilluminationv Difficultcalibrationv Needhighresolutionandexpensivecamerasv Interference:ExternalsourcesofIR:Sun
3
Other types:
4
GOAL
Designinganeyetrackingsystemashumancomputerinterfacewhichavoids:
ü specializedhardwareü infraredlightsourcesü expensivecameras
andü Increasestheaccuracyandspeedofthesystemtobeusableinreal-
timeapplications.
5
Eyetrackingsystem
Recently,variousmethodshavebeenproposed,someofthesemethodscansuccessfullytracktheeyegaze.However,theyalwaysrequirespecificcircumstances,trainingorarenotcapableofreal-timeperformance.
Wewanttotracktheeyegazeinreal-timebyusingasimpleandlowcostwebcammountedonordinarylaptops.
Itshouldbeableto:controlthemotionsofmousecursorandclickonanonscreenkeyboardinrealtime.
6
Implementation
Severalmethodscanbeusedforfacialandeyefeatureextraction.
Oneofthemis:Templatematching
TemplateofFeature• Theregionofimagewithmaximumsimilaritywiththetemplate,
referstopositionofourfeatureinimage.
7
Templatematching• SimilaritybetweenimageandTemplate??• Minimizingtheerror:
• Parametrictemplates
ü Correlationfunction8
|),(),(|),(1
0
1
0∑∑−
=
−
=
++−≡B
j
B
it jviuIjiTvuSAD
Correlationfunctionasameasureofsimilarity
9
∁ 𝑢, 𝑣 =
Fortemplatematching,thetemplatetslidesoverf(image)andCiscalculatedforeachcoordinate.Then,thepointwhichexhibitsmaximumC isreferredtoasthematchpoint.
Templatespecifications
• Size
• Thesize ofthetemplateisnottheonlyissue,butmoreimportantly,trackingperformancedependsonthe“complexity”ofthetemplate.
• Complexity:haveenough brightnessvariations,e.g.,textureorlines,toberecognizedas distinctfeatures
10
Templates
• Templateswithminsizewhichcontainenoughdetailsarechosen.
• Facetemplate:• 106x74pix
• Eyetemplate:• 15x25Pix
• Pupiltemplate:
• Eyecorners:
11
Reducingthesearchwindowsize
ü Reducethesearchareaineachstepbycropping.
ü Usingstructureoffacetodeterminethesearchregionforeachfeature.
12
240X320
106X75
45X36
15X25
Accuracy• Usingweightedcorrelationcoefficientcanincreasetheaccuracy.
• Consideringseveralframesofimages,themaximumindexincorrelationcoefficientmatrixcanbedetermined.
• Forthenextframes,weconsidermoreweightforthewinnerindex(andafewneighbors).
13
Trackingsystem
14
Thesystemconsistsoftwomainmodules:
1)thefacedetectorandtracker
2)theeyeanalysismodule
Theoutputfromtheeyemodulecanbe theinputtoasimplecomputercontrolinterfacebysimulatingakeypress.
Also:Differentmodesaredefinedcorrespondingtoeyeclosuretime.
EYEGAZEESTIMATION
15
• Horizontaldirection1-usingscleraregionchanges2-usingcenterofpupil
• VerticalDirection1-usingeyecorners2-usinguppereyelids
Eyegazeinhorizontaldirectionusingscleraregion
16
Dividingthemonitorinhorizontaldirectionto16pointsanddefinealphaasfollow:
( )( ) ( )RALA
LA+
=α
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
02468101214161820222426283032
alpha
position
ScleraRegionextraction
17
Croppedeyeimage:
Contrastenhancement:
Converttobinaryimageusingathreshold:
Morphological operations (opening – closing):
Makingcontoursandcalculatethenumberofpixelsinthemandfinding thecoordinates.
Eyegazedetection
18
• Horizontaldirection2-Usingpupilcentercoordinatewithrespecttoeyecorners
• Verticaldirection:1-eyelids:
2-eyecorners
Experimentandresults
19
ü Framesize240x320pix-30framepersec
ü Distanceofusertomonitor(17”):60cm
ü Dataset:realtimevideotakenfromregularwebcamonlaptops.ü Nospecificilluminationcondition
ü Testedontwoonscreenkeyboard:ü 3x7and5x5keys.
Result(facetrackermodule)
20
Robustnessto:differentusersandbackgroundchangesandheadtilt
Fastmoving ofuserinfrontofthecamera
Results:
Eyegazedetection(withmethod2)Using3x7onscreenkeyboard:
21
DEMO
Detectionrate
Faceillumination
Keyboard VerticalPre.
Horizontalprecision
Method
76.6% sensitive 5x5 4deg. 6.8deg. Method1
90% Notsensitive
3x7 2.8deg 11.3deg. Method2
22
Results:
Method1:usingscleraregion forhorizontaltrackingandeyecornersforvertical.
Method2:usingpupil centerforhorizontalandeyecornersforverticaltracking.
Eyetrackingsystem:
ü Realtimeü Notexpensiveü Moreaccuratecomparedtosimilarsystemsü Noneedtotrainingandoverwhelming calibration
Any Questions?
23