Finger Gesture Recognition through Sweep Sensor
description
Transcript of Finger Gesture Recognition through Sweep Sensor
![Page 1: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/1.jpg)
Finger Gesture Recognition through
Sweep SensorPong C Yuen1, W W Zou1, S B Zhang1, Kelvin K F Wong2 and Hoson H S
Lam2
1Department of Computer ScienceHong Kong Baptist University
2World Fair International Ltd
![Page 2: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/2.jpg)
OutlineMotivations
Design Criteria
Proposed Method
Experimental results
Conclusions
![Page 3: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/3.jpg)
MotivationsVision-based
interfaceInsert some images using face, expression, body movement…
Sensor-based interface
http://www.blogcdn.com/www.tuaw.com/media/2008/11/mac-101_-multi-touch-tips.jpg
http://www.fitbuff.com/wp-content/uploads/2007/10/wii-fitness.jpg
There should be a video about the body movement interface
Common objective: natural input to replace traditional physical input devices
![Page 4: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/4.jpg)
MotivationsWhile many sensor-based gesture input have been developed, there is no algorithm/system using sweep sensor
Why Sweep Sensor?low costNo latency problem (fingerprint recognition)popularity
![Page 5: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/5.jpg)
Design CriteriaUser friendliness
easily performed by a user. intuitive and easy to understand.
User independentGeneric for all users.
Robustnessdiversity of patterns captured.
Efficiency Real-time applicationmobile devices
![Page 6: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/6.jpg)
Classification
t > t0
left rightNo
left tick right tick
Yes
feature vector
D > 0.5 left
right
D > 1.3 left tick
right tickD > 1/1.3
D < -0.5
Feature Extractionnoise
reduction
envelope enhanceme
ntinput image
direction estimation
direction index D = Dleft
/Dright
envelope
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
yi
i
y
0
0
miiright
miileft
yD
yD
SegmentationInput image
CharacteristicsFormulate the noise
Proposed MethodInput image
CharacteristicsFormulate the noise
Segmentation
Feature Extractionnoise
reduction
envelope enhanceme
ntinput image
direction estimation
direction index D = Dleft
/Dright
envelope
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
yi
i
y
0
0
miiright
miileft
yD
yD
Classification
t > t0
left rightNo
left tick right tick
Yes
feature vector
D > 0.5 left
right
D > 1.3 left tick
right tickD > 1/1.3
D < -0.5
Input image
CharacteristicsFormulate the noise
Segmentation
Feature Extractionnoise
reduction
envelope enhanceme
ntinput image
direction estimation
direction index D = Dleft
/Dright
envelope
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
yi
i
y
0
0
miiright
miileft
yD
yD
Classification
t > t0
left rightNo
left tick right tick
Yes
feature vector
D > 0.5 left
right
D > 1.3 left tick
right tickD > 1/1.3
D < -0.5
![Page 7: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/7.jpg)
Input Image Characteristics
Different sensor characteristicsNoise level
![Page 8: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/8.jpg)
Figure 2. The block diagram of feature extraction
SegmentationOwing to different sensor characteristics, the gesture images obtained, even the gesture is the same, will be different
Segmentation by estimating the sweeping time
noise reduction
vertical gradient thresholding horizontal
projection
![Page 9: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/9.jpg)
Segmentation (cont.)
blank partsw
eeping part
noise reduction
vertical gradient
THthresholdin
g
horizontal projection
sth1t 't
] 0[on )( ), ))((1 ( 1)(0
2
1
1
ttEdttstcth ts
t
s
![Page 10: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/10.jpg)
Feature ExtractionTime information t (sweeping time)
Finger motion information d (direction)Left and rightLeft diagonal and right diagonal
![Page 11: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/11.jpg)
Feature Extraction (left / right)
noise reduction
Left
Right
direction
enhancement
input image
direction estimation
direction index D = Pleft - Pright
i-th fingerprint texture
A B
C D )(
)(
CBDAP
CBDAP
right
left
![Page 12: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/12.jpg)
Feature Extraction (left tick / right tick)
noise reduction
envelope enhancement
input image
direction estimation
direction index D = Dleft /Dright
0 40 80 120 160 2000
20
40
60
80
100
120
140
160
0 40 80 120 160 2000
20
40
60
80
100
120
140
160
0
0
miiright
miileft
yD
yD
envelope
yi
i
y
![Page 13: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/13.jpg)
ClassificationA very simple rule based on a combination of movement
Classification tree (decision tree)
left rightNo
left tick right tick
Yes
feature vector t > t0
D > 1.3 left tick
right tickD > 1/1.3
D > 0.5 left
rightD < -0.5
![Page 14: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/14.jpg)
Designed GesturesLeft and Right Left tick and Right
tick
![Page 15: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/15.jpg)
Experimental Results2 testing groups
3 technical users – Engineers, and technical managers, research staff (95.0%)3 Non-technical users – secretary, clerk (86.87%)
Test on different sensors4 different sensors manufacture at different period of time
![Page 16: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/16.jpg)
Experimental ResultsEvaluation interface
There should be a video here
![Page 17: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/17.jpg)
Experimental Results
Sensor1 Sensor2 Sensor3 Sensor40
102030405060708090
100
Results of fingerprint recognition using different sensors
success fail%
Results by 3 non-technical staff with 4 different sensors
![Page 18: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/18.jpg)
Experimental ResultsIntegrated application with an image viewer
There should be a video here
![Page 19: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/19.jpg)
Conclusions
![Page 20: Finger Gesture Recognition through Sweep Sensor](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816554550346895dd7d050/html5/thumbnails/20.jpg)
Thank You