INTERACTIVE DIALOGUE TECHNIQUE BASED COMPUTER VISION WITH PALM TRACKING
GIA MUHAMAD AGUSTA, NANDANG SUNANDAR, QURROTUL AINI
FACULTY OF SCIENCE AND TECHNOLOGY, STATE ISLAMIC UNIVERSITY SYARIF HIDAYATULLAHJAKARTA - INDONESIA
• Introduction about Interaction and Palm Tracking• Haar Cascade• Controlling Mouse Pointer Method• Controlling Cursor Key Method• Experimental Result• Conclusion and Future Work
PRESENTATION SUMMARY
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
• The needed of HCI Development
INTRODUCTION
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
• Why Palm Tracking?
INTRODUCTION
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
• Why Haar Cascade– Gaussian Mixture Model– SVM (Support Vector Machine)– Pyramidal Lucas Kanade– ANN (Artificial Neural Network)
• Haar Cascade– Integral Image– Haar-Like Feature– Ada Boost Algorithm– Cascade Classifier
HAAR CASCADE
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Computational Time(Andrew King, Survey of of Methods for Face Detection, 2003)
• Integral Image
HAAR CASCADE
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
𝐷=( 𝐴+𝐵+𝐶+𝐷 )− ( 𝐴+𝐵 )− ( 𝐴+𝐶 )+𝐴
• Haar-Like Feature
HAAR CASCADE
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
1 3 3 2
1 4 4 51 7 5 2
2 3 4 6
1 3 2 2
1 3 5 81 2 3 1
3 5 3 1
1 4 7 9
2 9 16 233 17 29 38
5 22 38 53
6 26 44 60
7 30 53 778 33 59 84
11 41 70 96
• Ada Boost Algorithm
HAAR CASCADE
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Weak or basic classifierLearning rateFinal classifier
𝑓 (𝑥 )=∑𝑡=1
𝑇
𝛼 𝑡h𝑡 (𝑥)
• Cascade Classifier
HAAR CASCADE
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Filter 1
Filter n
BukanTelapak Tangan
...
Telapak Tangan
Filter 2
Telapak Tangan
Citra Digital
Telapak Tangan
Telapak Tangan
Bukan Telapak Tangan
Bukan Telapak Tangan
Bukan Telapak Tangan
• How we do it• Training and Running(Detecting)
HAAR CASCADE
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Filter 1
Filter n
BukanTelapak Tangan
...
Telapak Tangan
Filter 2
Telapak Tangan
Citra Digital
Telapak Tangan
Telapak Tangan
Bukan Telapak Tangan
Bukan Telapak Tangan
Bukan Telapak Tangan
Training Detecting
.xml (Knowledge Based)Open and Close Palm
• Take x and y coordinate from detected palm (Hook)
CONTROLLING MOUSE POINTER
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Start
Is Object detected
?
Take coordinate from dectected
object
Use coordinate as parameter to
controlling pointer
End
CONTROLLING CURSOR KEY
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Start
End
Menghitung Frame
Jika Frame pada posisi 0
Jika Frame pada posisi 3
Set posisi frame 0 Mencatat koordinat akhir
Menghitung jarak Euclide
Distance >=40
Koordinat x awal < x akhir
Left Cursor Key Direction
Right Cursor Key Direction
Set Waiting Frame = 15 frame
Jika Waiting frame = 0
Mencatat koordinat awal
Count distance from last frame to current frame with Euclidean Distance
𝑑=√¿¿
EXPERIMENTAL RESULT
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Haar Cascade Detection
Palm Position
ExperimentPalm Position
Straight ± 250 ± 450 ± 900 Reverse1 ü ü ü û û2 ü ü û û û3 ü ü ü û û4 ü û û û û5 ü ü û û û6 ü û û û û7 ü ü û û û8 ü û û û û9 ü ü û û û
10 ü ü û û ûAverage
(%)100 70 20 0 0
EXPERIMENTAL RESULT
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Haar Cascade Detection
Palm Position
Tegak ± 25° ± 45° ± 90° Terbalik0
20
40
60
80
100
120
100
70
20
0 0
Measurement with Different Positions
Palm Position
Aver
age
(%)
EXPERIMENTAL RESULT
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Haar Cascade Detection
Distance Measurement
ExperimentDistance (cm)
40 60 80 100 120 140 160 180 200 220 240
1 ü ü ü ü ü ü ü ü ü ü û2 ü ü ü ü ü ü ü ü ü ü û3 ü ü ü ü ü ü ü ü ü û û4 ü ü ü ü ü ü ü ü ü û û5 ü ü ü ü ü ü ü ü ü û û6 ü ü ü ü ü ü ü ü ü ü û7 ü ü ü ü ü ü ü ü ü û û8 ü ü ü ü ü ü ü ü û û û9 ü ü ü ü ü ü ü ü ü û û
10 ü ü ü ü ü ü ü ü ü û ûAverage
(%)100 100 100 100 100 100 100 100 90 30 0
EXPERIMENTAL RESULT
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Haar Cascade Detection
Distance
40 60 80 100 120 140 160 180 200 220 2400
20
40
60
80
100
120
100 100 100 100 100 100 100 100
90
30
0
Measurement Result through Distances
Distances
Aver
age
(%)
EXPERIMENTAL RESULT
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Haar Cascade Detection
1 37 73 109145 181 217253289325 361397 433 469 505541 577613649 685721757 793 829 865 901937 9730.00
100.00
200.00
300.00
400.00
500.00
600.00
700.00
Processing Time of Palm Detection
Frame Number
Proc
essin
g Ti
me
(ms)
𝐹𝑃𝑆=1000𝑚𝑠139,5 =𝟕 ,𝟏𝟔
EXPERIMENTAL RESULT
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Timing Process of Palm Detection Mouse PointerDistance (cm) Detection Left Click Function
40 ü ü60 ü ü80 ü ü
100 ü ü120 ü ü140 ü ü160 ü ü180 ü û200 ü û220 ü û240 û û
Controlling Process
EXPERIMENTAL RESULT
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Timing Process of Palm DetectionControlling Process Cursor Keys
Distances (cm) DetectionCursor Keys Function
Left Right
40 ü ü ü60 ü ü ü80 ü ü ü
100 ü ü ü120 ü ü ü140 ü ü ü160 ü ü ü180 ü ü ü200 ü û û220 ü û û240 û û û
CONCLUSION
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
The results of a series test (position angle and distance between user's palm and screen) shows that the more upright position of hand movement, the detection results would be better.
Tracking process can run on 7.16 fps at maximum distance 200 cm with accuracy rate 90.9%.
The left click event (mouse pointer) enables to function at a maximum distance 160 cm with accuracy reaching 63.6%, while left and right cursor keys will be function at maximum distance 180 cm with accuracy 72.7%.
FUTURE WORK
GiaMuhammad | Singapore, January 11th, 2014International Conference on Computer Science and Human Computer Interaction (ICCSHCI) 2014 | Changi Village Hotel
Use Kalman Filter method for soflty mouse movement.
Adding more positive and negative image sample data training to increase it accuration.
Impementing a whole system with a good interface and add some option for a better tracking (brightness, contrast, etc.)
THANK YOU
Top Related