Movement Tracking in Real-time Hand Gesture Recognition
-
Upload
pranav-kulkarni -
Category
Science
-
view
52 -
download
0
Transcript of Movement Tracking in Real-time Hand Gesture Recognition
Movement Tracking in
Real-time Hand Gesture Recognition
PresentersPurohit Pankaj (W579074)Salagar Muaaz (W579080)Kulkarni Pranav (2010BCS203)Desale Ritesh (2010BCS211)
Seminar GuideMrs. S. S. Solapure
Agenda IntroductionReference Paper and Its Contents
ApplicationConclusionQuestions
Agenda IntroductionReferenced Paper and Its Contents
ApplicationConclusionQuestions
Introduction
Introduction
Gesture Recognition
Hand Gesture Recognition
Introduction (Contd.)
What is it?
How it works?
Introduction (Contd.)
What is its NEED?
Advantages Natural Interaction Builds a Richer Bridge Remote Interaction Wonderful Gaming Experience
Agenda IntroductionReference Paper and Its Contents
ApplicationConclusionQuestions
Reference Paper
Movement Tracking in Real-time Hand Gesture
RecognitionAuthored by
Hong-Min Zhu & Chi-Man PunDepartment of Computer and Information
ScienceUniversity of Macau, Macau SAR, China
{ma86560, cmpun} [at] umac.mo
9th IEEE/ACIS International Conference on Computer and Information Science
Reference PaperWhat does it say?This paper deals with overcoming of SCHD technique for Hand Gesture Recognition using newly improved Algorithm, IFDHD
Procedures in General Framework of Gesture Recognition
Previous Work DoneTemporal Hand GestureAssumptions
Camera User Synchronization Uniform Lightening Condition Simple Background Features Frame Rate – Gesture Speed
Coordination
S C H D Skin Color based Hand Detection
J. Kovac and P. Peer – Designed Skin Classifier
Rules Pixel is classified as a skin pixel if:
Value of Red > 95, Green > 40 and Blue > 20 & max{R, G, B} - min{R, G, B} > 15 & |R - G| > 15 and R > G and R > B
Proposed Solution Problems with SCHD
Computationally Expensive Skin-like Object Ambiguity Illumination Parameters Skin Color Variation
Solution – Motivated from BSHD
IFDHDInter-Frame Difference based Hand Detection
Proposed Solution
Hand DetectionModule
Motion TrackingModule
Hand Detection Module
Figure 3.1 Zoomed Mode for Hand Detection Module
Algorithm for Hand Detection
Input: Frames Fi = 1..N from video segmentSteps:
1. Convert frame F1 to grayscale2. Repeat (until end of video segment)
1. Convert frame Fi to grayscale2. Intensity difference image D0 = |Fi – F1|3. Binary image I = (D0 > T0)4. Do image opening on I followed by closing5. Splitting of Large regions into max size boundary box
as 60x806. Calculate center co-ordinate
Output: Center coordinate of each region in each frame
Experimental ResultsSCHD based Hand DetectionLightening Condition IFDHD based Hand DetectionSCHD based Movement Tracking IFDHD based Movement TrackingEfficiency Measurement
Fig. 4. Result of SCHD(a) original frame
Fig. 4. Result of SCHD(b) Skin Pixel Classification
Fig. 4. Result of SCHD(c) De-noise & Region
Connection
Fig. 4. Result of SCHD(d) Region Splitting
Fig. 4. Result of SCHD(e) Centers of Each Region
Experimental ResultsSCHD based Hand DetectionLightening Condition IFDHD based Hand DetectionSCHD based Movement Tracking IFDHD based Movement TrackingEfficiency Measurement
Fig. 5. Effect of Lightening Condition
(a) Original Frame
Fig. 5. Effect of Lightening Condition
(b) Skin Pixel Classification
Experimental ResultsSCHD based Hand DetectionLightening Condition IFDHD based Hand DetectionSCHD based Movement Tracking IFDHD based Movement TrackingEfficiency Measurement
Fig. 6. Result of IFDHD
(a) 1st Frame
Fig. 6. Result of IFDHD
(b) 11th Frame
Fig. 6. Result of IFDHD
(c) Subtraction: (b) - (a)
Fig. 6. Result of IFDHD
(d) Threshold of (c)
Fig. 6. Result of IFDHD
(e) De-noise
Fig. 6. Result of IFDHD
(f) Region Splitting
Fig. 6. Result of IFDHD
(g) Centers of Regions
System Domain (Contd.)
Movement Tracking Module
Figure 3.2 Zoomed Mode for Movement Tracking Module
Algorithm for Movement Tracking
Input: Region centers Detected in each frameSteps:
1. Initialize the start of frame2. Repeat (for each frame > 1)
1. Identify tail locations and store2. Calculate matrix of distances between centers and
tail locations3. Repeatedly select – min(Distance ( I ), Distance
( J )) 4. If Distance( I ) < Threshold then append Center to
Gesture and delete Distance( I ) Else initialize new Gesture start location
5. Select Gesture Frame that has the maximal standard deviation
6. Smooth movement track Gesture and interpolate it to Number of Center Coordinate falls coordinates
Output: Encoding of movement track
Experimental ResultsSCHD based Hand DetectionLightening Condition IFDHD based Hand DetectionSCHD based Movement Tracking IFDHD based Movement TrackingEfficiency Measurement
Fig. 7. SCHD Based Movement TrackingFirst Row: Last Frame of Video Segment
Fig. 7. SCHD Based Movement TrackingSecond Row: Detected Digit Track
Fig. 7. SCHD Based Movement TrackingThird Row: Smoothed Track
Experimental ResultsSCHD based Hand DetectionLightening Condition IFDHD based Hand DetectionSCHD based Movement Tracking IFDHD based Movement TrackingEfficiency Measurement
Fig. 8. IFDHD Based Movement TrackingFirst Row: Last Row of Video Segment
Fig. 8. IFDHD Based Movement TrackingSecond Row: Detected Digit Track
Fig. 8. IFDHD Based Movement TrackingThird Row: Smoothed Track
Experimental ResultsSCHD based Hand DetectionLightening Condition IFDHD based Hand DetectionSCHD based Movement Tracking IFDHD based Movement TrackingEfficiency Measurement
Efficiency Measurement
Table 1. Comparing the Efficiencies of SCHD and IFDHD
Outline IntroductionReferenced PaperApplication ScenarioConclusionQuestions
Reference PaperAmerican Sign Language Recognition System for Hearing Impaired People Using Cartesian
Genetic ProgrammingAuthored ByFahad Ullah
Department of Computer Systems Engineering,University Of Engineering & Technology,
Peshawar, Pakistan
Proceeding of 5th International Conference on Automation, Robotics and Applications, New
Zealand
Application Scenario
Why the interfaces are changing ?
How many Apps Out there? Have you tried X-box,PSP-2,Mac-OSX January 9, 2012, 66 million Xbox 360
consoles have been sold worldwide.
New era of Interfaces
Application Scenario What if you can’t speak? ASL CGP (Cartesian Genetic Programming) How it works? Genetic programming an Overview:
Probabilistic search Darwinian principle of natural
selection Naturally occurring genetic operations
such as crossover and mutation.
• Better individuals are preferred• Best is not always picked• Worst is not necessarily excluded• Nothing is guaranteed• Mixture of greedy exploitation and
adventurous exploration• Similarities to simulated annealing
(SA)
Probabilistic Selection Based On Fitness
Workflow
ASL using CGP
26 English language alphabets are trained and Identified
The system uses 26 binary images representing the different alphabets
Mentioned system with a Dictionary correction ability in order to increase the overall accuracy of the system.
Outline IntroductionReferenced PaperApplicationConclusionQuestions
ConclusionProposed IFDHDServing Feature Extraction Stage
Overcoming the pitfalls of SCHD
Outline IntroductionReferenced PaperApplicationConclusionQuestions
Questions, IF ANY?
Q?Thank You