Design Implementation of a Gesture Recognition System Isaac Gerg B.S. Computer Engineering The...
-
Upload
julianna-jasmin-wilkerson -
Category
Documents
-
view
216 -
download
0
description
Transcript of Design Implementation of a Gesture Recognition System Isaac Gerg B.S. Computer Engineering The...
Design & Implementation of a Gesture Recognition
System
Isaac Gerg
B.S. Computer EngineeringThe Pennsylvania State University
Necessity
KiosksVehicle ControlVideo Gaming
Large Screen OS ControlNovelty
Types of Gestures
Static Gestures
Dynamic Gestures
MTrackSoftware Characteristics
• Runs in Windows• COTS Hardware Support• Utilizes DirectX
Classifier Characteristics• Recognize four fundamental gestures plus
variations for a total of 9 actions.
System Architecture
RGB to HSVConversion
Image Thresholding(pdf)
Feature VectorGenerator
CAMSHIFT
MicrostateAssignment
Action Engine
MacrostateAssignment Win32 API
VideoStream
Display to User
5 Stages
System Architecture
Stages (in order or processing)1. RGB to HSV Colorspace conversion.2. Image Thresholding (pdf)3. CAMSHIFT4. Microstate Assignment5. Action Engine
• Macrostate Assignment• Win 32 API
Thresholding
Dealing with Noise
Mathematical Morphology Operations
DiscriminantHu Invariant Moments
Scale, Rotation, and Translation Invariant
20321
2123003211230
20321
21230123003217
03211230112
03212
123002206
20321
2123003210321
20321
21230123012305
20321
212304
20321
212303
2202202
02201
)()(3))(3()(3)())(3(
))((4)()()(
)()(3))(3()(3)())(3(
)()(
)3()3(
4)(11
I
I
I
I
I
I
I
Classification1st & 2nd Order Invariant Moments
0
100
200
300
400
600 700 800 900
1st Order
2nd
Ord
er
Open Hand, Open Fingers
Open Hand, Closed Fingers
Fist
1 Finger
Classification
The need for a Distance Metric.
ClassifierThe Mahalanobis Distance
Minimum Distance Classifier
)()()( 12 mxSmxxD ttt
xt = feature vector at time t of unknown class.
m = mean vector of samples.S = covariance matrix of samples.
Micro/MacrostatesStatistical physics paradigm
Last chance to correct before taking action
Provides contextual analysis
Implemented using order statistics
MTrack in Action
MTrack in Action
Tracker Settings
The FutureVideo Filtering (Wiener Filtering, Kalman Filtering)
Morphological Filtering
Trainable Data Sets
Macrostate Improvement
Referenceshttp://www.galactic.com/Algorithms/discrim_mahaldist.htm
J. Flusser and T. Suk, "Affine Moment Invariants: A New Tool for Character Recognition, " Pattern Recognition Letters, Vol. 15, pp. 433-436, Apr. 1994.
Bradski, G. R., “Computer Vision Face Tracking For Use In A Perceptual User Interface.” Intel Technology Journal, 1998(2).