Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.
-
date post
22-Dec-2015 -
Category
Documents
-
view
218 -
download
2
Transcript of Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.
![Page 1: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/1.jpg)
Real Time Gesture Recognition of Human Hand
Wu Hai
Atid Shamaie
Alistair Sutherland
![Page 2: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/2.jpg)
Overview:
• What are gestures?
• What can gestures be used for?
• How to find a hand in an image?
• How to recognise its shape?
• How to recognise its motion?
• How to find its position in 3D space?
![Page 3: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/3.jpg)
What is Gesture?
A movement of a limb or the body as an expression of thought or feeling.
--Oxford Concise Dictionary 1995
![Page 4: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/4.jpg)
Mood, emotion
• Mood and emotion are expressed by body language
• Facial expressions
• Tone of voice
• Allows computers to interact with human beings in a more natural way
![Page 5: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/5.jpg)
Human Computer Interface using Gesture
• Replace mouse and keyboard
• Pointing gestures
• Navigate in a virtual environment
• Pick up and manipulate virtual objects
• Interact with a 3D world
• No physical contact with computer
• Communicate at a distance
![Page 6: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/6.jpg)
Public Display Screens
• Information display screens
• Supermarkets
• Post Offices, Banks
• Allows control without having to touch the device
![Page 7: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/7.jpg)
Sign Language
• 5000 gestures in vocabulary• each gesture consists of a hand shape, a hand
motion and a location in 3D space• facial expressions are important• full grammar and syntax• each country has its own Sign language• Irish Sign Language is different from British Sign
Language or American Sign Language
![Page 8: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/8.jpg)
A C F
![Page 9: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/9.jpg)
Datagloves
![Page 10: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/10.jpg)
Datagloves
• Datagloves provide very accurate measurements of hand-shape
• But are cumbersome to wear
• Expensive
• Connected by wires- restricts freedom of movement
![Page 11: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/11.jpg)
Datagloves - the future
• Will get lighter and more flexible
• Will get cheaper ~ $100
• Wireless?
![Page 12: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/12.jpg)
• Our vision-based system
Wireless & Flexible No specialised hardware
Single Camera Real-time
![Page 13: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/13.jpg)
Coloured Gloves
• User must wear coloured gloves
• Very cheap
• Easy to put on
• BUT get dirty
• Eventually we wish to use natural skin
![Page 14: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/14.jpg)
Colour Segment
NoiseRemoval
Scale by Area
32
32
![Page 16: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/16.jpg)
Feature Space
Each point represents a different imageClusters of points represent different hand-shapesDistance between points depends on how similar the images are
![Page 17: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/17.jpg)
A continuous gesture creates a trajectory in feature space
We can project a new image onto the trajectory
![Page 18: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/18.jpg)
Multiple sub-spaces
Global space
Gesture 1Gesture 2
Classifying a new unknown image
![Page 19: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/19.jpg)
3D spatial position of hand
Subspaces and trajectories calculated with hand at origin
We know the image co-ordinates and the area of the hand in the original image
We can calculate depth and xy-position
camera
x
y
![Page 20: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/20.jpg)
A B C Y
Yes/No? Yes/No?Yes/No?
Yes/No?
![Page 21: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/21.jpg)
Hierarchical Search
• We need to search thousands of images
• How to do this efficiently?
• We need to use a “coarse-to-fine”search strategy
![Page 22: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/22.jpg)
Original image Blurring Factor = 1
Blurring Factor = 2Blurring Factor = 3
![Page 23: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/23.jpg)
Factor = 3.0
Factor = 2.0
Factor = 1.0
• Multi-scale Hierarchy
![Page 24: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/24.jpg)
• Hidden Markov Model ( HMM )--- time sequence of images modeling
Motion Recognition
HMM1 (Hello)
HMM2 (Good)
HMM3(Bad)
HMM4 (House)
P(f |HMM1)f
P(f |HMM2)
![Page 25: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/25.jpg)
Prediction and Tracking
•Given previous frames we can predict what will happen next
•Speeds up search.
•occlusions -
![Page 26: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/26.jpg)
Co-articulation
In fluent dialogue signs are modified by preceding and following signs
intermediate forms
A
B
![Page 27: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/27.jpg)
Future Work:
• Occlusions (Atid)
• Grammars in Irish Sign Language.
--- Sentence Recognition
• Body Language.
![Page 28: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/28.jpg)
Face Recognition
![Page 29: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/29.jpg)
A noisy environment
![Page 30: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/30.jpg)
Errors
![Page 31: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/31.jpg)
Model-based Recognition
![Page 32: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/32.jpg)
Pose-tracking
![Page 33: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/33.jpg)
Facial Expressions
Anger Fear Disgust Happy Sad Surprise
![Page 34: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/34.jpg)
Human Body Tracking
![Page 35: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/35.jpg)
Face Recognition
• Summary– Single pose– Multiple pose– Principal components analysis– Model-based recognition– Neural Networks
![Page 36: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/36.jpg)
Single Pose
• Standard head-and-shoulders view with uniform background
• Easy to find face within image
![Page 37: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/37.jpg)
Aligning Images
Alignment– Faces in the training set must be aligned with
each other to remove the effects of translation, scale, rotation etc.
– It is easy to find the position of the eyes and mouth and then shift and resize images so that are aligned with each other
![Page 38: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/38.jpg)
Nearest Neighbour
• Once the images have been aligned you can simply search for the member of the training set which is nearest to the test image.
• There are a number of measures of distance including Euclidean distance, and the cross-correlation
![Page 39: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/39.jpg)
Principal Components
• PCA reduces the number of dimensions and so the memory requirement is much reduced.
• The search time is also reduced
![Page 40: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/40.jpg)
Two ways to apply PCA (1)
• We could apply PCA to the whole training set.
• Then each face is represented by a point in the PC space
• We could then apply nearest neighbour to these points
![Page 41: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/41.jpg)
Two ways to apply PCA (2)
• Alternatively we could apply PCA to the set of faces belonging to each person in the training set
• Each class (person) is then reprented by a different ellipsoid and Mahalanobis distance can be used to classify a new unknown face
• You need a lot of images of each person to do this
![Page 42: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/42.jpg)
Problems with PCA
• The same person may sometimes appear differently due to– Beards, moustaches– Glasses,– Makeup
• These have to be represented by different ellipsoids
![Page 43: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/43.jpg)
-------(2)--------------(3)--------------(4)-------
-------(5)--------------(6)--------------(7)-------
-------(8)--------------(9)--------------(10)-------
![Page 44: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/44.jpg)
Problems with PCA
• Facial expressions– Differing facial expressions
• Opening and closing the mouth• Raised eyebrows• Widening the eyes• Smiling, frowing etc,
• These mean that the class is no longer ellipsoidal and must be represented by a manifold
![Page 45: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/45.jpg)
Facial Expressions
• There are six types of facial expression• We could use PCA on the eyes and mouth – so we
could have eigeneyes and eigenmouths
Anger Fear Disgust Happy Sad Surprise
![Page 46: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/46.jpg)
Multiple Poses
• Heads must now be aligned in 3D world space
• Classes now form trajectories in feature space
• It becomes difficult to recognise faces because the variation due to pose is greater than the variation between people
![Page 47: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/47.jpg)
![Page 48: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/48.jpg)
![Page 49: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/49.jpg)
Model-based Recognition
• We can fit a model directly to the face image
• Model consists of a mesh which is matched to facial features such as the eyes, nose, mouth and edges of the face.
• We use PCA to describe the parameters of the model rather than the pixels.
![Page 50: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/50.jpg)
Model-based Recognition
• The model copes better with multiple poses and changes in facial expression.
![Page 51: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/51.jpg)
Coarse Classification
• Fingerprints can be divided into 6 basic classes (some systems use other classes)– Arch– Tented Arch– Whorl– Right loop– Left Loop– Double Loop
![Page 52: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/52.jpg)
![Page 53: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/53.jpg)
![Page 54: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/54.jpg)
Orientation Field
• The orientation field gives the ridge direction at each point in the image
![Page 55: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/55.jpg)
Identifying Core and Delta
• The orientation field can be used to identify the core and delta in an image
![Page 56: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/56.jpg)
PCA applied to fingerprints
• We can align different images so that their cores line up
• We can then apply PCA to the orientation fields just as we did to face images
• We can project a new unknown image into the PC space and find the nearest matches in the training set.
![Page 57: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/57.jpg)
Accurate Matching
• Orientation fields and PCA are not good enough to give an accurate match
• They can reduce the number of possible candidates
• We can then apply a more accurate but time-consuming technique to the remaining candidate images
![Page 58: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/58.jpg)
Minutiae Matching
• Minutiae are fine details of the ridges in the fingerprint image such as– Ridge terminations,– Crossovers– Bifurcations etc.
• The pattern of minutiae is unique to each individual person
![Page 59: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/59.jpg)
Minutiae Types
• Different systems define different types of minutiae
• The most common are terminations (ridge endings) and bifurcations(forks)
![Page 60: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/60.jpg)
Binarisation and Thinning
• Binarisation– Every pixel is set to
either 0 or 1
• Thinning– Lines are thinned to a
width of 1 pixel
![Page 61: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/61.jpg)
Identifying minutiae
• Each black pixel in the image is classified using its “crossing number”.
• The crossing number of pixel p is defined as:
cn(p)= Half the sum of the differences between adjacent pixels in the 8-
neighbourhood of p
![Page 62: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/62.jpg)
Identifying Minutiae
• If the crossing number cn is equal to 2 then the pixel is not a minutia but a normal intra-ridge pixel
• If the crossing number is not equal to 2 then the pixel is some kind of minutia
![Page 63: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/63.jpg)
![Page 64: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/64.jpg)
Removing false-minutiae
![Page 65: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/65.jpg)
Minutiae on a real image
• The position of the minutiae is marked
• The direction of the ridge at each minutia is shown as a short line
![Page 66: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/66.jpg)
Matching minutiae between fingerprints
• We now have to compare minutiae between two different fingerprints to see if they came from the same person
• Remember the two images may have been rotated, translated or distorted with respect to one another
• We have to find the combination of rotations, translations and distortions which gives the largest number of matched pairs of minutiae
![Page 67: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/67.jpg)
Two different impressions of the same finger
![Page 68: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/68.jpg)
Matched pairs of minutiae
• Each pair must match position, direction and type
![Page 69: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/69.jpg)
Hough Transform
• Discretise the range of values for translation, rotation and distortion
• Set up an accumulator matrix A in which each element represents a different combination of translation, rotation and distortion
• For each possible pair of minutiae calculate the best values of translation, rotation and distortion which makes them match
• Increase the corresponding element of A by 1 • At the end of the process the element of A with the largest
value represents the best combination
![Page 70: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/70.jpg)
An alternative to minutiae-matching
• FingerCodes– 1. Centre image on core– 2. Divide image into circular zones– 3. Pass each zone through a set of 8 Gabor
filters (more about this next week)– 4. Compare the results using Euclidean distance
![Page 71: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/71.jpg)
![Page 72: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/72.jpg)
![Page 73: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/73.jpg)
Iris Recognition
![Page 74: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/74.jpg)
![Page 75: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/75.jpg)
John Daugman
• There is only one iris recognition algorithm in use
• The algorithm was developed mainly by John Daugman, PhD, OBE
• www.CL.cam.ac.uk/users/jgd1000/
• It is owned by the company Iridian Technologies
![Page 76: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/76.jpg)
Advantages of Iris Recognition
• Irises do not change with age – unlike faces
• Irises do not suffer from scratches, abrasions, grease or dirt – unlike fingerprints
• Irises do not suffer from distortion – unlike fingerprints
![Page 77: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/77.jpg)
Finding the Iris in the Image
• It is easy to find the circular boundaries of the iris
![Page 78: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/78.jpg)
Masking
• The boundaries of the eyelids can be found
• Eyelashes and specularities (reflections) can be found
• These areas can be masked out
![Page 79: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/79.jpg)
Gabor Wavelets
![Page 80: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/80.jpg)
Gabor Wavelets
• Gabor Wavelets filter out structures at different scales and orientations
• For each scale and orientation there is a pair of odd and even wavelets
• A scalar product is carried out between the wavelet and the image (just as in the Discrete Fourier Transform)
• The result is a complex number
![Page 81: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/81.jpg)
Phase Demodulation
• The complex number is converted to 2 bits
• The modulus is thrown away because it is sensitive to illumination intensity
• The phase is converted to 2 bits depending on which quadrant it is in
![Page 82: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/82.jpg)
![Page 83: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/83.jpg)
IrisCodes
• This process is carried out at a number of points throughout the image.
• The result is 2048 bits which describe each iris uniquely• Two codes from different irises can be compared by
finding the number of bits different between them – this is called the Hamming distance
• This is equivalent to computing an XOR between the two codes. This can be done very quickly
• To allow for rotation of the iris images the codes can be shifted with respect to each other and the minimum Hamming distance found
![Page 84: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/84.jpg)
Hamming Distance
bits of no. Total
different bits of No.HD
![Page 85: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/85.jpg)
Binomial Distribution
• If two codes come from different irises the different bits will be random
• The number of different bits will obey a binomial distribution with mean 0.5
![Page 86: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/86.jpg)
Identification
• If two codes come from the same iris the differences will no longer be random
• The Hamming distance will be less than expected than if the differences were random
• If the Hamming distance is < 0.33 the chances of the two codes coming from different irises is 1 in 2.9 million
• So far it has been tried out on 2.3 million people without a single error
![Page 87: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/87.jpg)
More Advantages of IrisCodes
• IrisCodes are extremely accurate
• Matching is very fast compared to fingerprints or faces
• Memory requirments are very low – only 2048 bits per iris
![Page 88: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/88.jpg)
Disadvantages of the Iris for Identification
•Small target (1 cm) to acquire from a distance (1 m) •Moving target ...within another... on yet another •Located behind a curved, wet, reflecting surface •Obscured by eyelashes, lenses, reflections •Partially occluded by eyelids, often drooping •Deforms non-elastically as pupil changes size •Illumination should not be visible or bright •Some negative (Orwellian) connotations
![Page 89: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/89.jpg)
![Page 90: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/90.jpg)
Fake Iris Attacks
![Page 91: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/91.jpg)
Fake Iris Fourier Spectrum
• Due to the dot matrix grid the Fourier Spectrum of the fake iris has 4 extra points
![Page 92: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/92.jpg)
![Page 93: Real Time Gesture Recognition of Human Hand Wu Hai Atid Shamaie Alistair Sutherland.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d825503460f94a670d2/html5/thumbnails/93.jpg)