Project 10Facial Emotion Recognition Based On Mouth Analysis
SSIP 08, Vienna1
http://www.we-hope-project10-will-win.info
The Project
Objective : To recognize emotional state / expression using mouth information
Input: Mouth images (no make-up)
Output: Emotional State/ Expression Happy, Neutral, Sad
2
http://www.we-hope-project10-will-win.info
The Team
3
Kornélprogrammer
Kornélprogrammer
PéterWeb programmer
PéterWeb programmer
Kamalprogrammer
Kamalprogrammer
Naiemresearcher
Naiemresearcher
Sofiaprogrammer
Sofiaprogrammer
http://www.we-hope-project10-will-win.info
The Tasks
Create facial expressions photographic database
Segment the mouth in the input image
Use suitable features for expression characterization
Design a reliable classifier to distinguish between different mouth expressions
4
http://www.we-hope-project10-will-win.info
SSIP Lips database
Happy, Neutral and Sad Photos of SSIP students and lecturers
Thank you all!!!
Happy Neutral Sad
5
http://www.we-hope-project10-will-win.info
Mouth Segmentation
6
Input Image HSV Space - Hue Thresholding
Morphological Operations
http://www.we-hope-project10-will-win.info
Lips Features Extraction
Detect the leftmost and rightmost lip points
Normalize images (rotation, translation and scaling)
Calculate features Eccentricity Convex Area Minor Axis Ratio of Upper to Lower Lip
8
http://www.we-hope-project10-will-win.info
Expression Classification
SVM Classifier
Two Stage Classification
Mouth Features
☺
9
http://www.we-hope-project10-will-win.info
Results 1
Differences between different classes were found to be statistically significant (p<0.01)
Classification Accuracy Stage 1 (Sad / Not Sad) 88% Stage 2 (Happy/ Neutral) 62%
10
http://www.we-hope-project10-will-win.info
Future Work Acquire larger database for training and testing
Test different facial expressions (such as anger and disgust)
Other classifiers: NN, FIS
Conclusion Mouth information is often insufficient for
recognizing facial expression / emotional state
Other face features such as eyes and eyebrows can contribute in emotional state recognition
12
http://www.we-hope-project10-will-win.info
References M. Gordan, C. Kotropoulos, I. Pitas, “Pseudoautomatic Lip Contour Detection Based on Edge
Direction Patterns”
J. Kim, S. Na, R. Cole, “Lip Detection Using Confidence-Based Adaptive Thresholding”
F. Tang, “Facial Expression Recognition using AAM and Local Facial Features”
M. Pantic, M. Tomc, L. Rothkrantz , “A Hybrid Approcah to Mouth Features Detection”
14
http://www.we-hope-project10-will-win.info
Top Related