RoshamRoboAlexander Ciccone – EEL4665 Spring 2014Oral Report 2 – Special Sensor

Image Credit – Author: Enzoklop URL: http://en.wikipedia.org/wiki/File:Rock-paper-scissors.svg

Special Sensor

• Laptop (Windows or Ubuntu environment) running OpenCV program to interpret images sent from an IP camera (Foscam FI8910W) via a wireless router.

RoshamRobo must:• Use sensors and IP camera to find and position itself in front of a target

human• Display countdown and game moves via LCD screen at appropriate times

The software and special sensor must:• Detect an outstretched hand of varying colors (foreground & background)• Differentiate amongst Rock, Paper, and Scissors hand gestures during play

Images: s-ln.in, wikipedia, amazon.com

Images courtesy of sparkfun, adafruit, amazon.com

Special Sensor Block Diagram

Inspiration• Computer vision algorithms are exciting!• OpenCV algorithms able to detect & track faces, people, hands,

images and many other objects

Images: bassemdy.wordpress.com, www.andol.info, www.creativeapplications.net,www.k2.t.u-tokyo.ac.jp/fusion/Janken/index-e.html

Initial Stages• Install and troubleshoot installation of OpenCV• Compile and study resources on hand gesture recognition and

OpenCV applications• The following resources were extremely helpful:• www.andol.info• anikettatipamula.blogspot.ro• blogs.ugidotnet.org/WetBlog• www.youtube.com/user/khounslow• www.youtube.com/user/18F4550videos

Initial Stages• Subtract background from image while preserving foreground

(hand)

• Find Convex Hull of hand region (the convex set enclosing the hand)

Convex Hull – Experimental Results

Finding Convexity Defects• Find, mark, and count (convexity) defects of the convex hull

Counting Defects & Fingers

Identifying Scissors & Rock Gesture

Further Experimentation