OPEN FRAMEWORKS + KINECT PART II

@GAFFTA

Thursday, March 7, 13

The Kinect for XBOX - released in

November 2010

Thursday, March 7, 13

why it matters http://memo.tv/archive/kinect_why_it_matters

Affordable Depth Sensor

Calibrated Depth + RGB Sensor

Nothing new technically, just making it accessible to a MUCH larger audience.

Thursday, March 7, 13

Microphone Array has advanced voice capabilities

Motorized tilt helps it to adjust to multiple environments

RGB camera for other data ( Facial Tracking, Color tracking )

Thursday, March 7, 13

what does the kinect see ?http://www.youtube.com/watch?v=dTKlNGSH9Po

Laser Projector casts a grid of infrared dots

Infrared lens interprets the size of the dots as distance and turns it into a “depth map”

This depth map makes it easy to separate parts of a scene

Computer learning is based off of these depth maps and large data sets

Thursday, March 7, 13

Hack The Kinect Method #1libfreenect is the driver / library

ofxKinect is the OF addon https://github.com/ofTheo/ofxKinect

Simplest wrapper for kinect. ( also the first )

Good at getting raw image + point cloud data + motor control

Extremely useful if combined with openCV

cross platform - PC / Mac / Linux

Thursday, March 7, 13

What is OpenCV ?

Open Source Computer Vision Library

A library that allows for real-time image manipulation and feature extraction. Including :

Gesture recognitionFacial recognition2D + 3D featuresMotion TrackingStereopsis

Originally developed by Intel Labs in 1999 for CPU intensive programs.

Released to the public in 2006. Now developed by Willow Garage

Thursday, March 7, 13

OpenCV with a webcam

Thursday, March 7, 13

OpenCV with a webcam

First there is a color image from a camera.

That color image is converted to grayscale so that it can be processed it faster. ( less channels = less data )

The background has a grayscale “snapshot” taken

A difference is applied to new frames after the background snapshot

OpenCV ContourFinder returns: a shape with many points, a bounding box, and centroid

Thursday, March 7, 13

OpenCV with a kinect

Thursday, March 7, 13

OpenCV with a kinect

There is already a grayscale depth feed, that can be used to isolate a range using OpenCV Threshold

Running OpenCV contour finder on this thresholded gray image gets us OpenCV blobs.

In addition to blob tracking we also get the 3d position of each point in space. Also known as a “point cloud”

Thursday, March 7, 13

LET’S START CODINGhttps://github.com/benMcChesney/OF_Kinect_Tutorials

Thursday, March 7, 13

ofxKinect + ofxUI

Thursday, March 7, 13