Kinect Part II Anna Loparev. OpenNI API Actually general purpose API Need 3 things (.
-
Upload
lydia-warren -
Category
Documents
-
view
220 -
download
1
Transcript of Kinect Part II Anna Loparev. OpenNI API Actually general purpose API Need 3 things (.
Kinect Part II
Anna Loparev
OpenNI API• Actually general purpose API• Need 3 things (http://cs.rochester.edu/courses/297/fall2011/kinect
/openni)– SensorKinect Driver– OpenNI Library– NITE Middleware (gesture support)
• Requirements:– Kinect– Visual Studio 2008/2010– C++/C#
• Requirements (From Tutorial):– Python– SWIG (Allow Python to interface with OpenNI)– OpenCV (Allow Python to work with image data)– NumPy
Features• Recording for debugging• Fully backwards compatible• Under the GNU Lesser General Public License (LGPL)
OpenNI vs Microsoft SDK• Microsoft
– Easy to install– Don’t have to do calibration pose but gets more false positives– Predictive tracking of joints (Good for when sensor losses player)
• Leads to false positives• Uses lots of processing power
– Gets full 1024x768 resolution from camera– Full access to microphone– Support for tilt
• OpenNI– Open source (Allows for commercial use)– Works across all platforms– Hand-only tracking– Gesture recognition– Rotation in joints– Tilt has to be done through motor drivers
Sample Applications (OpenNI)• General
– NiCRead (Prints out depth at center of image)– NiSimpleCreate (C++ version of NiCRead without using SamplesConfig
XML)– NiSimpleRead (C++ version of NiCRead using SamplesConfig XML)– SimpleRead (C# version of NiCRead)– NiSimpleViewer (Depth and video viewer)– SimpleViewer (C# version of NiSimpleViewer)– NiViewer (Shows a bunch of different views and allows recording)– NiUserTracker (Pose detection and skeletal tracker)– UserTracker (C# version of NiUserTracker)– NiUseExtension– NiAudioSample
• Recording– NiBackRecorder (Allows user to save certain number of seconds)– NiRecordRaw– NiRecordSynthetic (Opens recording, does transformation, and re-
records)– NiConvertXToONI (Copies recording)
Sample Applications (NITE)• General
– Sample-Boxes (Highlight boxes based on hand movement and gestures)
– Sample-Boxes.net (C# version of Sample-Boxes)– Sample-CircleControl (Perform a circle gesture and circle is drawn)– Sample-Players (Identifies multiple players and their skeletons)– Sample-PointServer– Sample-PointViewer (Tracks hand and leaves behind trail)– Sample-SceneAnalysis (Identifies people)– Sample-SingleControl (Identifies wave gesture)– Sample-StickFigure (Places skeleton and people in virtual room)– Sample-TrackPad (Clickable grid)
Overview
Overview
NITE Abilities• Focus gestures
– Click– Wave– Swipe left– Swipe right– Raise hand candidate– Hand candidate moved
• Gesture Status– Gesture started– Gesture completed
• Hand tracking• User segmentation (Identify and track people)• Skeletal tracking
Capabilities• Optional extensions (Providers choose if they want to implement
them.)
• Alternative View (Data appears to come from another sensor)• Cropping• Frame Sync (between multiple devices)• Mirror• Pose Detection• Skeleton• User Position (Optimize output depth map)• Error State• Lock Aware (Allows node to be locked outside of context)
Generators• Map Generator• Depth Generator• Image Generator• IR Generator• Scene Analyzer• Audio Generator• Gesture Generator• Hand Point Generator• User Generator
Generators (Cont)• xn::Generator::StartGenerating() - begin generating data• xn::Generator:StopGenerating() - stop generation without
destroying node• xn::Generator::WaitAndUpdateData() - Update data when it
becomes available• xn::Context::Wait[…]UpdateAll() - Update all generators ([…]
determines when)
Simple Program// Set status to OK
XnStatus nRetVal = XN_STATUS_OK;
// Create and initialize new context
xn::Context context;
nRetVal = context.Init();
// Create generators
xn::DepthGenerator depth;
nRetVal = depth.Create(context)
// Start generating data
nRetVal = context.StartGeneratingAll();
Simple Program (Cont)// Do main loop
while (bShouldRun)
{
// Wait for new data
nRetVal = context.WaitOneUpdateAll(depth);
// Make sure update worked
if (nRetVal != XN_STATUS_OK)
{
printf(“Failed updating data: %s\n”, xnGetStatusString(nRetVal));
continue;
}
// DO STUFF
}
Simple Program (Cont)
// Clean-up
context.Shutdown();
OR
depth.Release();
context.Release();
DEMOS
References/More InfoSetup:
http://cs.rochester.edu/courses/297/fall2011/kinect/openni
http://www.codeproject.com/Articles/148251/How-to-Successfully-Install-Kinect-on-Windows-Open.aspx
Extensions:
http://code.google.com/p/simple-openni/
Other:
http://www.openni.org/
http://www.brekel.com/?page_id=671
http://labs.vectorform.com/2011/06/windows-kinect-sdk-vs-openni-2/