BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect...
-
Upload
olivia-gilbert -
Category
Documents
-
view
220 -
download
4
Transcript of BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect...
![Page 1: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/1.jpg)
Kinect basics: Systems for Smart Home Automation
Nilanjan Banerjee
![Page 2: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/2.jpg)
AgendaOverviewCamera BasicsDepth DataSkeletal TrackingGestures & InteractionFace TrackingKinect Socket ServiceKinect FusionNext Version of Kinect
![Page 3: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/3.jpg)
What’s new with Kinect 1.6 & 1.7Kinect 1.6Improved Depth Data APICoordinate MapperMore Sensor data – IR, Accelerometer, Camera Settings
Kinect 1.7Kinect FusionKinect Interactions – Interaction Stream, Click, Grip, ScrollInteraction Controls – KinectRegion, KinectCursor, KinectItems Control, KinectTileButton, KinectCircleButton, KinectScrollViewer, UserViewer
![Page 4: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/4.jpg)
Getting Started
Kinect - Xbox 360 or Kinect for Windows
Kinect for Windows SDK 1.7
Kinect for Windows Developer Toolkit 1.7
Coding for fun Kinect WPF
![Page 5: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/5.jpg)
Kinect API BasicsSetup Kinect1. KinectSensorChooser – KinectChangedEvent or KinectSensor.Connected2. Enable Color, Depth, Skeleton, Audio3. Start Kinect
Get Data4. Interaction Controls5. Events – AllFramesReady, InteractionFrameReady6. Polling – OpenNextFrame
![Page 6: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/6.jpg)
Camera Basics
![Page 7: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/7.jpg)
Image Processing 101BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty
B G R Empty
255 255 255 0
Kinect ImageSizes: 80x60, 320x240, 640x480DPI: 96Stride: # of bytes per single line:
Width (320 or 640) x 4 bytes (B,G,R,E)
B G R Empty
255 255 255 0
320x240
640x480
80x60
![Page 8: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/8.jpg)
First order task: Retrieve a RGB image from the Kinect
We will be using WPF1. Next generation windows presentation foundation2. Helps you build pretty user interface in Windows3. Much more flexible than a windows form framework
1. Allow dynamic and flexible layouts --- sizes of views like textbox etc. changes dynamically
2. Separates the design of the UI and the logic: design of the UI is in a xaml file and the logic is in C# similar to web programming
Important data structure: WriteableBitmap4. Stores the bitmap of an image but unlike the ordinary bitmap, WriteableBitmap is
mutable5. Read and write to bitmap
![Page 9: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/9.jpg)
Camera BasicsDemoName
![Page 10: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/10.jpg)
Depth Data
![Page 11: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/11.jpg)
Depth Data
Distance and player index for every pixelDistance is in millimeters (2,000 mm = 6.56 feet)Player index 1-6 or 0 for no player
![Page 12: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/12.jpg)
DepthDataDemoName
![Page 13: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/13.jpg)
Skeletal Tracking
![Page 14: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/14.jpg)
Skeletal Tracking
Max of two skeletons per Kinect6 Player Proposals
HandLeft
WristLeft
ShoulderLeft
Head
ElbowLeft
ShoulderRight
HandRight
WristRight
ElbowRight
ShoulderCenter
FootLeft
AnkleLeft
KneeLeft
HipLeft
HipCenter
HipRight
FootRight
AnkleRight
KneeRight
Spine
Seated (10)
Default (20)
![Page 15: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/15.jpg)
Coordinates
Z AxisX Axis
Y AxisJoints in meters from camera
Negative
Positive
Positive Negative
Positive
![Page 16: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/16.jpg)
Joint Tracking & Smoothing
Each joint has associated tracking stateTracked, Not tracked, or InferredInferred - Occluded, clipped, or low confidence joints
Use TransformSmoothParameters to smooth joint data to reduce jitter
Kinect transform smoothing uses Holt Double Exponential Smoothing - http://en.wikipedia.org/wiki/Holt-Winters
Skeleton.ClippedEdges = FrameEdges.Top;
![Page 17: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/17.jpg)
Physical Interaction Zone
Fatigue Kills Gestures Use Coding4Fun ScaleTo extension Interaction Controls support
![Page 18: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/18.jpg)
Skeletal TrackingDemoName
![Page 19: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/19.jpg)
Gestures & Interaction
![Page 20: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/20.jpg)
Gesture Controls
Define Kinect controls in KinectRegion KinectItems Control, KinectTileButton, KinectCircleButton, KinectScrollViewer
Automatically get cursor, grip, and scroll support
![Page 21: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/21.jpg)
GesturesDemo
Name
![Page 22: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/22.jpg)
Face Tracking
![Page 23: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/23.jpg)
Face Tracking
2D/3D Mesh and Points3D Head PoseAnimation Units
Z Axis
![Page 24: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/24.jpg)
3D Head Pose
Pitch
+20
-20
Yaw
-45 +45
Roll
-90 +90
![Page 25: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/25.jpg)
Upper Lip Raiser
-1 Max pushed down lip
+1 Showing teeth fully
Jaw Lowerer
-1 Closed
+1 Fully open
Lip Stretcher
-1 Fully rounded (kissing mouth)
+1 Fully stretched (joker’s smile)
Brow Lowerer
-1 Raised almost all the way+1 Fully Lowered
+1 Very sad frown
Lip Corner Depressor
-1 Very happy smile
+1 raised in expression of deep surprise
Outer Brow Raiser
-1 fully lowered, very sad face
Animation Units
![Page 26: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/26.jpg)
FaceTrackingDemoName
![Page 27: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/27.jpg)
Kinect Service
![Page 28: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/28.jpg)
Kinect Socket ServiceKinect data over TCP SocketsColor, Depth, Skeleton, and AudioClient libraries for WPF, Windows Phone & WinRTConsole or Windows Service host
ScenariosWindows Store, Windows Phone, WPF1 Kinect to many PCs, Many Kinects to 1 PC
![Page 29: BGR32 Format – Every Pixel (0,0 | 0,1 | 0,2) has blue, green, red, empty BGREmpty 255 0 Kinect Image Sizes: 80x60, 320x240, 640x480 DPI: 96.](https://reader036.fdocuments.in/reader036/viewer/2022081519/56649d135503460f949e732b/html5/thumbnails/29.jpg)
KinectServiceDemoName