(Spring 2013) Detecting the Human Biometric Sensor Interaction with Microsoft Kinect
Kinect sensor
-
Upload
bhoomit-morkar -
Category
Technology
-
view
319 -
download
2
Transcript of Kinect sensor
![Page 1: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/1.jpg)
KINECT SENSOR
Group members
Bhoomit Morkar 14103180Sachin Keniya 14103170Vaishnavi Shah 14103182
Ears and Eyes of your application
![Page 2: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/2.jpg)
Content Introduction
• Basic Construction• Basic Specifications• Kinect’s Potential and applications
Working of Kinect• Classification of Depth Measurement Techniques
Triangulation Principles of TOF Imaging Principle of structured light
• Skeletal Tracking
Applications
References
![Page 3: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/3.jpg)
Introduction
• It was Designed by Microsoft for the Xbox 360 video game console and windows PC’s.
• While making the Kinect sensor , Microsoft’s aim was to enable interactions between users and a game without the need to touch a controller.
• Kinect is an RGB-D sensor providing synchronized color and depth images.
• It’s greatest differentials is depth sensing technology that could be extended far beyond gaming and at a much lower cost than traditional 3-D cameras.
![Page 4: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/4.jpg)
Why Kinect is revolutionary ?
• Kinect has proven to be an incredible piece of technology.
• It is the first practical natural user interface for machines communicating with people.
• It is combining acoustical and visual sense.
• It is recognizing full body movements, even complicated ones.
• It finds application not only in games but also in the field of medical sciences and robotics.
• After selling a total of 8 million units in its first 60 days, the Kinect holds the Guinness World Record of being the "fastest selling consumer electronics device".
![Page 5: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/5.jpg)
Basic Construction
![Page 6: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/6.jpg)
Power Comes from the Sum
• The Kinect Sensor is the outcome of several devices(or sensors) working together.
• It is comprised of a color camera (also known as RGB), a depth sensor, a 4-microphone array, and a motorized tilt.
• The depth sensor, in itself, consists of an infrared (IR) projector and an IR camera.
• Due to that Kinect being a motion sensing input device, able to capture, track, and interpret full-body movements(3D), and recognize gestures and voices.
![Page 7: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/7.jpg)
• The depth sensor is a monochrome complementary metaloxide semiconductor (CMOS) sensor.
• One of Kinect’s greatest differentials is its relatively low cost depth sensor.
• The depth-sensing technology is licensed from the Israeli company PrimeSense. Although the exact technology is not disclosed, it is based on the structured light principle.
• The IR projector is an IR laser that passes through a diffraction grating and turns into a set of IR dots.
• The sensor has an RGB (red-green-blue) camera for color video.
• It has four microphones that enable your application to respond to verbal input, in addition to responding to movement.
![Page 8: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/8.jpg)
Basic Specifications
Hardware
• Depth resolution : 640x480 px • RGB resolution : 1600x1200
px• FrameRate : 60 FP
Software• Depth;
30 FPS: 80x60 , 320x240 , 640x480
• Color;12 FPS : 1280x980 RGB15 FPS : Raw YUV 640x48020 FPS : 640x480
![Page 9: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/9.jpg)
Kinect’s Potential and applications
Depth sensor Skeletal tracking Microphone array
Background removalObject recognition
Multi-user Easy Gesture Recognition
Sound source detection Speech recognition
![Page 10: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/10.jpg)
How it works ?
![Page 11: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/11.jpg)
Audio input
Microphone array
Sound threshold
How does Kinect hear ?
Track me !
Or
Speak louder .
![Page 12: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/12.jpg)
what does it see?
![Page 13: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/13.jpg)
Near mode depth ranges
Default mode depth ranges
Angle of vision(RGB-D)
Area under Observation
![Page 14: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/14.jpg)
Classification of Depth Measurement Techniques
![Page 15: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/15.jpg)
Triangulation
• In trigonometry and geometry, triangulation is the process of determining the location of a point by forming triangles to it from known points.
• Here rather than measuring distances to the point directly as in trilateration; we use of both angles and distance
![Page 16: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/16.jpg)
Basic Mathematical model for Kinect
![Page 17: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/17.jpg)
Principles of TOF Imaging
• Time-of-Flight (TOF) Imaging refers to the process of measuring the depth of a scene by quantifying the changes that an emitted light signal encounters when it bounces back from objects in a scene.
![Page 18: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/18.jpg)
Pulsed ModulationContinuous Wave Modulation
![Page 19: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/19.jpg)
Principle of structured light
• Project a known light pattern into the 3D scene, viewed by cameras.
• Distortion of light pattern allows computing the 3D structure.
• This pattern has to be unique per position in order to recognize each point in the pattern.
• Kinect uses pseudo random pattern.
![Page 20: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/20.jpg)
speckles
Based on distance and orientation
First Region
High accurate depth.
(0.8 – 1.2 m)
Second Region
Medium accurate depth.
(1.2 – 2.0 m)
Third Region
Low accurate depth.
(2.0 – 3.0 m)
![Page 21: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/21.jpg)
Depthobserved patternCalibration
IR camera observes the scene
known pattern
projected
3D map of the
beginning frame
New 3D map of the
scene
Renew 3D map of the
scene
Between projector
and camera&
x-direction speckle shift
![Page 22: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/22.jpg)
Precision
Spatial x/y resolution
3mm @ 2m distance
Depth Z resolution
1cm @ 2m distance 10 cm @ 4m distance
operation range
0.8m ~ 4m | 0.5m ~ 3m
![Page 23: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/23.jpg)
Skeletal Tracking
In the beginning, it was justhands;
• Head and hands,• Added shoulder and elbows.
Today!• It can track 20 joints of our body and
rans fast.
![Page 24: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/24.jpg)
Process
Modes
![Page 25: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/25.jpg)
Applications
• The Kinect fusion project allows a user to use an off-the-shelf Kinect camera as a 3D scanner for producing high-quality 3D scans of small or large objects.
• Researchers in China, in collaboration with Microsoft Research, developed the Kinect
Sign Language Translator, a system that can translate sign language into spoken and written language in near real time.
• The Holoflector is an augmented reality mirror. Behind the mirror is a large LCD monitor and right above the mirror is the Kinect.
• Combining a Kinect, a few hacking tools and a 3D environment, Kinectasploit allows a
user to break into a computer’s security system using body gestures. The system works almost like a video game. In one demo, a hacker ‘shoots’ at Wi-Fi targets to break into them.
![Page 26: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/26.jpg)
• UBI Interactive developed their software UBI which combines your Windows 8 computer to Kinect and projector to create an interactive touchscreen off any surface.
• In another case of copying from the movies, many users have hacked their Kinect to allow users to control robots via body movement.
• We can pick out cloths and try them all out virtually with the help of Fitnect, an interactive dressing room.
![Page 27: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/27.jpg)
References http://mms.ecs.soton.ac.uk/2014/papers/2.pdf
http://lshao.staff.shef.ac.uk/pub/KinectReview_TC2013.pdf
https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/Microsoft20Kinect20Sensor20and20Its20Effect20-20IEEE20MM202012.pdf
http://mms.ecs.soton.ac.uk/2014/papers/2.pdf
http://campar.in.tum.de/twiki/pub/Chair/TeachingSs11Kinect/2011-DSensors_LabCourse_Kinect.pdf
http://www.slideshare.net/MatteoValoriani/introduction-to-kinect-update-v-18?qid=e70a04cc-282d-4c11-a0d0-cff7f58c2435&v=&b=&from_search=9
http://www.slideshare.net/rojizofrio/xbox-one-kinect?qid=e70a04cc-282d-4c11-a0d0-cff7f58c2435&v=&b=&from_search=15
![Page 28: Kinect sensor](https://reader033.fdocuments.in/reader033/viewer/2022061519/58a289141a28ab891a8b7a35/html5/thumbnails/28.jpg)
Videos https://youtu.be/uq9SEJxZiUg
http://youtu.be/bRgEdqDiOuQ?t=10m16s
http://www.google.com/patents/US20090
http://campar.in.tum.de/twiki/pub/Cha... (p. 33)
http://en.wikipedia.org/wiki/Range_im... (Stereo triangulation)
https://www.youtube.com/watch?v=-RELJD0LBg8&spfreload=5
https://youtu.be/uq9SEJxZiUg
http://www.hongkiat.com/blog/innovative-uses-kinect/