Platform for Situated Intelligence - microsoft.com · Platform for Situated Intelligence Sean...
28
Transcript of Platform for Situated Intelligence - microsoft.com · Platform for Situated Intelligence Sean...
Platform for Situated IntelligenceEnabling interactive, real-time AI
Platform for Situated Intelligence
Sean Andrist, Dan Bohus, John Elliott, Ashley Feniello, Don Gillett, Eric Horvitz, Mihai Jalobeanu, Anne Loomis Thompson, Nick Saw
MSR AI, Redmond
enabling interactive, real-time AI
Microphone array capture
Sound source localization
Speech recognition
Language understanding
Infrared proximity sensors
Badge sensors
Face detection and tracking
Head-pose tracking
Facial feature tracking
Face identity recognition
Gender detection
Attention models
Engagement models
Turn-taking models
Behavioral control
Dialog management
Natural language generation
Speech synthesis
Avatar synthesis
Robot motion control
Floor-plan models
User models
Microphone array capture
Sound source localization
Speech recognition
Language understanding
Infrared proximity sensors
Badge sensors
Face detection and tracking
Head-pose tracking
Facial feature tracking
Face identity recognition
Gender detection
Attention models
Engagement models
Turn-taking models
Behavioral control
Dialog management
Natural language generation
Speech synthesis
Avatar synthesis
Robot motion control
Floor-plan models
User models
ComponentsRepository of
AI technologies
RuntimeBasic primitives and programming model
Tools
Debugging
Visualization
Machine Learning
Components
RuntimeBasic primitives and programming model
Tools
Face
Tracking
Sound
Localization
…
Speech
source
identification
Camera
Microphone
Array
…
…
Face
Tracking
Sound
Localization
…
Speech
source
identification
Camera
Microphone
Array
…
…
Face
Tracking
Sound
Localization
…
Speech
source
identification
Camera
Microphone
Array
…
…
Speech
source
identification
Face
Tracking
Sound
Localization
…
Camera
Microphone
Array
…
…
Speech
source
identification
Face
Tracking
Sound
Localization
…
Camera
Microphone
Array
…
…
Components
Runtime
Tools
Debugging
Visualization
Machine Learning
Composited visualizations
Navigation
Live mode
Composited visualizations
Navigation
Live mode
Composited visualizations
Navigation
Live mode
Camera
Microphone
Array
Speech
source
identification
…
…
…
Face
Tracking
Sound
Localization
Composited visualizations
Navigation
Live mode
Annotation
Feature construction
End-to-end support
Speech
source
identification
Face
Tracking
Sound
Localization
…
Camera
Microphone
Array
…
…
ComponentsRepository of
AI technologies
Runtime
Tools
… …
SensorsCameras: USB, PointGrey, Kinect
Microphones: USB, Array
…
VisionImaging, transforms, optical flow
Foreground models
Blob tracking
…
Human perceptionFace tracking & identity
Gaze tracking
Body tracking
Affect recognition
Attention tracking
…
Speech Voice activity detection
Speech recognition
Natural language processing
Natural language generation
Speech synthesis
…
InteractionEngagement models
Turn-taking models
FSM-based controller
…
OutputSpeech synthesis
Avatar rendering
…
\psi
\psi
…
OpenCV
ComponentsRepository of
AI technologies
RuntimeBasic primitives and programming model
Tools
Debugging
Visualization
Machine Learning
