Perceptive Context for Pervasive Computing Trevor Darrell Vision Interface Group
description
Transcript of Perceptive Context for Pervasive Computing Trevor Darrell Vision Interface Group
Perceptive Context for Pervasive Computing
Trevor DarrellVision Interface Group
Human-centered Interfaces
• Free users from desktop and wired interfaces• Allow natural gesture and speech commands• Give computers awareness of users• Work in open and noisy environments
- Outdoors -- PDA next to construction site!- Indoors -- crowded meeting room
• Vision’s role: provide perceptive context
Perceptive Context
• Who is there? (presence, identity)• What is going on? (activity)• Where are they? (individual location)• Which person said that? (audiovisual grouping)• What are they looking / pointing at? (pose, gaze)
Virtual views
• Frontal •Profile silhouette:Head:
Examples: VH-generated views
• Faces:
• Gait:
Range-based stereo person tracking• Range can be insensitive to fast illumination change• Compare range values to known background• Project into 2D overhead view
IntensityRange
Foreground
Plan view
• Merge data from multiple stereo cameras..• Group into trajectories…• Examine height for sitting/standing…
Audio input in noisy environments
• Acquire high-quality audio from untethered, moving speakers
• “Virtual” headset microphones for all users
Vision guided microphone arrayCameras
Microphones
Audio-visual synchrony detection
• Audio weighting from video (detected face)
+
AVMI Applications• Image localization from audio
Audio associated with left face
Audio associated with right face
image variance AVMI
Brightness and depth motion constraints
I tI t + 1
I
Z
Z tZ t + 1 yt = yt-1
Parameter space
Closed-loop 3D tracker
Track users head gaze for hands-free pointing…
Head-driven cursor
Related Projects:• Schiele• Kjeldsen• Toyama
Current application for second pointer or scrolling / focus of attention…
“Look-to-talk”
Subject not looking at SAMASR turned off
Subject looking at SAMASR turned on