Your Noise is My CommandCHI 2011 – May 9, 2011 Your Noise is My Command Sensing Gestures Using the...
Transcript of Your Noise is My CommandCHI 2011 – May 9, 2011 Your Noise is My Command Sensing Gestures Using the...
CHI 2011 – May 9, 2011
Your Noise is My Command
Sensing Gestures
Using the Body as
an Antenna
Gabe Cohn1,2
Dan Morris1
Shwetak N. Patel1,2
Desney S. Tan1
1Microsoft Research 2University of Washington
ubicomp lab
2
Beyond the Keyboard and Mouse
3
Speech Recognition
4
Capacitive Touch Screen
5
Skinput Harrison et al., CHI 2010
6
Muscle-Computer Interfaces Saponas et al., CHI 2010
Saponas et al., UIST 2009
Saponas et al., CHI 2008
7
Computer Vision and Depth Cameras
8
9
10
11
Sensing Gestures
Using the Body as
an Antenna
no instrumentation to environment
touch gestures anywhere in the home
minimal instrumentation to user
Typical
“bunny ears”
TV antenna
12
Typical
“teenager”
human antenna
dielectric with
complex geometry
40 Hz – 400 MHz
13
“body antenna effect”
- body area networks (BAN)
- analyzing electrical activity on body
human body antenna
< 200 kHz
electromagnetic (EM) noise
from powerlines and appliances
14
15
Powerline Noise
Energy Monitoring and Activity
Recognition
Gupta et al., Ubicomp 2010
Patel at al., Ubicomp 2007
16
Hypothesis
Powerline noise differs at different
- locations in the home
- positions on the wall
Using the human body antenna to
receive this noise, we can determine
- location in the home
- position on the wall
of a user’s touch gestures
17
Apparatus measure voltage
sample with NI-DAQ
400 kS/s
16-bit resolution
store on laptop hard drive
Core Experiment
10 Homes
• single-family and townhouses
• 1 – 3 floors
• Area: 120 – 290 sq m (µ = 215)
1300 – 3100 sq ft (µ = 2310)
• Built: 1948 – 2006 (µ = 1981)
18
10 Participants
• 5 male / 5 female
• Age: 28 – 61 (µ = 38)
• Weight: 52 – 82 kg (µ = 64)
115 – 180 lbs (µ = 141)
• Height: 150 – 188 cm (µ = 169)
59 – 74 in (µ = 67)
19
Locations
6 locations in each home
5 light switches
2 in same room
1 blank wall
above an outlet
Procedure
6 gestures per location
hold each for 6 seconds
guided by computer
commands
6 locations
6 gestures per location
4 “rounds” (repetitions)
144 total gestures per
participant 20
21
Analysis 5 sec
Segmentation
22
Analysis 5 sec
82 ms Segmentation Windowing
23
Analysis Feature Extraction
Frequency (kHz)
Frequency (kHz)
Time (s)
Am
pli
tud
e (
V)
Po
wer
(dB
) P
ow
er
(dB
)
DC
RMS
Time Domain FFT 0 – 2 kHz
2 Features 332 Features
FFT 0 – 25 kHz FFT 0 – 200 kHz High Freq. Peaks
250 Features 400 Features 18 Features
1002 Total Features per 82 ms window
24
Analysis Machine Learning
1002 features per 82 ms time
window
classification using the Weka
SVM (SMO implementation)
4-fold cross validation
fold by “round”
training/testing sets in different
“rounds”
avoids over-fitting
25
Results Wall Touch
2-class classification
0.0% 20.0% 40.0% 60.0% 80.0% 100.0%
Chance Accuracy
98.5%
26
Results Touch Position on Wall
0% 20% 40% 60% 80% 100%
Chance Accuracy
87.4%
74.3%
All Walls
Blank Walls Only
5-position classification
27
Results Location in Home
0% 20% 40% 60% 80% 100%
Chance Accuracy
Touch Walls
Stand Near Walls
99.1%
99.5%
6-location classification
2 walls in same room
28
Results Location in Home and Position on Wall
30-class problem
0% 20% 40% 60% 80% 100%
Chance Accuracy
79.8%
un-optimized classification on each 82 ms window
29
Summary Core Experiment:
• Location in home - near 100%
• Position on around switch – 87%
Additional Exploration:
• Differentiate right/left hand
• Differentiate appliance touched
• Estimate proximity to wall
• Estimate continuous position on wall
30
Limitations
Improved feature set - relative features
- temporal features
Signal injection - on-body
- into power line
Real-time system
Smaller system (no laptop)
Contact point to body - contactless coupling to body
Explore gesture set
Explore application space
Generalizability of noise signals - Training procedure
- Locations with poor signals
- Strong signals overwhelm others
Changing state of home - Appliances switched on/off
Future Work
31
Your Noise is My Command
Sensing Gestures
Using the Body as an
Antenna
Gabe Cohn
Dan Morris
Shwetak N. Patel
Desney S. Tan
ubicomp lab
Thank You!
www.gabeacohn.com
32
Backup Slides
Applications
Long-term: • Home automation • Gaming
Short-term: • Impromptu interaction session • Meetings/presentations
33