ReachMedia
on-the-move interaction with everyday objects
Assaf Feldman, Emmanuel Muguia Tapia, Sajid Sadi, Pattie Maesand Chris Schmandt
The Media LaboratoryMassachusetts Institute of Technology
Question
Can we merge the virtual world with the physical
world?
Not enough
pixels
Too many clicks
An Interface Problem
Requirements
• Natural and seamless interaction
• Hands and eyes free
• Socially acceptable
IO
Input – RFID,Gestures,Keypad.
Output –audio, text
The smart phone as a general interface
Touch based interaction with objects
Gestures
Slight flick for menu navigation
Select
Next Previous
- Socially acceptable.
Demo
RFID
Multi modal Multi modal interface via interface via accessories accessories
MITe receiver
• Objects ID• Gestural input:
-Next-Previous-Select
RFID RFID Augmented Augmented
physical physical objectsobjects
Audio output:•Earcons •Voice menus
Interaction Interaction mediation via mediation via Mobile deviceMobile device
Object ID, user input
Audio & XML Content
Context Context ManagerManager
Hardware - Sensors
Accelerometers (House_n MITEs
–Cheap
–Small
–Low power
• RFID– Cheap– Great Context
• 25 features - Time domain feature extraction
– Cross correlation (3)
– Power (3)
– Turning points (18)
∑−
=
=1
0
21)(n
ii
an
aP
ji AA
jiji
AAAA
σσρ
),cov(),( =
Signal processing
Classification
• Online (Real Time) classification
• Good accuracy using Naïve Bayes and HMMs
• Preliminary results ~98% accuracy: Previous Next Select Noise
Previous 0.97101 0.014493 0 0.014493Next 0 1 0 0
Select 0 0 1 0Noise 0.017241 0 0 0.98276
Trut
h
The big picture
• Integrate digital world (information & services) and physical world (physical objects/environment)
• Make interfaces more responsive and proactive (objects & environment monitor user and (proactively) present information & services relevant to user’s current needs/interests)
Ambient Intelligence
3rd floor
Top Related