SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s...
Transcript of SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s...
![Page 1: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/1.jpg)
1
SurroundSense: Mobile Phone Localization
via Ambience Fingerprinting
Ionut Constandache
Co-authors:
Martin Azizyan and Romit Roy Choudhury
![Page 2: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/2.jpg)
2
Context
Pervasive wireless connectivity
+
Localization technology
=
Location-based applications
![Page 3: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/3.jpg)
3
Location-Based Applications (LBAs)
For Example:
GeoLife shows grocery list when near Walmart
MicroBlog queries users at a museum
Location-based ad: Phone gets coupon at Starbucks
iPhone AppStore: 3000 LBAs, Android: 500 LBAs
![Page 4: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/4.jpg)
4
Location-Based Applications (LBAs)
For Example:
GeoLife shows grocery list when near Walmart
MicroBlog queries users at a museum
Location-based ad: Phone gets coupon at Starbucks
iPhone AppStore: 3000 LBAs, Android: 500 LBAs
Location expresses context of user
Facilitates content delivery
![Page 5: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/5.jpg)
5
Location is an IP addressAs if for content delivery
![Page 6: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/6.jpg)
6
Thinking about Localization
from an application perspective…
![Page 7: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/7.jpg)
7
Emerging location based apps need
place of user, not physical location
Starbucks, RadioShack,
Museum, Library
Latitude, Longitude
![Page 8: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/8.jpg)
8
Emerging location based apps need
place of user, not physical location
Starbucks, RadioShack,
Museum, Library
Latitude, Longitude
We call this Logical Localization …
![Page 9: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/9.jpg)
9
Can we convert from
Physical to Logical Localization?
![Page 10: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/10.jpg)
10
Can we convert from
Physical to Logical Localization?
State of the Art in Physical Localization:
1. GPS Accuracy: 10m
2. GSM Accuracy: 100m
3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-100m
![Page 11: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/11.jpg)
11
Widely-deployable localization technologies
have errors in the range of several meters
Can we convert from
Physical to Logical Localization?
State of the Art in Physical Localization:
1. GPS Accuracy: 10m
2. GSM Accuracy: 100m
3. Skyhook (WiFi+GPS+GSM) Accuracy: 10m-100m
![Page 12: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/12.jpg)
12
Several meters of error is inadequate
to logically localize a phone
Physical Location
Error
![Page 13: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/13.jpg)
13
Several meters of error is inadequate
to logically localize a phone
RadioShackStarbucks
Physical Location
Error
The dividing-wall problem
![Page 14: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/14.jpg)
14
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
![Page 15: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/15.jpg)
15
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
![Page 16: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/16.jpg)
16
It is possible to localize phones by
sensing the ambience
Hypothesis
such as sound, light, color, movement, WiFi …
![Page 17: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/17.jpg)
17
Sensing over multiple dimensions extracts more
information from the ambience
Each dimension may not be unique,
but put together, they may provide a
unique fingerprint
![Page 18: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/18.jpg)
18
SurroundSense
Multi-dimensional fingerprint
Based on ambient sound/light/color/movement/WiFi
Starbucks
Wall
RadioShack
![Page 19: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/19.jpg)
19
B AC DE
Should Ambiences be Unique Worldwide?
FG
HJ
I
LMN
O
P
Q
Q
R
K
![Page 20: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/20.jpg)
20
Should Ambiences be Unique Worldwide?
B AC DE
FG
HJ
I
K
LMN
O
P
Q
Q
R
GSM provides macro location (strip mall)
SurroundSense refines to Starbucks
![Page 21: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/21.jpg)
21
+
Ambience Fingerprinting
Test Fingerprint
Sound
Acc.
Color/Light
WiFi
LogicalLocation
Matching
Fingerprint
Database
=
Candidate Fingerprints
GSM Macro Location
SurroundSense Architecture
![Page 22: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/22.jpg)
22
Fingerprints
Sound:
(via phone
microphone)
Color:
(via phone
camera)
Amplitude Values
-1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1
No
rma
lize
d C
ou
nt
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
Acoustic fingerprint
(amplitude distribution)
Color and light fingerprints on HSL space
Lig
htn
es
s1
0.5
0
Hue
0
0.5
1 00.2
0.40.6
0.81
Saturation
![Page 23: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/23.jpg)
23
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Moving
![Page 24: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/24.jpg)
24
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Moving
Queuing
![Page 25: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/25.jpg)
25
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Queuing Seated
Moving
![Page 26: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/26.jpg)
26
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Pause for product
browsing
Moving
![Page 27: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/27.jpg)
27
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Pause for product
browsing
Short walks between
product browsing
Moving
![Page 28: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/28.jpg)
28
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Walk more
Moving
![Page 29: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/29.jpg)
29
Fingerprints
Movement: (via phone accelerometer)
Cafeteria Clothes Store Grocery Store
Static
Walk more Quicker stops
Moving
![Page 30: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/30.jpg)
30
Fingerprints
Movement: (via phone accelerometer)
WiFi: (via phone wireless card)
Cafeteria Clothes Store Grocery Store
Static
ƒ(overheard WiFi APs)
Moving
![Page 31: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/31.jpg)
31
Discussion
Time varying ambience
Collect ambience fingerprints over different time windows
What if phones are in pockets?
Use sound/WiFi/movement
Opportunistically take pictures
Fingerprint Database
War-sensing
![Page 32: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/32.jpg)
32
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
![Page 33: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/33.jpg)
33
Evaluation Methodology
51 business locations
46 in Durham, NC
5 in India
Data collected by 4 people
12 tests per location
Mimicked customer behavior
![Page 34: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/34.jpg)
34
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
Ac
cu
rac
y (
%)
Cluster
Localization accuracy per cluster
![Page 35: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/35.jpg)
35
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
Ac
cu
rac
y (
%)
Cluster
Localization accuracy per cluster
Multidimensional
sensing
![Page 36: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/36.jpg)
36
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
Fault tolerance
Ac
cu
rac
y (
%)
Cluster
Localization accuracy per cluster
![Page 37: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/37.jpg)
37
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
Ac
cu
rac
y (
%)
Cluster
Localization accuracy per clusterSparse WiFi APs
![Page 38: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/38.jpg)
38
Evaluation: Per-Cluster Accuracy
Cluster
No. of Shops
1 2 3 4 5 6 7 8 9 10
4 7 3 7 4 5 5 6 5 5
No WiFi APsAc
cu
rac
y (
%)
Cluster
Localization accuracy per cluster
![Page 39: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/39.jpg)
39
Evaluation: Per-Scheme Accuracy
Mode WiFi Snd-Acc-WiFi Snd-Acc-Lt-Clr SS
Accuracy 70% 74% 76% 87%
![Page 40: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/40.jpg)
40
Evaluation: User Experience
Random Person Accuracy
Average Accuracy (%)
0 10 20 30 40 50 60 70 80 90 100
1
0.9
0.8
0.7
0.6
0.5
CD
F
0.4
0.3
0.2
0.1
0
WiFI
Snd-Acc-WiFi
Snd-Acc-Clr-Lt
SurroundSense
![Page 41: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/41.jpg)
41
Economics forces nearby businesses to be different
Not profitable to have 3 coffee shops
with same lighting, music, color, layout, etc.
SurroundSense exploits this ambience diversity
Why does it work?
The Intuition:
![Page 42: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/42.jpg)
42
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
![Page 43: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/43.jpg)
43
Limitations and Future Work
Energy-Efficiency
Localization in Real Time
Non-business locations
![Page 44: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/44.jpg)
44
Limitations and Future Work
Energy-Efficiency
Continuous sensing likely to have a large energy draw
Localization in Real Time
Non-business locations
![Page 45: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/45.jpg)
45
Limitations and Future Work
Energy-Efficiency
Continuous sensing likely to have a large energy draw
Localization in Real Time
User’s movement requires time to converge
Non-business locations
![Page 46: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/46.jpg)
46
Limitations and Future Work
Energy-Efficiency
Continuous sensing likely to have a large energy draw
Localization in Real Time
User’s movement requires time to converge
Non-business locations
Ambiences may be less diverse
![Page 47: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/47.jpg)
47
Contents
SurroundSense
Evaluation
Limitations and Future Work
Conclusion
![Page 48: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/48.jpg)
48
SurroundSense
Today’s technologies cannot provide logical localization
Ambience contains information for logical localization
Mobile Phones can harness the ambience through sensors
Evaluation results:
51 business locations,
87% accuracy
SurroundSense can scale to any part of the world
![Page 49: SurroundSense: Mobile Phone Localization via Ambience ...nadeem/classes/cs752-S12/s... · Today’s technologies cannot provide logical localization Ambience contains information](https://reader033.fdocuments.in/reader033/viewer/2022042305/5ed0f2a52a742537f26e9edf/html5/thumbnails/49.jpg)
49
Questions?
Thank You!
Visit the SyNRG research group @
http://synrg.ee.duke.edu/