Solution: Contact Sheets€¦ · Solution: Contact Sheets Two key factors: Cloud Older, Uploaded...
1
A Contact Sheet Approach to Searching Untagged Images on Smartphones Wolfgang Richter * , Kiryong Ha * , Alok Shankar * , Ardalan Amiri Sani ꝉ , Jan Harkes * , Lin Zhong ꝉ , Mahadev Satyanarayanan * ( * Carnegie Mellon University, ꝉ Rice University) Problem Low Fidelity Evaluation How do we efficiently search mobile devices in near real-time? Lessons and Contributions Solution: Contact Sheets Two key factors: Cloud Older, Uploaded Recent, not uploaded 0 50 100 150 200 250 300 350 400 Full 640x480 320x240 160x120 80x60 40x30 20x20 Bandwidth (KB) Fidelity To save energy and bandwidth, we degrade objects being searched and pull full fidelity versions in a just-in-time search. Diamond Modifications MIST Android Application C2DM Polling Camera UI Local Storage Mobile API Cloud API MIST Dataretriever Diamond GUI Session Server diamondd 0 2000 4000 6000 8000 10000 12000 14000 16000 18000 Full Fidelity 640x480 320x240 160x120 80x60 40x30 20x20 Energy Consumption (mJ) Fidelity No Batching Batching • Released Android Market App • Python Flask Web Application • Mobile API via HTTP • Cloud API via HTTP • Full Fidelity Retrieval • Push-based Google C2DM • Polling-based HTTP • Just-in-Time Search How far can we degrade images in a search before it loses meaning? Dataset: • 12, 963 Flickr Scraped Images Fidelity Reduction: • Resolution • JPEG Quality Factor Benchmarked Three Computer Vision Algorithms: • No One-Size-Fits-All • Significant Energy Savings • Significant Bandwidth Savings Lessons Help Us: Energy Bandwidth Potential Savings? Contributions 0 0 .2 0 .4 0 .6 0 .8 1 640x480 320x240 160x120 80x60 40x30 20x20 Filter Accuracy Fidelity 0.991 0.899 100 95 75 50 25 0 0 .2 0 .4 0 .6 0 .8 1 640x480 320x240 160x120 80x60 40x30 20x20 Filter False Negative Rate Fidelity 0.040 100 95 75 50 25 0 0.2 0.4 0.6 0.8 1 640x480 320x240 160x120 80x60 40x30 Filter Accuracy Fidelity 0.907 0.803 100 95 75 50 25 0 0.2 0.4 0.6 0.8 1 640x480 320x240 160x120 80x60 40x30 Filter False Negative Rate Fidelity 0.124 0.042 100 95 75 50 25 0 0 .2 0 .4 0 .6 0 .8 1 640x480 320x240 160x120 80x60 Filter False Negative Rate Fidelity 0.575 0.281 100 95 75 50 25 0 0 .2 0 .4 0 .6 0 .8 1 640x480 320x240 160x120 80x60 Filter Accuracy Fidelity 0.650 0.424 100 95 75 50 25 RGB Histogram Face Detection Brightness
Transcript of Solution: Contact Sheets€¦ · Solution: Contact Sheets Two key factors: Cloud Older, Uploaded...
A Contact Sheet Approach toSearching Untagged Images on Smartphones
Wolfgang Richter*, Kiryong Ha*, Alok Shankar*, Ardalan Amiri Saniꝉ, Jan Harkes*, Lin Zhongꝉ, Mahadev Satyanarayanan*
(*Carnegie Mellon University, ꝉRice University)
Problem
Low Fidelity Evaluation
How do we efficiently search mobile devices in near real-time?
Lessons and Contributions
Solution: Contact Sheets
Two key factors:
CloudOlder,Uploaded
Recent,not uploaded
0
50
100
150
200
250
300
350
400
Full 640x480 320x240 160x120 80x60 40x30 20x20
Bandw
idth
(K
B)
Fidelity
To save energy and bandwidth, we degrade objects being searched and pull full fidelityversions in a just-in-time search.
Diamond ModificationsMISTAndroid Application
C2DM
PollingCamera
UI
Local Storage
Mobile API
Cloud API
MISTDataretriever Diamond GUI
SessionServerdiamondd
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
Full Fidelity 640x480 320x240 160x120 80x60 40x30 20x20
Energ
y C
onsu
mptio
n (
mJ)
Fidelity
No BatchingBatching
• Released Android Market App• Python Flask Web Application • Mobile API via HTTP • Cloud API via HTTP• Full Fidelity Retrieval • Push-based Google C2DM • Polling-based HTTP• Just-in-Time Search
How far can we degrade images in a search before it loses meaning?
Dataset: • 12, 963 Flickr Scraped ImagesFidelity Reduction: • Resolution • JPEG Quality FactorBenchmarked Three Computer Vision Algorithms:
• No One-Size-Fits-All• Significant Energy Savings• Significant Bandwidth Savings
Lessons
Help Us:
Energy Bandwidth Potential Savings?
Contributions
0
0. 2
0. 4
0. 6
0. 8
1
640x480 320x240 160x120 80x60 40x30 20x20
Filt
er
Acc
ura
cy
Fidelity
0.991
0.899
10095755025
0
0. 2
0. 4
0. 6
0. 8
1
640x480 320x240 160x120 80x60 40x30 20x20
Filt
er
Fa
lse
Ne
ga
tive
Ra
te
Fidelity
0.040
10095755025
0
0. 2
0. 4
0. 6
0. 8
1
640x480 320x240 160x120 80x60 40x30
Filt
er
Acc
ura
cy
Fidelity
0.907
0.803
10095755025
0
0. 2
0. 4
0. 6
0. 8
1
640x480 320x240 160x120 80x60 40x30
Filt
er
Fa
lse
Ne
ga
tive
Ra
te
Fidelity
0.124
0.042
10095755025
0
0. 2
0. 4
0. 6
0. 8
1
640x480 320x240 160x120 80x60
Filt
er
Fa
lse
Ne
ga
tive
Ra
te
Fidelity
0.575
0.281
10095755025
0
0. 2
0. 4
0. 6
0. 8
1
640x480 320x240 160x120 80x60
Filt
er
Acc
ura
cy
Fidelity
0.650
0.424
10095755025
RGB Histogram Face DetectionBrightness
