SmartPhoto: A Resource-Aware Crowdsourcing Approach

for Image Sensing with Smartphones

Yi Wang, Wenjie Hu, Yibo Wu and Guohong CaoThe Pennsylvania State University

Photo Crowdsourcing Enabled by the popularity of smartphones

Equipped with cameras, sensors and network interfaces People are willing to share photos

The success of Flickr and Instagram A number of promising applications

Grassroots journalism, photo tourism, disaster recovery

Motivating Examples Post-earthquake recovery

First responders survey the area by taking photos Damaged/overloaded networks limit the bandwidth for

photo uploading Map service with virtual tours

Enhance user experience by showing street views Impractical to store and process billions of available photos

Key challenge: resource limitation

Outline

Introduction Photo utility model Max-utility with bandwidth constraint Achieving required utility with min-selection Testbed implementation Performance evaluation Summary

Photo Utility Model Characterize photo usefulness in a way that is both

meaningful and resource-friendly Different from traditional sensor coverage Utility: the amount of aspects a photo covers

Photo metadata Aspect coverage Coverage overlap

Outline

Introduction Photo utility model Max-utility with bandwidth constraint Achieving required utility with min-selection Testbed implementation Performance evaluation Summary

Max-Utility Problem Problem statement

With some known targets and photos, how to choose a given number of photos out of all the candidates to maximize the total utility?

Example: choose 3 out of 10 photos

Max-Utility Problem Conversion to maximum coverage problem

NP-hard!

Weighted maximum coverage problem

Max-Utility Problem Maximum coverage problem is NP-hard Greedy approximation

A multi-round selection process In each round, select the subset with the most weight

contribution to the total weight Once a subset is selected, the elements it covers are

removed from future consideration Approximation ratio

Outline

Introduction Photo utility model Max-utility with bandwidth constraint Achieving required utility with min-selection Testbed implementation Performance evaluation Summary

Min-Selection Problem Problem statement

With some known targets and photos, how to choose the minimum number of photos such that all the required intervals are covered?

Conversion to the NP-hard set cover problem Greedy approximation

In each round, select the subset with the most number of new elements

Approximation ratio

Outline

Introduction Photo utility model Max-utility with bandwidth constraint Achieving required utility with min-selection Testbed implementation Performance evaluation Summary

Metadata Acquisition Location: GPS Field-of-view: API Coverage range: depends on application; 50m as a

reference range Orientation: accelerometer, magnetic field sensor,

gyroscope

Improving Orientation Accuracy Hybrid method

Enhanced method Calibrate the result of hybrid method by an ortho-

normalization process Results

Occlusion & Out-of-Focus When the camera is focused at distance D, an object

only appears sharp if it is within range [Dnear, Dfar], where Dnear < D < Dfar

The length of the range is called depth-of-field (DOF)

Distance from camera to dictionary: 100cmLeft: Dnear =85cm, Dfar=105cmRight: Dnear =5cm, Dfar=10cm

Outline

Introduction Photo utility model Max-utility with bandwidth constraint Achieving required utility with min-selection Testbed implementation Performance evaluation Summary

Real-World Demo Max-utility vs. random selection

Performance of Max-Utility Left: our algorithm converges to the best achievable utility

much faster Right: our algorithm performs close to the best achievable

utility even when the bandwidth is heavily constrained

Performance of Min-Selection Left: our algorithm selects small number of photos to achieve

the required coverage, regardless of the increasing redundancy of related photos

Right: the increase of selected photos to cover more targets is slower for our algorithm

Summary SmartPhoto: a resource-aware framework to

optimize the selection of crowdsourced photos Photo utility model Optimization problems

Max-utility with bandwidth constraint Achieving required utility with min-selection Approximation ratio

Testbed based on Android smartphones Real-world demo and extensive simulations

Thank you!

http://mcn.cse.psu.edu

The paper and slides are also available at:http://www.cse.psu.edu/~yxw185

Online Max-Utility Problem Time is divided into transmission periods. Some new

photos are available in each period. Problem statement

Given the targets and the photos available at the beginning of each period, how to choose photos in each period such that the bandwidth constraint is satisfied and at the end of the period, all the selected photos up to now have the maximum utility

Solution Use the algorithm in max-utility problem to do greedy

selection in each period Aspects covered in previous periods are not considered

Performance of Online Max-Utility Left: for our algorithm, the utility is above 350 after t7 Right: our algorithm exploits the more number of new photos

each period to improve its result