ANSWERING COMPLEX LOCATION-BASED QUERIES WITH CROWDSOURCINGKarim Benouaret

Raman Valliyur-Ramalingam,

François Charoy

Inria – Université de Lorraine – CNRS

LORIA

Service and Cooperation Team

Nancy

?

Ask the crowd to contribute

How to do that Cost Effectively ?

• Express the problem (the query)• Transform it in something executable• Manage the execution• Evaluate the result

A Query• <

Object=roads, Context=need repair, Location=Nancy, Assessment = {not damaged, damaged, very

damaged}, Start Date = 10/19/2013, End Date = 10/25/2013, Strategy=Deadline

>

Collect

Clustering

Select

Assess

A Process• 3 crowdsourcing activities

Strategies

• Deadline• One after the other

• Buffer• Start the voting activities

when k photo are available

• FIFO• Start the voting activity

as soon as a data is available

• Wait for k/2 vote

C S A

C

S

A

C

S

A

Experimentation• Understand the behavior of each strategy

• Subset of the Gowalla Data Set• Checkins at different places

• The ground truth is generated• Participants have a probability to give a wrong answer.• Variable

• Number of days of the experiment• Number of votes required for each place and photo (k) to be

selected

Number of results vs Number of days

Evolution of the number of results

Quality vs Duration

Quality vs Number of vote

Conclusion• Promising Preliminary results• Interpretation of context aware crowdsourcing queries

requires more work• Crowdsourcing process orchestration is difficult

• Large scale• Not sequential ?

• Different strategies lead to different results• Quality vs number of results

• The problem of evaluation is an issue

Current work• Implementation of the process on a real BPM Systems• Deployment on AWS EC2 and S3• Prepare experimentation with the Lorraine Smart City

Living Lab

Questions ?

Current Structure of the system

Orchestration Engine

Data Production

Crowd Management

Service

Data Quality Service

Network serviceMobile app

serviceTask

Management

Collection

Selection

Assessment