ANSWERING COMPLEX LOCATION-BASED QUERIES WITH CROWDSOURCINGKarim Benouaret
Raman Valliyur-Ramalingam,
François Charoy
Inria – Université de Lorraine – CNRS
LORIA
Service and Cooperation Team
Nancy
?
Ask the crowd to contribute
How to do that Cost Effectively ?
• Express the problem (the query)• Transform it in something executable• Manage the execution• Evaluate the result
A Query• <
Object=roads, Context=need repair, Location=Nancy, Assessment = {not damaged, damaged, very
damaged}, Start Date = 10/19/2013, End Date = 10/25/2013, Strategy=Deadline
>
Collect
Clustering
Select
Assess
A Process• 3 crowdsourcing activities
Strategies
• Deadline• One after the other
• Buffer• Start the voting activities
when k photo are available
• FIFO• Start the voting activity
as soon as a data is available
• Wait for k/2 vote
C S A
C
S
A
C
S
A
Experimentation• Understand the behavior of each strategy
• Subset of the Gowalla Data Set• Checkins at different places
• The ground truth is generated• Participants have a probability to give a wrong answer.• Variable
• Number of days of the experiment• Number of votes required for each place and photo (k) to be
selected
Number of results vs Number of days
Evolution of the number of results
Quality vs Duration
Quality vs Number of vote
Conclusion• Promising Preliminary results• Interpretation of context aware crowdsourcing queries
requires more work• Crowdsourcing process orchestration is difficult
• Large scale• Not sequential ?
• Different strategies lead to different results• Quality vs number of results
• The problem of evaluation is an issue
Current work• Implementation of the process on a real BPM Systems• Deployment on AWS EC2 and S3• Prepare experimentation with the Lorraine Smart City
Living Lab
Questions ?
Current Structure of the system
Orchestration Engine
Data Production
Crowd Management
Service
Data Quality Service
Network serviceMobile app
serviceTask
Management
Collection
Selection
Assessment
Top Related