So human presentation
-
Upload
piero-fraternali -
Category
Technology
-
view
512 -
download
0
description
Transcript of So human presentation
CUbRIK Presentation 11/5/2013
Building social graphs from images through expert-based crowdsourcing
M. Dionisio, P. Fraternali, D. Martinenghi, C. Pasini, M. Tagliasacchi, S. Zagorac (Politecnico
Di Milano, Italy)
E. Harloff, I. Micheel, J. Novak (European Institute for Participatory Media,
Germany)
1/5/2013 CUbRIK Presentation 2
The CUbRIK project
CUbRIK is a research project financed by the European Union whose main goals are:1. Advance the architecture
of multimedia search2. Exploit the human
contribution in multimedia search
3. Use open source components provided by the community
4. Start up a search business ecosystem
1/5/2013 CUbRIK Presentation 3
The CUbRIK architecture
The CUbRIK architecture is layered in four main tiers
1. Content and user acquisition tier
2. Content processing tier3. Query processing tier4. Search tier
1/5/2013 CUbRIK Presentation 4
History Of Europe use case
HoE Dataset(3924 pictures
shot from the end of World War II to the most recent
years of EU history)
Automatic face
recognition tool+
Crowdsourced validation
of face matches
Social Graph
1/5/2013 CUbRIK Presentation 5
Content processing pipeline
In the initial proof of concept we designed a prototype for a face recognition service that combined automatic mechanisms for face detection/recognition and a general purpose crowd.
Group photos
Face detection
Bounding boxes
Face matching
Annotated portraits
Face detection
Bounding boxes
Top – 10 similarities for crowd validation
1/5/2013 CUbRIK Presentation 6
Limits of a purely automatic processing
False negatives
False positives
1/5/2013 CUbRIK Presentation 7
Limits of a purely automatic processing
Matching score = 0.185
Matching score = 0.210
The matching score between two faces of the same person is not always the highest
one
1/5/2013 CUbRIK Presentation 8
Using general purpose crowds We interfaced a general purpose crowd for the validation
of the top-10 matches.
1/5/2013 CUbRIK Presentation 9
Results of the first proof of concept
574 faces extracted from group photos Only 17% of them were identified by the
crowd Of this 17% the 66% of the matches were
correct The automatic tool identified the 80% of the
faces correctly
1/5/2013 CUbRIK Presentation 10
Results of the first proof of concept
These weak results were influenced by several factors:
1. Influence of image taking times2. Limited size of the ground truth3. Image resolution constraints4. Replicability and trustworthiness of the
results
1/5/2013 CUbRIK Presentation 11
Interfacing the expert based crowd
The deficiencies encountered using a general purpose crowd can be overcome by adopting an expert-based crowdsourcing.
combined implicit and explicit expert-based crowdsourcing
interface
1/5/2013 CUbRIK Presentation 12
Interfacing the expert based crowd
Indications suggest that the expert-based strategy can succeed:
1. Experts’ knowledge can overcome the drawbacks both of the automatic tool and of the general purpose crowd
2. They can use the already existing community means to contact colleagues and cooperate to fulfill the task.
1/5/2013 CUbRIK Presentation 13
Interfacing the expert based crowd
Thank you!