2012.09.26.CUbRIK at CHORUS + (the progress)
-
Upload
cubrik-project -
Category
Technology
-
view
339 -
download
3
description
Transcript of 2012.09.26.CUbRIK at CHORUS + (the progress)
CUbRIK Project
Vincenzo Croce Engineering Ingegneria Informatica S.p.a.
“In the same way Stanley Kubrick was one of the most innovative and pattern-breaking Directors in the Cinema industry, so the CUbRIK project will introduce real innovative patterns inside the Multimedia search domain”
CUbRIK – International Workkshop Search Computing
36 month large-scale integrating project started in October 2011
partially funded by the European Commission’s 7th Framework ICT Programme for Research and Technological Development
www.cubrikproject.eu
CUbRIK at a glance
226/9/2012CUbRIK – International Workkshop Search
Computing
The CUbRIK Project Concept: the Cube Metaphor
26/9/2012CUbRIK – International Workkshop Search
Computing 3
project concept is based a on cube metaphor progress proceeds along three dimensions: technological, human and
business ones At the same time, the cube metaphor recalls the open
composability of the Rubik-cube puzzle
Objective: Advance the architecture of multimedia search
Multimedia search: coordinated result of three main processes: Content processing: acquisition, analysis,
indexing and knowledge extraction from multimedia content
Query processing: derivation of an information need from a user and production of a sensible response
Feedback processing: quality feedback on the appropriateness of search results
426/9/2012CUbRIK – International Workkshop Search
Computing
Objective: Advance the Human platform
The CUbRIK vision is to incorporate human capacities and social interaction
CUbRIK aims at harvesting different forms of human computation:
Implicit, such as emotion recognition Individual, such as intelligence and decision making in gaming Social, such as in crowdsearch
526/9/2012CUbRIK – International Workkshop Search
Computing
What achieved so far
Scenario definition History of Europe – entities Fashion Application
Requirements Architecture H-Demos
CUbRIK Pipelines CrowdSearch Frameworks
Q&A GWAP
26/9/2012CUbRIK – International Workkshop Search
Computing 7
Objective: Advance the architecture of multimedia search
Content processing, query processing and feedback processing phases implemented by means of independent components
Components are organized in CUbRIK pipelines Each application defines ad-hoc pipelines that provide
unique multimedia search capabilities in that scenario
826/9/2012CUbRIK – International Workkshop Search
Computing
Objective: Advance the architecture of multimedia search
Example of CUbRIK Pipeline
926/9/2012CUbRIK – International Workkshop Search
Computing
conflict - situation during the analysis where absence or contradictions of facts recorded in Annotations may arise
Conflict Resolutoin Task -> Application – GWAP -> application-performer pair –
Q&A application
CUbRIK architecture
1026/9/2012CUbRIK – International Workkshop Search
Computing
Objective: Humans in the loop
Problem: the uncertainty of analysis algorithms leads to low confidence results and conflicting opinions on automatically extracted features
Solution: humans have superior capacity for understanding the content of audiovisual material
State of the art: humans replace automatic feature extraction processes (human annotations)
Our contribution: integration of human judgment and algorithms
Goal: improve the performance of multimedia content processing
1126/9/2012CUbRIK – International Workkshop Search
Computing
Human activities
26/9/2012CUbRIK – International Workkshop Search
Computing 12
CrowdSearch Framework
26/9/2012CUbRIK – International Workkshop Search
Computing 13
Types of tasks•Automatic tasks•Crowd tasks: tasks that are executed by an open-ended community of performers
Types of tasks•Automatic tasks•Crowd tasks: tasks that are executed by an open-ended community of performers
GWAP Framework
26/9/2012CUbRIK – International Workkshop Search
Computing 14
Objective• extract the core aspects of a media refinement exploiting a videogame.
• task gamification - converting a task into a game• task injection - integrates the task goals inside pre existing games.
Challenges• Drive user’s motivation • Select the proper users
• based on their profile, competence and reliability.
• Validate the results provided by the crowd
Human-powered trademark logo detection demo
26/9/2012CUbRIK – International Workkshop Search
Computing 15
Problems in automatic logo detection: Object recognition is affected by the quality of the
input set of images
Uncertain matches, i.e., the ones with low matching score, could not contain the searched logo
Human-powered trademark logo detection demo
Goal: integrate human and automatic computation to increase precision and recall w.r.t. fully automatic solutions
1626/9/2012CUbRIK – International Workkshop Search
Computing
Human-powered trademark logo detection demo
26/9/2012CUbRIK – International Workkshop Search
Computing 17
Human-powered entities creations
26/9/2012CUbRIK – International Workkshop Search
Computing 18
Problems in automatic entities : Entity creation can be affected by wrong
synset or gloss relation between synset and concept relation between concepts entity names entity attribute attribute value
Human-powered entities creations
26/9/2012CUbRIK – International Workkshop Search
Computing 19
Solution: Word Games Popular Fun Educational use Played with:
Words real world entity names definitions (glosses)
Finalist - crossword: extremely popular fun words, clues, relations (in clues) maximum mistake coverage easy personalization themes or topics
Human-powered entities creations
26/9/2012CUbRIK – International Workkshop Search
Computing 20
Crossword generation select a layout select a slot pick a word check grid consistency
Feedback , check of Etype relations attributes values
Vincenzo Croce
Thanks for your attention
www.cubrikproject.eu
2126/9/2012CUbRIK – International Workkshop Search
Computing