2013.10.22 Tom De Nies - Assessing Content Value for Digital Publishing through Relevance and...
Click here to load reader
-
Upload
tdenies -
Category
Technology
-
view
398 -
download
1
description
Transcript of 2013.10.22 Tom De Nies - Assessing Content Value for Digital Publishing through Relevance and...
ELIS – Multimedia Lab
Tom De Nies Supervised by: Erik Mannens and Rik Van de Walle
Ghent University – iMinds – MMLab
http://users.ugent.be/~tdenies
@TomDeNies (this presentation has links, click them to find out more!)
Assessing Content Value for Digital Publishing through Relevance and
Provenance-based Trust
Presented at ISWC 2013 Doctoral Consortium
2
ELIS – Multimedia Lab
Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013
Digital publishers are facing an information overload … but they get less time to deal with it!
They need a way to select content that is valuable to themselves, and to their target audience!
Problem Description
3
ELIS – Multimedia Lab
Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013
Most work on quality assessment of machine-observed data
Only limited number of works[1] for human-generated content
State of the Art
Very diverse: worthiness, novelty, readability, pageranking, …
No integrated, automated approach to assess content value through multiple aspects.
4
ELIS – Multimedia Lab
Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013
Research Questions and Hypotheses
How can we automatically assess the value of content on the Web?
Relevance assessment? Trustworthiness assessment?
Through provenance + reputation?
Retrieval? Reconstruction?
Through context and semantic similarity
5
ELIS – Multimedia Lab
Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013
Proposed Approach
A contextual model is used to generate the content's relevance, reconstruct its provenance, and assess its trustworthiness.
6
ELIS – Multimedia Lab
Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013
Content Value depends on end-user and use case
makes a general evaluation infeasible
representative use cases! E.g. online news
Relevance: benchmarks and established metrics (e.g. MediaEval)
Provenance reconstruction:
Currently no benchmarks or gold standard datasets!
-> used preliminary dataset -> make our own (e.g. with Git2PROV)
Trust & Value: human evaluation & crowdsourcing (e.g. Amazon MT)
Evaluation Plan
7
ELIS – Multimedia Lab
Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013
Preliminary Results
- Linking videos based on textual content - Supervised method (2012): MAP 0.17 - Unsupervised method (2013): MAP 0.0375
HOWEVER: MAP is wrong metric! - Precision: 35% of top-10 links relevant
[APROVeD] - 420 news items - Reconstruct their provenance
(find the original source) - Using our clustering method - 73% of sources found - With 68.2% precision
[AVALON]
- 224 news abstracts - Newsworthiness criteria identified
(from expert literature) - Detect these criteria automatically - Detected with 83,9% precision
8
ELIS – Multimedia Lab
Assessing Content Value for Digital Publishing through Relevance and Provenance-based Trust Tom De Nies 22/10/2013
Reflections
Trust Relevance
valu
e
Combination Fine-grained Cross-domain
Benchmarks
- Our points of attention:
+ Our strong points:
Annotate uncertainty! [2] Supervised vs. unsupervised