Post on 01-Jun-2020
Global Media Monitoring http://eventregistry.org/
Marko Grobelnik
Marko.Grobelnik@ijs.si
Artificial Intelligence Lab, Jozef Stefan Institute
Ljubljana, Slovenia
Contributions from Gregor Leban, Blaz Fortuna, Janez Brank, Jan Rupnik,
Andrej Muhic, Aljaz Kosmerlj, Evgenia Belyaeva, Pat Moore
GESIS Computational Social Science Winter Symposium 2016, Nov 30th 2016, Cologne
Outline
• Introduction
• Collecting Media Data
• Document Enrichment
• Cross-linguality
• News Reporting Bias
• Event Representation
• Event Visualization
• Event API
• Future/Ongoing Projects
Introduction
What questions we’ll try to answer?
• Where to get global media data?
• What is extractable from media documents?
• How to connect information across languages?
• What is an event?
• How to approach diversity in news reporting?
• How to query & visualize global event dynamics?
Systems/Demos used within the presentation
• NewsFeed (http://newsfeed.ijs.si/) • News and social media crawler
• Wikifier (http://wikifier.org/) • Language and Semantic annotation
• XLing (http://xling.ijs.si/ • Cross-lingual document linking and categorization
• Event Registry (http://eventregistry.org/) • Event detection and topic tracking
Main stream news
Blogs
Cross-lingual article matching
Article semantic annotation
Event formation
Cross-lingual cluster matching Event registry
API Interface
Event info. extraction
Input data
Pre-processing steps
Event construction
Event storage & maintenance
Extraction of date references
Article clustering
Identifying related events
Detection of article duplicates
Global Media Monitoring pipeline
GUI/Visualizations
http
://EventR
egistry.org
Where to get references to news publishers? • Good start is Wikipedia list of newspapers:
• http://en.wikipedia.org/wiki/Lists_of_newspapers
From a newspaper home-page to an article http://www.nytimes.com/ HTML
RSS Feed (list of articles)
Article to be retrieved
Example: North Rhine Westphalia Local Newspapers Aachener Nachrichten Aachener Zeitung Aegidienberger Bote Ahlener Zeitung Sonntags Rundblick Altenaer Kreisblatt Sonntags Rundblick Werbepost Bergisches Handelsblatt Bergisches Sonntagsblatt Neue Westfalische Zeitung Bocholt Borkener Volksblatt Die Tageszeitung Bonner Rundschau General Anzeiger Rheinischer Merkur Blickpunkt Schaufenster Borkener Zeitung Briloner Anzeiger Bruhler Schlossbote
Hellerthaler Zeitung Allgemeine Zeitung Dattelner Morgenpost Lippische Landes Zeitung Ruhr Nachrichten Dulmener Zeitung Dusseldorfer Nachrichten Handelsblatt Rheinische Post Terz Westdeutsche Zeitung Emsdettener Volkszeitung Neue Ruhr Zeitung Blickpunkt am Sonntag Wochenende Neue Regionale Anzeigen Echo Westfalen Post Haller Kreisblatt Allgemeiner Anzeiger
Sonntags Rundblick Westfalischer Anzeiger Hertener Allgemeine Ibbenburener Volkszeitung Echo Nord Iserlohner Kreisanzeiger Kevelaerer Blatt Aktiv Express Kolner Stadt Anzeiger Kolnische Rundschau Krefelder Zeitung Der Neue Postillon Lippische Wochenschau Siegerland Kurier Lokale Informationen Der Patriot Ludenscheider Nachrichten Marler Zeitung Blickpunkt Schaufenster
Mendener Zeitung Mindener Tageblatt Munster am Sonntag Munstersche Zeitung Westfalische Nachrichten Hellerthaler Zeitung Neuss Grevenbroicher Zeitung Die Glocke Stimberg Zeitung Sauerland Kurier Sonntags Post Recklinghauser Zeitung Remscheider General Anzeiger Munsterlandische Volkszeitung Extra Blatt Rhein Sieg Rundschau Sontagsanzeiger Solinger Tageblatt Steinheimer Blickpunkt Hellweger Anzeiger Lokalanzeiger Waltroper Zeitung Suderlander Volksfreund Werbekurier Meerbuscher Nachrichten General Anzeiger
Example: news publishers from Rio de Janeiro
General News Media
• Agencia O Globo - agency
• Correio do Brasil - newspaper
• Destak - newspaper
• Extra Rio de Janeiro - newspaper
• Globo News - TV
• Ilha Noticias - newspaper
• Jornal Cidadania - newspaper
• Jornal da Barra - newspaper
• Jornal do Brasil - newspaper
• O Dia Rio de Janeiro - newspaper
• O Globo - newspaper
• Posto Seis - newspaper
• Povo do Rio - newspaper
• Tribuna da Imprensa - newspaper
Business Media
• Jornal do Commercio - newspaper
• Monitor Mercantil - newspaper
Entertainment Media
• Canal Brasil - TV
• Canal USA - TV
• Fox Life South America - TV
• Multishow - TV
• Piaui Magazine - magazine
• Rede Globo - TV
• TV Brasil - TV
Sports Media
• Diario Lance - newspaper
• Jornal dos Sports - newspaper
• Sportv - TV
• TV Esporte Interativo - TV
Society Media
• Folha Universal - newspaper
Collecting global media data
• Data collection service News-Feed • http://newsfeed.ijs.si/ • …crawling global main-stream and social media
• Monitoring • ~150k main-stream publishers (RSS feeds+special feeds) • ~250k most influential blogs (RSS feeds) • free Twitter feed
• Data volume: ~350k articles & blogs per day (+5M tweets)
• Languages: eng (50%), ger (10%), spa (8%), fra (5%), …
Downloading the news stream (1/2)
• The stream is accessible at http://newsfeed.ijs.si/stream/
• To download the whole stream continuously, you can use the python script (http://newsfeed.ijs.si/http2fs.py)
• The script does the following:
Downloading the news stream (2/2) • News Stream Contents and Format
• The root element, <article-set>, contains zero or more articles in the following XML format:
• …more details: • Mitja Trampus, Blaz Novak: The Internals Of
An Aggregated Web News Feed. Proceedings of 15th Multiconference on Information Society 2012 (IS-2012). [PDF]
Document Enrichment http://wikifier.org/
http://enrycher.ijs.si/
What can extracted from a document? • Lexical level
• Tokenization – extracting tokens from a document (words, separators, …) • Sentence splitting – set of sentences to be further processed
• Linguistic level • Part-of-Speech – assigning word types (nouns, verbs, adjectives, …) • Deep Parsing – constructing parse trees from sentences • Triple extraction – subject-predicate-object triple extraction • Name entity extraction – identifying names of people, places, organizations
• Semantic level • Co-reference resolution – replacing pronouns with corresponding names;
merging different surface forms of names into single entity • Semantic labeling – assigning semantic identifiers to names (e.g.
LOD/DBpedia/Freebase) including disambiguation • Topic classification – assigning topic categories to a document (e.g. DMoz) • Summarization – assigning importance to parts of a document • Fact extraction – extracting relevant facts from a document
Semantic annotation with Wikifier.Org (operates for 100 languages)
Annotated text in original language and English
Plain clean text
Original text
Key problem in semantic annotation (wikification) is concept disambiguation
Many Candidates
disambiguation
Enrycher (http://enrycher.ijs.si/) Plain text
Text Enrichment
Diego Maradona Semantics: owl:sameAs: http://dbpedia.org/resource/Diego_Maradona owl:sameAs: http://sw.opencyc.org/concept/Mx4rvofERZwpEbGdrcN5Y29ycA rdf:type: http://dbpedia.org/class/yago/ArgentinaInternationalFootballers rdf:type: http://dbpedia.org/class/yago/ArgentineExpatriatesInItaly rdf:type: http://dbpedia.org/class/yago/ArgentineFootballManagers rdf:type: http://dbpedia.org/class/yago/ArgentineFootballers
Robbie Keane Semantics: owl:sameAs: http://dbpedia.org/resource/Robbie_Keane rdf:type: http://dbpedia.org/class/yago/CoventryCityF.C.Players rdf:type: http://dbpedia.org/class/yago/ExpatriateFootballPlayersInItaly rdf:type: http://dbpedia.org/class/yago/F.C.InternazionaleMilanoPlayers
Extracted graph of triples from text
“Enrycher” is available as as a web-service generating Semantic Graph, LOD links, Entities, Keywords, Categories, Text Summarization, Sentiment
Enrycher Architecture
• Enrycher is a web service consisting of a set of interlinked modules…
• …covering lexical, linguistic and semantic annotations
• …exporting data in XML or RDF
• To execute the service, one should send an HTTP POST request, with the raw text in the body: • curl -d “Enrycher was developed at JSI, a research institute in Ljubljana. Ljubljana is the capital of Slovenia.” http://enrycher.ijs.si/run
Plain text
Annotated document
• Cross-linguality is a set of functions on how to transfer information across the languages • …having this, we can track information independent of the language borders
• Machine Translation is expensive and slow, so the goal is to avoid machine translation to gain speed and scale
• The key building block is the function for comparing and categorization of documents in different languages • http://XLing.ijs.si is an open web service to bridge information across 100
languages
Cross-linguality How to operate in many languages?
Languages covered by XLing (top 100 Wikipedia languages)
XLing (http://XLing.ijs.si) service for comparing and categorization of documents across 100 languages
Chinese Text
English Text
Automatically Extracted Keywords
Automatically Extracted Keywords
Similarity Between Two Documents
Selection Of 100 Languages
Example: Cross-lingual News Recommendation
• What local media (e.g. German) is writing about the topic we are reading in English?
• Usual fear of publishers: are users we are sending away coming back? • …evaluation shows they
all come back
Example: Social Media Recommendation (http://aidemo.ijs.si/xlike/hsdemo/)
• Usual problem publishers have what and when to publish to which social media channel? • …the demo shows what Bloomberg articles should be the most relevant
based on what is most ‘popular’ in a particular geographical area
News Reporting Bias http://aidemo.ijs.si/diversinews/
News Reporting Bias example
Detecting News Reporting Bias
• The task: • Given a news story, are we able to say from which news source it came?
• We compared CNN and Aljazeera reports about the same events from the war in Iraq • …300 aligned articles describing the same story from both sources
• The same topics are expressed in both sources with the following keywords: • CNN with:
• Insurgents, Troops, Baghdad, Iran, Militant, Police, Suicide, Terrorist, United, National, Hussein, Alleged, Israeli, Syria, Terrorism…
• Aljazeera with: • Attacks, Claims, Rebels, Withdrawing, Report, Fighters, President, Resistance, Occupation,
Injured, Army, Demanded, Hit, Muslim, …
DiversiNews iPad App (1/2)
• DiversiNews iPad App is using newsfeed.ijs.si and enrycher.ijs.si services
• …in its initial screen is shows list of current hot topics and current trending events
Hot Topics
Trending Events
DiversiNews iPad App (2/2) • DiversiNews “diversity search”
screen allows dynamic reranking of articles describing an event along three dimensions: • Geography – where is a content
being published from
• Subtopics – what are subtopics of an event
• Sentiment – what are good and what are bad news
• For each query it provides • Automatically generated
summary
• List of corresponding articles
Geography
Subtopics
Sentiment
Summary
Articles
New Bias Experiment
• 30 news publishers selected
• All articles from Dec 15th 2014 – Aug 15 2014
• General: BBC, USA Today, Daily Mail, The Guardian, Globe and Mail, Huffington Post, Washington Post, CNN Europe, The Independent, Time, The Hindu, The Moscow Times,
• Bussiness related: Business Insider, Boston Business Journal, Economic Times, Financial Times, WSJ Blogs
• Tech related: Gizmodo, GigaOM, The Next Web
• Foreign: ABC.es, El Mundo, Die Welt, Stern Magazine
Dimensions of news bias to be observed
• Analysis of article length differences
• Analysis of grammatical differences
• Readability differences
• Newswire citations
• Geographical bias
• Topic (category) bias
• Speed of reporting
• Predicting article news source
• Similarity between news publishers
Article length differences
Number of tokens in articles per topic
Number of tokens in article headlines
Grammatical differences (% of adjectives in articles)
Grammatical differences (% of adverbs in articles)
Readability differences
• Dale-Chall readability formula.
0,1579𝑑𝑖𝑓𝑓𝑖𝑐𝑢𝑙𝑡 𝑤𝑜𝑟𝑑𝑠
𝑤𝑜𝑟𝑑𝑠× 100 + 0,0496
𝑤𝑜𝑟𝑑𝑠
𝑠𝑒𝑛𝑡𝑒𝑛𝑐𝑒𝑠
• Flesch-Kincaid Reading Ease • This measure is a standard way for many US agencies for evaluating technical
documents.
206,835 − 1, 015 𝑡𝑜𝑡𝑎𝑙 𝑤𝑜𝑟𝑑𝑠
𝑡𝑜𝑡𝑎𝑙 𝑠𝑒𝑛𝑡𝑒𝑛𝑐𝑒𝑠− 84, 6
𝑡𝑜𝑡𝑎𝑙 𝑠𝑦𝑙𝑙𝑎𝑏𝑙𝑒𝑠
𝑡𝑜𝑡𝑎𝑙 𝑤𝑜𝑟𝑑𝑠
Readability in articles (Flesch-Kincaid test)
Readability in articles (Flesch-Kincaid test on Washington Post topics)
Publishers citing news agencies
Agencies Citations
Geographical bias
• What events are being reported by news publishers?
USA Today geographical coverage
ABC.es geographical coverage
“Die Welt” geographical coverage
Separability between news sources
• How well we can determine which news publisher is the author of the article
• Tested on pairs of news publishers
• Used 10.000 articles from each news publisher
• Removed a custom-built set of tokens (like huffpost, mailonline, ...)
• Used keywords as features, used 1.000 top ranking features
• Linear SVM
• Used 70% articles for training, 30% for testing
• Use accuracy as a distance measure use MDS for plotting
News source
Keywords
Time selfie, uber, rapper, minister, researchers, Hillary, sex, party, relationships
Washingon post
Maryland, redskins, Georgetown, Montgomery, reelection, capital, Arlington, officials, capitol
FOX News Jerusalem, Christians, minister, dynasty, officials, prosecutors
Daily mail Mr, uk, centre, behavior, football, Scotland, British, premier, Cameron
BBC Correspondents, Belfast, council, Scotland, UK, Wales, Ireland
CNN Editor, commentary, opinions, affiliate, according, London, official, Pakistan
Sydney Morning Herald
Australian, Australia, Melbourne, Fairfax, Canberra, rugby, Queensland, Brisbane,
Moscow times
Russian, Russia, Interfax, soviet, Barack, Petersburg, Siberia, Sergei, Kremlin, Duma, Yury, Oleg
DNA India Mumbai, indian, singh, Bollywood, Kapoor, cricket, crore, Bangalore, lakh, Pune, Kumar, Tata, Gandhi
Separability on reports about Ukraine
• Analyze articles reporting about Russia-Ukraine incident
• Selected articles • Mention Ukraine and Russia
• between Jan 15th, 2014 and June 1st, 2014
• Categorized into appropriate Dmoz category
• Selected publishers that reported about 500 articles
News source Keywords
Non-Russian sources
troops, allies, invasion, aggression, excellency, insurgents, investigation, defense, ethnic, seized
The Moscow Times
overthrow, authorities, western, election, khodorkovsky, detained, annexation, pressure, seizure, fanning, imposed, domestic
Similarity between news publishers regarding the events they cover • For each article we know the corresponding event in the Event
Registry
• Compute a list of events covered by each news publisher
• Compute the pairwise similarity between the lists
• We used cosine similarity as the similarity measure
What is an event? (abstract description)
• …more practical question: what definition of is computationally feasible?
• In general, an event is something which “sticks out” from the rest of information in some kind of (high dimensional) data space • …could be interpreted as an “anomaly” • …densification of data points (e.g. many similar documents) • …significant change of distribution (e.g. a trend on Twitter)
• In practice, the event could be: • A cluster od documents / change of a distribution in data
• Detected in an unsupervised way
• A fit to a pre-built model • Detected in a supervised way
How to represent an event?
• Baseline data for a news event is usually a cluster of documents • …with some preprocessing we extract linguistic and semantic annotations
• …semantic annotations are linked to ontologies providing possibility for multiresolution annotations
• Three levels of event representation: • Feature vector event representation:
• …light weight representation that can be easily represented as a set of feature vectors augmented with external ontologies – suitable for scalable ML analysis
• Structured event representation: • Infobox representation (slots filling) using open schema or event taxonomy
• Deep event representation • Semantic representation linked to a world-model (e.g. CycKB common sense knowledge)
– suitable for reasoning and diagnostics
Feature vector event representation
• Feature vectors easily extractable from news documents: • Topical dimension – what is being talked about? (keywords) • Social dimension – which entities are mentioned? (named entities) • Temporal aspect – what is the time of an event? (temporal distribution) • Geographical aspect – where an event is taking place? (location) • Publisher aspect – who is reporting? (publisher identifiers) • Sentiment/bias aspect – emotional signals (numeric estimates)
• Scalable Machine Learning techniques can easily deal with such representation • …in “Event Registry” system we use this representation to describe events
Example of “feature vector” event representation: Event Registry “Chicago” related events
Where? (geography)
When? (temporal distribution)
Who? (named entities)
What? (keyword/ topics)
Query: “Chicago”
Structured event representation
• Structured event representation describes an event by its “Event Type” and corresponding information slots to be filled
• Event Types should be taken from “Event Taxonomy”
• …at this stage of development this level of representation still requires human intervention to achieve high accuracy (Precision/Recall) extraction
• Example on the right – Wikipedia event infobox: • 2011 Tōhoku earthquake and tsunami
“Event Taxonomy” – preview to the current development
Prototype for event Infobox extraction: XLike annotation service
• The goal is to build a system for economically viable extraction of event infoboxes • …using crowd-sourcing
• …aiming at high Precision & Recall for a small cost
Event sequences & Hierarchical events
• Once having events identifies and represented we can connect events into “event sequences” (also called story-lines)
• “Event sequences” include events which are supposedly related and constitute a larger story
• Collection of interrelated events can be also organized in hierarchies (e.g. World Cup event consists from a series of smaller events)
An example event: Microsoft Windows 9
Similar events example: similar events to Microsoft Windows 9 event
Event sequence identification
Hierarchy of events
Example Microsoft hierarchy of events
Zoom-in Example Microsoft hierarchy of events
Event description through entities and Semantic keywords
Collection of events described through Entity relatedness
Collection of events described through trending concepts
Collection of events described through three level categorization
Events identified across languages
Collection of events described through Reporting dynamics
Collection of events described through a story-line of related events
Event Registry API
Python code to access “Event Registry”
• https://github.com/gregorleban/event-registry-python
Searching for events using Python
>>> from EventRegistry import *
>>> er = EventRegistry()
>>> q = QueryEvents()
# get events related to Barack Obama
>>> q.addConcept(er.getConceptUri("Obama"))
# and are related to issues in society
>>> q.addCategory(er.getCategoryUri("society issues"))
# and have been reported by the BBC
>>> q.addNewsSource(er.getNewsSourceUri("bbc"))
# return event details for first 30 events
>>> q.addRequestedResult(RequestEventsInfo(page = 0, count = 30))
# execute query and obtain results
>>> res = er.execQuery(q)
Result of the query
‘events’: ‘resultCount’: 122,
‘results’: [
'articleCounts': 'eng': 54.0, 'total': 54.0,
'categories': [...],
'concepts': [..., ...],
'eventDate': '2014-08-29',
'eventDateEnd': '',
'multiLingInfo': 'eng':
'title': ..., 'summary': …,
'uri': '1211229', 'wgt': 9.0
, …
]
Getting info for a particular event
# get information about event with ID 123
>>> q = QueryEvent("123");
# return concept labels in 3 languages
>>> q.addRequestedResult(RequestEventInfo(["eng", "spa", "slv"]))
# get 10 most central articles
>>> q.addRequestedResult(RequestEventArticles(0, 10))
# get information how articles about the event were trending
>>> q.addRequestedResult(RequestEventArticleTrend())
# get top keywords
>>> q.addRequestedResult (RequestEventKeywordAggr())
>>> eventRes = er.execQuery(q);
Searching for articles
>>> q = QueryArticles(); # articles should be from a particular time period >>> q.setDateLimit(datetime.date(2014, 4, 16), datetime.date(2014, 4, 28)) # they should mention apple >>> q.addKeyword("apple") # they should also mention iphone >>> q.addKeyword("iphone") # get top 30 articles that match criteria >>> q.addRequestedResult(RequestArticlesInfo(page=0, count = 30)); >>> res = er.execQuery(q)
Event Registry exports event data through API and RDF/Storyline ontology • API to search and export event information
• Export of all the system data in JSON
• Event data is exported in a structured form • BBC Storyline ontology • http://www.bbc.co.uk/ontologies/storyline/2013-05-01.html
• SPARQL endpoint: • http://eventregistry.org/rdf/search
• http://eventregistry.org/rdf/event/eventID • http://eventregistry.org/rdf/article/articleD • http://eventregistry.org/rdf/storyline/storylineID • Example: http://eventregistry.org/rdf/event/1234
Future / Follow-up projects
Some of the follow-up projects/research
• Understanding global social dynamics • How global society functions?
• Integrating text-based media with TV channels • …requires speech recognition, video processing, visual object recognition,
face recognition, …
• Event prediction / Event-Consequence prediction • …requires understanding of causality in the social dynamics and much more
• Micro-reading / Machine-reading • …full understanding of individual documents – the goal for 10+ years