WP5.4/3.1/4.2/5.5 meeting
29th of November 2007, DFKI
Agenda
9:00 - 9:15 welcome, intro and agenda 9:15 - 10:00 Jan Nemrava: current work in WP5.4/3.1
– video analysis, extended tickers coverage 10:00 - 10:30 Nikos Simou: FiRE reasoner 10:30 - 11:00 discussion on connection WP4.2 (FiRE
reasoner) with cross-wp work on 5.4/3.1 11:00 - 11:30 coffee break 11:30 - 13:00 wp5.5 - IPTC manipulator, KAT
integration, discussion about WP5.4 demo
13:00 - 14:00 lunch
Current work in WP5.4/WP3.1
• Textual Sources
• Video Analysis
• OCR
• Reasoning (FiRE)
• KAMC paper
WP5.4
• semantically annotate complementary resources
• extract semantically organized ‘cross-media features’from the aligned text and video data sets
• use in event type classification of video segments by use of fuzzy reasoning
Textual Resources
• Structured– Players names, numbers, attendance, place, date etc.
• Unstructured– 3 German and 3 English minute by minute reports for
all World Cup matches– SPRouT grammar differs for each language– Ita-Fra events extracted from text
shortest longest german english allplayeraction 31/31 104/112 76/89 230/315 308/404player 68/73 134/157 176/259 186/310 286/569
without duplicate concepts / all extracted concepts
OCR (with TUB and JRS)
• Time synchronization
• Additional primary resource
• Merging 16 frames to recognize moving vs. Static objects in the image
Text region detection example
audio-video analysis
• S
audio-video analysis (DCU)
• Crowd image detector• Speech-Band Audio Activity• On-Screen Graphics Tracking• Motion activity measure• Field Line orientation• Close-up
AUDIO3NEXT
0
20
40
60
80
100
120
0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
27 0 0 0 0 0 0
AUDIO3NEXT
audio-video analysis (DCU)
• Experiments with machine learning– Linear regression used to classify
• Midfield shot• Endzone shot• Other (replays, shot on crowd, etc)
– K-Nearest Neighbour to classify • Cornerkicks
– KL-Miner• Part of LispMiner developed at UEP• Time Series analysis
• By considering seconds as isolated instances we loose a lot of information
IPTC manipulator (UEP)• Extraction of semantic concepts from
unstructured texts using DFKI ontology based information extraction tool
IPTC and extraction problems
• Inability to insert multiple values into certain fields (in case they are duplicate) - city, country, date
• SPRouT identifies concepts that are in the text but are not depicted on the picture
• Which information are really relevant for annotation (information about future, past?)(possibility to use sentence syntax analysis to solve this problem)
• Domain dependency
ITPC manipulator demo
ITPC manipulator
• Swiss forward Alexander Frei (R) goes airborne challenged by Ukrainian midfielder Oleg Gusev (L) during the World Cup 2006 round of 16 football game Switzerland vs. Ukraine, 26 June 2006 at Cologne stadium. AFP PHOTO / ODD ANDERSEN
• A boy shows a football autographed by French midfielder Zinedine Zidane at the entrace of at Weserbergland Stadium in Hameln, 03 July 2006, where the French national team are training ahead of their next match. France on 01 July defeated Brazil 1-0 and will play Portugal 05 July in Munchen in a 2006 World Cup semifinal showdown. AFP PHOTO / DANIEL GARCIA
• SCHUG output??
Japan's Keisuke Tsuboi, left, and Singapore's Indra Sahdan Daud battle
for the ball during the first half in their Asian zone first round of
the World Cup soccer soccer qualifying match at Saitama Stadium in
Saitama, near Tokyo, Wednesday, Nov. 17, 2004. Japan leads Singapore 1-0 at half-time.
Discussion
• Demo during the K-Space review (Berlin)
• Complementary Resources Architecture– KAT implementation– COMM ontology with descriptors to express
results of comp. resources analysis.
• IPTC manipulator– Future plans
WP5.4 demo
Top Related