MUMIS User Group Workshop P. Wittenburg Max-Planck-Institut für Psycholinguistik Nijmegen.
-
Upload
alex-hawksley -
Category
Documents
-
view
212 -
download
0
Transcript of MUMIS User Group Workshop P. Wittenburg Max-Planck-Institut für Psycholinguistik Nijmegen.
MUMIS
User Group Workshop
P. Wittenburg
Max-Planck-Institut für PsycholinguistikNijmegen
Agenda
• 11.00 Introduction Peter WittenburgASR Aspects Helmer
StrikNLP/IE Aspects Thierry
Declerck• 11.45 Demo of Prototype Peter Wittenburg• 12.30 Lunchbreak• 13.15 Potential of MUMIS Technology Discussion• 14.00 Typical Questions Discussion• 15.00 Other Points of Interest Discussion• 15.30 End
Objectives
MUMIS is about
• Technology development to automatically index lengthy multimedia recordings
Find and mark interesting scenes
• Technology development to exploit indexed multimedia archives
Search for interesting scenes and play them via Internet
• Test Domain: Soccer Games / UEFA Tournament 2000
Off-line Tasks
Indexing by
• Automatic Speech Recognition (Radio/TV Broadcasts)
Transfer the speech signals into texts automatically
• Natural Language ProcessingTake all texts (newspapers, speech transcript, tickers, ...)and determine interesting scenes
• Create a database with formal annotations
• Use video processing to adjust time marks
Off-line Tasks
indexed video recording
Radio Commenting3 Languages
Radio Commenting3 Languages
Radio Commenting3 Languages
Audio Commenting (TV, Radio)3 Languages
NewspaperText
NewspaperText
NewspaperText
NewspaperTexts
3 Languages
NewspaperText
NewspaperText
NewspaperText
Close Captions3 Languages
Freekick Goal Pass Defense
17 min 18 min 24 min 28min
Foul Freekick Dribbling
Kohler Basler Matthäus Wörns
Basler Bierhoff
25 m 25 m 60 m
Event Table
Event ID Time Player1 Player2 Location Destination Cause Consequence Count RefEnd-Signal ID Time Ref
Offside ID Time Player1 Location RefGoal-Kick ID Time Player1 Player2 Location Destination RefStop-of-Attack
ID Time Player1 Location Cause Consequence Ref
Kick-Off ID Time Player1 Player2 RefChest Ball ID Time Player1 Player2 Location Destination Consequence RefWall Pass ID Time Player1 Player2 Location Destination Consequence RefDribbling ID Time Player1 Location Consequence Ref
Corner ID Time Player1 Player2 Location Destination Cause Consequence RefOwn-Goal ID Time Player1 Location Cause Score RefThrow-In ID Time Player1 Player2 Location Destination Cause Consequence RefNice Ball ID Time Player1 Player2 Location Destination Consequence Ref
Foul ID Time Player1 Player2 Location Cause Consequence RefFreekick ID Time Player1 Player2 Location Destination Cause Consequence Ref
Yellow Card ID Time Player1 Player2 Cause RefHalftime Signal
ID Time Ref
Hands Ball ID Time Player1 Location Cause Consequence RefHead Ball ID Time Player1 Player2 Location Destination Consequence Ref
Top-Post-Ball ID Time Player1 Location Cause RefPass ID Time Player1 Player2 Location Destination Consequence Ref
Post Ball ID Time Player1 Location Cause RefSave ID Time Player1 Player2 Location Cause Ref
Red Card ID Time Player1 Player2 Cause Consequence RefBicycle Kick ID Time Player1 Location Destination Consequence Ref
Sliding ID Time Player1 Player2 Location Consequence RefPenalty ID Time Player1 Player2 Cause Consequence Ref
Goal ID Time Player1 Player2 Cause Score RefShot-on-Goal ID Time Player1 Player2 Location Cause Consequence RefKeeper-Save ID Time Player1 Player2 Location Cause Ref
Injury ID Time Player1 Player2 Location Cause Consequence RefChange ID Time Player1 Player2 Cause Ref
On-line Tasks
Searching and Displaying
• Search for interesting events with formal questions
Give me all goals from Overmars shot with his head in 1. Half.
Event=Goal; Scorer=Overmars; Cause=Head; Time<=45
• Indicate hits by thumbnails & let user select scene
• Play scene via the Internet & allow scrollingOf course: slow motion, fast play, start/stop, etc
On-line Tasks
Knowledge GuidedUser Interface
&Search Engine
München - Ajax1998
München - Porto1996
Deutschland - Brasilien1998
PlayMovie
Fragmentof that Game
Freekick Goal Pass Defense
17 min 18 min 24 min 28min
Foul Freekick Dribbling
Kohler Basler Matthäus Wörns
Basler Bierhoff
25 m 25 m 60 m
On-line SW Architecture
first interaction with Web-server
download of appletcontact between applet
and distributed Java server via RMI
contact via media server via RMI and media streaming
annotations from DB or file server
JMF serves for merging of streams on client
Abstract Corpus Model
this kernel is operational(distributed and local)
On-line HW Architecture
• efficient & reliable storage management (near-line capacity, media change, 2. Location)
• high storage capacity (n TB, 1 h MPEG1 = 1 GB)• powerful media servers / powerful network
2nd locationMirror site
RAID
TapeLibrary3.5 TB
FC Switch
Media Server
Media Server
GB Switch
Internet
MM Stations Notebook
1Gbps
100Mbps
Gb-Switch
TapeLibrary25 TB
Gain
• What gets lost? Is it necessary?• Potential: direct Internet Service, less
dependencies
Current Procedure MUMIS Procedure
Manual Video Annotation Automatic Video Annotation and DB
IntegrationIntegration Central DB
Query via PC Query via PC
Results on PC
Results on PCAnd
Select & Play
Contact Video Archive
Get Video Tapes
Search on Tape on VCR
Segment & Play
Formal Aspects
• Start: 1.7.2000• Duration: 30 Months• EC-Funds: 2.4 Mio Euro
Consortium• CTIT University of Twente, Enschede, NL NLP/IE• TSI University of Nijmegen, Nijmegen, NL ASR• DFKI Saarbrücken, D NLP/IE• MPI Nijmegen, NL Online
Part• DCS University of Sheffield, UK
NLP/IE• ESTEAM Gothenburg, SE (location Athens, GR)
Translation Software
• VDA Hilversum, NLDissemination
Acknowledgements
• UEFA
• DFB, FA, KNVB
• EBU, WDR, NOS