TF-Media meeting, March 2011University of Oslo
Streaming systems today... View a video sequentially
Once produced, video never changes
Make and distribute the same video to everybody
Emerging:− topic based composition using search− personalized playlists− recommendation− integration with social networking− anytime, everywhere...
timeline
TF-Media meeting, March 2011University of Oslo
Topic based composition
Query:- premier league - April 2010 - Liverpool- Goal by Steven Gerrard- 30 seconds duration Users select events
from multiple sources,played out as one video
TF-Media meeting, March 2011University of Oslo
Personalized video:
Personalized ordering
Query:- premier league - April 2010 - Liverpool- Goal by Steven Gerrard- 30 seconds duration
Users select eventsfrom multiple sources, arbitrary ordered,played out as one video
TF-Media meeting, March 2011University of Oslo
Recommendations and social networking
Personalized video:
Enabling user (re)publishing:− create directory service
(user generated content)− social network
Recommendations:− recommend personalized content− user interest profile stored−match user profile against
interesting content
TF-Media meeting, March 2011University of Oslo
Torrent-like HTTP streaming For load-balancing and scaling
multiple servers, taking the best from several worlds….
Downloads segments
Tracker manages information about segment locations
The user contacts the tracker for segment locations
Users send HTTP GET requests to download video segments
Video object:
TF-Media meeting, March 2011University of Oslo
Torrent-like HTTP streaming
playout time
quality
Based on experiments, we use 2-second segments(2-hour movie 3600++ small, indexed videos)
FFMPEG encoded:− H.264 (GOP = IP48I)− MP3− Custom made container
To support adaptation to available resources, each segment is coded in many quality levels
TF-Media meeting, March 2011University of Oslo
DAVVI: Idea
Present a 2-minutes video of highlights from last month games combined from
• goals by Dirk Kuyt• sliding tackles • tip over the bar• …
• TV broadcasters, etc. have huge repositories of sports content
o full videos, short events, highlights etc. o should be searchable
• Multimedia search and delivery systemsstill lacks precision and flexibility
TF-Media meeting, March 2011University of Oslo
Web-servers fromBBC, Yahoo, VG, …
video analysislive-text crawling
live commentary
DAVVI
search / recommendation
feedback
tran
scod
er/
chop
per
HTTP Get
Video segments
stor
age
web
-ser
vers
tracker
DAVVI: system architecture
TF-Media meeting, March 2011University of Oslo
Annotation: sports event analysis
Audio-video analysis is difficult - why so hard?− Video data at 25 fps, an event may last 2,000 frames− Variation among & within sport broadcasts− Complex video quality, camera angles and on screen graphics
− Many different events to detect, e.g., in the context of soccer• yellow cards / red cards / goals / penalties / free kicks / fouls / corners / throw-ins / tackles / headings / passes / player numbers...
− Identify the beginning and end of the event− Find all the events – not miss any, no false positives
− Computationally expensive
How can an event be identified??
TF-Media meeting, March 2011University of Oslo
Annotation: sports event analysis
Event identification figures:− Huang et al. (U. Illinois) – text & video analysis (2000): 57%− Hanjalic et al. (Delft U.)– audio based analysis (2002): 52%− Sadlier et al. (DCU) – audio & video analysis (2005): 64%
Initial evaluation of visual/aural approach we developed for iAD: 67% - 83%
TF-Media meeting, March 2011University of Oslo
Annotation: live text commentaries Many online TV-stations and newspapers
provide live text commentaries
DAVVI uses a semi-automatic live-text crawler and parser to improve the automatic annotations
uk.eurosport.yahoo.com:
news.bbc.co.uk:
TF-Media meeting, March 2011University of Oslo
Search and recommendation Solr/Lucene open-source search
engine which has indexed thevideos
Users can query for video clips using a rich set of keywords, specifying values for tags or as free text
Each result is returned as a playlist of video segments, and playlists can be combined to make an topic-based, personalized video
The playlist describing the personalized video can be submitted to the social network
TF-Media meeting, March 2011University of Oslo
DAVVI demo systemsearch box automatically generate an X-minute playlist
search results – horizontally
scrollable
video quality
indicator
player controls
playlist generated by drag-and-drop or automatically
generated
each clip can be
adjusted
textual description of
the event which can be
expanded
TF-Media meeting, March 2011University of Oslo
vESP: Idea
Present a video of explanations about TCP congestion control techniques combining slides from talks/lectures given by
• Van Jacobson• Vinton G. Cerf • Mark Allman• Jitendra Padhye• …
• Companies/Schools/Universities/etc. have huge repositories of presentation content
o presentations, training videos, etc. o must be searchableo part of the content is multimedia
• Enterprise multimedia search still lacks precision and flexibility
TF-Media meeting, March 2011University of Oslo
tran
scod
er/
chop
per
talk transcript indexing
HTTP Get
Video segments
stor
age
web
-ser
vers
tracker
search
video analysis
slide indexing
vESP platform
TF-Media meeting, March 2011University of Oslo
vESP platform
Slide number
Start End
1 0:00 2:34
2 2:34 5:43
Query: “Windows 7”Slides
Transcript
Slide timing
Video of presentation
Search index
Video serverCustom presentation
Search results
TF-Media meeting, March 2011University of Oslo
vESP custom video presentation
PPT-file A, slide 3 PPT-file D, slide 2
Video-file: A, seg. 161 - 198
PPT-file C, slide 4
Video-file: D, seg. 20 - 28 Video-file: C, seg. 40 - 61
TF-Media meeting, March 2011University of Oslo
vESP demo system
slide playlist generated by drag-and-drop or
automatically generated
video presentationselect slides for playlist
document preview
TF-Media meeting, March 2011University of Oslo
Summary
DAVVI and vESP are scalable prototypes that−
− give you a new way to access video content
− integrate video streaming, search, personalization and recommendation(with social networking potential)
− well evaluated by subjective assessment group
Our next generation systems aim for 3D and free-view video experiences
TF-Media meeting, March 2011University of Oslo
Questions?? Comments??
Contact information:
Pål Halvorsen
[email protected]://home.ifi.uio.no/paalh
Top Related