SAMBITS: Interactive Digital TV and Rich Media Streaming, based on DVB-MHP
description
Transcript of SAMBITS: Interactive Digital TV and Rich Media Streaming, based on DVB-MHP
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 1
SAMBITS:SAMBITS: Interactive DigitalInteractive Digital TV and TV and Rich Media Rich Media Streaming, based on DVB-MHPStreaming, based on DVB-MHP
Limitations of A/V-Streaming on the Internet today SAMBITS:
System System architecturearchitecture Combination of broadcast and Internet delivery Combination of broadcast and Internet delivery System proposal for Rich Media Streaming, using System proposal for Rich Media Streaming, using
MPEG-4 and MPEG-7 in DVB-MHPMPEG-4 and MPEG-7 in DVB-MHP Multimedia Studio SystemMultimedia Studio System: New production tools: New production tools Multimedia User TerminalMultimedia User Terminal SAMBITS Demonstration ScenariosSAMBITS Demonstration Scenarios
Eurovision Songcontest 2001Eurovision Songcontest 2001BBC‘s Big AlBBC‘s Big Al
Gerhard StollInstitut für Rundfunktechnik, München, Germany
ITU-Workshop on Multimedia Convergence, Geneva, 14 March, 2002
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 2
18%
82%
July 1998
The growths of A/V-streaminThe growths of A/V-streaming @g @ Internet: Internet: InterestInterest in in Internet-Streaming in Internet-Streaming in the USthe US
Source: edison media research, Arbitron, 2001
Nearly half of the Internet users in the US was listening to Internet-Radio already
35%
65%
January 2000
January 200110 %: during last week
21%: during last month56%
44%
10%: during last month
27%
73%
July 1999
4%: during last week
36%
20%
0%
5%
10%
15%
20%
25%
30%
35%
40%
Listen Audio Online Watch Video Online
22% ofUS citizens
areAudio
Streamies
12% ofUS citizens
areVideo
Streamies
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 3
TV and Internet:TV and Internet:Convergence of Broadcast & Telecom networksConvergence of Broadcast & Telecom networks
CoreCoreTransportTransport
ServiceServiceProviderProvider
UserUserTerminalTerminal
AccessAccess
Internet
Internet / Telecom Services
Core NetworkIP, ATM, SDH,
WDM
NodeNode
POTS/ISDNxDSL/fibrePMP/UMTS
POTS/ISDNxDSL/fibrePMP/UMTS
BroadcastNetwork
Broadcaster
HeadendHeadend
HFCLMDSHFC
LMDS
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 4
Broadcast@Internet: Broadcast@Internet: Progress with Seven-League Boots or just a Hype?Progress with Seven-League Boots or just a Hype?
Streaming Audio/Video on the Internet gets more and more attention,
but
Streaming of “events“ does not work properly today, because Internet is not a “broadcast network“ Multicasting is not implemented widely Serving millions of viewers is still expensive for the content
provider QoS on the Internet is not guaranteed
What happened during the EBU‘s Webcast of the Eurovision Song Contest 2000 & 2001? Please have a look!
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 5
InternetTV: InternetTV: Convergence of TV and InternetConvergence of TV and Internet
A successful web-experience with live-streams of events needs an intelligent management of - Internet (unicast) and - Broadcast (multicast) channels- Links between the two channels
A system is needed, which combines the broadcast and Internet delivery can manage both channels produces and tailors the content for both channels provides an easy to use multimedia terminal to the end-
user
Solution:
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 6
S A M B I T SS A M B I T S
System for Advanced
Multimedia Broadcast &
InformationTechnology
Services
Project Coordination:
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 7
Internet@DTV or DTV@Internet:Internet@DTV or DTV@Internet:„The value chain“„The value chain“
Content owner/ provider
Content owner/ provider
Broadcaster/ TV Company
Broadcaster/ TV Company
USERUSER
Set-top boxSet-top boxNetwork
operator
Networkoperator
Dig. Platformservice provider
Dig. Platformservice provider
Content owner
/ provider
Content owner
/ providerPortalPortal
EPGEPG
SoftwareSoftware
CACA
ISPISP TelcoTelco PCPC
DTV
Internet
ContentCreation
ContentCreation
PackagingAgg.tion
Navigation
PackagingAgg.tion
Navigation
Multiplex, Customer
interface
Multiplex, Customer
interfaceDeliveryDelivery
Usage, Consumer
device
Usage, Consumer
device
Internet@DTV or DTV@Internet : How to cross the DTV bridge?Internet@DTV or DTV@Internet : How to cross the DTV bridge?
Involvement
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 8
SAMBITS: SAMBITS: providing a perfect TV and Web providing a perfect TV and Web experience for the viewerexperience for the viewer
Combines MPEG-2, MPEG-4, MPEG-7, DVB-MHP and the Internet for high-quality interactive multimedia broadcasting, including Rich Media Streaming
Develops tools for integration of broadcast and internet value chain
Enables interactive personalised information through broadcast and internet channels
Objectives:
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 9
SAMBITS:SAMBITS:Concept of complete System and Reference ChainConcept of complete System and Reference Chain
BroadcastService
BroadcastService
InternetService
InternetService
Broadcast ChannelDVB-S, DVB-C, DVB-T
Broadcast ChannelDVB-S, DVB-C, DVB-T
Internet ChannelPSTN/ISDN,
Cable (Coax & Fiber)Satellite
Internet ChannelPSTN/ISDN,
Cable (Coax & Fiber)Satellite
SAMBITSIntegratedReceiverDecoder
SAMBITSIntegratedReceiverDecoder
Bro
adca
st N
etw
ork
Ad
apte
rIn
tern
et N
etw
ork
Ad
apte
r
Inte
rnet
In
terf
ace
Mo
du
leB
road
cast
In
terf
ace
Mo
du
le
ForwardInteraction
path
ReturnInteraction
path SAMBITSUser Terminal
(STB)
Content Producer Content Provider Consumer
Local interaction
Local interaction
BroadcastService
BroadcastService
InternetService
InternetService
Broadcast ChannelDVB-S, DVB-C, DVB-T
Broadcast ChannelDVB-S, DVB-C, DVB-T
Internet ChannelPSTN/ISDN,
Cable (Coax & Fiber)Satellite
Internet ChannelPSTN/ISDN,
Cable (Coax & Fiber)Satellite
SAMBITSIntegratedReceiverDecoder
SAMBITSIntegratedReceiverDecoder
Bro
adca
st N
etw
ork
Ad
apte
rIn
tern
et N
etw
ork
Ad
apte
r
Inte
rnet
In
terf
ace
Mo
du
leB
road
cast
In
terf
ace
Mo
du
le
ForwardInteraction
path
ReturnInteraction
path SAMBITSUser Terminal
(STB)
Content Producer Content Provider Consumer
Local interaction
Local interaction
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 10
SAMBITS:SAMBITS:Systems for Advanced Multimedia Broadcast and IT ServicesSystems for Advanced Multimedia Broadcast and IT Services
Main Directions of the ProjectMain Directions of the Projectdeveloped a multimedia studio system (example
implementation) for real-time and offline content manipulation, annotation/event characterisation and insertion/integration
developed a consumer type multimedia terminal interactive multimedia services based on MPEG-4 and
MPEG-7 including synchronising broadcast and Internet content
demonstrated integrated (DVB/Internet) services using a consumer type terminalEurovision Song Contest 2001 Scientific program: Story of “Big Al”
Project duration: 1 January 2000 – 31 December 2001
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 11
SAMBITS:SAMBITS: Concept of Interactive Concept of Interactive Programme Production Sub-SystemProgramme Production Sub-System
Content integration
Contentcreation/editing
Content display and pseudo- consumer
interaction
Raw A/V
Storage
Off-lineProducer
Internet
Broadcastcontent
Internetcontent
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 12
Interactive Programme Production SystemInteractive Programme Production System
ContentCapture
Tools
ContentCreation
Tools
EncodingDecoding
Tools
IndexingTools
Database& Search
Tools
ProgrammeCreation &
VisualisationTools
ContentCapture
ContentCapture
MHPWorkshop:Jpad Pro
MHPWorkshop:Jpad Pro
AdobePremiere
5.1
AdobePremiere
5.1
3DStudio
Max 3.0
3DStudio
Max 3.0
VRML toMPEG-4Scene
Translator
VRML toMPEG-4Scene
Translator
HTMLContent Creation
HTMLContent Creation
MPEG-2A/V
Codecs
MPEG-2A/V
Codecs
MPEG-4AV
Codec
MPEG-4AV
Codec
MPEG-7Codec
MPEG-7Codec
HighLevelIndex
HighLevelIndex
SegmentLevelIndex
SegmentLevelIndex
ImageSearch
Tool
ImageSearch
Tool
ProgrammeCreation
ProgrammeCreation
ProgrammeVisualisation
ProgrammeVisualisation
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 13
SAMBITS: Studio and Server SystemsSAMBITS: Studio and Server Systems
aa
ISP
InternetService
BroadcastService
On-lineService
ProvisionSubsystem
BroadcastIntermediate
Storage
WebIntermediate
Storage
Live MPEG-2 TS
TSPL
Live A/V
MPEG-2 TS
ServiceProvider
ContentProducer
InteractiveProgrammeProductionSubsystem
Off-LineService
ProvisionSubsystem
MPEG-2 PES
MPEG-4 ES
MPEG-4 File
MPEG-7
Webcontent
ContentDatabase
WebAccess
RawA/V
Content
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 14
Online Service Provision Sub-system Online Service Provision Sub-system Hardware and Software ComponentsHardware and Software Components
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 15
SAMBITS:SAMBITS:Broadcast MechanismsBroadcast Mechanisms
Mu
ltip
lex
ing
Traditional content
Multi-protocolencapsulated
real time events
MPEG-4 files,MPEG-7 files,
and web contentin object carousel
MPEG-4 streamsin PES packets
Added value contentsynchronised with traditional DTV service
Added valuecontent
MPEG-2 A/Vmaterial
MPEG-2/DVBtransport
stream
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 16
Composition of DVB-Transport stream and Composition of DVB-Transport stream and Internet deliveryInternet delivery
MPEG-2 AV stream
MPEG-7 fileannotation of main program
repeated transmission
MPEG-4 AV stream (synchronisable to MPEG-2)
MPEG-4 files2D/3D scenes
MPEG-7 filesassociated to MPEG-4 files
HTML pages
MPEG-4 files2D/3D scenes
MPEG-7 filesassociated to MPEG-4 files
description of segmentURLs to related material
segment 1 segment 2 segment N
DV
B-M
UX
Inte
rne
t
MPEG-4 files2D/3D scenes
HTML pages
HTML pages
main
dig
ital
TV
pro
gram
description of segmentURLs to related material
User feedback / requests
description of segmentURLs to related material
ad
ditio
na
lm
ultim
ed
ia c
on
tent
retu
rnc
ha
nn
el
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 17
Relationship to Key Action IV:Relationship to Key Action IV:Essential Technologies and InfrastructuresEssential Technologies and Infrastructures
SAMBITS as major European contributor to MPEG-7 Standard for interaction in combined broadcast and Internet applications Contributions to “Multimedia Content Description Interface” Participation in Core Experiment (CE) and Experimentation
Model (XM) development Creation of a production & postproduction studio system to
enter and edit metadata-based High Level -semantic- and Low Level -syntactic- MPEG-7 descriptions
Development of a multimedia receiver terminal with metadata based search functionality and user interaction
Large-scale shared virtual and augmented environments
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 18
SAMBITS:SAMBITS:Contributions to MPEG-7 standardisationContributions to MPEG-7 standardisation
Transmission/Storage Medium
IP MP4Delivery
Layer
Multiplex
MPEG-2 ATM ...
Multiplexed Streams
MultiplexMultiplex
Schemastreams
Descriptionstreams
CompressionLayer
Elementary Streams
Multimediastreams
UpstreamData
Application
APIs
Defines Describe
Reconstruction
DescriptionDecoder
SchemaDecoder
BiM/ TextualParsing
BiM/ TextualDecoding
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 19
SAMBITS and MPEG-7 StandardSAMBITS and MPEG-7 Standard
Generation of Descriptions Structure (e.g. Segments) Content (low level and high level)
Searching and Filtering Support of Program Production Preparation for Delivery
Use on studio site:
Searching Content oriented search
(e.g. classification and structured annotation)User Interface Management
Providing information about related material Triggering user interface events, based on
scene change and new related material
Use on terminal site:
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 20
SAMBITS Consumer Terminal:SAMBITS Consumer Terminal:FunctionalityFunctionality
Enable consumers to receive multimedia broadcasting (MPEG-2, MPEG-4, MPEG-7)
Enable user to retrieve additional content from the Internet (HTML)
let users interact with multimedia content consumers have a back-channel available via Internet based on MPEG-7 meta-data user can search and
retrieve content consumers can create their individual reception
experience
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 21
SAMBITS Multimedia Terminal:SAMBITS Multimedia Terminal:Types of content through broadcast channelTypes of content through broadcast channel
MHP Section FilterMHP Section Filter
Packet Elementary StreamPacket Elementary Stream SectionsSections
DVB-Board DriverDVB-Board Driver
CableCable SatelliteSatellite
MPEG-2 Transport StreamMPEG-2 Transport Stream
MPEG-2A/V
Streams
MPEG-2A/V
Streams
MPEG-4A/V
Streams
MPEG-4A/V
Streams
MPEG-4BIFS
Streams
MPEG-4BIFS
Streams
MPEG-4Files
MPEG-4Files
MPEG-7Files &
Streams
MPEG-7Files &
Streams
HTMLPages
HTMLPages
MHPProgram
Data
MHPProgram
Data
DVB-Board DriverDVB-Board Driver
SoftwareLayer
LogicalChannels
PhysicalNetwork
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 22
SAMBITS Multimedia Terminal:SAMBITS Multimedia Terminal:Types of content through Internet Types of content through Internet && local storage local storage
Internet ClientInternet Client
EthernetEthernet
InternetInternet
MPEG-4A/V Files
& Streams
MPEG-4A/V Files
& Streams
MPEG-7Metadata
MPEG-7Metadata
HTMLPages
HTMLPages
MP
G-4
File
sM
PG
-4Fi
les
MP
EG
-7Fi
les
MP
EG
-7Fi
les
HTM
LP
ages
HTM
LP
ages
MH
PP
rogr
am D
ata
MH
PP
rogr
am D
ata
Local DiskLocal Disk
MP
EG
-2P
ES
MP
EG
-2P
ES
Extended MHP Storage ManagerExtended MHP Storage Manager SoftwareLayer
LogicalChannels
PhysicalNetwork
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 23
System outline of a conventional System outline of a conventional set-top boxset-top box
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 24
The SAMBITS terminal can be interpreted as an The SAMBITS terminal can be interpreted as an extension of a standard compliant MHP STB.extension of a standard compliant MHP STB.
conventional STBMHP (IRT)
MHP compliantSTB
conventionalSTB
SAMBITSterminal
MPEG-7 MPEG-4
MHP (IRT)
Fujitsu-Siemens
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 25
SAMBITS: System outline of the SAMBITS: System outline of the multimedia consumer terminalmultimedia consumer terminal
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 26
Interaction of applications with the differentInteraction of applications with the differentmodules of DVB-MHPmodules of DVB-MHP
Application
User InteractionStorage Media
Control Remote Program
TCP/IP
Others overUDP/IP
DSM-CCServiceInformation
MPEG-2Section
Filter
TunerControl
CAControl
MediaControl
Graphics
Tuner
CA
Demux
Remote Control keyboard
API
MediaDecoder
+
AudioOutput
VideoOutput
InteractionChannel
Network
© IRT/AS G. Stoll SAMBITS: ITU Mediacom 14.03.2002 Folie: 27
Answer: SAMBITS!Answer: SAMBITS!Systems for Advanced Multimedia Broadcast and IT ServicesSystems for Advanced Multimedia Broadcast and IT Services
Interactive Digital TV, based on MPEG-2 System as transport medium for broadcast and individually addressed traffic in connection with return channel
MPEG-4 synchronised services to MPEG-2 main program
MPEG-4 and MPEG-7 based push servicesLocal and full, i.e. remote interactivity in user terminalHandover/roaming of network functionalitiesCompatibility with the standardised DVB API:
The Multimedia Platform (DVB-MHP)
Features: