T-111.500 Multimedia Seminar Mobile Internet Technology Architecture (MITA) Tietoliikenne- ja...
-
Upload
eleanor-brundage -
Category
Documents
-
view
212 -
download
0
Transcript of T-111.500 Multimedia Seminar Mobile Internet Technology Architecture (MITA) Tietoliikenne- ja...
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Java Video and Audio in Consumer Devices: JMF and MM API
Pablo César and J. Luc Lamadon
{pcesar, lamadon}@tml.hut.fi
Helsinki University of Technology
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Outline
• Introduction
• Java Overview
• Personal Computer
• Digital Television Receiver
• Mobile Phone
• Conclusions
• References
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Introduction (1/3)
• What is multimedia?– Multiple media (e.g., audio, video, images, graphics,
text) or multimedia objects – Synchronisation of multimedia elements– Interaction between user and system
• Multimedia in Consumer Devices, user expectation– Latest mobile phones with camera, MMS, Internet
connection, streaming– Digital TV based on audio visual stream + interactive
services– PC with 3D games, videoconference, streaming
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Introduction (2/3)
• Video objects in programming tools:– Synchronized Markup
Integration Language (SMIL)
– Flash presentations
– Java Media Framework (JMF)
• Stand alone video player:– Proprietary players:
Real One, Windows Media Player...
– Open Source players: winamp, MPlayer...
Audio and Video in Consumer Devices, two alternatives:
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Introduction (3/3)
• The focus of this presentation is video objects in a programming tool
• Specifically Java, why?– Consumer devices includes Java
• Digital TV = Multimedia Home Platform (MHP) and JDK 1.1.8
• PCs = Java 2 Standard Edition• Mobile phones = Connected Limited Device
Configuration (CLDC) + Information Device Profile (MIDP)
– Independent of Operating System
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Java Overview (1/2)
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Java Overview (2/2)
Decodes and plays the media data
Retrieves the actual media data
Implements the state machine
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Personal Computer (1/4)
• Services:– Internet, videoconference– Games– …
• Physical Characteristics:– Pointer device (e.g., mouse) and keyboard as major
input mechanisms– Screen resolution: 640x480 to 1600x1200 pixels– Runtime memory: around 128 to 256 MB
• Java Environment– Java 2 Standard Edition (J2SE) including Abstract
Windowing Toolkit (AWT) and Swing
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Personal Computer (2/4)
• JMF as an optional package either version 1.0 or 2.0
• JMF relies on the mentioned classes: Manager, Player, Data Source, and Controller.
• Player behaviour:– Unrealised: when it does not have all the information to
acquire the needed resources– Realised: when it has all the information to acquire the
needed resources– Prefetched: when it has all the needed resources, and has
already prefetched enough media data to start playing immediately
– Started: when it is actually playing the media
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Personal Computer (3/4)
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Personal Computer (4/4)
• Time model: Clock defines the basic timing and synchronisation operations, which control the presentation of media data (e.g. forward, reward)
• Data model: DataSource encapsulates both the location and protocol of media
• JMF 2.0 includes as well a low level API– A Processor performs user-defined processing on the
media data using JMF plug-ins (e.g. Codec, Demultiplexer, Effects, Multiplexer, Renderer)
• Render is done in an AWT Component:– Player.getVisualComponent()– Player.getControlPanelComponent()
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Digital Television Receiver (1/5)
• Multimedia platform in the living room• Services:
– Audio visual stream (normal passive watching)
– Interactive services (active behaviour)
• Physical Characteristics– Remote control as major input mechanism
– Screen resolution: 720x576 pixels (minimal)
– Runtime memory: at least 16 MB
• Java Environment– MHP
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Digital Television Receiver (2/5)
• MHP includes JMF 1.0 (obviously, not as an optional package) with some clarifications, restrictions, and extensions
• The basic classes are again: – Manager, Player, DataSource, and Controller
• The player behaviour is the same as in JMF 1.0:– Unrealised, Realised, Prefetched, and Started
• JMF can be used both for broadcast stream and local files (only in high-end receivers)
• Video drips presents I-frame and consecutive P-frames (high quality interactive graphics)
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Digital Television Receiver (3/5)
• Locators (reference to a particular piece of media) basically three main types:– javax.media.MediaLocator (and subclasses): media clip– org.davic.net.Locator (and subclasses): DVB transport
stream or service– javax.tv.locator.Locator: any type of digital TV system
• Time based (Clock) is useless in broadcast– Solution to synchronise = stream events
• A JMF player cannot access directly services in other transport streams (unless explicitly done)
• Includes new player events (e.g.,):– Media Presented: The media actually starts showing
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Digital Television Receiver (4/5)
• Includes a number of specific digital TV controls:– Media Select: changes the actual stream presented (e.g.,
change the angle of a camera)– Language: intended to control the audio and subtitles
(if present) language
• Television display is composed of:– Background + Video + Graphics Layers
• Video media is, normally, hardware decoded and demultiplexed. So, rendered in video layer (no AWT container)– getVisualContent() returns null– getControlPanelComponent() returns null
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Digital Television Receiver (5/5)
• Still, video can be controlled:– BackgroundVideoPresentationControl: location
and size– VideoFormatControl: picture format and aspect
ratio (e.g., 4:3)
• Maybe in the future (high-end receivers) a real integration of media in graphics layer can be obtained.
• External audio clips can be played in JMF-MHP
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Mobile Phone (1/5)
• Mobile phones are becoming powerful Multimedia platforms
• Provide a number of services:– Internet (e.g., Nokia + Opera), MMS, Video Player– Office capabilities
• Physical characteristics– Key Pad as major input mechanism– Screen resolution: 84x48 to 120x130 pixels– Runtime memory: 160 to 512 KB
• Java Environment– Mobile Information Device Profile (MIDP) version 1.0
or 2.0
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Mobile Phone (2/5)
• Mobile Multimedia API (MMAPI) version 1.0• Latest release was on 26th June, 2003 http://
java.sun.com/products/mmapi/ • MMAPI Description:
– It extends MIDP functionality by providing audio, video and other time-based multimedia support
– It is a thin Java layer completely platform dependent– It is not JMF– It is an optional package
• MIDP 2.0 includes the audio-only subset from MMAPI (i.e. Audio Building Block)
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Mobile Phone (3/5)
• Same concepts as JMF: Player, Controller, Manager and DataSource.
• Player behaviour same (Unrealised, Realised, Prefetched, Started) as JMF but one more state:– Closed: the player cannot be used again, it has
released most of the resources
• Controller is embedded in Player class
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Mobile Phone (4/5)
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Mobile Phone (5/5)
• The types of media supported depends on the controls associated to the Player– Player.getControls() returns the supported controls– Some examples: VolumeControl, ToneControl…
• The Player is content and protocol agnostic• A Player renders media data in a component
dependent on the device configuration, two options:– AWT Component– MIDP Canvas or Items
• VideoControl manages the location and the size of the video
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Conclusions (1/2)
• The actual capabilities of the targeted device is the cause of the differences between standards
• Low level versus high level control of the media:– In MM API and JMF (MHP) the actual control of the
media is done at the native level since they are resource-constrained devices (e.g. decoding)
– JMF uses two profiles, JMF plug-ins enables developers to process the data (e.g. multiplexing)
• Behaviour of the player– All the standards have the same player behaviour. But
MM API defines a Closed state to make explicit that all the resources are freed
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Conclusions (2/2)
• MM API is influenced by the design of JMF, and have number of similarities: Manager, Player, Data Source, and Controller concepts.– MM API hides Controller within Player
• Video render– JMF = AWT Component (complete integration)– MHP = A layer (transparency can be applied) – MM API = MIDP canvas (minimal integration)
• Controls are different depending on the targeted device (e.g., subtitles language in television)
• MHP includes specific television requirements (e.g., Clock not needed in broadcast, Locators)
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
References (1/2)
• Multimedia – P. Vuorimaa, Multimedia Technology Course’s Slides, Helsinki
University of Technology, accessed on November 7th 2003, http://www.tml.hut.fi/Opinnot/T-111.350/index_uk.html
– S. V. Raghavan, and S. K. Tripathi, Networked Multimedia Systems: Concepts, Architecture, and design, Upper Saddle River (NJ), Prentice Hall, 1998
• Digital Television– ETSI TS 102 812 v1.1.1, Digital Television Broadcasting (DVB):
Multimedia Home Platform (MHP) Specification 1.1, ETSI, November 2001
– Steven Morris, The Interactive TV Web, http://www.mhp-interactive.org , 2002
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
References (2/2)
• PC– R. Gordon, and S. Talley, Essential JMF: Java Media Framework,
Upper Saddle River (NJ), Prentice Hall, 1999– Sun Microsystems, JMF Home Page, accessed on November 7th
2003, http://java.sun.com/products/java-media/jmf/index.html
• Mobile Phones– Sun Microsystems, J2ME: Java 2 Platform Micro Edition,
accessed November 7th 2003, http://java.sun.com/products/j2me/ – Sun Microsystems, CLDC, accessed November 7th 2003, http://
java.sun.com/products/cldc/ – Sun Microsystems, Mobile Information Device Profile (MIDP),
accessed November 7th 2003 http://wireless.java.sun.com/midp/ – Sun Microsystems, Mobile Media API (MMAPI), accessed
November 7th 2003, http://java.sun.com/products/mmapi/
T-111.500 Multimedia SeminarMobile Internet Technology Architecture (MITA)
Tietoliikenne- ja multimediatekniikan laboratorio (TML)
Thank you!
Questions?
Comments?