Designing the future of media
Copyright © 2009 by PacketVideo Corporation.
This document and the information contained herein is CONFIDENTIAL INFORMATION of PacketVideo Corporation (PacketVideo) and any client that has been granted a license of this document by PacketVideo. This document shall not be used, published, disclosed, or disseminated outside of PacketVideo and licensed client in whole or in parts without written consent of PacketVideo.
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIESPACKETVIDEO CONFIDENTIAL & PROPRIETARY
OpenCORE Multimedia Framework CapabilitiesOHA 2.07, rev 2December 15, 2009
Abstract:This document describes the capabilities of PacketVideo’s OpenCORE multimedia framework on the Android Platform
Table of Contents
1 Summary ...................................................................................................................................................................... 5
2 OpenCORE Multimedia Framework Architecture ........................................................................................................ 5
3 Container Formats ....................................................................................................................................................... 6
4 Network Protocols ....................................................................................................................................................... 7
4.1 Streaming ............................................................................................................................................................... 7
4.2 Progressive Download and Progressive Playback ..................................................................................................... 7
5 Codecs ......................................................................................................................................................................... 8
5.1 Video Decoders ...................................................................................................................................................... 9
5.2 Audio Decoders ..................................................................................................................................................... 11
5.3 Video/Audio Encoders ........................................................................................................................................... 12
6 Engines ...................................................................................................................................................................... 14
6.1 Author ................................................................................................................................................................... 14
6.2 2-Way ................................................................................................................................................................... 15
6.3 Frame and Metadata Utility (FMU) .......................................................................................................................... 15
6.4 PV Metadata Engine (PVME) ................................................................................................................................. 15
7 Advanced Capabilities ................................................................................................................................................. 16
7.1 Trick Mode - local playback .................................................................................................................................... 16
7.2 Gapless Audio Playback (under Planning) .............................................................................................................. 16
8 Content Policy Manager ............................................................................................................................................. 17
................................................................................................................................................................................... 18
................................................................................................................................................................................... 18
Appendix 1 – Container Format Specifications ............................................................................................................. 19
Appendix 2 - Support for 3GPP TS 26.234 specifications .............................................................................................. 20
Appendix 3 - Support for 3GPP TS 26.234 Release 6 Annex A.1 specifications ............................................................ 22
Appendix 4 – Support for 3GPP TS 26.234 Release 6 Annex A.2 specification ............................................................. 24
Appendix 5 – Decoder Specifications ............................................................................................................................ 26
Appendix 6 – 2-way Specifications ................................................................................................................................ 27
Appendix 7 – Feature level summary of elements supported by 2-way .......................................................................... 28
Feature Group 1: H.223 .............................................................................................................................................. 28
Feature Group 1: H.245 .............................................................................................................................................. 28
Feature Group 3: H.324/3G-324M ............................................................................................................................... 30
Feature Group 5: Video ............................................................................................................................................... 31
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
2
List of Tables and Figures
Table 1 - Container formats support................................................................................................................................6
Table 2 - Streaming support.............................................................................................................................................7
Table 3 - Decoder support................................................................................................................................................8
Table 4 - Encoder support...............................................................................................................................................9
Table 5 – Video Decoder support...................................................................................................................................10
Table 6 – Video Decoder sample performance on OMAP 2420 processor......................................................................10
Table 7 - Audio Decoder Support...................................................................................................................................12
Table 8 - Audio Decoder sample performance on OMAP 2420 processor......................................................................12
Table 9 - Video Encoder support....................................................................................................................................13
Table 10 - Audio Encoders sample performance on OMAP 2420 processor..................................................................13
Table 11 - Author Engine support.................................................................................................................................14
Table 12 - Author Engine input output streams.............................................................................................................15
Table 13 - Trick Mode local playback............................................................................................................................16
Table 14 - Content Policy Manager support...................................................................................................................18
Figure 1 - Block Diagram OpenCORE Multimedia Framework.....................................................................................5
Figure 2 - OpenCORE video decoders modular design....................................................................................................9
Figure 3 - OpenCORE audio decoders modular design.................................................................................................11
Figure 4 - OpenCORE encoders modular design...........................................................................................................13
Figure 5 - Content Policy Manager...............................................................................................................................17
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
3
References
1. “PVPlayer SDK Developer’s Guide” - pvplayer_developers_guide.pdf
2. “PVAuthor Developer’s Guide” - pvauthor_developers_guide.pdf
3. “PV2Way Developer’s Guide “ - pv2way_developers_guide.pdf
4. “OMX Core Integration Guide” - omx_core_integration_guide.pdf
5. “Media I/O Developer’s Guide” - mio_developers_guide.pdf
6. “Content Policy Manager Developer’s Guide” pv_cpm_developers_guide.pdf
The above set of OpenCORE documents are available at the following Android Open Source Project location: http://android.git.kernel.org/?p=platform/external/opencore.git;a=tree;f=doc;hb=HEAD
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
4
1 Summary
This document describes the capabilities of PacketVideo’s OpenCORE multimedia framework for the Android platform.
The OpenCORE multimedia framework covers the following areas:
• Container Formats
• Network Protocols
• Codecs
• Engines
• Advanced Capabilities
• Content Policy Manager
This document is intended as a starting point for potential integrators of OpenCORE multimedia framework on the Android platform. It covers high-level summary of features available in OpenCORE.
This document is NOT intended to be a developers guide or an integration guide. For further detailed information regarding OpenCORE development and integration guides, please consult the OpenCORE documents listed in the Reference section of this document.
2 OpenCORE Multimedia Framework Architecture
Figure 1 - Block Diagram OpenCORE Multimedia Framework
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
5
OpenCORE
Content Policy Manager
Multimedia Engines
Player, Author, 2-Way, FMU, PVME
Data Formats
Audio Codecs
Linux Interface
Audio Codecs
AACAMR (NB, WB)
HE-AACHE-AACV2
MP3
Android Application Layer
Video Codecs
AVC H.263 MPEG-4
Data Formats
Parser MPEG-4 3GPP3GPP2 MP3 AAC AMR
ComposerMPEG-43GPPAACAMR
Streaming3GPP
Shoutcast
DownloadMPEG-43GPPMP3AAC
2-way3G324-M
H.223H.245
Video Codecs
Linux Operating System
3 Container Formats
The OpenCORE multimedia framework has support for the following container formats:
Co
nta
iner
Fo
rmat
Mim
e ty
pe
Par
ser
Com
pose
r
Typ
e
Tim
ed T
ext
Met
adat
a ta
gs
Rep
ositi
on
Loca
l Pla
ybac
k
Pro
gres
sive
D
ownl
oad
Pro
gres
sive
Pla
ybac
k
MPEG4audio/mp4video/mp4 audio + video
1
3GPPaudio/3gpvideo/3gp audio + video 2 3
3GPP2 audio/3gpp2video/3gpp2
4 audio + video 3
MP3 audio/mpeg audio N/A 2
AAC audio/aac 5 audio N/A 2 6 U/P U/P
AMR audio/amr 7 audio N/A
WAV audio/wav 8 audio N/A
ASF Available under feature pack “PV Android Windows Media Package”.
DivX Available under feature pack “PV Android DivX Package”
Table 1 - Container formats support
N/A – Not Applicable; U/P – Under Planning
1 . MPEG4 progressive download and playback - The moov atom is expected to be present at the beginning of the MP4/3GP files in order to perform progressive download and progressive streaming.
2. ID3 Tag support – Support for ID3 tags up to version ID3v2.43 . 3GPP Progressive playback – Progressive playback for 3GPP clips is supported, except in the case
when the clip has movie fragments. Support for 3GPP clips with movie fragments is under planning.4 . 3GPP2 Parser - The 3GPP2 parser in addition to supporting the 3GPP video and audio streams, it also
supports audio streams of types EVRC, EVRC-B, EVRC-WB, 13K (QCELP), SMV, and VMR-WB.5 . AAC Parser - The AAC parser supports AAC, HE-AAC v1, and HE-AAC v2 audio only streams. They
can be in either ADIF or ADTS format. 6 . AAC Reposition support - The repositioning of ADTS format is allowed. The repositioning of ADIF
format is NOT allowed.7 . AMR Parser - The AMR parser supports the parsing of AMR-NB and AMR-WB audio file formats. It
supports file extensions of .amr for AMR-NB and .awb for AMR-WB.8. WAV Parser - The WAV parser supports the formats: PCM, µ -law, and A-law data.
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
6
The specifications describing these Container Formats are listed in Appendix 1.
4 Network Protocols
4.1 STREAMING
The OpenCORE multimedia framework provides support for streaming network protocols. The supported streaming protocols are listed below:
Streaming Protocol Supported Comments
3GPP RTSP 9 RTP UDP
RTSP-T RTP TCP
Shoutcast 10 MP3 content type supported
ASF Available under feature pack “PV Android Windows Media Package”
Real Media Available under feature pack “PV Android Real Media Package”
Flash VideoUnder
PlanningPlease see feature pack “PV Android Flash Video Package”
Table 2 - Streaming support
4.2 PROGRESSIVE DOWNLOAD AND PROGRESSIVE PLAYBACK
The OpenCORE multimedia framework provides support for progressive download and progressive playback of various multimedia container formats. The supported container formats are listed in Table 1 “Container Formats support”.
Progressive download involves download of media file from a generic web server using the HTTP protocol. In case of progressive download, the playback starts while the file is downloading in the background. The OpenCORE parser depends on the metadata available in the container format of the media file.
Progressive playback is similar to progressive download with the exception that during progressive playback the complete file is not saved on the client. During a progressive playback session, only a finite portion of the file is resident in the device’s cache at any given point.
9 . 3GPP RTSP Streaming - All mandatory features described in RTSP TS 26.234 Release 6 are supported The 3GPP Release 6 specifications cover optional features. The supported features from 3GPP TS 26.234 v6.8.0 specifications are listed in Appendix 2. The support for 3GPP TS 26.234 Release 6 Annex A.1 specifications is listed in Appendix 3. The support for 3GPP TS 26.234 Release 6 Annex A.2 specifications is listed in Appendix 4.
10 . Shoutcast Streaming support - The OpenCORE multimedia framework provides client-side support for parsing and playback of Shoutcast streams. The supported content type is MP3. The common use case for Shoutcast is internet radio. It uses HTTP as the transport protocol, along with special ICY extension headers. The media data is interleaved with metadata tags, such as song title and station name, within the stream.
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
7
5 Codecs
The OpenCORE multimedia framework has adopted OpenMAX Integration Layer (IL) interface to integrate the codecs. The modular design allows the use of multiple alternate codecs, as long as they are OpenMAX IL 1.1 compatible.
The OpenCORE multimedia framework provides its own set of optimized software codecs, which are OpenMAX IL 1.1 compatible. Where applicable, third party hardware accelerated codecs may also be used. In cases when both software and hardware codecs are present on a device, the OpenCORE framework allows either decoder to be loaded and used at run time. The choice of which underlying decoder to use may depend on a use case scenario or on the availability of resources.
Additional information on how OpenMAX IL components can be integrated into OpenCORE framework is available in “omx_core_integration_guide.pdf” and other related documents at the following location: Android Open Source Project – OpenCORE documentation
The OpenCORE multimedia framework provides the following optimized software decoders:
Decoder Type Support
H.263 video Baseline Profile and Wireless Profile (Annex I, J, K, and T)
MPEG-4 video Simple Profile
AVC/H.264 video Constrained Baseline Profile
MP3 audioSampling rates: 8 KHz to 48 KHz Bitrates: 8 kbps to 320 kbps
AAC audioAAC (MPEG4 AAC-LC and LTP)HE-AAC v1 (AAC+)HE-AAC v2 (Enhanced AAC+)
AMR-NB audioSampling rate: 8 KHz (mono)Bitrates: 4.75 kbps to 12.20 kbps
AMR-WB audioSampling rate: 16 KHz (mono)Bitrates: 6.60 kbps to 23.85 kbps
Windows Media video/audioAvailable under feature pack “PV Android Windows Media Package”.
Real Media video/audioAvailable under feature pack “PV Android Real Media Package”
Table 3 - Decoder support
The specifications describing these Decoders are listed in Appendix 5
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
8
The OpenCORE multimedia framework provides the following optimized software encoders:
Encoder Type Support
H.263 video Baseline Profile
MPEG-4 video Simple Profile
AVC/H.264 video Constrained Baseline Profile
AMR-NB audioSampling rate: 8 KHz (mono)Bitrates: 4.75 kbps to 12.20 kbps
AAC audioAvailable under feature pack “PV Android AAC Encoder Package”.
Table 4 - Encoder support
5.1 VIDEO DECODERS
The OpenCORE multimedia framework includes the H.263, MPEG-4 part 2, and AVC/H.264 software video decoders. The following figure shows the modular OpenCORE design that takes advantage of the OpenMAX IL interface.
PV OpenMAX Video Codecs Node
OpenMAX Integration Layer
H.263 / MPEG4 Part 2OMX Component
AVCOMX Component
H.263 / MPEG4 Part 2Software Codecs
AVCSoftware Codec
Figure 2 - OpenCORE video decoders modular design
The modular design of the video codec node allows for the existence of multiple alternate codecs to support a particular codec standard. For example, a particular mobile device may have hardware AVC decoder support, in addition to the optimized software AVC decoder provided as part of the Android package. The Video Codecs Node may provide support for both decoder options, and may allow either decoder to be loaded and used at run time.
The table below includes, but not limited to, some typical supported levels (according to 3GPP TS 26.234 Rel. 7). The actual level of support varies depending on the capabilities of the particular hardware platform, such as the CPU availability of the processor and memory map characteristics. On many newer platforms, the optimized codecs available in OpenCORE can decode at levels beyond these ranges. Furthermore, hardware-accelerated codecs, if available on the device, may provide higher levels of performance.
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
9
Video Decoder
Profile Level Frame SizeFrame Rate
(fps)Bit Rate (Kbps)
H.263
0
(Baseline)
10 QCIF 15 64
20 CIF 15 128
30 CIF 30 384
45 QCIF 15 128
3
(Wireless) annex I, J, K, T
10 QCIF 15 64
20 CIF 15 128
30 CIF 30 384
45 QCIF 15 128
MPEG-4
Part 2 Simple
0 QCIF 15 64
0b QCIF 15 128
1 QCIF 15 64
2 CIF 15 128
3 CIF 30 384
AVC/H.264Constrained Baseline
1 QCIF 15 64
1b QCIF 15 128
1.1 QCIF 30 192
1.2 CIF 15 384
Table 5 – Video Decoder support
For reference, the following performance measurements were obtained from OpenCORE MPEG-4 and AVC video decoders running on an OMAP 2420 processor:
Video Decoder Frame SizeBit rate(kbps)
Frame Rate (Fps)
CPU(MHz)
MPEG-4QCIF 64 15 13.9
CIF 384 30 97.7
AVC/H.264QCIF 64 15 42.5
CIF 384 30 307.3
Table 6 – Video Decoder sample performance on OMAP 2420 processor
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
10
5.2 AUDIO DECODERS
The OpenCORE provides support for MP3, AAC, HE-AAC, AMR-NB, and AMR-WB audio decoders. The following figure shows the modular OpenCORE design that takes advantage of the OpenMAX IL interface.
Figure 3 - OpenCORE audio decoders modular design
Where applicable, the audio codec libraries have a configurable design, which allows optimization of the memory footprint. For instance, the AAC library uses separate layers to accommodate baseline AAC, Spectral Band Replication (SBR) and Parametric Stereo (PS) tools. Thus when selecting among the AAC and HEAAC codec options at build time, it is possible to configure the code to exclude the layers which are not needed.
The table below includes, but not limited to, some typical supported levels. The actual level of support varies depending on the capabilities of the particular hardware platform, such as the CPU availability of the processor and memory map characteristics. On many newer platforms, the optimized audio codecs available in OpenCORE can decode at levels beyond these ranges. Furthermore, hardware-accelerated codecs, if available on the device, may provide higher levels of performance.
Audio DecoderTypical Bit Rate
(Kbps)Sampling Rate
(KHz)Channels Extensions
MP3 1/2/2.5 8 to 320 8 ~ 48 2/1 .mp3
MPEG4 AACAAC-LC and LTP
level 2Up to 256 Max: 48 2/1 .aac, .mp4, .3gp, .m4a, .3ga
HE-AACv1 (AAC+) level 2
16 to 96 Up to 48 2/1 .aac, .mp4, .3gp, .m4a, .3ga
HE-AACv2 (AAC++) or Enhanced AAC+
level 216 to 64 Up to 48
ParametricStereo
.aac, .mp4, .3gp, .m4a, .3ga
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
11
AMR-NB 4.75 to 12.20 8 1 .amr, .3gp, .mp4
AMR-WB 6.60 to 23.85 16 1 .amr, .3gp, .mp4
Table 7 - Audio Decoder Support
For reference, the following performance measurements were obtained from OpenCORE audio decoders running on an OMAP 2420 processor:
Audio DecoderSampling Rate (KHz)
ChannelsBit Rate (Kbps)
CPU (MHz)
MP3 44.1 2 128 29.5
AAC 44.1 2 96 20.47
HE-AAC v1 44.1 2 48 45.09
HE-AAC v2 44.1 2 48 63.39
AMR-NB 8 1 12.2 10.94
AMR-WB 16 1 23.85 33.14
Table 8 - Audio Decoder sample performance on OMAP 2420 processor
5.3 VIDEO/AUDIO ENCODERS
The OpenCORE provides H.263, MPEG4 and AVC/H.264 software video encoders, and AMR-NB software audio encoder. The following figure shows the modular OpenCORE design that takes advantage of the OpenMAX IL interface. An encoder node is associated with one OMX component. In order to encode multiple tracks, multiple instantiations of OMX Encoder Node is required. For example, a typical audio/video bitstream requires two encoder nodes, one for video and one for audio.
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
12
PV OpenMax Encoder Node
OpenMax Integration Layer
MPEG4/H.263 Enc OMX comp
AVC Enc OMX comp
s/w MPEG4 H.263 encoder
s/w AVC H.264 encoder
AMR Enc OMX comp
s/w AMR NB encoder
Figure 4 - OpenCORE encoders modular design
As in decoder case, the modular design of the encoder node allows for the existence of multiple alternate codecs to support a particular codec standard. For example, a particular mobile device may have hardware AVC encoder support, in addition to the optimized software AVC encoder provided as part of the Android package. The following table shows examples of the software encoder performance measured on OMAP 2420 processor: For other hardware platform, the actual level of support is, but not limited to, a function of the CPU availability of the processor and memory map characteristics.
Video EncoderFrame Size
Bit rate
(kbps)Fps
CPU(MHz)
MPEG-4 Simple ProfileEncoder
QCIF 64 15 68.0
CIF 384 30 521.9
AVC Constrained Baseline Profile Encoder
QCIF 64 15 146.7
CIF 384 30 1209.4
Table 9 - Video Encoder support
Audio Encoder KHz Channels Kbps CPU (MHz)
AMR-NB Encoder 8 1 12.2 53.68
Table 10 - Audio Encoders sample performance on OMAP 2420 processor
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
13
6 Engines
6.1 AUTHOR
The OpenCORE multimedia framework has support for Authoring capabilities that allows recording of video and audio content.
The Author accepts both compressed and uncompressed input media. The input bit streams supported by the Author are listed below:
Input Bit Streams supported
Compressed/
Uncompressed Input FormatInput Source File Format
WAV UncompressedAudio: PCM8, PCM16
Video: N/A.wav
AVI UncompressedAudio: PCM8, PCM16
Video: RGB12, RGB24,YUV4:2:0 planar
.avi
AMR CompressedAudio: AMR-NB, AMR-WB
Video: N/A
.amr
.awb
MPEG4 CompressedAudio: AAC, AAC Plus, eAAC
Plus
Video: MPEG-4 Part 2, H.264.mp4
3GPP Compressed
Audio: AMR, AAC, AAC Plus, eAAC Plus
Video: MPEG-4 Part 2, H.263Timed-Text
.3gp
Table 11 - Author Engine support
The Author creates a graph of input and output formats based on the container MIME type selected. The following table lists the combinations of input and output streams:
MIME Type INPUT TRACKS Output Container FormatOutput File
Format
. "/x-pvmf/ff-mux/3gp”
Any combination of supported input
formats of audio, video, and timed-text
3GPP .3gp
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
14
"/x-pvmf/ff-mux/amr-nb" AMR-NB AMR-NB .amr
"/x-pvmf/ff-mux/amr-wb" AMR-WB AMR-WB .awb
"/x-pvmf/ff-mux/adif" AAC-ADIF AAC-ADIF .aac
"/x-pvmf/ff-mux/adts" AAC-ADTS AAC-ADTS .aac
Table 12 - Author Engine input output streams
NOTE: The application has the freedom to choose the output file extensions. For example, the application may decide to name file extensions as .m4a or .mp4 for the same content.
6.2 2-WAY
The OpenCORE multimedia framework supports a 2-way subsystem that provides capabilities to create, configure, and control a video telephony session. The 2-way is capable of executing 3G-324M standard video telephony calls.
A client of 2-way SDK can be an application or an adapter layer that adapts the 2-way interface to a higher-level interface used by an application. The input media data are typically provided by live source(s) such as camera and microphone. The output media data is forwarded to presentation sinks such as display and speaker. The sources and sinks are provided by the client applications.
The feature-level summary of the elements supported by 2-way is listed in Appendix 7.
6.3 FRAME AND METADATA UTILITY (FMU)
The OpenCORE multimedia framework supports a Frame and Metadata Utility (FMU) that retrieves metadata and thumbnail information from batches of multimedia files.
The FMU can retrieve DRM protected metadata where applicable. It uses the Content Policy Manager (CPM) framework for extracting the DRM protected metadata.
The FMU supports dynamic loading of the parser libraries in order to reduce the runtime memory footprint.
6.4 PV METADATA ENGINE (PVME)
The OpenCORE multimedia framework supports a PV Metadata Engine (PVME) that retrieves metadata information from multimedia files. The PVME does not support extraction of thumbnails from multimedia files.
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
15
The PVME is built for speed and targeted for processing large batches of content. Applications that need to extract thumbnails as well as metadata, are recommended to use FMU instead.
The PVME can retrieve DRM protected metadata where applicable. It uses the Content Policy Manager (CPM) framework for extracting the DRM protected metadata.
The PVME supports dynamic loading of the parser libraries in order to reduce the runtime memory footprint.
7 Advanced Capabilities
7.1 TRICK MODE - LOCAL PLAYBACK
In addition to the basic support of opening, playback, pause, resume, stop, and metadata query, the OpenCORE multimedia framework also supports Trick Mode playback. Trick mode refers to the playback rate of anything other than 1X.
Feature DescriptionSupporte
dFile Format Supported
Fast Forward Playback of the file forward, at a uniform rate that is faster than realtime. Actual rate is capped by system decode and render capability.
Yes All supported OpenCORE file formats
Fast Rewind Playback of the file backward at a uniform rate that is faster than realtime. Only key frames are rendered and audio is usually disabled.
Yes MP4 and ASF11
Slow Motion - Forward
The slower playback rate (e.g. slower than realtime playback) allows the user more time to examine each frame and stop or pause at a specific point.
Yes All supported OpenCORE file formats
Slow Motion – Backwards
Slow motion playback in backward direction Yes MP4 and ASF10
Frame Stepping – Forward
Frame stepping forward refers to the process of advancing the playback in a non-uniform way, usually based on a direct user request.
Yes All supported OpenCORE file formats
Frame Stepping – Backwards
Frame stepping backward refers to the process of reversing the playback in a non-uniform way, usually based on a direct user request.
Yes MP4 and ASF10
Table 13 - Trick Mode local playback
7.2 GAPLESS AUDIO PLAYBACK (UNDER PLANNING)
The gapless audio feature is under planning for the OpenCORE multimedia framework. The gapless audio playback allows uninterrupted playback of consecutive audio tracks from a continuous media presentation without intervening silence or clicks at the point of track change. This feature supports gapless local playback of AAC (in MPEG4 container) and MP3 formats.
11. The trick mode playback support for ASF is available as part of a feature pack from PV. Please see the feature pack “PV Android Windows Media Package”
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
16
Gapless playback support includes two components: compression artifact removal and track queuing. The OpenCORE multimedia framework will remove compression artifacts for the iTunes encoded MP3, iTunes encoded AAC, and LAME encoded MP3
Track queuing allows an application to present a set of separate tracks comprising a continuous media presentation as a single data source to PV Player engine. The gapless audio playback feature assumes consecutive audio tracks to be of the same file format and encoding parameters for best performance (minimum audible gap between clips).
8 Content Policy Manager
The OpenCORE multimedia framework supports a content policy manager (CPM) subsystem that provides a generic framework for controlling and enforcing content usage rules. The CPM provides a well-defined interface
between consumers and providers/managers of content for mutually establishing the identities of the parties involved (authenticating), authorizing content usage, and providing access to the content.
A popular use for the CPM involves providing access to files protected with Digital Rights Management (DRM). The CPM itself does not directly implement DRM algorithms and protocols, but instead provides a framework for developing and registering plugins that provide these implementations. These plugins are provided to CPM in form of a registry.
The CPM allows for rules that extend beyond a single piece of content. Since the content management rules can by layered, it is possible to implement a wide range of rules by combining a set of plugins. For example, a common use-case for the CPM involves the implementation of parental controls to restrict access to content based on ratings and user profile.
MULTIMEDIA PLAYER
CONTENT POLICY
MANAGER
SERVICE PLUGIN 0
SERVICE PLUGIN 1
SERVICE PLUGIN N
Figure 5 - Content Policy Manager
The following table lists the file formats for which CPM support exists:
File Format CPM SUPPORT Comments
3GP Yes
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
17
AAC Yes
AMR Yes
MP3 Yes
WAV No
DivX YesSee “PV Android DivX Package”
SDC YesSee “PV Android SDC Package”
ASF YesSee “PV Android Windows Media Package”
REAL Media No
Table 14 - Content Policy Manager support
DRM plugins support (Feature Packs ONLY)
The support for actual DRM plugins, such as OMA2, Janus, DivX DRM, Windows Media PlayReady, and SDC is available in feature packs from PV.
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
18
Appendix 1 – Container Format Specifications
Container Format Specifications
Container Format
Specifications
MPEG4 ISO/IEC 14496-12: "Information technology — Coding of audio-visual objects — Part 12: ISO base media file format"
3GPP 3GPP TS 26.244: "Packet switched streaming service (PSS); 3GPP file format (3GP)"
3GPP2 3GPP2 C.S0050-B “3rd Generation partnership project 2: 3GPP2 file formats for multimedia services”
MP3 ISO/IEC 11172-3: “Information technology -- Coding of moving pictures and associated audio for digital storage media at up to about 1,5 Mbit/s -- Part 3: Audio", 1993
ISO/IEC 3818-3 “MPEG-2 audio”
AAC ISO/IEC 14496-3:2004: “Information Technology – Generic coding of moving pictures and associated audio information – Part 3: Advanced Audio Codec (AAC)”, (3rd edition)
AMR IETF RFC 3267: “RTP payload format and file storage format for the Adaptive Multi-Rate (AMR) Adaptive Multi-Rate Wideband (AMR-WB) audio codecs ", March 2002
WAV Wave Form audio file format
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
19
Appendix 2 - Support for 3GPP TS 26.234 specifications
The support for 3GPP TS 26.234 v6.8.0 specification is listed below:
SectionOpenCORE
SupportComments
5.1 Session establishment Yes SDP and plain RTSP URL is supported
No HTTP GET requests with byte range to support 3GP files NOT authored for PD is NOT supported
SMIL is NOT supported
5.2 Capability exchange No
5.3.2 RTSP Yes DESCRIBE and Range header are supported. (see also minimal on-demand playback RTSP implementations); SET_PARAMETER is supported; "Bandwidth" header is NOT supported; "3GPP-Link-Char" is NOT supported; "3GPP-Adaptation" is supported.
5.3.2.4 Video buffering headers x-predecbufsize:x-initpredecbufperiod:x-initpostdecbufperiod:3gpp-videopostdecbufsize:are all NOT supported
5.3.3.1 SDP "b=TIAS:" supported;
"a=maxprate" supported
5.3.3.2 Additional SDP fields NOT supported
5.3.3.3 The “alt” and “alt-default-id” attributes
Yes
5.3.3.4 The session level grouping attribute, “alt-group”
Yes
5.3.3.5 The bit-rate adaptation support attribute, “3GPP-Adaptation-Support”
Yes
5.3.3.6 The Quality of Experience support attribute, "3GPP-QoE-Metrics"
No
5.3.3.7 The asset information attribute, "3GPP-Asset-Information"
Yes
5.4 MIME types AMR, AAC, MP4A-LATM, MP4V-ES, H263-1998,H263-2000, H264 are supported; 3GPP timed-text NOT supported;OMA DRM NOT supported; RTP retransimission NOT supported
6. Data transport Support
6.2 RTP over UDP/IP AMR-NB(rfc 3267),
AMR-WB(rfc 3267),
AAC/LATM(rfc 3016),
m4v(rfc 3016),
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
20
H.263(rfc 2429)
6.2.2 RTP profiles RTP/AVP is supported;
RTP/AVPF and RTP/SAVP are NOT supported
6.2.3 RTP and RTCP extensions
6.2.3.1 RTCP extended reports No
6.2.3.2 RTCP App packet for client buffer feedback (NADU APP packet)
Yes
6.2.3.3 RTP retransmission No
6.2.4 RTP payload formats Extended AMR-WB codec,
3GPP timed text, DRM encrypted RTP pkt, RTP retransimission payload are NOT supported
7 Codecs Support
7.2 Speech Codecs AMR-NB is supported
AMR-WB is supported
7.3 Audio Codecs AAC-LC, AAC-LTP?
7.3a Synthetic audio No
7.4 Video H.263 profile 0 level 10 and profile 3 level 10; MP4 Simple Profile Level 0; H.264 (AVC) Baseline Profile Level 1b decoder [90][91] with constraint_set1_flag=1 and without requirements on output timing conformance
7.5 Still images No
7.6 Bitmap graphics No
7.7 Vector graphics No
7.8 Text No
7.9 Timed text local playback is supported
7.10 3GPP file format Basic profile and Progressive-download profile
10 Adaptation of continuous media
Yes supported
11 Quality of Experience No
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
21
Appendix 3 - Support for 3GPP TS 26.234 Release 6 Annex A.1 specifications
The support for 3GPP TS 26.234 Release 6 Annex A.1 specifications is listed below:
Type DescriptionRequirement according to
RFC 4566
Requirement according to
3GPP TS 26.234 Rel 6
OpenCORE Support
Comments
Session Description
V Protocol version R R Yes
O Owner/creator and session identifier R R Yes
S Session Name R R Yes
I Session information O O Yes
U URI of description O O No Ignored
E Email address O O No Ignored
P Phone number O O No Ignored
C Connection Information R R Yes Ignored. Uses "a=control" session attribute in "Media Description"
B Bandwidth information
AS O O Yes
RS ND O Yes
RR ND O Yes
TIAS ND O Yes
One or more Time Descriptions (See below)
Z Time zone adjustments O O No Ignored
K Encryption key O O No Ignored
A Session attributes control O R Yes
range O R Yes
alt-group ND O Yes
3GPP-QoE-Metrics
ND O No
3GPP-Asset-Information
ND O Yes
3GPP-Integrity-Key
ND O No
3GPP-SDP-Auth ND O No
maxprate ND O
One or more Media Descriptions (See below )
Time Description
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
22
T Time the session is active R R No Ignored
R Repeat times O O No Ignored
Media Description
M Media name and transport address R R Yes
I Media title O O No Ignored
C Connection information R R Yes
B Bandwidth information
AS O R Yes
RS ND R Yes
RR ND R Yes
TIAS ND R Yes
K Encryption Key O O No Ignored
A Attribute Lines
control O R Yes
range O R Yes
fmtp O R Yes
rtpmap O R Yes
X-predecbufsize ND O No Ignored
X-initpredecbufperiod ND O No Ignored
X-initpostdecbufperiod ND O No Ignored
X-decbyterate ND O No Ignored
framesize ND R 12 Yes
Alt ND O Yes
alt-default-id ND O Yes
3GPP-Adaptation-Support
ND O Yes
3GPP-QoE-Metrics ND O No Ignored
3GPP-Asset-Information ND O Yes
3GPP-SRTP-Config ND O No
Rtcp-fb O O No
maxprate ND R Yes
12 NOTE: The “framesize” attribute is only required for H.263 streams
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
23
Appendix 4 – Support for 3GPP TS 26.234 Release 6 Annex A.2 specification
The support for 3GPP TS 26.234 Release 6 Annex A.2 specifications is listed below:
MethodRequirement client
according to RFC 2326.
Requirement for a PSS client according to 3GPP
TS 26.234 Rel 6.
OpenCORESupport
Comments
OPTIONS O O Yes
REDIRECT Respond Respond Partial Client handles 3xx error codes as REDIRECT, but currently it does not accept REDIRECT request.
DESCRIBE O Generate Yes
SETUP Generate Generate Yes
PLAY Generate Generate Yes
PAUSE Generate Generate Yes
TEARDOWN Generate Generate Yes
SET_PARAMETER O O Yes Client responds with "200 OK" but does NOT parse the particular parameter
NOTE 1: O = Support is optional
NOTE 2: 'Generate' means that the client/server is required to generate the request where applicable.
NOTE 3: 'Respond' means that the client/server is required to properly respond to the request.
Header
Requirement for a minimal on demand
playback client according to RFC 2326.
Requirement for a PSS client according to 3GPP
TS 26.234 Rel 6.
PVSupport
Comments
Bandwidth O O Ignored
Connection include/ understand include/ understand Yes
Content-Encoding understand understand No Ignored
Content-Language understand understand No Ignored
Content-Length understand understand Yes
Content-Type understand understand Yes
Cseq include/ understand include/ understand Yes
Date include include Yes
Location understand understand Yes
Public O O No
Range O include/ understand Yes
Require O O Yes
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
24
RTP-Info understand understand Yes
Server4 O O Yes
Session include include Yes
Timestamp O O No
Transport include/ understand include/ understand Yes
Unsupported include include Yes
User-Agent4 O O Yes Argument can be passed in through KVP
3GPP-Adaptation N/A O Yes
3GPP-Link-Char N/A O No
3GPP-QoE-Metrics N/A O No
NOTE 1: O = Support is optional
NOTE 2: 'include' means that the client/server is required to include the header in a request or response where applicable.
NOTE 3: 'understand' means that the client/server is required to be able to respond properly if the header is received in a request or response.
NOTE 4: According to RFC 2326 the "Server" and “User-Agent” headers are not strictly required for a minimal RTSP implementation, although it is highly recommended that they are included with responses and requests. The same applies to PSS servers and clients according to the present document.
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
25
Appendix 5 – Decoder Specifications
Decoder Specifications
Codec Specifications
H.263 video ITU-T Recommendation H.263: “Video coding for low bit rate communication”
MPEG-4 video ISO/IEC 14496-2:2001, “Information technology – Coding of audio-visual objects – Part 2: Visual”
AVC video AVC/H.264: “Advanced video coding for generic audiovisual services” ITU-T recommendation H.264, 05/2003
MP3 audio MPEG 1/2/2.5 Standars according to ISO/IEC 11172-3 and ISO/IEC 3818-3
AAC audio AAC LC and LTP: “Information Technology – Generic coding of moving pictures and associated audio information – Part 3: Advanced Audio Codec (AAC)”, ISO/IEC 14496-3:2004 (3rd edition)
HE-AAC v1: 3GPP TS 26.401, v6.10: “General audio codec audio processing functions; Enhanced aacPlus general audio codec; General description
HE-AAC v2: 3GPP TS 26.401, v6.10: “General audio codec audio processing functions; Enhanced aacPlus general audio codec; General description
AMR-NB audio 3GPP TS 26.071: “Mandatory Speech Codec speech processing functions; AMR Speech Codec; General description”
AMR-WB audio 3GPP TS 26.171: “AMR speech codec, wideband; General description”
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
26
Appendix 6 – 2-way Specifications
2-way Specifications
3GPP TS 26.110 v6.0.0: "Codec for Circuit Switched Multimedia Telephony Service; General description".
3GPP TS 26.111 v6.1.0: “Codec for Circuit Switched Multimedia Telephony Service; Modifications to H.324”
ITU-T Recommendation H.324: "Terminal for low bitrate multimedia communication", February 1998
ITU-T Recommendation H.223: "Multiplexing protocol for low bit rate multimedia communication", March 1996
ITU-T Recommendation H.223 - Annex A: "Multiplexing protocol for low bit rate multimedia mobile communication over low error-prone channels", February 1998
ITU-T Recommendation H.223 - Annex B: "Multiplexing protocol for low bit rate multimedia mobile communication over moderate error-prone channels", February 1998
ITU-T Recommendation H.245, Version 6: "Control protocol for multimedia communication”, February 2000
3GPP TR 26.911 v6.0.0: "Codec for circuit switched multimedia telephony service; Terminal Implementor’s Guide"
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
27
Appendix 7 – Feature level summary of elements supported by 2-way
The feature-level summary of the elements supported by 2-way is listed below. These features are classified by feature groups, which relate to specific areas of the 3G-324M standard. In general, features that are listed as supported may be used and/or configured through 2-way API’s, i.e. no special build time provisions is necessary to activate such features. For other features (either absent from the list, or listed as “Not Supported”), some customization would be needed in order to add or utilize the feature.
FEATURE GROUP 1: H.223
Group # Feature Support Direction Comment
H.223 1 Level 0 Yes In/Out
H.223 2 Level 1 (H.223 Annex A)
Yes In/Out Support for Annex A in single or double flag modes.
H.223 3 Level 2 (H.223 Annex B)
Yes In/Out Support for Annex B with or without optional header.
H.223 4 Level 3a (H.223 Annex C)
No --
H.223 5 Level 3b (H.223 Annex D)
No --
H.223 6 Control over AL1
Yes In/Out Framed transfer mode, per H.324 recommendation
H.223 7 Audio over AL2
Yes In/Out CRC and Optional SN are both supported
H.223 8 Video over AL2
Yes In/Out CRC and Optional SN are both supported
H.223 9 Video over AL3
Yes In/Out Support for 0, 1 or 2 control field octets. Retransmission not supported (this is in line with the recommendation in 3GPP TS 26.911)
FEATURE GROUP 1: H.245
Group # Feature Supported? Direction Comment
H.245 1 Terminal Yes In/Out
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
28
Capability Set
H.245 2 Master Slave Determination
Yes In/Out
H.245 3 Multiplex Entry Send
Yes In/Out
H.245 4 Request Multiplex Entry
Yes In/Out
H.245 5 Open Logical Channel
Yes In/Out For video case, both unidirectional and bidirectional are supported
H.245 6 Request Channel Close
Yes In/Out
H.245 7 End Session Yes In/Out
H.245 8 Request Mode No -- Currently no outgoing Mode Requests are sent. Incoming Mode Requests are rejected, consistent for case with no Transmit Capabilities in outgoing TCS.
H.245 9 User Input Indication
Yes In/Out
H.245 10 Round Trip Delay
Yes In/Out
H.245 11 Flow Control Yes In/Out H.245 signaling support only
H.245 12 Video Spatial/Temporal Tradeoff (Command)
Yes In/Out
H.245 13 Video Fast Update
Yes In/Out Picture update only
H.245 14 Multiplex Reconfiguratin(level change)
No --
H.245 15 H.223 Skew Indication
Yes In/Out
H.245 16 Vendor Identification Indication
Yes In/Out
H.245 17 Mux PDU size restriction signaling
Yes In/Out As recommended in Section 5 of 3GPP TS 26.911.
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
29
H.245 18 Timer/Counter configuration
Yes -- Various H.245 timer and counter values may be configured via API
H.245 19 Fast call setup Yes -- Optimal standards-compliant bundling of H.245 messages to reduce setup time.
FEATURE GROUP 3: H.324/3G-324M
Group # Feature Supported Direction Comment
H.324 1 H.324 Annex C Yes -- Mobile annex, as required in 3G-324M specs
H.324 2 SRP Yes In/Out
H.324 3 NSRP Yes In/Out
H.324 4 WNSRP Yes In/Out
H.324 5 CCSRL Yes In/Out
H.324 6 Audio Channels Yes In/Out
H.324 7 Video Channels Yes In/Out
H.324 8 Data Channels No --
H.324 9 Loopback Mode Yes -- External loopback support provided for development and testing. Multiplex bitstream looped within terminal prior to transmission. H.245 Maintenance loop not currently supported.
H.324 10 Encryption No --
H.324 11 Pause/Resume Yes In/Out Option to stop sending A/V data on outgoing side, or to pause rendering of data on incoming side.
H.324 12 MONA-MPC Optional In/Out Standards Based Enhancement for faster Call Setup.
Feature Group 4: Audio
Group # Feature Supported Direction Comment
Audio 1 AMR-NB Yes In/Out All modes including SID
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
30
Audio 2 G.723.1 No -- Not required for 3G-324M
FEATURE GROUP 5: VIDEO
Group # Feature Supported? Direction Comment
Video 1 H.263 baseline(Profile 0, Level 10)
Yes In/Out
Video 2 MPEG-4 Simple Profile
Yes In/Out Level 0 is baseline support; Could support higher levels as well.
Video 3 H.261 No -- Not required for 3G-324M
Video 4 H.263 additional annexes
No -- Not required for 3G-324M
Video 5 MPEG-4 Resync Markers
Yes In/Out
Video 6 MPEG-4 HEC Yes In/Out
Video 7 MPEG-4 Data Partitioning
Yes In/Out
Video 8 MPEG-4 RVLC’s
Yes In/Out
Video 9 Error concealment
Yes In As recommended in 3GPP TS 26.911. Advanced error concealment provided by PacketVideo software decoders; this may or may not be available if a non-PV video decoder is used.
OPENCORE MULTIMEDIA FRAMEWORK CAPABILITIES Last Saved on: 12/16/2009 09:36:00 AMPACKETVIDEO CONFIDENTIAL & PROPRIETARY
31
Top Related