Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer...
-
Upload
juliet-young -
Category
Documents
-
view
219 -
download
1
Transcript of Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer...
![Page 1: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/1.jpg)
Embedded Streaming Media with GStreamer
and BeagleBoard
ESC-321Presented by Todd Fischer
![Page 2: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/2.jpg)
Agenda
•Introduction to BeagleBoard multimedia architecture features.
•GStreamer concepts
•OMAP GStreamer hands on exercises
•DMAI and GStreamer
•OpenMAX and GStreamer
•Questions
![Page 3: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/3.jpg)
Introduction
•Primary objective for this class is to introduce audience to GStreamer on OMAP3 with hands on exercises
•This doesn’t class won't cover extensively the APIs available and assumes basic knowledge on C programming, Linux and computer graphics.
![Page 4: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/4.jpg)
BeagleBoard and OMAP3 architecture
•Provides architecture with several multimedia features:
•Cortex A8 with Neon
•C64x DSP
•Video accelerators
•How to utilize the hardware features with the software stack?
![Page 5: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/5.jpg)
BeagleBoard and OMAP3 architecture
•Provides architecture with several multimedia features:
•Cortex A8 with Neon
•C64x DSP
•Video accelerators
•How to utilize the hardware features with the software stack?
![Page 6: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/6.jpg)
BeagleBoard and OMAP3 architecture
•Provides architecture with several multimedia features:
•Cortex A8 with Neon
•C64x DSP
•Video accelerators
•How to utilize the hardware features with the software stack?
![Page 7: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/7.jpg)
GStreamer•Streaming media framework – audio and
video
•Close to 200 plug-ins available
•Higher level than just input / filters / output
Networking, audio/video mixed streams, auto data handling
•Various options utilizing hardware accelerators
![Page 8: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/8.jpg)
GStreamer Overview•Elements
•Sources, filters, sinks
•Bins and Pipelines
•Containers, pipeline is the overall bin
•Pads
•Element source / sink connection points
•Caps
•Capabilities organized by stream type with a set of properties
![Page 9: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/9.jpg)
GStreamer Overview•Elements
•Sources, filters, sinks
•Bins and Pipelines
•Containers, pipeline is the overall bin
•Pads
•Element source / sink connection points
•Caps
•Capabilities organized by stream type with a set of properties
•Plugin•Collection of elements
![Page 10: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/10.jpg)
Simple MP3 Player•Create dynamically using gst-launch
•Source element reads from a file
•Filter element converts MP3 to PWM
•Sink element passes to ALSA output
gst-launch filesrc location=a.mp3 ! mad ! alsasink
filesrc mad alsasink
location=music.mp3
src sink sinksrc
pipeline
![Page 11: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/11.jpg)
Hands On Exercise 0
•Start up Windowing environment with two terminals
cd gst
. src
Period character to source the filee
![Page 12: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/12.jpg)
Hands On Exercise 0
•Play audio file
a2•Actual command
gst-launch filesrc
location=bbb.flac !
flacdec ! alsasink
![Page 13: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/13.jpg)
Simple Audio PlayerSource Code
•Create pipeline, source, filter, sink
•Set element properties
•Build into pipeline
•Connect src and sink pads
•Setup pipeline event handler
•End of stream
•Set pipeline state to play
•Run
![Page 14: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/14.jpg)
Simple PA System•Create dynamically using gst-launch
•Source element ALSS audio in
•No filters
•Sink element passes to ALSA output
gst-launch alsasrc num-buffers=100 ! alsasink
alsasrc alsasinksrc sink
pipeline
![Page 15: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/15.jpg)
Keeping Plug-ins Organized
•Each known plug-in is added to registry
•Most aspects of plug-in are tracked in the registry
•Registry support run-in pipeline creation and dynamic filter selection
•Use gst-inspect to list plug-ins
gst-inspect | lessgst-inspect filesrc
![Page 16: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/16.jpg)
Hands On Exercise 1
•Using gst-inspect, list
•All plug-ins
•All video plug-ins
•Element properties for filesrc plug-in
![Page 17: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/17.jpg)
Hands On Exercise 2•GStreamer video pipelines
gst-launch videotestsrc ! ffmpegcolorspace ! fbdevsink
![Page 18: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/18.jpg)
PerformanceData Passing
•Stream held in buffers with data, timestamp, other info
•When possible, buffer memory allocated by sink pad
•Use hardware when data copy is necessary
![Page 19: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/19.jpg)
PerformanceData Transformation•Cortex A8 compiler optimization
•NEON
•Single Instruction Multiple Data
•C64
•Video accelerator
•DMA and other data movers
![Page 20: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/20.jpg)
Davinci MultimediaApplication Interface
AR
M C
orte
x w
ith N
EO
N
GStreamer Libraries
gstplug-in
gstplug-in
DMAIplug-in
Davinci MultimediaApplication Interface
audio video displayframecopy
Codec Engine
VISA OSAL CMEM
DSP Link
Cortex Linux Application
IVA
2 w
ith C
64 D
SP
video hardware accelerator
DSP Link
DSP BIOS
Framework Components
codec codecvideocodec
![Page 21: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/21.jpg)
DMAI and GStreamer
•Davinci Multimedia Application Interface
•Exposes OMAP/Davinci hardware using high level of abstraction
•Stream audio / video
•Graphics display
•Hardware optimized frame/data copy
![Page 22: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/22.jpg)
OMAP3 Codec Engine•Isolates users for audio/video codecs
from those implementing the codecs
•Codec can run in several places without calling application being aware
•Cortex A8, NEON, C64, hardware accelerator
•Uses DSPLink and DSPBios conventions to support DSP based algorithms dynamically
![Page 23: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/23.jpg)
Video Phone
•Vision – video phone application can create audio and video GStreamer pipelines for capture / rendering
•DMAI, Codec engine, DSPLink, C64 algorithms perform heavy lifting
•OMAP3 hardware capabilities utilized
•Video phone application focused on call management, not streaming data manipulation
![Page 24: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/24.jpg)
OpenMAX and GStreamer
•Addresses codecs using hardware accelerators
•GStreamer integration via gst-openmax
•All OpenMax codecs available as GStreamer plug-ins
•Initial focus on high performance codecs
Application LayerDefines high-level playback
and recording API
Hardware
Multimedia Application
Integration LayerDefines media component interfaces
Development LayerDefines media primitives
and concurrency constructs
![Page 25: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/25.jpg)
OpenMAX and OMAP3
•ARM Inc has Cortex-A8 / NEON DL implementation
•Texas Instruments has AAC and MPEG4 IL / DL support
Application LayerDefines high-level playback
and recording API
Hardware
Multimedia Application
Integration LayerDefines media component interfaces
Development LayerDefines media primitives
and concurrency constructs
![Page 26: Embedded Streaming Media with GStreamer and BeagleBoard ESC-321 Presented by Todd Fischer todd.fischer@ridgerun.com.](https://reader030.fdocuments.in/reader030/viewer/2022032707/56649e555503460f94b4be07/html5/thumbnails/26.jpg)
GStreamer in3 Layers
API for Super Fast Hardware
GStreamer Media Handling
Your Way Cool Application