Sharing Userspace IO devices for fast access to multimedia...
Transcript of Sharing Userspace IO devices for fast access to multimedia...
![Page 1: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/1.jpg)
Sharing Userspace IO devices for fast access to multimedia hardware
Conrad ParkerRenesas
![Page 2: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/2.jpg)
Sharing Userspace IO devices for access to fast multimedia hardware
Conrad ParkerRenesas
![Page 3: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/3.jpg)
UIO is a Linux kernel interface for accessing devices, such that the driver logic can be implemented in userspace. It is very lightweight, and provides no mechanisms for co ordinating access to the device, or management of resources associated with it.
We introduce UIOMux: a resource management layer for UIO devices, providing fine-grained locking, memory management and interrupt dispatch. It is used in the Linux multimedia architecture for SH-Mobile, and an example will be given for managing an image rescaling and colorspace conversion hardware unit (VEU). We demonstrate how multiple VEUs can be shared, how the driver logic is implemented as a shared library (libshveu), and how this is used in higher level software such as GStreamer plugins.
![Page 4: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/4.jpg)
This architecture allows coordination of access to multiple hardware units, so that zero-copy pipelines combining the VEU with camera capture and hardware video encoding/decoding can be constructed by conventional GStreamer applications.
UIOMux is implemented as a stateful shared library, allowing in-process device access by applications without context-switching.
![Page 5: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/5.jpg)
![Page 6: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/6.jpg)
![Page 7: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/7.jpg)
![Page 8: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/8.jpg)
![Page 9: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/9.jpg)
![Page 10: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/10.jpg)
![Page 11: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/11.jpg)
Ken Burns effect
Zoom and pan ...
![Page 12: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/12.jpg)
![Page 13: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/13.jpg)
![Page 14: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/14.jpg)
![Page 15: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/15.jpg)
![Page 16: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/16.jpg)
![Page 17: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/17.jpg)
IP Blocks
● VIO: CEU, VEU, BEU● JPU: JPEG● VPU: Video Processing Unit● LCDC: LCD controller● VOU: Video Output● SIU: Sound I/O● USB
![Page 18: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/18.jpg)
VPU
Video Processing Unit● MPEG-4 and H.264 accelerators● Encoding and decoding (half-duplex)● H.264 slice processing
![Page 19: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/19.jpg)
CEU
● Camera interface:● YCbCr 4:2:2 or 4:2:0● horiz/vertical sync
![Page 20: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/20.jpg)
Video Engine Unit: image processing in memory
● Scaling, cropping● Filtering: mirror, point symmetry, 90degree,
deblocking, median● Colorspace conversion
● YCbCr → RGB → YCbCr● Dithering (in RGB color subtraction)
![Page 21: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/21.jpg)
![Page 22: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/22.jpg)
![Page 23: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/23.jpg)
![Page 24: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/24.jpg)
![Page 25: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/25.jpg)
Colorspace conversion
![Page 26: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/26.jpg)
Some standard values
ITU-R BT.601
![Page 27: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/27.jpg)
Which you can substitute
![Page 28: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/28.jpg)
Convert to 8bit digital values
Scaling and rounding into the range [16,235], leaving some headroom
![Page 29: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/29.jpg)
(alternatively)
![Page 30: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/30.jpg)
Or you can solve for RGB
![Page 31: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/31.jpg)
HD video has different coefficients
ITU-R BT.709
![Page 32: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/32.jpg)
JPEG is different again
It uses the full [0,255] range
![Page 33: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/33.jpg)
Colorspace conversion
● Repetitive calculation for every pixel value● Required in all video applications● Hence it gets its own dedicated hardware block● Hey we can also rescale and crop!
![Page 34: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/34.jpg)
Linux kernel interfaces
We use existing Linux kernel interfaces to provide access to various IP blocks:
● v4l2: Video for Linux 2● fbdev: Framebuffer device● UIO: User space IO
![Page 35: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/35.jpg)
v4l2
● v4l2 (Video for Linux 2) is the standard Linux kernel interface for video capture. It contains specific interfaces for selecting capture dimensions and color format. Applications can use read() to read frame data, or alternatively can use an ioctl() to stream frame data directly into memory buffers.
● Applications can also provide buffers for v4l2 to capture into, using USERPTR
● Camera parameters like brightness and contrast can also be changed while the device is open.
![Page 36: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/36.jpg)
fbdev
● fbdev (Framebuffer device) is a standard Linux kernel interface for handling video output. It specifies routines for negotiationg screen dimensions, the color map and pixel format, and for using any available acceleration functions.
![Page 37: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/37.jpg)
UIO
UIO (Userspace IO) is a fairly recent Linux kernel mechanism for allowing device driver logic to be implemented in userspace. The kernel level UIO interface itself is generic and does not have functionality specific to any kind of device. It is currently only available for char drivers (not block or network).
Using UIO, the implementation of a device driver is split into two parts:
● a small kernel module which provides memory maps, and stubs for interrupt handling and IO
● a userspace device driver which uses normal read(), mmap() system calls to work with the device.
http://lwn.net/Articles/232575/
![Page 38: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/38.jpg)
UIO: kernel module
On the kernel side, the module provides a struct specifying:
● the IRQ number, flags and a handler() function which acknowledges the interrupt
● an array of memory areas which can be mapped into userspace
● mmap(), open(), release() functions which are called from the corresponding file_operations members
![Page 39: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/39.jpg)
UIO: user space
● The user space portion is a normal process which opens eg. /dev/uio0. This returns a pollable file descriptor from which the number of available events can be read. Normal system calls can be used to interact with this device:
● Can use poll() to wait until the file descriptor is ready to perform I/O.
● calling read() on this file descriptor returns the number of events (interrupts) since the last read. Both blocking and non-blocking reads are possible.
● mmap() should be used to map the memory areas described by the kernel space module.
![Page 40: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/40.jpg)
SH-Mobile multimedia libraries
● libuiomux
● libshveu
● libshcodecs
● omxil-sh (OpenMAX components)
● gst-shmobile (Gstreamer components)
These libraries are available under the GNU LGPL
http://github.com/kfish
![Page 41: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/41.jpg)
UIOMux
UIOMux multiplexes access to named resources, including devices which are available via UIO.
It can also be queried for whether or not a resource is available on the currently running system.
UIOMux consists of a user-level shared library, libuiomux, which manages a shared memory segment containing allocation information for each managed resource. It provides locking and memory allocation.
![Page 42: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/42.jpg)
UIOMux locking
UIOMux allows simultaneous locking of access to multiple resources, with deterministic locking and unlocking order to avoid circular waiting. Processes or threads requring simultaneous access to more than one resource should lock and unlock them simultaneously via libuiomux.
UIOMux will save and restore of memory-mapped IO registers associated with a UIO device. Registers are saved on uiomux_unlock() and restored on uiomux_lock(), if intervening users have used the device.
![Page 43: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/43.jpg)
Map management
The entire range mapped by the UIO device is implicitly available to all its users, so they must co-operatively share it and ensure not to overwrite each other's regions. The kernel is unable to provide more finely grained protection as these regions are not fixed in size at the time of UIO initialization; for example, the size of an image buffer depends on the requested dimensions.
![Page 44: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/44.jpg)
Shared state
The resource info segment and its mutexes are shared amongst all processes and threads on the system, to provide system-wide locking. In this way, libuiomux can be used to manage contention across multiple simultaneous processes and threads.
● No context-switching
![Page 45: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/45.jpg)
![Page 46: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/46.jpg)
![Page 47: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/47.jpg)
MMU provides:● Virtual ↔ physical translation● Cache control● Bus arbitration
![Page 48: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/48.jpg)
![Page 49: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/49.jpg)
● V4L2 USERPTR to capture into physically-contiguous buffer, previously allocated with uiomux_malloc()
![Page 50: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/50.jpg)
● Convert to a physical address with uiomux_virt_to_phys()
![Page 51: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/51.jpg)
● The VEU also needs the physical address of the destination buffer
![Page 52: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/52.jpg)
● Which you can convert back to a virtual address for the framebuffer using uiomux_phys_to_virt()
![Page 53: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/53.jpg)
Tunneling
In a real application you want to use various IP blocks together, like displaying the output of the VEU onto the LCD. However we don't want to have to write all this glue code each time; we want to be able to use normal media frameworks.
OpenMAX and GStreamer are abstract interfaces which generally make buffers available for software processing. However if you happen to connect two plugins together for hardware IP blocks, then you want them to share buffers and just pass pointers around and synchronize buffer access.
![Page 54: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/54.jpg)
GStreamer negotiation
GStreamer has great facilities for discovering plugin capabilities and negotiating data formats for exchange between plugins.
GStreamer-OpenMAX can use this negotiation to trigger OpenMAX tunneling between components.
![Page 55: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/55.jpg)
Plain GStreamer pipeline
![Page 56: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/56.jpg)
H/W accelerated pipeline
![Page 57: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/57.jpg)
Comparison: in-kernel framework
● In-kernel routing proposed for v4l2● In development for OMAP● Takes over a predefined set of processor
functions / hardware blocks● Appropriate for a programmable DSP pipeline● Instead we use the locking and sync that is
conventional for a particular framework
![Page 58: Sharing Userspace IO devices for fast access to multimedia ...oss.renesas.com/presentations/uiomux-lca2010.pdf · UIO (Userspace IO) is a fairly recent Linux kernel mechanism for](https://reader034.fdocuments.in/reader034/viewer/2022042808/5f84eb8578e2eb3cbe67a216/html5/thumbnails/58.jpg)
Summary
● SH-Mobile is cool hardware with nice features● We provide simple UIO interfaces to these● UIOMux lets us use these in conventional
multimedia frameworks like OpenMAX and Gstreamer