Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get...
Transcript of Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get...
![Page 1: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/1.jpg)
Technology Consulting Company IGEL Co.,Ltd.
Technology Consulting Company Research, Development & Global Standard
Using Linux Media Controller for Wayland/Weston Renderer
Takanari Hayama [email protected] http://www.igel.co.jp/
1
![Page 2: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/2.jpg)
• Wayland/Weston Overview
• Porting Weston to R-Car
• Why Linux Media Controller Renderer?
• Linux Media Controller Framework
• V4L2 Renderer Design
• Conclusions
2
Agenda
![Page 3: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/3.jpg)
WAYLAND/WESTON OVERVIEW �
3
![Page 4: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/4.jpg)
Weston Architecture
Wayland Client
Weston
Wayland Client
Compositor Core
Compositor (e.g. DRM)
Renderer (e.g. GL)
Shell (e.g. desktop)
Shell Client (e.g. desktop-shell)
Wayland IPCLocal API Call
Confidential 4
![Page 5: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/5.jpg)
Weston w/ DRM Backend
5
Wayland Client
Wayland Client Stub
Weston (Wayland Server)
Wayland Server StubOpenGL/ES
OpenGL/ES
compositor-drm
gl-renderer
shell
libgbm/libdrm
compositor-core
pixman-renderer
libpximan
GPU Kernel Driver
DRM/KMS Driver
Qt, EFL, etc.
Weston Components
Wayland Components
Other Components
IPC API Call
User Space
Kernel Space
![Page 6: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/6.jpg)
wa
yla
nd
p
roto
co
l
6
Rendering and Composition: Overview (GL-Renderer)
Wayland Client
OpenGL/ES + wayland_egl
GPU driver for Wayland
Weston / DRM Compositor
OpenGL/ES + WL Ext. GBM
DRM/KMS
GPU
GPU Driver for Wayland
Display
wl_egl_window Type
wl_egl_window Type
RENDERING �
DISPLAYING �
wl_buffer Type
wl_buffer Type
FRAME BUFFERS �gbm_surface Type
CLIENT BUFFERS �wl_buffer Type
FULL SCREEN�wl_buffer Type DRM/KMS BO Type
SCANOUT BUFFERS �
Software�
Hardware�
Wayland/Weston Components
Khronos w/ Wayland Ext.
OSS Components
Hardware Specific
![Page 7: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/7.jpg)
wa
yla
nd
p
roto
co
l
7
Rendering and Composition: Window Composition
Wayland Client
OpenGL/ES + wayland_egl
GPU driver for Wayland
Weston / DRM Compositor
OpenGL/ES + WL Ext. GBM
DRM/KMS
GPU
GPU driver for Wayland
Display
wl_egl_window Type
wl_egl_window Type
RENDERING �
DISPLAYING �
wl_buffer Type
wl_buffer Type
FRAME BUFFERS �gbm_surface Type
CLIENT BUFFERS �wl_buffer Type
FULL SCREEN�wl_buffer Type DRM/KMS BO Type
SCANOUT BUFFERS �
Software�
Hardware�
Wayland/Weston Components
Khronos w/ Wayland Ext.
OSS Components
Hardware Specific
1. Render w/OpenGL/ES
2.Commit buffers w/ eglSwapBuffers()
3. Import w/ eglCreateImageKHR()
6. Set composed buffers as KMS BOs.
5. Compose w/ OpenGL/ES
4. Register destination frame buffers allocated with GBM.
![Page 8: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/8.jpg)
wa
yla
nd
p
roto
co
l
8
Rendering and Composition: Full Screen or Sprite Rendering
Wayland Client
OpenGL/ES + wayland_egl
GPU driver for Wayland
Weston / DRM Compositor
OpenGL/ES + WL Ext. GBM
DRM/KMS
GPU
GPU driver for Wayland
Display
wl_egl_window Type
wl_egl_window Type
RENDERING �
DISPLAYING �
wl_buffer Type
wl_buffer Type
FRAME BUFFERS �gbm_surface Type
CLIENT BUFFERS �wl_buffer Type
FULL SCREEN�wl_buffer Type DRM/KMS BO Type
SCANOUT BUFFERS �
Software�
Hardware�
Wayland/Weston Components
Khronos w/ Wayland Ext.
OSS Components
Hardware Specific
1. Render w/OpenGL/ES
2.Commit buffers w/ eglSwapBuffers()
3. Import w/ gbm_bo_import()
4. Set composed buffers as KMS BOs.
![Page 9: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/9.jpg)
PORTING WESTON TO R-CAR �
9
![Page 10: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/10.jpg)
1. OpenGL/ES for Wayland/Weston
2. Zero Copy Mechanism for Native Buffer
“Typically, hardware enabling includes modesetting/display and EGL/GLES2. On top of that, Wayland needs a way to share buffers efficiently between processes.”http://wayland.freedesktop.org/architecture.html
10
What Are Required?
![Page 11: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/11.jpg)
• Must support the following Native Display Types for eglGetDisplay(): – wl_display for clients – gbm handle for Weston
• Must support the following EGL_EXTENSIONs: – EGL_KHR_image_pixmap – EGL_WL_bind_wayland_display
• Must support the following Native Pixmap Type for eglCreateImageKHR(): – EGL_WAYLAND_BUFFER_WL
• Must support the following Wayland extension APIs: – eglBindWaylandDisplayWL – eglUnbindWaylandDisplayWL – eglQueryWaylandBufferWL
11
Wayland Requirements for OpenGL/ES
![Page 12: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/12.jpg)
WSEGL for Wayland
Weston for Renesas R-Car
12
Wayland Client
Renesas OpenGL/ES
libgbm w/ KMS Backend
wayland-kms
Wayland/Weston Components
New OSS Components
Standard OSS Components IPC API Call
Wayland Client Support Wayland Server Support
Weston (Wayland Server)
Wayland Server Stub
EGL with Wayland Extension
Wayland Client Stub
libdrm & libkms
Proprietary
wayland-egl
![Page 13: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/13.jpg)
1. A client creates a wl_surface on the server.
2. The client attach a wl_buffer to the created surface.
3. The client submit the wl_buffer to the server.
4. The server takes the wl_buffer and compose to the screen.
All of above should happen in zero-copy manner!�
13
Wayland Composition Revisited
![Page 14: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/14.jpg)
• An abstract data type that represents a reference to a pixel buffer.
• 2 open source implementations: – wl_shm : wayland standard
• Based on Linux shared memory. Not physically contiguous.
– wl_drm : Mesa standard • Based on DRI. Possibly physically contiguous.
• Weston understands wl_shm only. Wl_drm is Mesa specific. Thus, wl_drm is not handled by Weston, but by Mesa internally.
14
What is wl_buffer by the way?
![Page 15: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/15.jpg)
• Requirements – End-to-end Buffer Zero Copy – Physically Contiguous Memory
• wl_drm? – Implementation is too Mesa dependent.
• Need more generic implementation.
15
Which wl_buffer implementation to use?
![Page 16: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/16.jpg)
• KMS BO buffer type. – https://github.com/thayama/wayland-kms – Based on wl_drm in Mesa.
• Imports DMABUF via PRIME, a dma-buf
interface layer in DRM. – Originally, we used DRM Handle, but we now
use DMABUF instead.
• Can directly pass video output from V4L2.
16
wl_kms
![Page 17: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/17.jpg)
17
Buffer Zero Copying with wl_kms
Wayland Client
Wayland Client Stub
Weston (Wayland Server)
Wayland Server StubOpenGL/ES gl-renderer
compositor-drm
KMS BO
User Space
Kernel Space
Wayland GPU Driver
KMS Driver
OpenGL/ESWayland
GPU Driver
![Page 18: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/18.jpg)
18
Buffer Zero Copying with wl_kms
Wayland Client
Wayland Client Stub
Weston (Wayland Server)
Wayland Server StubOpenGL/ES gl-renderer
compositor-drm
KMS BO
User Space
Kernel Space
Wayland GPU Driver
KMS Driver
OpenGL/ESWayland
GPU DriverBuffer Allocation. Client creates rendering surface with EGL API. WSEGL allocates memory with KMS BO, and make avaialable to GPU.
![Page 19: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/19.jpg)
19
Buffer Zero Copying with wl_kms
Wayland Client
Wayland Client Stub
Weston (Wayland Server)
Wayland Server StubOpenGL/ES gl-renderer
compositor-drm
KMS BO
User Space
Kernel Space
Wayland GPU Driver
KMS Driver
OpenGL/ESWayland
GPU DriverRendering to the allocated buffer. GPU gets all details needed about the buffer via WSEGL.
![Page 20: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/20.jpg)
20
Buffer Zero Copying with wl_kms
Wayland Client
Wayland Client Stub
Weston (Wayland Server)
Wayland Server StubOpenGL/ES gl-renderer
compositor-drm
KMS BO
User Space
Kernel Space
Wayland GPU Driver
KMS Driver
OpenGL/ESWayland
GPU Driver
When a client calls eglSwapBuffers(), WSEGL commits a buffer to the server via Wayland. The details of the buffer is DMABUF fd, a stride, a size, and a pixelf ormat.
![Page 21: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/21.jpg)
21
Buffer Zero Copying with wl_kms
Wayland Client
Wayland Client Stub
Weston (Wayland Server)
Wayland Server StubOpenGL/ES gl-renderer
compositor-drm
KMS BO
User Space
Kernel Space
Wayland GPU Driver
KMS Driver
OpenGL/ESWayland
GPU Driver
When the server receives the buffer, it imports with eglCreateImageKHR().
![Page 22: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/22.jpg)
22
Buffer Zero Copying with wl_kms
Wayland Client
Wayland Client Stub
Weston (Wayland Server)
Wayland Server StubOpenGL/ES gl-renderer
compositor-drm
KMS BO
User Space
Kernel Space
Wayland GPU Driver
KMS Driver
OpenGL/ESWayland
GPU Driver
WSEGL gets details of the buffer from wayland-kms, and asks to import the given DMABUF. The buffer is then made available to GPU.
![Page 23: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/23.jpg)
23
Buffer Zero Copying with wl_kms
Wayland Client
Wayland Client Stub
Weston (Wayland Server)
Wayland Server StubOpenGL/ES gl-renderer
compositor-drm
KMS BO
User Space
Kernel Space
Wayland GPU Driver
KMS Driver
OpenGL/ESWayland
GPU Driver
Gl-renderer can now refer the buffer passed by the client, and composes a final output.
![Page 24: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/24.jpg)
WHY LINUX MEDIA CONTROLLER RENDERER? �
24
![Page 25: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/25.jpg)
• Applications are heading towards more and more GPU intensive.
• People want to use GPU for more advanced UI, rather than a simple window composition. – On the other hand, some people want to do
more complex composition using GPU. J
• GPU Offloading is one way. – https://archive.fosdem.org/2014/schedule/
event/wayland_gpu/ – But, still premature for real products.
25
Motivation
![Page 26: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/26.jpg)
• Not many embedded SoC has multiple GPUs.
• However, they often have sophisticated hardware for video signal processing that allow to do 2D blending.
• Why not use them?
26
Simpler Approach?
![Page 27: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/27.jpg)
LINUX MEDIA CONTROLLER FRAMEWORK �
27
![Page 28: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/28.jpg)
• Just a Video4Linux2 device.
• Make V4L2 media device parameters and pipelines configurable from user space. – http://linuxtv.org/downloads/presentations/
summit_jun_2010/20100206-fosdem.pdf
• Important keywords: Entities, Pads, and Links.
28
What is Media Controller?
![Page 29: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/29.jpg)
29
Media Controller
Entity
Entity
1
0
0
1
Entity
0
2
1
Link
Entity
Source Pad
Sink Pad
May have multiple sink
pads
![Page 30: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/30.jpg)
• A standard Linux API to configure complicated media devices from user space.
• On Renesas R-Car, VSP1, a device for video signal processing, is exposed via Media Controller API in Linux.
• Zero-copy could be easily achieved via DMABUF. Ideal for our use case. – Can use wl_kms!
30
Why Media Controller?
![Page 31: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/31.jpg)
• Supported Features – Scaling – Cropping – Pixel Format Conversion – 4 to 1 Blending
• As pipelines in VSP1 are configurable, best suits to Media Controller Framework!
31
Renesas R-Car VSP1
![Page 32: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/32.jpg)
V4L2 RENDERER FOR WESTON�
32
![Page 33: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/33.jpg)
V4L2 Renderer Support in Weston
33
Wayland Client
Wayland Client Stub
Weston (Wayland Server)
Wayland Server StubOpenGL/ES
v4l2-renderer device for VSPD
compositor-drm
v4l2-renderer
shell
libgbm/libdrm
compositor-core
GPU Kernel Driver
DRM/KMS Driver
Qt, EFL, etc.
Weston Components
Wayland Components
Other Components
IPC API Call
User Space
Kernel Space VSP V4L2
Driver
New Components
New components added to Weston. Plus Minimum changes to the existing compositor-drm to host a v4l2-renderer.
![Page 34: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/34.jpg)
• Kept as minimum as possible.
• Changes are to load v4l2-renderer, and pass output buffers to v4l2-renderer. Almost same as those of for pixman-renderer.
• Minor changes on output buffer allocations; DMABUF export and a read permission to mmap’d output buffer are added.
34
Changes to compositor-drm
![Page 35: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/35.jpg)
• Media device agnostic layer.
• Does everything needed to import wl_kms and wl_shm buffer to V4L2 Media Controller arena, i.e. DMABUF.
• Calculations required to figure out source regions and destination regions are done in v4l2-renderer.
• Anything that are not media device specific is handled in v4l2-renderer.
35
v4l2-renderer
![Page 36: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/36.jpg)
• Media device specific layer.
• Does everything specific to the media devices. – Media Controller Framework requires the background
knowledge of the underlying media devices.
• Does actual job to compose surfaces specified as DMABUF from v4l2-renderer.
36
V4L2 renderer device
![Page 37: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/37.jpg)
API � Descriptions�init Initialize a v4l2 media controller device.
create_output Create an output. No buffer passed yet.
set_output_buffer Set an output buffer for the output.
create_surface Create a surface. No buffer passed yet.
attach_buffer Set a buffer for the surface.
begin_compose Begin a new composition.
finish_compose Finish the composition.
draw_view Compose the surface.
get_capabilities Get capabilities of the V4L2 Renderer Device.
37
V4L2 Renderer Device API
![Page 38: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/38.jpg)
• V4L2 Renderer is not official yet. – https://github.com/thayama/weston
38
Current Status
![Page 39: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/39.jpg)
CONCLUSIONS �
39
![Page 40: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/40.jpg)
• Yes and No.
• Configuring parameters and links are not free. Not really great if we need to configure media device very often. – DRI could be alternative.
• On the other hand, use of the Linux standard features including DMABUF is great for extensibility and flexibility.
40
Was Media Controller the Right Choice?
![Page 41: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/41.jpg)
• Yes and No.
• Other renderers help you implementing new renderer.
• Geometry was the most complex part of Weston. – Global coordinate, Output local coordinate, and
view coordinate.
• Renderers are responsible for understanding these coordinates and rendering views to the correct location. – Pixman-renderer can give you some idea; how
complicated it is.
41
Was implementing new renderer easy?
![Page 42: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/42.jpg)
42
Coordinate System in Weston
Output 0Output 1
opaque region
View position is in Global Coordinate.
Rendering needs to take care of output local coordinate.
Opaque region is specified in View Coordinate.
![Page 43: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/43.jpg)
APPENDIX �
43
![Page 44: Using Linux Media Controller for Wayland/Weston Renderer · PDF fileget_capabilities Get capabilities of the V4L2 Renderer Device. 37 ... wayland-kms A subclass of wl_buffer for to](https://reader034.fdocuments.in/reader034/viewer/2022051522/5a721e537f8b9a93538d7903/html5/thumbnails/44.jpg)
Components used in Renesas OpenGL/ES for Wayland
Components � Descriptions�EGL with Wayland Extension
A thin layer to support Wayland specific EGL APIs and a native buffer type for eglCreateImageKHR(). https://github.com/thayama/libegl
wayland-kms A subclass of wl_buffer for to pass KMS BO. Defines a wl_kms wayland protocol, and server side codes. https://github.com/thayama/wayland-kms
libgbm w/ KMS Backend GBM frontend extracted from Mesa and used in Weston with a KMS Backend support. https://github.com/thayama/libgbm
WSEGL for Wayland A bridge component between GPU and Wayland.
44