Camera2 API, SHIM, and HAL 3.2 in Android 5.1

OUTLINE

Android camera API history

What’s new of Camera API 2

HAL 3.2

Titan Camera API 2 situation

Solutions & Hacks

Conclusions

API Lv 21

Camera API 1Camera API 1(camera)Camera API 2(camera2)

API Lv ##

Camera API 2

HISTORY

ICS

HAL 1.0

JB 4.2 JB 4.3

HAL 1.0 - 3.1HAL 3.2

HAL 1.0HAL 2.0HAL 3.0

KK

HAL 1.0HAL 2.0HAL 3.0HAL 3.1

L

HISTORY

WHAT’S NEW OF CAMERA API 2?

camera & camera2 are both available to App Developer

BUT - only HAL 3.2 enables all features/properties of camera2

Shim layer

being used to as the Facade layer between new camera2 API and older HAL 1.0-3.1 by using camera API



much more metadata, and per-frame metadata.

more direct controls on sensor and Post Processing

Synchronisation of setting and captures

Full rate, full-res captures in various formats

Support raw image capturing and saving

Camera API 1 is necessary for legacy applications

HAL 3.2

support vendor tags by camera module HAL

support partial metadata result

enables all features/properties of camera2

Not support stream configurations for still captures

Framework assumes high-res JPEG stream can be configured at all times without showing down preview framerate.

HAL 3.2

HAL 3.2

FULL mode

LIMITED mode

HAL 1.0 - 3.1

LEGACY Mode

HAL 3.2

Step 1Step 2Step 3

TITAN CAMERA API 2 SITUATION


CaptureRequest

frame meta

CaptureRequest

frame meta

camera2 HAL 3

CaptureRequest

frame meta

CaptureRequest

frame meta

CaptureRequest

frame meta

CaptureRequest

frame meta


CaptureRequest

frame meta

CaptureRequest

frame meta

camera2 HAL 1

CaptureRequest

frame meta

CaptureRequest

frame meta

SHIM

CaptureRequest

frame meta

CaptureRequest

frame meta


Arcsoft involved to use camera2 only for its camera app.

QCT never support YUV sensor with HAL 3, and Titan’s camera solution was forced to use HAL 1.

Vertu decide to use HAL 1 + shim + camera 2 in Apr 2015

Vertu would like to have a Legacy+ device.


Arcsoft involved to use camera2 only for its camera app.

QCT never support YUV sensor with HAL 3, and Titan’s camera solution was forced to use HAL 1.

Vertu decide to use HAL 1 + shim + camera 2 in Apr 2015

Vertu would like to have a Vertu specific Legacy+ mode for Titan


LEGACY+ mode

slowmo/HFR support

Cont. shot/ZSL/BurstCap support (abandoned by Vertu at wk 41)

PICAUTO support

AS HDR support

YUV420 to NV12/21, not YV12 format


mediaserver

android.hardware.Camera

android.hardware.camera2.*

android.hardware.camera2.legacy.*

CameraService(Native)

Camera Module & Device HAL

App

Camera2 Legacy Mode

SHIM Layer


CameraDeviceUserShimCameraManager

LegacyCameraDevice

RequestThreadManager

GLThreadManager

CaptureCollector


RequestThreadManager

GLThreadManager

LegacyMetadataMapper

SurfaceTextureRender

LegacyRequestMapper

LegacyResultMapper

LegacyXXXMapper

LegacyCameraDevice

SOLUTIONS & HACKS

LegacyMetadataMapper

be used for create camera static characteristic from HAL 1

Legacy+ mode

convert HFR support from HAL 1 to HIGH_SPEED_VIDEO property and export it to camera characteristic

export AS HDR, PICAUTO to camera characteristic

export scene mode FOOD/BACKLIT to camera characteristic

SOLUTIONS & HACKS

LegacyRequestMapper

be used for mapping camera2 request to parameters for HAL 1

Legacy+ mode

adjust fps and HFR mode from HIGH_SPEED_VIDEO setting

set AS HDR and PICAUTO flag

wrapper and convert VERTU specific scene mode to pass CTS

Cont. shot/ZSL/Burst shot request process

SOLUTIONS & HACKS

LegacyResultMapper

be used for mapping camera2 request to parameters for HAL 1

Legacy+ mode

based on HIGH_SPEED_VIDEO setting, convert correct camera2 result back.

based on AS HDR and PICAUTO flag, convert correct camera2 result back.

wrapper and convert VERTU specific scene mode to pass CTS

SOLUTIONS & HACKS

More and more?

camera2 settings are stored in native space and extensions are supported by vendor tags. (not available for Legacy mode)

We must revise the accessor to native space for extra supported features with Java space support

camera native framework need to be extended (LegacyXXXMapper)

HAL 1 must be greatly modified for supports of all extra features.

SOLUTIONS & HACKS

More and more?

Performance for shim is really not good

parts of shim are not completed, cause most of all stability problem

not only HAL 1, but underly QCOM camera system also need to be customised repetitively

CONCLUSIONS

Shim is just a start, what is the next?

Performance tuning, shim is really not good in performance

Complete the shim, parts of shim are not done by Google, which cause most of all stability problem

Go to lower layer. Not only HAL 1, but underly QCOM camera system also need to be customised repetitively

PICAUTO Integration

AS HDR Integration

Camera2 API, SHIM, and HAL 3.2 in Android 5.1

Software

Transcript of Camera2 API, SHIM, and HAL 3.2 in Android 5.1