INTRODUCTION TO HAL
Android Architecture:
Android is an open source, Linux-based software stack created for a wide array of
devices and form factors. The following diagram shows the major components of
the Android platform
Hardware Abstraction Layer (HAL):
The hardware abstraction layer(HAL) provides standard interfaces that expose
device hardware capabilities to the higher-level Java API framework.
The HAL consists of multiple library modules, each of which implements an
interface for a specific type of hardware component, such as the camera or
bluetooth module.
When a framework API makes a call to access device hardware, the Android
system loads the library module for that hardware component.
Android HaL acts as a bridge between hardware and software.
Android app/Framework communicates with the underlying hardware through the
JAVA API’s but not system calls.
But the kernel has the ability to handle only system calls from application.
So for the interface purpose we need a layer in between both F.W and kernel.
Camera HAL1 overview:
Note: As Camera HAL1 has been deprecated, Camera HAL3 is recommended for
devices launching on Android 9 or higher.
Version 1 of the camera subsystem was designed as a black box with high-level
controls and the following three operating modes:
• Preview
• Video Record
• Still Capture
Each mode has slightly different and overlapping capabilities. This made it hard to
implement new features such as burst mode, which falls between two of the
operating modes.
Figure 2. Camera components
Android 7.0 continues to support camera HAL1 as many devices still rely on it. In
addition, the Android camera service supports implementing both HALs (1 and 3),
which is useful when you want to support a less-capable front-facing camera with
camera HAL1 and a more advanced back-facing camera with camera HAL3.
There is a single camera HAL module (with its own version number), which lists
multiple independent camera devices that each have their own version number.
Camera module 2 or newer is required to support devices 2 or newer, and such
camera modules can have a mix of camera device versions (this is what we mean
when we say Android supports implementing both HALs).
Limitations of HAL1:
• Based on black-box-style camera subsystem
• Limited camera control
• Dififficult to......
-->implement new modes(burst)
-->Expose deep controls(manual focus).
To overcome all the above limitations developers came up with the much
advanced version of HAL .And the main advantages of this are:
• Camera 2:not just update
• Based on update to previous camera1
• it completly depricates camera1
• More subsystem control i.e,...focus,conrol,post-processing,pre-
prosessing...etc.
• More capabilities
• Api is more efficient and maintainable.
Camera HAL3 overview:
• The app framework issues requests for captured results to the camera
subsystem. One request corresponds to one set of results.
• A request encapsulates all configuration information about the capturing
and processing of those results.
• This includes things such as resolution and pixel format; manual sensor,
lens, and flash control; 3A operating modes; RAW to YUV processing control;
and statistics generation.
• This allows for much more control over the results' output and processing.
Multiple requests can be in flight at once, and submitting requests is non-
blocking.
• The requests are always processed in the order they are received.
Summary of API use :
This is a brief summary of the steps for using the Android camera API. See the
Startup and expected operation sequence section for a detailed breakdown of
these steps, including API calls.
[Link] for and enumerate camera devices.
[Link] device and connect listeners.
[Link] outputs for target use case (such as still capture, recording, etc.).
[Link] request(s) for target use case.
[Link]/repeat requests and bursts.
[Link] result metadata and image data.
[Link] switching use cases, return to step 3.
HAL operation summary :
• Asynchronous requests for captures come from the framework.
• HAL device must process requests in order. And for each request, produce output
result metadata, and one or more output image buffers.
• First-in, first-out for requests and results, and for streams referenced by
subsequent requests.
• Timestamps must be identical for all outputs from a given request, so that the
framework can match them together if needed.
• All capture configuration and state (except for the 3A routines) is encapsulated
in the requests and results.
Figure. Camera HAL overview
Framework request sending to HAL:
• Enumerating, opening camera devices and creating an active session
[Link] initialization, the framework starts listening for any present camera
providers that implement the Iinterface. If such provider or providers are present,
the framework will try to establish a connection.
[Link] framework enumerates the camera devices via
CameraProvider::getCameraIdList().
[Link] framework instantiates a new cameraDevice by calling the
respectiveCameraProvider::getCameraDeviceInterface_VX_X().
[Link] framework calls cameraDevice::open() to create a new active capture
session IcameraDeviceSession.
Using an active camera session
[Link] framework calls CameraDeviceSession::configureStreams() with a list
of input/output streams to the HAL device.
[Link] framework requests default settings for some use cases with calls to
CameraDeviceSession::constructDefaultRequestSettings(). This may occur
at any time after the cameraDeviceSession is created by
CameraDevice::open.
[Link] framework constructs and sends the first capture request to the HAL with
settings based on one of the sets of default settings, and with at least one output
stream that has been registered earlier by the framework. This is sent to the HAL
with CameraDeviceSession::processCaptureRequest(). The HAL must block
the return of this call until it is ready for the next request to be sent.
[Link] framework continues to submit requests and calls
CameraDeviceSession::constructDefaultRequestSettings() to get default
settings buffers for other use cases as necessary.
[Link] the capture of a request begins (sensor starts exposing for the capture),
the HAL calls CameraDeviceCallback::notify() with the SHUTTER message,
including the frame number and the timestamp for start of exposure. This notify
callback does not have to happen before the firstprocessCaptureResult() call
for a request, but no results are delivered to an application for a capture until
afternotify() for that capture is called.
[Link] some pipeline delay, the HAL begins to return completed captures to the
framework with CameraDeviceCallback::processCaptureResult(). These are
returned in the same order as the requests were submitted. Multiple requests can
be in flight at once, depending on the pipeline depth of the camera HAL device.
[Link] some time, one of the following will occur:
• The framework may stop submitting new requests, wait for the existing
captures to complete (all buffers filled, all results returned), and then call
cameraDeviceSession::configureStreams() again. This resets the camera
hardware and pipeline for a new set of input/output streams. Some streams may
be reused from the previous configuration. The framework then continues from
the first capture request to the HAL, if at least one registered output stream
remains. (Otherwise, CameraDeviceSession::configureStreams() is required
first.)
• The framework may call cameraDeviceSession::close() to end the camera
session. This may be called at any time when no other calls from the framework
are active, although the call may block until all in-flight captures have completed
(all results returned, all buffers filled). After the close() call returns, no more calls
to CameraDeviceCallback are allowed from the HAL. Once the close() call is
underway, the framework may not call any other HAL device functions.
• In case of an error or other asynchronous event, the HAL must call
CameraDeviceCallback::notify() with the appropriate error/event message.
After returning from a fatal device-wide error notification, the HAL should act as if
close() had been called on it. However, the HAL must either cancel or complete
all outstanding captures before calling notify(), so that once notify() is called
with afatal error, the framework will
[Link] close() should return
ENODEV or NULL afterthe notify() method returns from a fatal error message.
HAL Logs(CAMX):
12-13 [Link].175 3741 3990 D SnapCam_CaptureModule: openCamera 0
12-13 [Link].177 3741 3990 E libc : Access denied finding property
"[Link]"
12-13 [Link].169 3741 3741 W CameraBackgroun: type=1400 audit(0.0:122):
avc: denied { read } for name="u:object_r:camera_prop:s0" dev="tmpfs"
ino=15424 scontext=u:r:priv_app:s0:c512,c768
tcontext=u:object_r:camera_prop:s0 tclass=file permissive=0
12-13 [Link].177 986 1199 I CameraService: CameraService::connect call
(PID -1 "[Link]", camera ID 0) for HAL version default and
Camera API version 2
12-13 [Link].178 986 1199 I Camera2ClientBase: Camera 0: Opened. Client:
[Link] (PID 3741, UID 10025)
12-13 [Link].178 986 1199 I CameraDeviceClient: CameraDeviceClient 0:
Opened
12-13 [Link].178 3741 3741 D CAM_Activity: send the turn off Flashlight
broadcast
12-13 [Link].179 655 655 I CamX : [ INFO][HAL ]
[Link] SetTorchModeInternal() Torch notify state is
(inReleaseMode = 0, dropCallbacks = 0)
12-13 [Link].180 986 1199 I CameraProviderManager: Camera device
device@3.3/legacy/0 torch status is now NOT_AVAILABLE
12-13 [Link].180 986 1199 I CameraService: onTorchStatusChangedLocked:
Torch status changed for cameraId=0, newStatus=0
12-13 [Link].180 986 1199 E CameraService: onTorchStatusChangedLocked:
cannot get torch status of camera 0: No such file or directory (-2)
12-13 [Link].180 655 655 I CamX : [ INFO][HAL ] [Link]
open() Open: overrideCameraClose is 0 , overrideCameraOpen is 1
12-13 [Link].180 655 655 I CamX : [ INFO][HAL ] [Link]
initialize() initialize(): 0xf3985304, 0xf4588cb0
12-13 [Link].182 3741 3990 D SnapCam_CaptureModule: onOpened 0
12-13 [Link].182 3741 3990 D SnapCam_CaptureModule: createSession 0
12-13 [Link].182 655 655 I CamX : [ INFO][HAL ] [Link]
construct_default_request_settings() Override specific tags added to
construct default settings
12-13 [Link].182 3741 4002 D OpenGLRenderer: HWUI GL Pipeline
12-13 [Link].190 3741 3741 D CameraStorage: External storage
state=mounted
12-13 [Link].192 3741 3741 D CameraStorage: External storage
state=mounted
12-13 [Link].201 3741 4002 I Adreno : QUALCOMM build :
81746dd, Ie5c8ceec2f
12-13 [Link].201 3741 4002 I Adreno : Build Date : 08/11/18
12-13 [Link].201 3741 4002 I Adreno : OpenGL ES Shader Compiler Version:
EV031.23.04.01_03
12-13 [Link].201 3741 4002 I Adreno : Local Branch :
12-13 [Link].201 3741 4002 I Adreno : Remote Branch :
12-13 [Link].201 3741 4002 I Adreno : Remote Branch :
12-13 [Link].201 3741 4002 I Adreno : Reconstruct Branch :
12-13 [Link].203 3741 4002 I Adreno : PFP: 0x016ee170, ME: 0x00000000
12-13 [Link].205 3741 4002 I zygote64:
android::hardware::configstore::V1_0::ISurfaceFlingerConfigs::hasWideColorDispla
y retrieved: 0
12-13 [Link].206 3741 4002 I OpenGLRenderer: Initialized EGL, version 1.4
12-13 [Link].206 3741 4002 D OpenGLRenderer: Swap behavior 2
12-13 [Link].219 3741 3741 V SnapCam_CaptureUI: surfaceCreated
12-13 [Link].219 3741 3741 I SnapCam_CaptureModule: Preview Surface is
ready!
12-13 [Link].219 3741 3741 V SnapCam_CaptureUI: surfaceChanged: width
=1280, height = 960
12-13 [Link].221 986 1199 W CameraDeviceClient: createSurfaceFromGbp:
Camera 0 with consumer usage flag: 2304: Forcing asynchronous mode for stream
12-13 [Link].221 986 1199 W CameraDeviceClient: createSurfaceFromGbp:
Camera 0: Overriding format 0x4 to IMPLEMENTATION_DEFINED
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() Number of streams: 2
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() stream[0] = 0xf5ba3274 - info:
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() format : 34,
HAL_PIXEL_FORMAT_IMPLEMENTATION_DEFINED
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() width : 1280
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() height : 960
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() stream_type : 00000000, CAMERA3_STREAM_OUTPUT
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() usage : 00000900
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() max_buffers : 0
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() rotation : 00000000,
CAMERA3_STREAM_ROTATION_0
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() data_space : 00000000, HAL_DATASPACE_UNKNOWN
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() priv : 0x0
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() stream[1] = 0xf5ba32d4 - info:
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() format : 33, HAL_PIXEL_FORMAT_BLOB
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() width : 3264
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() height : 2448
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() stream_type : 00000000, CAMERA3_STREAM_OUTPUT
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() usage : 00000003
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() max_buffers : 0
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() rotation : 00000000,
CAMERA3_STREAM_ROTATION_0
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() data_space : 00000101, HAL_DATASPACE_JFIF
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() priv : 0x0
12-13 [Link].223 655 655 I CamX : [ INFO][HAL ] [Link]
configure_streams() operation_mode: 0
12-13 [Link].283 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() frame_number 0
12-13 [Link].283 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() num_output_buffers 1
12-13 [Link].283 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() output_buffers 0xf3ae8b48
12-13 [Link].283 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() output_buffers[0] : 0xf3ae8b48 - info
12-13 [Link].283 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() buffer : 0xf3ae8b40
12-13 [Link].283 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() status : 00000000
12-13 [Link].283 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() stream : 0xf5ba3274
12-13 [Link].283 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() input_buffer 0x0
12-13 [Link].283 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() settings 0xf067d600
12-13 [Link].283 655 655 I CamX : [ INFO][HAL ]
[Link] ProcessCaptureRequest() CHIOverride: Original
framework framenumber 0 contains 1 output buffers
12-13 [Link].283 655 655 E CHIUSECASE: [Link] FillCameraId()
FillCameraId 0
12-13 [Link].283 655 655 I CamX : [ INFO][HAL ] [Link]
ProcessCaptureRequest() Submit to pipeline index: 0 / number of pipelines: 1
batched index 0
12-13 [Link].283 655 655 I CamX : [ INFO][HAL ] [Link]
ProcessCaptureRequest() Added Sequence ID 0 framework framenumber 0 to
request queue and launched job withrequest id 1
12-13 [Link].283 655 837 I CamX : [ INFO][HAL ] [Link]
ProcessRequest() Sequence ID 0 assigned for framework frameNumber 0,
RequestId: 1 and m_requestQueueDepth 8
12-13 [Link].284 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() frame_number 1
12-13 [Link].284 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() num_output_buffers 1
12-13 [Link].284 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() output_buffers 0xf3ae8bf0
12-13 [Link].284 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() output_buffers[0] : 0xf3ae8bf0 - info
12-13 [Link].284 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() buffer : 0xf3ae8be8
12-13 [Link].284 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() status : 00000000
12-13 [Link].284 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() stream : 0xf5ba3274
12-13 [Link].284 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() input_buffer 0x0
12-13 [Link].284 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() settings 0x0
12-13 [Link].284 655 655 I CamX : [ INFO][HAL ]
[Link] ProcessCaptureRequest() CHIOverride: Original
framework framenumber 1 contains 1 output buffers
12-13 [Link].298 655 836 I CamX : [ INFO][HAL ] [Link]
ProcessResults() Processing result for Sequence ID 1 and metadataReady 0
bufferReady = 0
12-13 [Link].298 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() num_output_buffers 1
12-13 [Link].298 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() output_buffers 0xf3ae8c38
12-13 [Link].298 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() output_buffers[0] : 0xf3ae8c38 - info
12-13 [Link].298 655 655 I CamX : [ INFO][HAL ] [Link]
process_capture_request() buffer : 0xf3ae8c18
Call flow:
• Before opening the camera the camera manager which is having the
getCameraIdList() will get the all the characteristics of the camera by using
the api getCameraCharacteristics().
• After getting all the characteristics of the camera the framework will now
call Camera_device_open() which is in [Link].
Opencamera():
• By checking which hal the device is supporting then approximately those
API’s are being called.
• If the device is suppoting HAL1:
-->Then gQCameraMuxer which is an object for the class QcameraMuxer
points to a sub API camera_device_open present in [Link].
• If the device is suppoting HAL3 then:
-->gQcamera2Factory which is an object to the Qcamera2Factory points
to an sub API cameraDeviceOpen.
• After this API is defined now the framework is going to call Initialize() which
is a member function like all the remaining sub API’s.
• The list of those all sub API’s present in the HAL are as given below.
• Here functions on the left side are the function pointers in the
Camera3_device_ops_t structure.
• where as the functions calling on the right side are the member functions of
the Qcamera3HardwareInterface class.
• After calling opencamera() API framework now will call Initialize() API.
Initialize():
• Here initalize(callback_ops) is the sub API pointed by the hw variable.
• After initialization construct_default_request_settings() can be called at any
instance from the framework.
construct_default_request_settings:
• After calling the construct_default_request_settings() the framework will
now call configure_streams().
configure_streams():
• In this configure_streams() function, streams initialization and
channel_initialization will be done depending on the stream type and stream
format.
• It will then check for the pending request and then reset them.
• Taking all this configurations and stream_list the framework will move
forward for calling the process_capture_request() function.
• Now the sub API configurestreams() which has a single argument is called.
Process_capture_request:
Dump:
➢ Print out debugging state for the camera device.
➢ This will be called by the framework when the camera service is asked for a
debug dump, which happens when using the dumpsys tool, or when
capturing a bugreport.
➢ The passed-in file descriptor can be used to write debugging text using
dprintf() or write(). The text should be in ASCII encoding only.
➢ This must be a non-blocking call. The HAL should return from this call in
1ms, must return from this call in 10ms.
➢ This call must avoid deadlocks, as it may be called at any point during
camera operation.
➢ Any synchronization primitives used (such as mutex locks or semaphores)
should be acquired with a timeout.
static void dump(const struct camera3_device *, int fd);
This function is static member function to the class
“Qcamera3HardwareInterface”.
• void dump(int fd);It is a member function to the class
“Qcamera3HardwareInterface”.
• The sub API dump(fd) is called for the further process.
Flush:
• Flush all currently in-process captures and all buffers in the pipeline on the
given device.
• The framework will use this to dump all state as quickly as possible in order
to prepare for a configure_streams() call.
• No buffers are required to be successfully returned, so every buffer held at
the time of flush() (whether successfully filled or not) may be returned with
CAMERA3_BUFFER_STATUS_ERROR. Note the HAL is still allowed to return
valid (CAMERA3_BUFFER_STATUS_OK) buffers during this call, provided they
are successfully filled.
• All requests currently in the HAL are expected to be returned as soon as
possible. Not-in-process requests should return errors immediately. Any
interruptible hardware blocks should be stopped, and any uninterruptible
blocks should be waited on.
• flush() may be called concurrently to process_capture_request(), with the
expectation that process_capture_request will return quickly and the
request submitted in that process_capture_request call is treated like all
other in-flight requests. Due to concurrency issues, it is possible that from
the HAL's point of view, a process_capture_request() call may be started
after flush has been invoked but has not returned yet. If such a call happens
before flush() returns, the HAL should treat the new capture request like
other in-flight pending requests.
static int flush(const struct camera3_device *);
This function is static member function to the class
“Qcamera3HardwareInterface”.
• After calling the flush() function there will be a sub API flush(hw-
>mFlushRestart) is called.
Call back functions:
There are 4 call back functions to the [Link] are:
1)captureResultCb()
2)dataNotifyCB()
3)dataProcRoutine()
4)streamCbRoutine().
Reference Documents:
[Link]
[Link]
summary
[Link]
e
