S5pv210 Android Overlay System (video output system) Analysis

Overlay is the name that android often sees. It can be understood as video superposition, or overlay as video output.

Video output is closely related to video superposition. Generally, the output video background is the system UI Layer, while video output is superimposed on the UI Layer.

Video output and system UI generally use an independent video memory area. The system UI is displayed through framebuffer, while video output can be implemented in the driver layer in two ways: 1. framebuffer method, 2. the output device Implementation Method of v4l2 is closely related to the platform, and even some platform implementation methods, such as Freescale imx51, the application layer can achieve video output through/dev/fb2, and set/dev/video16 through output to achieve video output. Which of the following is the work of overlayhal.

The Samsung s5pv210 platform uses the output device to output videos. The device node/dev/video1 (corresponding to the fimc1 Controller) does not have to use the fimc1 controller. fimc0 fimc2 can also be used to output videos, the specific fimc used for video output is determined by the implementation of overlayhal.

See the device/Samsung/proprietary/liboverlay/v4l2_utilc.c file.

 76 int v4l2_overlay_open(int id) 77 { 78     LOG_FUNCTION_NAME 79     return open("/dev/video1", O_RDWR); 80 }

The node of the hardcode overlay device is/dev/video1.

The superposition of the primary display area and the Overlay display area is achieved through hardware. As for the hierarchical relationship between the primary FB and overlay, the blending mode is generally implemented by the driver layer; of course, the application layer can control the hierarchy, the position of the stack layer, the size rotation and transparency.

Android video output System Structure

Android video output systems include: Driver layer (output driver, framebuffer driver), overlayhal, overlay server surfaceflinger, and local framework

The overlay video output system structure is as follows:

1. Overlay driver layer: framebuffer driver, v4l2 output driver

2. overlayhal: Hardware/libhardware/include/hardware/overlay. H, and Samsung s5pv210 overlayhal are implemented under the device/Samsung/proprietary/liboverlay/directory.

3. The layerbuffer In the overlay server surfaceflinger: surfaceflinger is used on camera preview/video playback. layerbuffer provides two implementation methods: POST buffer and overlay.
4. Overlay local framework code:

Header file path: Frameworks/base/inlcude/UI

Source File Path: Frameworks/base/libs/UI

The overlay system framework is the library libui. as part of so, the method provided is used by the camera viewfinder and the render of video output. For details about how to call the camera of the Samsung platform, see device/Samsung/proprietary/libcamera/seccamerahwinterface. CPP, video output render see Samsung/proprietary/libstagefrighthw/sechardwarerenderer. CPP. You must call the overlay system in these two files.

Overlay driver layer

S5pv210 overlay involves the output driver and framebuffer driver in the driver layer.

The output driver is actually a special form driven by the fimc controller. fimc is an image processing unit on the s5pv210 platform. The input of the fimc controller can be camera sensor or MEM, and the output can be mem.

If the input is camera sensor, fimc is fimc_capture. If the input is MEM, fimc is fimc_ouput.

When fimc is used for overlay, the workflow is as follows:

1. Set the fimc input to camera preview or video decoding output data address.

2. Set the output address of fimc to the mem_base of framebuffer.

3. Start the fimc controller for rotation, scaling, and color space conversion. The fimc controller stores the processing results to the output address.

4. after fimc completes the conversion, fb_set_par is called. This function is used to indirectly call the numbers of the columns to be displayed. This step is very important. Step 2, step 3, converts the content to the memory pointed to by the framebuffer, but it is not refreshed to display. fb_set_par aims to refresh the content of framebuffer to the LCD.

Note: In Step 4, framebuffer stores the content displayed on the screen. After the content of framebuffer is rewritten, It is not automatically updated to the LCD. You must set the display controller of the CPU to display the content address register, the content will be flushed to the screen only after it points to the physical memory address of framebuffer.

The address register of the display controller only supports physical addresses, which also explains why framebuffer requires a large block of continuous physical memory.