Android graphics subsystem details

Graphical operations can be implemented in two ways: one is to use a general CPU to simulate graphical operations, and the other is to use a GPU to perform graphic operations. The former will increase the CPU burden. When high resolution is now a common phenomenon, it is unrealistic to let the general-purpose processor complete a large amount of graphic computing. The development process of the Android graphics system also proves this point.

To achieve efficient graphic processing, software and hardware must be closely integrated. This article mainly introduces the graphics subsystem of Android. These topics may be further explored in the future.

Software Architecture of Android Graphics System

The following shows the software module for Graphic Processing on Android.

Relationship Diagram of various graphic system components in a typical Android Application

GPU:

GPU is designed to accelerate graphic operations. Unlike CPUs, a gpu is designed for highly parallel processing, which is a common feature of most graphic computing.

When Android was just launched, GPU was still optional. In the latest version, GPU is already a required hardware. If there is no GPU in the system, the OpenGL ES used by the system includes libagl and pixelflinger. the OpenGL ES protocol interface is implemented through software, and sometimes CopyBit is supported by hardware. Unfortunately, Android simulates OpenGL through software and does not support OpenGL ES 2.0. Currently, many components in the Android system use OpenGL ES 2.0, such as HWUI, Renderscript, and SurfaceTexture. Tablet computers have a high resolution. The simulation support of pure software does not ensure that the filling of graphics is required, so it cannot provide users with a smooth UI experience. If a vendor wants to create a device based on the ICS or later Android system, it must have a GPU that supports OpenGL ES 2.0.

Canvas:

A canvas is the place where an application is used to draw widgets, graphics, and other elements. On Froyo and Gingerbread, the canvas is drawn using Skia. Honeycomb and later versions, HWUI is added to provide GPU acceleration support. In Ice Cream Sandwich and later versions, HWUI is used for drawing graphics by default.

Skia:

Skia is a set of 2D plotting APIs that are fully implemented by software. Due to performance, Skia is gradually replaced by HWUI.

HWUI

HWUI enables the UI component to use GPU acceleration. HWUI is introduced in Honeycomb to facilitate fast interaction, timely response, and smoothness. Using Skia to draw an animation on a tablet with a large resolution will occupy a high CPU resource and slow down the entire system. HWUI must support OpenGL ES 2.0 GPUs and cannot be simulated by software.

Renderscript

Renderscript is also a new API introduced by Honeycomb. It is designed to solve both porting and performance problems. The application programmer uses Renderscript to write code based on C99), and then a LLVM cross compiler compiles it into a machine-independent bit code, and the application programmer packs it into the apk. When the user downloads the apk, the compiler on the device is based on LLVM and located in/system/lib/libbcc. so) to compile the bit code as the instruction on the target machine.

Renderscript also exists on Froyo and Gingerbread, but it is not a public API. Only some wallpaper of Android uses it. At that time, its implementation was also very rough and its functions were limited.

Surface:

A Surface corresponds to an off-screen buffer, which is used by applications to render window content. A game program that may use OpenGL to draw 3D objects on the Surface. A common application may use Skia to draw widgets or text, it may also use the HWUI library to enable GPU acceleration. Starting from ICS, the Surface is implemented through a back-end SurfaceTexture, which means that the Surface is no longer a buffer, but a texture ).

Image stack of Android platform

SurfaceFlinger:

SurfaceFlinger is a synthesizer that manages the Surface of different applications. For example, many applications may exist at the same time. Correspondingly, many independent surfaces need to be rendered. SurfaceFlinger determines the content displayed on the screen, which needs to be overwritten and cropped.

SurfaceFlinger uses functions in the OpenGL ES 1.1 Standard. Why? If OpenGL ES 2.0 is used, you must support the hardware GPU of OpenGL ES 2.0. This will make the system startup more complex and make the implementation of the simulator more difficult.

HW Composer:

The hardware synthesizer is a HAL introduced by Honeycomb. SurfaceFlinger uses it to accelerate Surface synthesis using hardware resources, such as 3D GPUs and 2D graphics engines.

CopyBit:

CopyBit is also a HAL. It allows special hardware to accelerate some graphic operations, such as copying blitting ). It was designed to accelerate the rendering process of software on systems without 3D GPUs. CopyBit is deleted from ICS because GPU has become a required hardware and there is no need to design a dedicated acceleration part.

Libagl/PixelFlinger:

Libagl is a component that implements OpenGL ES 1.0 and 1.1 APIs through software. It uses PixelFlinger to implement OpenGL calls. To accelerate the rendering process using PixelFlinger, JIT is introduced, called CodeFling. CodeFling generates machine code, which greatly accelerates many types of pixel operations.

We can see that the Android graphics system is constantly being adjusted to provide a faster and smoother UI experience. This is the reason why the graphic-related code changes greatly in the Android version.