Key issues about display on Qualcomm Android platform

Several Problems in the Display Section have been clarified through actual tests over the past few days, mainly including the usage of each module and the calling process of several modules in the Hal layer. The problem is described as follows:

0. surfaceflinger Main Functions

Surfaceflinger is only responsible for the control of merge surface. For example, to calculate the overlapping areas of two surfaces, as for the content to be displayed on the surface, it is calculated through skia, OpenGL, and pixflinger.

 

1. How is the AP displayed?

Surfaceflinger is responsible for the display of all the upper layers. For AP (2D or 3D applications), as long as the surface is created in surfaceflinger and the parameters are set, surfaceflinger is responsible for processing.

 

2. How does surfaceflinger manage multiple surfaces?

No matter how many surfaces there are, only the screen size data is sent to the display part. surfaceflinger uses MDP or GPU to synthesize multiple surfaces, and the common synthesis of MDP can be completed, however, if it is complex, such as 3D applications, the GPU must be used, and the final synthesized data will be sent to framebuffer.

 

3. What is framebuffer?

Framebuffer is a display memory allocated for the display data in Linux (in the FB device). Generally, the size is twice the size of the entire screen data. For the upper-layer AP, you only need to drop the data to be displayed in framebuffer, but the data is not actually sent to the LCD, but saved in framebuffer.

 

4. How does the upper layer send the displayed content to framebuffer?

You can use either of the following two methods ):

A. For normal display, copybit (MDP) is used (GPU is not used)

Surfaceflinger sends the data to be displayed through copybit to framebuffer.

Note: copybit can be seen as the mdp ppp interface. It provides MDP functions, such as multiple layer synthesis, scale, and rotate.

The interfaces are Android/hardware/msm7k/libcopybit/copybit. cpp.

B. Use GPU (that is, use graphics driver in the figure)

For complex display processing, such as 3D applications, GPU directly drops the processed data to framebuffer, which has nothing to do with MDP.

 

5. How is the data in framebuffer sent to the LCD display?

Gralloc is completed in the figure.

Gralloc has two functions:

1) One is the same as copybit, which contains the mdp ppp interface (not used currently)

2) The other is the interface for screen swiping (full screen swiping), which sends data in framebuffer to the LCD and calls the MDP DMA interface.

This part of the code in the android/hardware/msm7k/libgralloc-qsd8k directory, did not pay attention to before, that is not used. Now we can see that the disp_loop thread is created after the initialization, and the operation is to call the system interface.

IOCTL (m-> framebuffer-> FD, fbioput_vscreeninfo, & M-> info)

Send data to LCD

Note: When sending data, two buffer switches.

In addition, surfaceflinger on the upper layer learns the screen size through the interface in gralloc. The called interface is

IOCTL (FD, fbioget_vscreeninfo, & info). The screen width and height in info correspond to the wide and high values set by the underlying driver.

 

6. What is OpenGL?

It is an image processing engine that is used when complex display (2D/3d) operations are required. It is divided into SW solution and HW solution. The software solution is libagl in the figure. so, which corresponds to libgles_android.so in the current project. It can complete simple 2D (text, icon, and so on) processing. It can be seen through trace that most of the current display operations are completed by it.

Note: It is a software solution. The processed data is sent to framebuffer through copybit, rather than GPU.

For more information about the interfaces, see Android/frameworks/base/OpenGL/libagl.

The HW scheme is the graphics driver in the figure. It uses GPU hardware to complete image processing, and the processed data is directly sent to framebuffer. For details about the interfaces, see Android/frameworks/base/OpenGL/libs (several versions are available)

 

7. How is OpenGL configured in the project?

There is an EGL. cfg file in the android/vendor/qcom/msm7627_ffa Directory, which specifies the OpenGL information in the current version. The current version is as follows:

0 0 android

0 1 adreno200

The first line indicates that the codebase supports OpenGL of SW solution, which is Android default

The second line indicates that the codebase also supports the HW solution OpenGL, Which is Qualcomm's Adreno engine.

If the cfg file is empty, only the default SW scheme is supported.

If both solutions exist, the upper layer will choose to use one based on the actual application.

For more information, see Android/frameworks/base/OpenGL/libs/EGL/loader. cpp.

0. surfaceflinger Main Functions

Surfaceflinger is only responsible for the control of merge surface. For example, to calculate the overlapping areas of two surfaces, as for the content to be displayed on the surface, it is calculated through skia, OpenGL, and pixflinger.

 

1. How is the AP displayed?

Surfaceflinger is responsible for the display of all the upper layers. For AP (2D or 3D applications), as long as the surface is created in surfaceflinger and the parameters are set, surfaceflinger is responsible for processing.

 

2. How does surfaceflinger manage multiple surfaces?

No matter how many surfaces there are, only the screen size data is sent to the display part. surfaceflinger uses MDP or GPU to synthesize multiple surfaces, and the common synthesis of MDP can be completed, however, if it is complex, such as 3D applications, the GPU must be used, and the final synthesized data will be sent to framebuffer.

 

3. What is framebuffer?

Framebuffer is a display memory allocated for the display data in Linux (in the FB device). Generally, the size is twice the size of the entire screen data. For the upper-layer AP, you only need to drop the data to be displayed in framebuffer, but the data is not actually sent to the LCD, but saved in framebuffer.

 

4. How does the upper layer send the displayed content to framebuffer?

You can use either of the following two methods ):

A. For normal display, copybit (MDP) is used (GPU is not used)

Surfaceflinger sends the data to be displayed through copybit to framebuffer.

Note: copybit can be seen as the mdp ppp interface. It provides MDP functions, such as multiple layer synthesis, scale, and rotate.

The interfaces are Android/hardware/msm7k/libcopybit/copybit. cpp.

B. Use GPU (that is, use graphics driver in the figure)

For complex display processing, such as 3D applications, GPU directly drops the processed data to framebuffer, which has nothing to do with MDP.

 

5. How is the data in framebuffer sent to the LCD display?

Gralloc is completed in the figure.

Gralloc has two functions:

1) One is the same as copybit, which contains the mdp ppp interface (not used currently)

2) The other is the interface for screen swiping (full screen swiping), which sends data in framebuffer to the LCD and calls the MDP DMA interface.

This part of the code in the android/hardware/msm7k/libgralloc-qsd8k directory, did not pay attention to before, that is not used. Now we can see that the disp_loop thread is created after the initialization, and the operation is to call the system interface.

IOCTL (m-> framebuffer-> FD, fbioput_vscreeninfo, & M-> info)

Send data to LCD

Note: When sending data, two buffer switches.

In addition, surfaceflinger on the upper layer learns the screen size through the interface in gralloc. The called interface is

IOCTL (FD, fbioget_vscreeninfo, & info). The screen width and height in info correspond to the wide and high values set by the underlying driver.

 

6. What is OpenGL?

It is an image processing engine that is used when complex display (2D/3d) operations are required. It is divided into SW solution and HW solution. The software solution is libagl in the figure. so, which corresponds to libgles_android.so in the current project. It can complete simple 2D (text, icon, and so on) processing. It can be seen through trace that most of the current display operations are completed by it.

Note: It is a software solution. The processed data is sent to framebuffer through copybit, rather than GPU.

For more information about the interfaces, see Android/frameworks/base/OpenGL/libagl.

The HW scheme is the graphics driver in the figure. It uses GPU hardware to complete image processing, and the processed data is directly sent to framebuffer. For details about the interfaces, see Android/frameworks/base/OpenGL/libs (several versions are available)

 

7. How is OpenGL configured in the project?

There is an EGL. cfg file in the android/vendor/qcom/msm7627_ffa Directory, which specifies the OpenGL information in the current version. The current version is as follows:

0 0 android

0 1 adreno200

The first line indicates that the codebase supports OpenGL of SW solution, which is Android default

The second line indicates that the codebase also supports the HW solution OpenGL, Which is Qualcomm's Adreno engine.

If the cfg file is empty, only the default SW scheme is supported.

If both solutions exist, the upper layer will choose to use one based on the actual application.

For more information, see Android/frameworks/base/OpenGL/libs/EGL/loader. cpp.