Overview of the relationship between Android applications and surfaceflinger services and learning plans

The surfaceflinger Service is responsible for drawing the UI of the Android Application. Its implementation is quite complex. It is not easy to analyze its implementation from the front. Since we cannot analyze the data from the front, we can try to analyze the data from the side. In the end, no matter how complicated the surfaceflinger service is, it serves Android applications. Therefore, we start with the relationship between Android applications and surfaceflinger services, to outline and develop a learning plan for the surfaceflinger service.

The surfaceflinger service runs in the system process of the Android system. It manages the frame buffer of the Android system ). For more information about the frame buffer of the Android system, see the previous two articles on the display process of the boot screen of the Android system and the hardware abstraction layer (HAL) of the android frame buffer) the implementation principle of the gralloc module is analyzed. Android applications must communicate with the surfaceflinger service in order to draw their UI on the system's frame buffer, as shown in Figure 1:

Figure 1 Relationship Between Android applications and surfaceflinger services

Note: Android applications and surfaceflinger services run in different processes. Therefore, they use the binder inter-process communication mechanism for communication. For more information about the binder inter-process communication mechanism in the Android system, see the Introduction and learning plan of the Binder Mechanism in the Android system.

In Figure 1, each Android Application and surfaceflinger service have a connection, which is described by a client-Type binder object. These client objects are created by the surfaceflinger service when the Android Application connects to the surfaceflinger service. After the Android Application successfully connects to the surfaceflinger service, you can obtain the binder proxy interface of the corresponding client object. With these binder proxy interfaces, the android application can notify the surfaceflinger service to draw its own UI.

When the android application notifies the surfaceflinger service to draw its own UI, it needs to pass the UI metadata to the surfaceflinger service, for example, to draw information such as the UI region and location. An android application may have many windows, and each window has its own UI metadata. Therefore, the UI metadata that the android application needs to pass to the surfaceflinger service is considerable. In this case, it is inappropriate to transmit UI metadata between Android applications and surfaceflinger services through the binder inter-process communication mechanism, at this time, the android system's anonymous shared memory mechanism (anonymous shared memory) will come in handy. For more information about the anonymous shared memory mechanism of Android systems, see the anonymous shared memory (anonymous shared memory) Introduction and learning plan series of articles in the Android system.

Add an anonymous shared memory to the connection between each android application and the surfaceflinger service to transmit UI metadata. Then we can see Figure 2, as shown below:

Figure 2 anonymous shared memory used to transmit UI metadata between Android applications and surfaceflinger services

From the perspective of application and client, the native anonymous shared memory block between them is a cool. Therefore, the application and client cannot see the native anonymous shared memory. As a result, this native anonymous shared memory was furious at the time and determined to turn the attack into Bai fumei, as shown in 3:

Figure 3 structured anonymous shared memory block used to pass UI metadata

After the tufuyuan attack, it became a white rich beauty named sharedclient. Since then, it has lived a happy life with application and client.

How white, how rich, and beautiful are sharedclient? See Figure 4:

Figure 4 sharedclient used to describe the UI metadata of the Android Application

Each sharedclient contains up to 31 sharedbufferstacks. Literally, sharedbufferstack is a shared buffer stack. How can this problem be solved? First, shared indicates that the stack is shared. So who will share it? Of course, it is the Android Application and surfaceflinger service. Second, buffer indicates that the content of this stack is a buffer. What kind of buffer? Of course, it is the buffer used to describe the UI metadata. Furthermore, stack indicates that the buffer used to describe the UI metadata needs to be accessed according to certain rules. In combination, we can think that each sharedbufferstack is used to describe a series of buffers that need to be accessed according to certain rules.

Does it seem that I still cannot understand sharedbufferstack? Well, recall that we usually use a technology called "Double Buffering" when drawing the UI. Two buffers are used, one of which is called the front buffer and the other is called the back buffer. The UI is always drawn in the back buffer first, and then switched with the front buffer and rendered to the display device. Now I can understand the meaning of sharedbufferstack? The surfaceflinger service is simply to sublimate and abstract the traditional "Double Buffering" technology into a sharedbufferstack. Don't underestimate this sublimation and abstraction. With sharedbufferstack, The surfaceflinger service can use N Buffer technology to draw the UI. The value range of N is 2 to 16. For example, in Android 2.3, the value of N is equal to 2, and in Android 4.1, it is said to be equal to 3.

We can further understand sharedbufferstack. In the surfaceflinger service, each sharedbufferstack corresponds to a surface, that is, a window. In this way, we can know why each sharedclient contains a series of sharedbufferstacks instead of a single sharedbufferstack: A sharedclient corresponds to an android application, and an Android Application may contain multiple windows, surface. It can be seen from this that an android application can contain up to 31 surfaces.

What does a sharedbufferstack look like? See Figure 5:

 

Figure 5 structure of sharedbufferstack

In Figure 5, for ease of description, we assume that the sharedbufferstack in the figure has five buffers. Buffer-1 and buffer-2 are already used, buffer-3, buffer-4, and buffer-5 are idle. The pointer head and tail point to the header and tail of the idle buffer list, and the pointer queue_head points to the header of the used buffer list. From this we can see that the buffer from the pointer tail to the head is an idle buffer table, and the buffer from the pointer head to the queue_head is a list of used buffers. Note that the five buffers in the figure are used cyclically.

The idle buffer zone is easy to understand. Next we will focus on the buffer zones that have been used, namely buffer-1 and buffer-2 in Figure 5.

As we mentioned earlier, the buffer in sharedbufferstack is only used to describe the UI metadata, which means that they do not contain real UI data. The real UI data is stored in graphicbuffer, And we will describe it later. Therefore, to fully describe a UI, each used buffer in sharedbufferstack has a graphicbuffer to describe real UI data. When the surfaceflinger service suspends the buffer-1 and buffer-2, it will find the graphicbuffer corresponding to them, so that the corresponding UI can be drawn.

When the android application needs to update a surface, it will find the sharedbufferstack corresponding to it and retrieve an idle buffer from the end of its idle buffer list. Let's assume that the number of the idle buffer obtained is index. Next, the android application requests the surfaceflinger service to allocate a graphic buffer graphicbuffer for the buffer numbered index. After the surfaceflinger service allocates the graphic buffer graphicbuffer, it sets its number to index, and then returns the graphic buffer graphicbuffer to the Android Application for access. After the Android Application obtains the graphic buffer graphicbuffer returned by the surfaceflinger service, it writes the UI data. After writing, insert the buffer corresponding to it, that is, the buffer numbered index, to the header of the used buffer list of the corresponding sharedbufferstack. After this step is completed, the android application notifies the surfaceflinger service to draw the graphic buffer graphicbuffer described in the used buffer. In the example in figure 5, the surfaceflinger service needs to draw the graphic buffer graphicbuffer corresponding to the buffer numbers 1 and 2. Because the surfaceflinger service knows where the graphicbuffer corresponding to the buffer numbers 1 and 2 is, the android application only needs to tell the surfaceflinger service the number of the buffer to be drawn. When a buffer that has been used is drawn, it becomes a idle buffer again.

The process described above is complex. We will use several articles to describe it in detail later.

Sharedbufferstack is shared between Android applications and surfaceflinger services. However, the android applications and surfaceflinger services use sharedbufferstack in different ways. Specifically, the Android Application is concerned with the list of idle buffers in it, while the surfaceflinger service is concerned with the list of buffers used in it. From the surfaceflinger service perspective, the buffer that has been used in the sharedbufferstack is actually waiting for rendering in the queue.

To facilitate the access of sharedbufferstack In the Android Application and surfaceflinger service, the android system uses sharedbufferclient and sharedbufferserver to describe sharedbufferstack respectively, sharedbufferclient is used to access the list of idle buffers of sharedbufferstack on the Android Application side, while sharedbufferserver is used to access the list of queuing buffers of sharedbufferstack on the surfaceflinger Service side.

In the view of sharedbufferclient, sharedbufferstack looks like 6:

Figure 6 sharedbufferstack in the eyes of the sharedbufferclient

As long as the number of available buffers in the sharedbufferstack is greater than 0, the sharedbufferclient will move the pointer tail one step forward and reduce the value of available to obtain an idle buffer. After the Android Application writes data to the idle buffer, it adds the data to the end of the Queuing Buffer buffer list in the sharedbufferstack through the sharedbufferclient, that is, the next position of the pointer queue_head.

In the view of sharedbufferserver, sharedbufferstack looks like 7:

Figure 7 sharedbufferstack in the eyes of the sharedbufferserver

When the android application notifies the surfaceflinger service to update the UI, as long as the number of queued buffers in the corresponding sharedbufferstack is greater than 0, the sharedbufferserver will draw the next buffer of the pointer head, move the pointer head one step forward and subtract 1 from the queued value.

We have mentioned the graphic buffer graphicbuffer many times. What is it? Let's see Figure 8:

Figure 8 graph buffer Graphic Structure

Each graphicbuffer contains a buffer for storing UI data, which is described using a buffer_handle_t object. Are you familiar with buffer_handle_t? In the previous article about the implementation principle of the hardware abstraction layer (HAL) module gralloc of the android frame buffer, we have said, the graphic buffer allocated by the gralloc module of the Hal layer is described using a buffer_handle_t object, and the graphic buffer described by the buffer_handle_t object is either in the system frame buffer (frame buffer) or allocated in the anonymous shared memory (anonymous shared memory. In this way, we can associate the surfaceflinger service with the gralloc module in the Hal layer.

Now, the relationship between the ndroid application and the surfaceflinger service has been summarized, but our task has not been completed yet. We need to learn more about it, for example:

1. How does an Android Application establish a connection with the surfaceflinger service?

2. How is the sharedclient used to describe the UI metadata of the android application created?

3. How does an Android Application request the surfaceflinger service to create a surface?

4. How does the Android app request the surfaceflinger service to render a surface?

After answering these four questions, I believe we can have a deep understanding of the surfaceflinger service and help us analyze the implementation of the surfaceflinger service from the front. Next we will take the Android system boot animation as an example and use four articles to answer these four questions. Stay tuned!

Lao Luo's Sina Weibo: http://weibo.com/shengyangluo. welcome to the attention!