Analysis on the creation process of shared UI metadata (SharedClient) between Android applications and SurfaceFlinger services

In the previous article, we analyzed the connection process between Android applications and SurfaceFlinger services. After the Android Application successfully connects to the SurfaceFlinger service, an anonymous shared memory is required to share its UI metadata with the SurfaceFlinger service so that the SurfaceFlinger service can correctly create and render the Surface for it. In this article, we will analyze in detail the process of creating an anonymous shared memory that stores UI metadata.

As mentioned in the overview and learning plan of the relationship between Android applications and SurfaceFlinger services, the anonymous shared memory used to save the UI metadata of Android applications is finally structured as a SharedClient object for access. Each UI-related Android application process has only one SharedClient object, and these SharedClient objects are created by the Android application requesting SurfaceFlinger service: the Android Application first obtains a Binder proxy interface of the SurfaceFlinger service, and then obtains another Binder proxy interface of UserClient type through this proxy interface, finally, you can use the last Binder proxy interface to obtain a SharedClient object.

Because each Android application process related to the UI has only one SharedClient object, the Android system creates and manages this SharedClient object through a class in the singleton mode. The class name is SurfaceClient, which is defined in the frameworks/base/libs/surfaceflinger_client/Surface. cpp file, as shown below:

[Cpp]
Class SurfaceClient: public Singleton <SurfaceClient>
{
// All these attributes are constants
Sp <ISurfaceComposer> mComposerService;
Sp <ISurfaceComposerClient> mClient;
Status_t mStatus;
SharedClient * mControl;
Sp <IMemoryHeap> mControlMemory;
 
SurfaceClient ()
: Singleton <SurfaceClient> (), mStatus (NO_INIT)
{
Sp <ISurfaceComposer> sf (ComposerService: getComposerService ());
MComposerService = sf;
MClient = sf-> createClientConnection ();
If (mClient! = NULL ){
MControlMemory = mClient-> getControlBlock ();
If (mControlMemory! = NULL ){
MControl = static_cast <SharedClient *> (
MControlMemory-> getBase ());
If (mControl ){
MStatus = NO_ERROR;
}
}
}
}
Friend class Singleton <SurfaceClient>;
Public:
Status_t initCheck () const {
Return mStatus;
}
SharedClient * getSharedClient () const {
Return mControl;
}
Ssize_t getTokenForSurface (const sp <ISurface> & sur) const {
// TODO: we cocould cache a few tokens here to avoid an IPC
Return mClient-> getTokenForSurface (sur );
}
Void signalServer () const {
MComposerService-> signal ();
}
};
When the static member function getInstance of the SurfaceClient class is called for the first time, the system creates a SurfaceClient object in the corresponding application process, that is, the SurfaceClient class constructor is called. The constructor of the SurfaceClient class first calls the static member function getComposerService of the ComposerService class to obtain the proxy interface of the SurfaceFlinger service and stores it in the member variable mComposerService of the SurfaceClient class for future use. The static member function getComposerService of the ComposerService class has been analyzed in the connection process analysis article between the Android Application and SurfaceFlinger service. With the proxy interface sf of SurfaceFlinger service, the constructor of the SurfaceClient class can then call its member function createUserClient to obtain a Binder proxy interface of UserClient type, this Binder proxy interface implements the ISurfaceComposerClient interface. Therefore, we can save it in the mClient member variable of the SurfaceClient class. Finally, the SurfaceClient class constructor calls the previously obtained Binder proxy interface mClient member function getControlBlock for ISurfaceComposerClient to obtain an anonymous shared memory mControlMemory that describes the UI metadata of the application, in addition, these anonymous shared memories are forcibly converted into a SharedClient object mControl, so that you can easily access the UI metadata later.
The above is the general description of the creation process of the shared UI metadata (SharedClient) between the Android Application and SurfaceFlinger service. Next we will analyze the implementation of each step in detail. Now, let's continue to analyze the implementation of the rest of the SurfaceClient class member functions:

1. The member function initCheck is used to check whether an Android Application Process has successfully requested the SurfaceFlinger service to create a SharedClient object that describes UI metadata.

2. The member function getSharedClient is used to return the SharedClient object mControl that describes UI metadata.

3. The member function getTokenForSurface is used to return the Token value of a Surface described by the sur parameter. This Token value is created and managed by the SurfaceFlinger service, and can be obtained through the getTokenSurface member function of mClient, the Binder proxy interface of the UserClient type obtained previously.

4. the member function signalServer is used to notify the SurfaceFlinger service to update the Android application UI. This is achieved by calling the proxy interface mComposerService of SurfaceFlinger service, namely, sending a signal to SurfaceFlinger, so that you can wake it up and update the UI.

After introducing the implementation of the SurfaceClient class, we also need to understand the implementation of the two classes, namely the implementation of the UserClient class and the SharedClient class, this helps us understand the creation process of anonymous shared memory used to save the UI metadata of Android applications, as well as the analysis of the Surface creation and rendering process in the next two articles.

Next, we will first analyze the implementation of the UserClient class, and then analyze the implementation of the SharedClient class.

In Figure 2 of the connection process analysis between the Android Application and SurfaceFlinger service, we introduce the Client class used to connect the Android Application and SurfaceFlinger service, the UserClient and Client classes are similar. They all implement the same interface, but the focus is different. In Figure 2 of the connection Process Analysis Between Android applications and SurfaceFlinger services, replace the Client class with the UserClient class to obtain the implementation structure of the UserClient class, as shown in Figure 1:

 

Figure 1 implementation structure of the UserClient class

The most important difference between the UserClient class and the Client class is that the former implements the ISurfaceComposerClient interface member function getControlBlock, while the latter implements the ISurfaceComposerClient interface member function createSurface. Later, we will analyze how the UserClient class implements the getControlBlock member function of the ISurfaceComposerClient interface.

The implementation of the UserClient class is now introduced here. Next let's look at the implementation of the SharedClient class. To facilitate the description, we will describe the relationship between the Android Application and the SurfaceFlinger service and the figure 4 and figure 5 of the learning plan, as shown in figure 2 and Figure 3:

 

Figure 2 SharedClient used to describe the UI metadata of the Android Application

 

Figure 3 structure of SharedBufferStack

Each SharedClient contains up to 31 sharedbufferstacks, and each SharedBufferStack corresponds to a Surface in the process of an Android application.

The SharedClient class is defined in the file frameworks/base/include/private/surfaceflinger/SharedBufferStack. h, as shown below:

[Cpp]
Class SharedClient
{
Public:
SharedClient ();
~ SharedClient ();
......
 
Private:
......
 
SharedBufferStack surfaces [SharedBufferStack: NUM_LAYERS_MAX];
};
It has a SharedBufferStack array with the size of SharedBufferStack: NUM_LAYERS_MAX. The value of SharedBufferStack: NUM_LAYERS_MAX is 31, which is defined in the SharedBufferStack class.
The SharedBufferStack class is also defined in the file frameworks/base/include/private/surfaceflinger/SharedBufferStack. h, as follows:

[Cpp]
Class SharedBufferStack
{
......
 
Public:
// When changing these values, the COMPILE_TIME_ASSERT at the end of this
// File need to be updated.
Static const unsigned int NUM_LAYERS_MAX = 31;
Static const unsigned int NUM_BUFFER_MAX = 16;
Static const unsigned int NUM_BUFFER_MIN = 2;
Static const unsigned int NUM_DISPLAY_MAX = 4;
 
......
 
Struct SmallRect {
Uint16_t l, t, r, B;
};
 
Struct FlatRegion {
Static const unsigned int NUM_RECT_MAX = 5;
Uint32_t count;
SmallRect rects [NUM_RECT_MAX];
};
 
Struct BufferData {
FlatRegion dirtyRegion;
SmallRect crop;
Uint8_t transform;
Uint8_t reserved [3];
};
 
SharedBufferStack ();
......
Status_t setDirtyRegion (int buffer, const Region & reg );
Status_t setCrop (int buffer, const Rect & reg );
Status_t setTransform (int buffer, uint8_t transform );
Region getDirtyRegion (int buffer) const;
Rect getCrop (int buffer) const;
Uint32_t getTransform (int buffer) const;
 
// These attributes are part of the conditions/updates
Volatile int32_t head; // server's current front buffer
Volatile int32_t available; // number of dequeue-able buffers
Volatile int32_t queued; // number of buffers waiting for post
......
 
// Not part of the conditions
......
Volatile int8_t index [NUM_BUFFER_MAX];
......

Int8_t headBuf; // last retired buffer
......
BufferData buffers [NUM_BUFFER_MAX];
};

Next we will briefly analyze the SharedBufferStack class.
First, the SharedBufferStack class defines four constants internally:

NUM_LAYERS_MAX -- indicates that an Android application can have a maximum of NUM_LAYERS_MAX layers. A Layer can be understood as a Surface.

NUM_BUFFER_MAX -- indicates that a SharedBufferStack can have up to NUM_BUFFER_MAX UI metadata buffers.

NUM_BUFFER_MIN -- indicates that a SharedBufferStack must have at least UM_BUFFER_MIN UI metadata buffers.

NUM_DISPLAY_MAX -- indicates that the Android system supports up to NUM_DISPLAY_MAX display screens.

From these constants, we can see that:

1. The Android system supports up to four display screens.

2. An Android application can create up to 31 surfaces at the same time.

3. A Surface can have 2 ~ 16 UI metadata buffers, that is, 2 ~ 16 Buffer technology to render the Surface.

Second, the SharedBufferStack class internally defines three structs:

SmallRect: used to describe a rectangular area. The member variables l, t, r, and B represent the positions of the Left and bottom right corners, respectively.

FlatRegion -- used to describe a SmallRect array rects. The array size is NUM_RECT_MAX, but the actual number is count.

BufferData -- used to describe a UI metadata buffer. It has four member variables dirtyRegion, crop, transform, and reserved. dirtyRegion is used to describe the region where the Surface needs to be updated, that is, the cropping area, crop is used to describe the texture coordinates of a Surface, and transform is used to describe the rotation direction of a Surface, such as Rotating 90 degrees or turning up or down, and reserved is reserved for future use. Through this UI metadata buffer, the SurfaceFlinger service can correctly render the image described in the graphic buffer of a Surface to the screen.

The SharedBufferStack class has a BufferData array buffers whose size is NUM_BUFFER_MAX, that is, 16, which is used for a set of UI metadata buffers, the content of the UI metadata buffer can be accessed through the setDirtyRegion, setCrop, setTransform, getDirtyRegion, getCrop, and getTransform member functions. The first parameter of these six member functions is an int value, which is used to describe which BufferData data is to be accessed.

The SharedBufferStack class has another array index of the int8_t type, and its size is also NUM_BUFFER_MAX. This index array is a real Stack, which is accessed according to certain rules. The value of each element in the index array is an index value, which is used to map to the array buffers. For example, if the value of index [0] is equal to 2, it corresponds to the 2nd elements in the array buffers, that is, buffers [2].

The meanings of other important member variables of the SharedBufferStack class are as follows:

Head -- used to describe the header of A SharedBufferStack. It is an index value and is mapped to an array index.

Available -- used to describe the number of idle UI metadata buffers in a SharedBufferStack.

Queued is used to describe the number of completed UI metadata buffers in a SharedBufferStack, that is, the UI metadata buffers waiting for SurfaceFlinger service in queue.

HeadBuf -- used to describe the ID of the UI metadata buffer corresponding to the header of A SharedBufferStack, Which is mapped to the array buffers.

We will introduce the implementation of the SharedBufferStack class for the moment. When I analyze the Surface creation process of an Android Application in the next article, we can further understand the implementation of the SharedBufferStack class through the implementation of the SharedBufferServer class and the SharedBufferClient class.

Now, we will start to analyze in detail the process of creating the shared UI metadata between the Android Application and the SurfaceFlinger service, as shown in Figure 4:

 

Figure 4 creation of shared UI metadata for Android applications

Next, we will analyze each step in detail.

Step 1. SurfaceFlinger: createClientConnection

[Cpp]
Sp <ISurfaceComposerClient> SurfaceFlinger: createClientConnection ()
{
Sp <ISurfaceComposerClient> bclient;
Sp <UserClient> client (new UserClient (this ));
Status_t err = client-> initCheck ();
If (err = NO_ERROR ){
Bclient = client;
}
Return bclient;
}
The createClientConnection function of the SurfaceFlinger class is implemented in the file frameworks/base/services/surfaceflinger/SurfaceFlinger. in cpp, its implementation is very simple. it just creates a Binder object client of the UserClient type and obtains one of its ISurfaceComposerClient interfaces. Finally, this ISurfaceComposerClient interface is called a UserClient proxy object, returns the process to the Android Application.
Next, we will continue to analyze the creation process of the UserClient object, that is, the implementation of the UserClient class constructor.

Step 2. new UserClient

[Cpp]
UserClient: UserClient (const sp <SurfaceFlinger> & flinger)
: Ctrlblk (0), mBitmap (0), mFlinger (flinger)
{
Const int pgsize = getpagesize ();
Const int cblksize = (sizeof (SharedClient) + (pgsize-1 ))&~ (Pgsize-1 ));
 
MCblkHeap = new MemoryHeapBase (cblksize, 0,
"SurfaceFlinger Client control-block ");
 
Ctrlblk = static_cast <SharedClient *> (mCblkHeap-> getBase ());
If (ctrlblk) {// construct the shared structure in-place.
New (ctrlblk) SharedClient;
}
}

The UserClient member variable mFlinger is a strong pointer of SurfaceFlinger type. It points to the SurfaceFlinger service. Another UserClient member variable mBitmap is an int32_t value, it is used to assign a Token value to the Surface of the Android application. If its nth digit is equal to 1, it indicates that the Token whose value is equal to n has been allocated for use.
The UserClient class constructor first obtains the size of a SharedClient object, and then alignment the size to the page boundary. Therefore, it obtains the cblksize of the anonymous shared block to be created next. This anonymous shared memory is described by a MemoryHeapBase object and stored in mCblkHeap, a member variable of the UserClient class. The MemoryHeapBase class is a C ++ interface used to create Anonymous Shared Memory. For the implementation principle, refer to the analysis article on the C ++ call interface of the Android system Anonymous Shared Memory (Anonymous Shared Memory.

After the UserClient class constructor obtains an anonymous shared memory, it creates a SharedClient object on this anonymous shared memory and stores it in the member variable ctrlblk of the UserClient class, this allows later users to access the UI metadata of Android applications.

Return to the createClientConnection function of the SurfaceFlinger class, which returns an ISurfaceComposerClient interface pointing to a UserClient object to the Android application process, the Android application process can encapsulate it into a Binder proxy object of the BpSurfaceComposerClient type.

Step 3. return BpSurfaceComposerClient

To encapsulate a Binder proxy object into a BpSurfaceComposerClient, refer to Step 4 in the previous analysis of the connection process between the Android Application and the SurfaceFlinger service.

Step 4. UserClient: getControlBlock

[Cpp]
Sp <IMemoryHeap> UserClient: getControlBlock () const {
Return mCblkHeap;
}
As you can see from Step 2 above, the member variable mCblkHeap of the UserClient class points to an anonymous shared memory. After the UserClient class returns this anonymous shared memory to the Android Application, the Android Application constructs a SharedClient object for access and stores it in the member variable mControl of the SurfaceClient class, this structured process can be referred to the constructor of the SurfaceClient class described above.
So far, the creation process of a SharedClient object that describes the UI metadata of the Android Application is analyzed. When the Android Application requests the SurfaceFlinger service to create a Surface, the SurfaceFlinger service extracts a SharedBufferStack from the SharedClient object so that it can be used as the UI metadata buffer of the Surface. In the next article, we will describe in detail the process of the Android application requesting the SurfaceFlinger service to create the Surface!