Display buffer management between the sharedclient-client (surface) and server (layer) in Android surfaceflinger

Surfaceflinger is loaded as a system service in the system startup phase. Each window in the application corresponds to the surface in the local code, and the surface corresponds to each layer in surfaceflinger. The main function of surfaceflinger is to apply for memory for these layers, manage the layer display, hide, and re-draw operations based on application requests. surfaceflinger combines all the layers and displays them on the display. When an application needs to draw a graph on a surface, it first needs to get the starting address of the surface in the memory, and the memory is allocated in surfaceflinger, because surfaceflinger and the application are not running in the same process, how do I transmit and synchronize display buffers between the application client (surface) and the server (surfaceflinger-layer? This is exactly what we will discuss in this article.

Surface creation process

Let's first look at how Android creates a surface. The following sequence diagram shows the entire creation process.

Figure 1 creation process of Surface

The process of creating a surface is basically divided into two steps:

1. Create a surfacesession

The first step is generally executed only once to create a surfacecomposerclient instance. The Java layer calls the local code through JNI, and the Local Code creates a surfacecomposerclient instance. The surfacecomposerclient calls the createconnection of surfaceflinger through the isurfacecomposer Interface, surfaceflinger returns an isurfaceflingerclient interface to surfacecomposerclient. During the createconnection process, surfaceflinger creates a sharedclient for managing the buffer switchover. We will introduce the sharedclient later, the local layer returns the surfacecomposerclient instance to the Java layer to complete surfacesession creation.

 

2. Use surfacesession to create a surface

The Java layer calls the local code surface_init () through JNI. The local code first obtains the surfacecomposerclient instance created in step 1, and calls the createsurface method of the isurfaceflingerclient interface through surfacecomposerclient to access surfaceflinger and Surface, create different layers, call the setbuffers () method of the layer, create two buffers for the layer, and return the isurface interface of the layer, surfacecomposerclient uses this isurface interface to create a surfacecontrol instance and returns this surfacecontrol to the Java layer.

 

The following result is obtained:

• The surface of the Java layer actually corresponds to the surfacecontrol object of the local layer. Later, the local code can use the surfacecontrol object passed in by Java to obtain the local surface object through the getsurface method of surfacecontrol;
• Android assigns two graphic buffers to each surface to implement page-flip operations;
• When a surfacesession is created, surfaceflinger creates a sharedclient object used to manage two graphical buffer switches. surfacecomposerclient can obtain the sharedclient object through the getcontrolblock () method of the isurfaceflingerclient interface and view the member function _ init of surfacecompo:

 Void surfacecomposerclient: _ Init (<br/> const sp <isurfacecomposer> & SM, const sp <isurfaceflingerclient> & conn) <br/> {<br/> ...... <br/> mclient = conn; <br/> If (mclient = 0) {<br/> mstatus = no_init; <br/> return; <br/>}</P> <p> mcontrolmemory = mclient-> getcontrolblock (); <br/> msignalserver = Sm; <br/> mcontrol = static_cast <sharedclient *> (mcontrolmemory-> getbase (); <br/>}< br/>

Obtain the display buffer corresponding to the surface.

Although surfaceflinger has applied two buffers for each layer when creating a layer, but the two buffers are not visible at the Java layer at this time, the Java layer wants to draw images on the surface, you must first bind a buffer to the canvas, and then all painting operations on the canvas will be painted to the buffer. Shows the buffer binding process:

Figure 2 bind a buffer

Start drawing surface. java will first call lockcanvas () to obtain the canvas for drawing operations. lockcanvas will further call the surface_lockcanvas on the local layer. The local code uses the surfacecontrol object passed in at the Java layer and getsurface () obtain the surface object of the local layer, and then call the lock () method of the Surface object. Lock () returns the information for modifying the surface, including the first vaddr address of the available buffer zone, this vaddr creates a bitmap in the android 2D graphics library skia, and then uses the canvas API in the skia Library: canvas. setbitmapdevice (Bitmap), bind the bitmap to the canvas, and finally return the canvas to the Java layer, so that the Java layer can draw images on the canvas, these drawing operations will eventually be led by vaddr. Address Buffer.

Let's look at what the lock () method of the surface has done:

• Dequeuebuffer (& backbuffer) to obtain backbuffer
  • Sharedbufferclient-> dequeue () to obtain the number of the current idle Buffer
  • Obtain the real graphicbuffer: backbuffer using the buffer number.
  • If you have not mapped the buffer in the layer (Mapper), getbufferlocked remaps through the isurface interface.
• Obtain frontbuffer
• The frontbuffer content is copied to the backbuffer according to the update regions of the two buffers. This ensures the synchronization of the content displayed in the two buffers.
• Backbuffer-> lock () obtain the first vaddr address of the backbuffer.
• Vaddr is returned using the info parameter.

Release the display buffer corresponding to the surface.

After drawing, to display the surface content to the screen, you need to release the bound buffer in the canvas and change the buffer from being shipped (because there are only two buffers by default, so it is actually a frontbuffer), surfaceflinger's working thread will mix all the frontbuffer in the system at an appropriate refresh time, and then refresh it to the screen through OpenGL. Shows the process of Unbinding the buffer zone:

Figure 3 unbind a buffer

• Java layer calls unlockcanvasandpost
• Enter the local code surface_unlockcanvasandpost.
• The Local Code uses the surfacecontrol object passed in at the Java layer to get the surface object at the local layer through getsurface ().
• Bind an empty bitmap to the canvas.
• Call the surface unlockandpost Method
  • Call the unlock () of graphicbuffer to unlock the buffer.
  • In queuebuffer (), the queue () of sharedbufferclient is called to update the buffer to a shipping status.

Sharedclient and sharedbufferstack

As we can see from the previous discussion, when binding a buffer on the canvas, we need to use the dequeue method of the sharedbufferclient to obtain the idle buffer. When unbinding and submitting the buffer shipping, finally, you must call the queue method of the sharedbufferclient to notify the surfaceflinger worker thread. In fact, in surfaceflinger, each layer is associated with a sharedbufferserver. The surfaceflinger worker thread manages the layer buffer through the sharedbufferserver and establishes a connection at surfacecomposerclient, surfaceflinger has already created a sharedclient object for the connection. The sharedclient object contains a sharedbufferstack array with the size of 31. Each time a surface is created, a sharedbufferstack in the array is occupied, then, the surfacecomposerclient's surface will create a sharedbufferclient and the sharedbufferstack, And the surfaceflinger layer will also create sharedbufferserver and sharedbuff Erstack association. In fact, each pair of sharedbufferclient/sharedbufferserver controls the same sharedbufferstack object. Through sharedbufferstack, it ensures that the application responsible for surface drawing operations and the server responsible for screen refreshing (surfaceflinger) you can use different buffers and let them know when the other side Locks/releases the buffer.

The code and header files of sharedclient and sharedbufferstack are located:

/Frameworks/base/libs/surfaceflinger_client/sharedbufferstack. cpp

/Frameworks/base/include/private/surfaceflinger/sharedbufferstack. h

 

 

Figure 4 client and server Buffer Management

Continue to study the birth process of sharedclient, sharedbufferstack, sharedbufferclient, and sharedbufferserver.

1. sharedclient

• In the createconnection stage, surfaceflinger creates the client object:

Sp <isurfaceflingerclient> surfaceflinger: createconnection () <br/>{< br/> mutex: autolock _ L (mstatelock); <br/> uint32_t token = mtokens. acquire (); </P> <p> sp <client> client = new client (token, this); <br/> If (client-> ctrlblk = 0) {<br/> mtokens. release (token); <br/> return 0; <br/>}< br/> status_t err = mclientsmap. add (token, client); <br/> If (ERR <0) {<br/> mtokens. release (token); <br/> return 0; <br/>}< br/> sp <bclient> bclient = <br/> New bclient (this, Token, client-> getcontrolblockmemory (); <br/> return bclient; <br/>}

• Then go to the constructor of the client, which allocates 4 K shared memory and builds the sharedclient object on this memory:

Client: client (clientid, const sp <surfaceflinger> & flinger) <br/>: ctrlblk (0), CID (clientid), MPID (0 ), mbitmap (0), mflinger (flinger) <br/>{< br/> const int pgsize = getpagesize (); <br/> const int cblksize = (sizeof (sharedclient) + (pgsize-1 ))&~ (Pgsize-1); </P> <p> mcblkheap = new memoryheapbase (cblksize, 0, <br/> "surfaceflinger client control-block "); </P> <p> ctrlblk = static_cast <sharedclient *> (mcblkheap-> getbase (); <br/> If (ctrlblk) {// construct the shared structure in-place. <br/> New (ctrlblk) sharedclient; <br/>}< br/>}

• Return to createconnection, use the getcontrolblockmemory () method of the client to obtain the imemoryheap interface of the shared memory block, and then create a subclass bclient of isurfaceflingerclient. The member variable mcblk of bclient saves the imemoryheap interface pointer;
• Return the bclient to the surfacecomposerclient. The surfacecomposerclient obtains the imemoryheap interface pointer through the getcontrolblock () method of the isurfaceflingerclient interface, and saves it in the mcontrolmemory member variable of surfacecomposerclient;
• Continue to get the first address of the shared memory through the getbase () method of the imemoryheap interface, convert it to the sharedclient pointer, and save it in the member variable mcontrol of surfacecomposerclient;
• So far, the member variables mcontrol and surfaceflinger: client. ctrlblk of surfacecomposerclient point to the same memory block, which is the sharedclient object.

2. sharedbufferstack, sharedbufferserver, and sharedbufferclient

The sharedclient object has a sharedbufferstack array:

Sharedbufferstack surfaces [num_layers_max];

Num_layers_max is defined as 31, which ensures that the size of the sharedclient object is exactly 4 kb. When creating a new surface, enter the createsurface function of surfaceflinger, obtain the client object created in the createconnection stage, and obtain an unused number between 0--num_layers_max using the client, this number is actually the index of the sharedbufferstack array:

Int32_t id = client-> generateid (PID );

Then, we use the client object, index value, and other parameters to create different types of layer objects. A common layer object is used as an example:

Layer = createnormalsurfacelocked (client, D, ID, <br/> W, H, flags, format );

Create a layer object in createnormalsurfacelocked:

Sp <layer> layer = new layer (this, display, client, ID );

When constructing a layer, the parent class layerbaseclient is constructed first. The sharedbufferserver object is created in layerbaseclient. The index value and sharedclient of the sharedbufferstack array are passed into the sharedbufferserver object.

Layerbaseclient: layerbaseclient (surfaceflinger * flinger, displayid display, <br/> const sp <client> & client, int32_t I) <br/>: layerbase (flinger, display ), lcblk (null), client (client), mindex (I), <br/> midentity (uint32_t (android_atomic_inc (& sidentity ))) <br/>{< br/> lcblk = new sharedbufferserver (<br/> client-> ctrlblk, I, num_buffers, <br/> midentity); <br/>}

 

Since then, the layer has been associated with the sharedbufferserver through the lcblk member variable (sharedbufferserver) and the sharedclient shared memory zone. Each layer corresponds to one of the sharedbufferstack arrays.

Return to surfaceflinger's client surface. cpp. The surface constructor is as follows:

Surface: surface (const sp <surfacecontrol> & surface) <br/>: mclient (surface-> mclient), msurface (surface-> msurface ), <br/> mtoken (surface-> mtoken), midentity (surface-> midentity), <br/> mformat (surface-> mformat), mflags (surface-> mflags ), <br/> mbuffermapper (graphicbuffermapper: Get (), msharedbufferclient (null), <br/> mwidth (surface-> mwidth), mheight (surface-> mheight) <br/>{< br/> msharedbufferclient = new sharedbufferclient (<br/> mclient-> mcontrol, mtoken, 2, midentity ); </P> <p> Init (); <br/>}< br/>

The sharedbufferclient constructor mclient-> mcontrol is the sharedclient object in the shared memory block, and mtoken is the sharedbufferstack array index value.

Here we finally know that the msharedbufferclient member in the surface and the lcblk member in the layer (sharedbufferserver) Manage the two buffers in the surface through the same sharedbufferstack in the sharedclient.