The Graphicbuffer synchronization mechanism in Android-fence

Fence is a synchronization mechanism that is used primarily in Android for Graphicbuffer synchronization in a graphics system. What does it have to do with the existing synchronization mechanism? It is primarily used to handle cross-hardware situations, especially synchronization between Cpu,gpu and HWC, and it can also be used for synchronization between multiple points in time. A big difference between GPU programming and pure CPU programming is that it is asynchronous, that is, when we call GL command to return, this command is not necessarily completed, but it is placed in the local command buffer. Exactly when this GL command is really done CPU is not known, unless the CPU uses glfinish () to wait for these commands to finish, another method is based on the fence mechanism of the synchronization object. Here's an example of a producer handing Graphicbuffer to consumers. If the producer is the renderer in the app, the consumer is Surfaceflinger. The Graphicbuffer queue is placed in the buffer queue bufferqueue. Bufferqueue to the app-side interface is igraphicbufferproducer, the implementation of the class is surface, the interface to the Surfaceflinger end is Igraphicbufferconsumer, The implementation class is Surfaceflingerconsumer. In Bufferqueue, each graphibuffer has a bufferstate flag with its state:

This state partly describes the Graphicbuffer, but only indicates the state in the CPU, and the real user of Graphicbuffer is the GPU. That is, when a producer puts a graphicbuffer into bufferqueue, it only completes the transfer of ownership at the CPU level. But the GPU might still be in use, and if it's still in use, the consumer can't get it. At this time Graphicbuffer and production of consumer relations is more ambiguous, consumers have ownership of Graphicbuffer, but no use, it needs to wait for a signal, tell it the GPU ran out, consumers really have the right to use. A simplified model is as follows:

This notification Graphicbuffer by the last user used the signal is done by fence. The existence of fence is very simple, starting from the birth is to send a signal at the right time. On the other hand, why not call Glfinish () and other GPUs when the producer gives the Graphicbuffer to the consumer? Such ownership and use of rights are passed together, without fence. It is possible to do this functionally, but performance can have an impact because Glfinish () is blocked, and the CPU is unable to work for the GPU itself. If you use fence, you can wait for the graphicbuffer to actually be blocked by the consumer, and before that CPU and GPU can work in parallel. This is equivalent to the lazy passing that implements the critical resource.

Finish the basic function of fence, and then say its realization. Fence, as the name implies is the first to stop, and so on later, both Unison and then go forward. abstractly, fence contains multiple points of time on the same or different time axes, and fence is triggered only when those points arrive simultaneously. For a more detailed introduction, refer to this article (http://netaz.blogspot.com/2013/10/android-fences-introduction-in-any.html)。

Fence can be implemented by hardware (Graphic driver) or by software (Sw_sync in Android kernel). An extension of the synchronization object is provided in EGL Khr_fence_sync (Http://www.khronos.org/registry/vg/extensions/KHR/EGL_KHR_fence_sync.txt)。 It provides Eglcreatesynckhr (), Egldestroysynckhr () to produce and destroy synchronization objects. This synchronization object is a special operation inserted into the GL command queue and, when executed to it, signals that the command in front of the queue has been fully executed. The function Eglclientwaitsynckhr () allows the caller to block the wait signal from occurring.

On top of that, ANDROID has extended the-android_native_fence_sync (Http://www.khronos.org/registry/egl/extensions/ANDROID/EGL_ANDROID_native_fence_sync.txt), the new Interface egldupnativefencefdandroid () is added. It can convert a synchronization object into a file descriptor (in turn, Eglcreatesynckhr () can turn the file descriptor into a synchronization object). This extension is equivalent to allowing the CPU to have a handle to the synchronization object in the GPU, which can be passed between processes (through the IPC mechanisms such as binder or domain sockets), which provides the basis for synchronization between multiple processes. We know that UNIX systems are all files, so the versatility of the fence is greatly enhanced after this extension.

Android has further enriched the fence software stack. Mainly distributed in three parts: C + + Fence class is located in/frameworks/native/libs/ui/fence.cpp; C's Libsync library is located in/SYSTEM/CORE/LIBSYNC/SYNC.C; The Kernel driver section is located in/DRIVERS/BASE/SYNC.C. In all, the kernel driver part is the main implementation of synchronization, Libsync is the encapsulation of the driver interface, fence is a further C + + package for Libsync. Fence will be used as a subsidiary of Graphicbuffer along with Graphicbuffer transmission between producer and consumer. Another fence software implementation is located in the/DRIVERS/BASE/SW_SYNC.C. Syncfeatures is used to query the synchronization mechanism supported by the system:/frameworks/native/libs/gui/syncfeatures.cpp.

Below is an analysis of the specific usage of fence in Android. Its main role is to Graphicbuffer in the app, GPU and HWC between the three to synchronize.

First, Graphicbuffer the journey from app to display. Graphicbuffer is first drawn by the app-side as a producer and then put into Bufferqueue, waiting for the consumer to take out the next rendering composition. Surfaceflinger as a consumer, the corresponding graphicbuffer of each layer is taken to generate Eglimagekhr objects. The processing of Graphicbuffer is divided into two cases when synthesizing. For the overlay layer, Surfaceflinger will place its buffer handle directly into the HWC layer list. For layers that require GPU drawing (exceeding the number of HWC processing layers or having complex transformations), Surfaceflinger will synthesize the previously generated eglimagekhr through Gleglimagetargettexture2does () as a texture (http:/ /snorp.net/2011/12/16/android-direct-texture.html). After the synthesis of Surfaceflinger and as a producer, the GPU synthesized framebuffer handle to HWC Framebuffertarget (HWC The last slot in the _layer_1_t list is used to place the GPU's render result in buffer. HWC finally overlay overlay layer and then throw to display, then HWC is the consumer. The entire approximate process

As you can see, for non-overlay layers, Graphicbuffer has undergone two production consumer models. We know that the Graphicbuffer core contains the buffer_handle_t structure, which points to the native_handle_t containing the file descriptors and other basic properties of the graphics buffers requested in the Gralloc. This file descriptor is mapped to both the client and the server as shared memory.

Because both the service and the client process can access the same physical memory, the absence of synchronization causes an error. In order to reconcile the client and server side, when transferring graphicbuffer, there is also a fence, which marks whether it was used by the previous user. There are two kinds of fence by function: Acquirefence and releasefence. The former is used by producers to inform consumers that production has been completed, the latter for consumers to inform producers that consumption has been completed. The following is a look at the production and use of these two fence process. The first is the use of the acquirefence process:

When the app side inserts Graphicbuffer through Queuebuffer () to Bufferqueue, a fence is passed, which indicates whether the fence has been used by the producer. The graphicbuffer is then taken away by the consumer through Acquirebuffer () and the acquirefence is removed. After the consumer (i.e. Surfaceflinger) wants to render it, it needs to wait for fence to be triggered. If the layer is rendered through the GPU, the place to use it is Layer::ondraw (), where the texture is bound through bindtextureimage ():

486    status_t err = Msurfaceflingerconsumer->bindtextureimage ();

The function will finally call doglfencewaitlocked () to wait for the acquirefence trigger. Because the next step is to draw, if there is no waiting to go straight down, then the rendering is the wrong content.

If the layer is the overlay layer of the HWC rendering, then it does not need to pass through the GPU, then the corresponding acquirefence of these layers need to be uploaded to HWC. In this way, HWC can confirm that the buffer has been used by the producer before compositing, so a normal point of HWC needs to wait for these acquirefence to be fully triggered to draw. The work of this setting is done in Surfaceflinger::d ocomposesurfaces (), which invokes the layer::setacquirefence () function of each layer:

428    if (layer.getcompositiontype () = = Hwc_overlay) {429        sp<fence> Fence = msurfaceflingerconsumer-> Getcurrentfence ();.. 431            FENCEFD = Fence->dup (); 437    LAYER.SETACQUIREFENCEFD (FENCEFD);

You can see that the non-overlay layer is ignored because the HWC does not need to be synchronized directly with the non-overlay layer, as long as it is synchronized with the results of these non-overlay-layer compositions. After the GPU renders the non-overlay layer, queuebuffer () puts the graphicbuffer into framebuffersurface corresponding bufferqueue, and then Framebuffersurface::o Nframeavailable () is called. It first takes a graphicbuffer from the Bufferqueue through the Nextbuffer ()->acquirebufferlocked () and comes with the acquirefence to get it. Then call Hwcomposer::fbpost ()->setframebuffertarget (), which will have just acquire Graphicbuffer and acquirefence set to the HWC layer In the Framebuffertarget slot in the list:

580        acquirefencefd = Acquirefence->dup (); 586    disp.framebuffertarget->acquirefencefd = ACQUIREFENCEFD;

In conclusion, the precondition of the final processing of HWC is that the acquirefence and Framebuffertarget acquirefence of the overlay layer are triggered.

After reading the acquirefence, look at the use of releasefence process:

The previous process of compositing is that the GPU works by synthesizing non-overlay layers in the docomposition () function, and the results are placed in framebuffer. Then Surfaceflinger will call Postframebuffer () to let HWC start working. The most important thing in Postframebuffer () is to call HWC's set () interface to notify HWC to perform a composition display, and then synchronize Releasefence (if any) generated in HWC to Surfaceflingerconsumer. Implement the Onlayerdisplayed () function located in the layer:

151        msurfaceflingerconsumer->setreleasefence (Layer->getandresetreleasefence ());

The above is primarily for overlay layers, what about the GPU-drawn layer? When the invalidate message is received, Surfaceflinger calls Handlemessageinvalidate ()->handlepageflip ()->layer::latchbuffer ()- >surfaceflingerconsumer::updateteximage (), which invokes the glconsumer::updateandreleaselocked () function of the layer corresponding to consumer. The function releases the old Graphicbuffer, which is inserted through the syncforreleaselocked () function before releasing, which means that the Graphicbuffer consumer is already in use if triggered. Then call releasebufferlocked () back to Bufferqueue, of course, with this releasefence. Thus, when the Graphicbuffer is taken out again by the producer through Dequeuebuffer (), the releasefence can be used to determine whether the consumer is still in use.

On the other hand, after the HWC is synthesized, Surfaceflinger calls displaydevice::onswapbufferscompleted (), in turn

Framebuffersurface::onframecommitted (). The onframecommitted () core code is as follows: 148    sp<fence> Fence = mhwc.getandresetreleasefence (Mdisplaytype); 151        status_t err = addreleasefence (mcurrentbufferslot,152                mcurrentbuffer, fence);

Here you get the releasefence of the framebuffertarget generated by HWC to the corresponding Graphicbuffer slots in Framebuffersurface. So the framebuffersurface corresponding Graphicbuffer can also be released back to Bufferqueue. When EGL gets this buffer in the future, it will routinely wait for the releasefence trigger to be used.

The Graphicbuffer synchronization mechanism in Android-fence