The Graphicbuffer synchronization mechanism in Android-fence

Fence is a synchronization mechanism that is used primarily in Android for Graphicbuffer synchronization in a graphics system. What does it have to do with the existing synchronization mechanism? It is primarily used to handle cross-hardware scenarios. Especially the CPU. Synchronization between the GPU and HWC, plus it can be used for synchronization between multiple points in time. A very big difference between GPU programming and pure CPU programming is that it is asynchronous. This means that when we call GL command to return, this command is not necessarily complete. Just put this command in the local commands buffer. In detail when this GL command is actually run over the CPU is not known, unless the CPU uses glfinish () to wait for these commands to run out, the second method is based on the fence mechanism of the synchronization object. Here is an example of a producer handing Graphicbuffer to consumers. If the producer is the renderer in the app. Consumers are surfaceflinger. The Graphicbuffer queue is placed in the buffer queue bufferqueue.

Bufferqueue to the app-side interface is igraphicbufferproducer, the implementation of the class is surface, the interface to the Surfaceflinger end is Igraphicbufferconsumer, The implementation class is Surfaceflingerconsumer.

Each graphibuffer in Bufferqueue has a bufferstate mark of its state:

This state partly describes the Graphicbuffer, but only indicates the state of the CPU, and the real user of Graphicbuffer is the GPU. That is, when a producer puts a graphicbuffer into bufferqueue, the transfer of ownership is completed only at the CPU level.

But the GPU is probably still in use, assuming that the consumer is not able to synthesize it. At this time the relationship between Graphicbuffer and production consumers is more ambiguous. Consumers have ownership of Graphicbuffer. But without the right to use it, it needs to wait for a signal to tell it that the GPU is running out and consumers really have access. A simplified model such as the following:

This notice graphicbuffer by the last user used the signal is finished by fence. The existence of fence is simple, from the beginning of the birth is to send a signal at the right time.

There is also a point of view, why not the producer of the Graphicbuffer to the consumer when the call Glfinish () and other GPU finished it? So the ownership and the right to use are passed together. No fence required. Functionally this is possible, but performance can have an impact because glfinish () is clogged. At this point the CPU is unable to work for the GPU itself. Assuming that the fence will be able to wait for the graphicbuffer to actually be plugged by the consumer, the CPU and GPU will be able to work in parallel. This is equivalent to the lazy passing that implements the critical resource.

Finish the basic function of fence, and then say its realization.

Fence. As the name implies is the first to stop, and so on later. The two unison and go forward. To say abstractly. Fence includes multiple points of time on the axis at the same or not the same time. Only when these points arrive at the same time will the fence be triggered. A more specific introduction is available for this article (http://netaz.blogspot.com/2013/10/android-fences-introduction-in-any.html).

Fence can be implemented by hardware (Graphic driver) and can be implemented by software (Sw_sync in Android kernel).

The extension Khr_fence_sync (http://www.khronos.org/registry/vg/extensions/KHR/EGL_KHR_fence_sync.txt) of the synchronization object is provided in EGL.

The Eglcreatesynckhr () is provided. Egldestroysynckhr () generates and destroys synchronization objects. This synchronization object is a special operation inserted into the GL command queue and, when run to it, signals that the command in front of the queue has all run completed. The function Eglclientwaitsynckhr () allows the caller to block the wait signal from occurring.

On this basis. ANDROID has extended the-android_native_fence_sync (http://www.khronos.org/registry/egl/extensions/ANDROID/EGL_ANDROID_ Native_fence_sync.txt). The new Interface Egldupnativefencefdandroid () is added.

It can convert a synchronization object into a file descriptive descriptor (in turn, Eglcreatesynckhr () can turn a file descriptor into a synchronization object). This extension is equivalent to allowing the CPU to have a handle to the synchronization object in the GPU, which can be passed between processes (through the IPC mechanisms such as binder or domain sockets), which provides the basis for synchronization between multiple processes.

We know that UNIX systems are all files, so the versatility of the fence is greatly enhanced after this extension.

Android has further enriched the fence software stack. Mainly distributed in three parts: C + + Fence class is located in/frameworks/native/libs/ui/fence.cpp; C's Libsync library is located in/SYSTEM/CORE/LIBSYNC/SYNC.C; The Kernel driver section is located in/DRIVERS/BASE/SYNC.C.

It's gotta be. The kernel driver part is the main implementation of synchronization, Libsync is the encapsulation of the driver interface. Fence is a further C + + package for Libsync.

Fence will be used as a subsidiary of Graphicbuffer along with Graphicbuffer transmission between producer and consumer.

Another fence software implementation is located in the/DRIVERS/BASE/SW_SYNC.C. Syncfeatures is used to query the synchronization mechanism supported by the system:/frameworks/native/libs/gui/syncfeatures.cpp.

Below is an analysis of the detailed use of fence in Android.

Its basic role is to synchronize graphicbuffer between apps, GPUs and HWC.

First, Graphicbuffer the journey from app to display. Graphicbuffer is first drawn by the app side as a producer. Then put it into the bufferqueue. Wait for the consumer to take out the next render composition. Surfaceflinger as a consumer. The corresponding graphicbuffer of each layer is taken to generate the Eglimagekhr object. The processing of Graphicbuffer is divided into two cases when synthesizing. For the overlay layer. Surfaceflinger will place its buffer handle directly into the HWC layer list.

For layers that require GPU drawing (more than HWC processing layers or complex transformations). Surfaceflinger will synthesize the previously generated eglimagekhr through Gleglimagetargettexture2does () as a texture (http://snorp.net/2011/12/16/ android-direct-texture.html).

After the synthesis of Surfaceflinger and as a producer. Handle the GPU-crafted framebuffer into framebuffertarget in HWC (HWC in hwc_display_contents_1_t The last slot in the T list is used to place the GPU's render result in buffer).

HWC finally overlay overlay layer and then throw to display, then HWC is the consumer.

The entire approximate process

Can see, for non-overlay layer of graphicbuffer successively through two production consumer model. We know that the Graphicbuffer core consists of the buffer_handle_t structure, which points to native_handle_t that includes the file descriptor and other basic properties of the graphics buffers applied to the gralloc. This descriptive descriptor is mapped to the client and server at the same time. As shared memory.

Because both the service and the client process are able to access the same physical memory, it can cause errors without synchronization. In order to reconcile the client and the service side, when transmitting Graphicbuffer. Also comes with fence, which marks whether it was used by a previous user. There are two kinds of fence by function: Acquirefence and releasefence. The former is used for producers to inform consumers that production has been completed, the latter for consumers to inform producer consumption has been completed. The following is a look at the production and use of these two fence. The first is the use of the acquirefence process:

When the app side inserts Graphicbuffer through Queuebuffer () to Bufferqueue, a fence is passed, which indicates whether the fence has been used by the producer. The graphicbuffer is then taken away by the consumer through Acquirebuffer () and the acquirefence is taken out at the same time. After the consumer (i.e. Surfaceflinger) wants to render it, it needs to wait for fence to be triggered. Assuming that the layer is rendered through the GPU, the place to use it is Layer::ondraw (). The textures are bound through bindtextureimage ():
486 status_t err = Msurfaceflingerconsumer->bindtextureimage ();
The function will finally call doglfencewaitlocked () to wait for the acquirefence trigger. Because the next step is to draw. If you don't wait to go straight down here, it's the wrong thing to render.

Assuming that the layer is the overlay layer of the HWC rendering, then it is not necessary to pass the GPU, which requires the corresponding acquirefence of these layers to be uploaded to HWC. Such HWC can confirm that the buffer has been used by the producer before synthesizing it, so a normal point of HWC needs to wait for these acquirefence to be fully triggered to draw. The work of this setting is completed in Surfaceflinger::d ocomposesurfaces (). The function invokes the Layer::setacquirefence () function of each layer:
428 if (layer.getcompositiontype () = = Hwc_overlay) {
429 sp<fence> Fence = Msurfaceflingerconsumer->getcurrentfence ();
...
431 FENCEFD = Fence->dup ();
...
437 LAYER.SETACQUIREFENCEFD (FENCEFD);
Can see that ignores the non-overlay layer, because HWC does not need to be directly and non-overlay layer synchronization, it just and these non-overlay layer synthesis Results framebuffertarget synchronization can be. After the GPU renders the non-overlay layer, the graphicbuffer is placed framebuffersurface the corresponding Bufferqueue by Queuebuffer (). Then Framebuffersurface::onframeavailable () is called. It first takes a graphicbuffer from the Bufferqueue through the Nextbuffer ()->acquirebufferlocked () and comes with the acquirefence to get it.

Then call Hwcomposer::fbpost ()->setframebuffertarget (), which will have just acquire Graphicbuffer and acquirefence set to the HWC layer In the Framebuffertarget slot in the list:
580 ACQUIREFENCEFD = Acquirefence->dup ();
...
586 disp.framebuffertarget->acquirefencefd = ACQUIREFENCEFD;
In conclusion, the precondition of the final processing of HWC is that the acquirefence and Framebuffertarget acquirefence of the overlay layer are triggered.

Read Acquirefence. Take a look at Releasefence's use flow:

The previous process of compositing is that the GPU works by synthesizing non-overlay layers in the docomposition () function, and the results are placed in framebuffer. Then Surfaceflinger will call Postframebuffer () to let HWC start working.

The most important thing in Postframebuffer () is to call HWC's set () interface to notify HWC to perform a composition display, and then synchronize Releasefence (if any) generated in HWC to Surfaceflingerconsumer. Implement the Onlayerdisplayed () function located in the layer:
151 Msurfaceflingerconsumer->setreleasefence (Layer->getandresetreleasefence ());
The above is primarily for overlay layers, what about the GPU-drawn layer? When the invalidate message is received, Surfaceflinger calls Handlemessageinvalidate ()->handlepageflip ()->layer::latchbuffer ()- >surfaceflingerconsumer::updateteximage (), in which the corresponding consumer glconsumer::updateandreleaselocked () function of the layer is called.

The function will release the old Graphicbuffer, which will be inserted through the syncforreleaselocked () function before releasing the releasefence, representing the assumption that the Graphicbuffer consumer has finished using the trigger. Then call releasebufferlocked () back to Bufferqueue, of course, with this releasefence.

Such When this graphicbuffer is taken out again by the producer through Dequeuebuffer (). This releasefence can be used to infer whether the consumer is still in use.

On the other hand, when the HWC synthesis is complete, Surfaceflinger will call Displaydevice::onswapbufferscompleted (), in turn framebuffersurface::o Nframecommitted (). onframecommitted () core code such as the following:
148 sp<fence> Fence = mhwc.getandresetreleasefence (Mdisplaytype);
...
151 status_t err = Addreleasefence (Mcurrentbufferslot,
Mcurrentbuffer, fence);
Here you get the releasefence of the framebuffertarget generated by HWC to the corresponding Graphicbuffer slots in Framebuffersurface. In this way framebuffersurface corresponding Graphicbuffer can also be released back to Bufferqueue. When EGL gets this buffer in the future, it will routinely have to wait for the releasefence to trigger the ability to use it.

The Graphicbuffer synchronization mechanism in Android-fence