GraphicBuffer synchronization mechanism in Android-Fence

GraphicBuffer synchronization mechanism in Android-Fence

Fence is a synchronization mechanism. It is mainly used for GraphicBuffer synchronization in the graphics system in Android. What are the characteristics of the synchronization mechanism? It is mainly used to process cross-hardware scenarios, especially the synchronization between CPU, GPU, and HWC. It can also be used for synchronization between multiple time points. A big difference between GPU programming and pure CPU programming is that it is asynchronous. That is to say, when we call the GL command to return, this command is not necessarily complete, just put this command in the local command buffer. The CPU does not know when the GL command is actually executed. Unless the CPU uses glFinish () to wait until these commands are executed, another method is based on the Fence mechanism of the synchronization object. The following is an example of a producer giving GraphicBuffer to a consumer. For example, the producer is the renderer in the App and the consumer is SurfaceFlinger. Put the GraphicBuffer queue in the Buffer Queue BufferQueue. The interface of BufferQueue to the App is IGraphicBufferProducer, the implementation class is Surface, the interface of SurfaceFlinger is IGraphicBufferConsumer, and the Implementation class is SurfaceFlingerConsumer. Each GraphiBuffer in BufferQueue has a BufferState marked with its status:

This State indicates the ownership of GraphicBuffer to a certain extent, but only indicates the status in the CPU, while the real user of GraphicBuffer is GPU. That is to say, when the producer puts a GraphicBuffer into BufferQueue, it only completes the ownership transfer at the CPU level. However, GPUs may still be in use. If they are still in use, consumers cannot merge them. At this time, the relationship between GraphicBuffer and the production consumer is ambiguous. The consumer has the right to GraphicBuffer but has no right to use it. It needs to wait for a signal to tell it that the GPU is used up before the consumer has the right to use it. A Simplified Model is as follows:

Release + ttPB0Meww + release/K/release/ycjDtffTw9Xf1 + jI + release + qGjPGJyPgo8YnI + CtTatMu7 + release/sPoyva3 + release/ztcS + 5LH6o6zOxLz + w + examples/Bytes/By6GjPGJyPgo8YnI + examples/samples + rL61d + 6zc/examples/samples + Cs/Cw + samples + 38zl08O3qKGjy/found + found/found + 7fR1d/iobp21./found + 7fR1d + found/found + m/found/fOqs7GwO29 + NDQus + found + 6687X + logs/7t9HV36Gj1fu49rTz1sLB97PMyOfNvKO6PGJyPgo8L3A + CjxwPjxpbWcgc3JjPQ = "http://www.2cto.com/uploadfile/Collfiles/20141001/201410010908579.png" alt = "\">

We can see that for non-Overlay layers, GraphicBuffer goes through two production consumer models successively. We know that the core of GraphicBuffer includes the buffer_handle_t structure. The native_handle_t it points to contains the file descriptor and other basic attributes of the graphic buffer applied in gralloc, this file descriptor is mapped to both the client and server as the shared memory.

Because both the service and client processes can access the same physical memory, errors may occur without synchronization. In order to coordinate the client and the server, the transmission of GraphicBuffer also carries Fence, indicating whether it is used by the previous user. Fence can be divided into acquireFence and releaseFence. The former is used to notify the consumer that production has been completed, and the latter is used to notify the consumer that consumption has been completed. The following describes the generation and use of these two Fence types. The first is the use process of acquireFence:

When the App inserts GraphicBuffer to BufferQueue through queueBuffer (), a Fence is introduced, which indicates whether the GraphicBuffer has been used by the producer. Then the GraphicBuffer is taken away by the consumer through acquireBuffer (), and The acquireFence is also taken out. When the consumer (SurfaceFlinger) wants to render it, it needs to wait until Fence is triggered. If the Layer is rendered by GPU, the Layer used is Layer: onDraw (), where the texture is bound through bindTextureImage:
486 status_t err = mSurfaceFlingerConsumer-> bindTextureImage ();
This function will finally call doGLFenceWaitLocked () to wait for acquireFence to trigger. The next step is to draw a picture. If you don't wait to go down, the rendered content is the wrong content.

If this layer is an Overlay layer of HWC rendering, you do not need to pass through the GPU, then you need to upload the acquireFence corresponding to these layers to HWC. In this way, HWC can confirm whether the buffer has been used by the producer before merging. Therefore, a normal HWC must wait for all the acquireFence to be triggered before drawing. This setting is completed in SurfaceFlinger: doComposeSurfaces (). This function calls the layer: setAcquireFence () function of each layer:
428 if (layer. getCompositionType () = HWC_OVERLAY ){
429 sp Fence = mSurfaceFlingerConsumer-> getCurrentFence ();
...
431 fenceFd = fence-> dup ();
...
437 layer. setAcquireFenceFd (fenceFd );
We can see that the non-Overlay layer is ignored, because HWC does not need to be synchronized directly with the non-Overlay layer, as long as it synchronizes with the results of the non-Overlay layer synthesis FramebufferTarget. After the GPU finishes rendering a non-Overlay layer, it uses queueBuffer () to put GraphicBuffer into the BufferQueue corresponding to FramebufferSurface, and then FramebufferSurface: onFrameAvailable () is called. It first obtains a GraphicBuffer from the BufferQueue through nextBuffer ()-> acquireBufferLocked (), with its acquireFence. Next, call HWComposer: fbPost ()-> setFramebufferTarget (), where the GraphicBuffer of acquire is set to FramebufferTarget slot in the Layer list of HWC together with acquireFence:
580 acquireFenceFd = acquireFence-> dup ();
...
586 disp. framebufferTarget-> acquireFenceFd = acquireFenceFd;
To sum up, the precondition for HWC to perform final processing is that the acquireFence of the Overlay layer and the acquireFence of FramebufferTarget are all triggered.

After reading acquireFence, let's look at the procedure of using releaseFence:

As mentioned above, the merging process is GPU-based. In the doComposition () function, a non-Overlay layer is formed, and the result is placed in framebuffer. SurfaceFlinger then calls postFramebuffer () to start HWC. In postFramebuffer (), the most important thing is to call the set () interface of HWC to notify HWC for merging and display, and then synchronize the releaseFence (if any) generated in HWC to SurfaceFlingerConsumer. Implemented in the onLayerDisplayed () function of the Layer:
151 mSurfaceFlingerConsumer-> setReleaseFence (layer-> getAndResetReleaseFence ());
The above is mainly for the Overlay layer. What about the GPU-drawn layer? When receiving the INVALIDATE message, SurfaceFlinger calls handleMessageInvalidate ()-> handlePageFlip ()-> Layer: latchBuffer ()-> SurfaceFlingerConsumer: updateTexImage (), the GLConsumer: updateAndReleaseLocked () function corresponding to the Consumer of the layer is called. This function will release the old GraphicBuffer. Before the function is released, it will use the syncForReleaseLocked () function to insert releaseFence, which means that if the GraphicBuffer consumer is used up at the time of triggering. Call releaseBufferLocked () and return it to BufferQueue. Of course, this releaseFence is also carried. In this way, when the GraphicBuffer is taken out by the producer again through dequeueBuffer (), the releaseFence can be used to determine whether the consumer is still in use.

On the other hand, after HWC synthesis, SurfaceFlinger will call DisplayDevice: onSwapBuffersCompleted ()-> FramebufferSurface: onFrameCommitted () in sequence (). The onFrameCommitted () Core code is as follows:
148 sp Fence = mHwc. getAndResetReleaseFence (mDisplayType );
...
151 status_t err = addReleaseFence (mCurrentBufferSlot,
152 mCurrentBuffer, fence );
Get the releaseFence of FramebufferTarget generated by HWC and set it to the corresponding GraphicBuffer Slot in FramebufferSurface. In this way, the GraphicBuffer corresponding to FramebufferSurface can be released back to BufferQueue. When EGL obtains the buffer from the buffer in the future, it also needs to wait for the releaseFence to be triggered before it can be used.