Chromium Graphics: Let's talk about the evolution of Chromium WebView hardware rendering mode, chromiumwebview

Chromium Graphics: Let's talk about the evolution of Chromium WebView hardware rendering mode, chromiumwebview

Original article, reprinted in the form of a link to indicate the original source for http://blog.csdn.net/hongbomin/article/details/40896313

Abstract: starting from the first WebView using the Chromium kernel in the Android KitKat system, Android WebView has been continuously evolving since Chromium M38, webView has undergone major changes in the hardware rendering mode. The most obvious change is the support of WebGL and the use of ubercompositor. In addition, to adapt to the rendering model changes of Android L, the DrawGL function is executed in the rendering thread of the Android system.

Hardware rendering for Android 4.4 WebView

For Chromium WebView, it is a single-process model. At least CommandBuffer can be used to save unnecessary cross-process IPC overhead, although the IPC overhead is not as big as you think (according to performance evaluation, it probably takes up to 5% of the overhead ). More importantly, WebView must be a common View in the Android window. It shares the same Android Surface with other views in the window. Therefore,

• WebView responds to the onDraw event on the UI thread, starts the page rendering pipeline, and draws the new frame content to the Canvas through onDraw;
• WebView uses SynchronousCompositor to synthesize page content. "synchronization" refers to running on the UI thread, and merging is triggered by the onDraw event of the UI thread, instead of being triggered by VSYNC. From the thread-based synthesizer perspective, the UI thread is actually a Compositing thread;
• WebView provides a hardware acceleration version of the callback function DrawGL (that is, AwContents: DrawGL), the Chromium rendering assembly line to the rendering path of AndroidView. When hardware acceleration is used, WebView. the Canvas parameter accepted by onDraw is invisible to the Android SDK. view. hardwareCanvas object. HardwareCanvas allows the client to provide its own GL callback function. The Android View system calls this callback function when the View hierarchy is drawn in the playback display list. (Note: In the internal implementation of AndroidFramework, when HardwareCanvas is drawn, the current GL context is bound to the framebuffer object of HardwareCanvas. Therefore, all GL commands in the GL callback function are actually operating the framebuffer object of HardwareCanvas.).

In the Android 4.4 system, when the View level is drawn, the update records and replay of the list are displayed on the UI thread. Accordingly, WebView. in onDraw event processing, the page layers are merged, and the merged content is drawn to HardwareCanvas through the DrawGL function, all of which are executed in the UI thread. After WebView receives the invalidate message, ViewRoot traverses the entire View tree to determine which views are sent onDraw events. When WebView responds to the onDraw event, it directly requests HardwareCanvas to execute the DrawGL callback function, drawGL then requests the synchronous synthesizer to synthesize a new frame of content in hardware acceleration mode, and draws the new content to HardwareCanvas using the GL command. The Blink thread is not mentioned here, because the Blink thread is relatively independent. After the Blink thread is started, it will gradually execute DOM parsing and formatting, and JS code execution, and compute the state information of the rendering layer. After these operations are completed, if the Compositor internal state machine discovers that the page still needs to update the content and execute the animation, the Blink thread notifies the UI thread (that is, the Compositing thread) to set the WebView to the invalidate state, trigger the onDraw event, and then repeat the above process. For details about the process, see:

Necessary explanations:

• The Blink thread enables the Impl-Side-Painting method. When drawing the page content, first record the draw command to a SkPicture, and then replay the draw commands in the Raster thread later, to minimize the load of Blink threads;
• The Raster thread uses the zero-copy Technology to directly draw rasterization into the GPU memory, thus reducing the overhead of data copying from the CPU to the GPU;
• There is only one Compositor throughout the rendering process. In the UI thread, the updated Web page content is merged and written to the fbo of the HardwareCanvas backend, then, Android SurfaceFlinger displays the final UI on the screen.
• Steps 1, 2, 6, and 3, 4, 5 can be performed at the same time. Steps 3, 4, 5 are always preparing the next frame of content.

Here, we need to note that in Android 4.4, WebView does not have a separate GPU thread to execute the GL command, and everything occurs on the UI thread, as a result, it is difficult to provide support for WebGL. Even if we make some effort to support WebGL, it is not satisfactory because performance is a big problem, executing WebGL on the UI thread may prevent the UI from responding at all.

WebView improvement for Android L System

Chromium WebView is still evolving quite fast. From the latest Android L Developer Preview version, there are many improvements to the page rendering pipeline and thread model, as shown in:

Use of ubercompositor

To put it simply, ubercompositor is a unified synthesizer. No matter how many Synths exist in the Chromium system, the final synthesis is always completed by a Compositor called Parent, the only thing other Compositor has to do is to package its managed GPU resources into a CompositorFrame and send it to the Parent Compositor. The Parent Compositor uses the RenderPass list and metadata contained in the CompositorFrame, execute the merging operation to write the final content to the Native window. When the Parent Compositor completes the merging operation, it notifies the Child compositor to clean up resources, as shown in:

To match the rendering thread introduced by Android L (Note: to distinguish it from the rendering thread of the Android system, the rendering thread of Chromium is called a Blink thread.) To seamlessly integrate the Chromium rendering process into the Android L system,Two-level synthesizer is introduced in the new architecture of WebView.:

• Child Compositor: The page content synthesizer is the synchronization synthesizer mentioned earlier. It runs on the UI thread and is a threaded synthesizer. The Blink thread is responsible for generating the status information of the rendering layer, the UI thread is responsible for merging the rendering layer. After receiving the onDraw event, WebView uses ChildCompositor to synthesize a new frame and package the content into the CompositorFrame struct. Later, the Android rendering thread will access this struct. ChildCompositor is created by android_webview: BrowserViewRenderer;
• Parent Compositor: Similar to the Browser process synthesizer in Chrome on Android, it is a single-threaded synthesizer. Its function is to render the thread in the Android system. Note that it is not a UI thread, use the DrawGL callback function to draw a CompositorFrame that has been synthesized by ChildCompositor to HardwareCanvas. ParentCompositor is created by android_webview: HardwareRender;

Unlike Android 4.4, WebView can be logically divided into two steps to process onDraw events on Android L:

• Step 1: Call the Child Compositor merging operation on the UI thread to generate a new CompositorFrame, and then request the Android system to execute the DrawGL callback function. However, DrawGL will not be executed immediately. The UI thread is responsible for generating a display list for the View that has an invalidate operation, and synchronizing the display list to the ThreadedRenderer Renderer on the rendering thread;
• Step 2: the Android View system submits the display list to the GPU driver on the rendering thread to draw the View level content. The DrawGL function will be called when the display list is drawn, on the rendering thread, request Parent Compositor to draw the merged page content to HardwareCanvas.

Note: about how the DrawGL callback function is dispatched to the Android rendering thread for execution, because the source code of AndroidL is not completely disclosed, it cannot be 100% determined whether the above description is completely correct. According to the information analysis, the UI thread will initiate a DrawGL call request to HardwareCanvas. At this time, the DrawGL may not be executed immediately, but is saved as a drawOp in HardwareCanvas first, when the rendering thread starts to replay the HardwareCanvas display list, call DrawGL.

The above two steps are exactly the same as the rendering thread model used by Android L, which is also the reason for introducing a two-level synthesizer. AwContents: DrawGL is an extremely critical method to integrate the Chromium graphics system into Android. It is the only entry for the Android L rendering thread to enter the Chromium graphics system, drawGL is called in Android L in the rendering thread for the HardwareCanvas object replay display list. On the other hand, Chromium can only be passively called by the rendering thread by the DrawGL operation, the only information that can be obtained is the FBO object of the current HardwareCanvas backend, and all the content is drawn to this Canvas. Other information, such as message queue and the ability to execute tasks asynchronously, is hard to be obtained, as a result, Chromium can only wait for SyncPoint, refresh CommandBuffer, and execute all the tasks in the task queue in DrawGL, this is why the WebGL mentioned below needs to be executed in a dedicated GPU thread.

In addition, the new rendering model WebView is incompatible with the Android4.4 system, that is, libwebviewchromium. so compiled from AndroidL cannot be run on Android 4.4.

Support for WebGL and hardware-accelerated Canvas 2D

Taking WebGL as an example, WebView creates a dedicated GPU thread for rendering WebGL. All gl commands in the WebGL program are generated on this thread. The egl context for rendering WebGL is created on this thread. However, this egl context is an independent context and does not share resources with any other context in the WebView process, that is, the context of the AndroidView system is not in the same share group. For more information about Chromium context and share group, see ChromiumGraphics: 3D context and virtualization. The context of the Android View system refers to the context when DrawGL is called to draw HardwareCanvas. Therefore, the content of WebGL must be drawn to HardwareCanvas, solve the Problem of texture sharing and synchronization between WebGL context and AndroidView System context. First, both contexts belong to the same process, so you can use EGLImage for cross-thread texture sharing. Again, the mailbox texture mechanism provided by the Chromium graphic stack helps solve texture sharing. The Mailbox texture mechanism generates a 64-byte unique name for each generated texture and creates a global ing table named texture id. Therefore, no matter which context the texture is created, the corresponding texture id can be retrieved through the mailbox name. The advantage of the mailbox mechanism is that the global alias System of textureid is provided in a multi-context environment. In other words, with this alias system, all contexts seem to belong to the same share group ".

So how does the EGLImage and mailbox texture mechanisms be used to render WebGL for resource sharing?

The content of WebGL in Chromium is rendered to texture through the framebuffer object on the off-screen. When this texture is created in the WebGL context of the GPU thread, it is bound to a unique mailbox name, you will also call eglCreateImageKHR to create an EGLImage for this texture id that can be shared by other contexts. When the rendering thread of the AndroidView system executes the DrawGL function in another context, by using the mailbox name, you can determine the EGLImage In the GPU thread and bind it to a texture in the context to directly use the texture of the WebGL context. This solves the texture sharing problem. There is another problem that needs to be explained. When sharing texture in multiple contexts, the synchronization problem between the producer and the consumer must be well handled, that is, when the DrawGL function is executed to draw a new frame, make sure that all gl commands in the WebGL context have been executed. Chromium also provides A syncpoint mechanism to ensure the execution order of GL commands between multiple contexts (also called CommandBuffer in Chromium) and insert A synchronization point in context, then, wait for the synchronization point in context B to ensure that the GL command in context B can be executed only after all the GL commands before the context synchronization point A are executed.

Therefore, after WebGL generates a new frame, a synchronization point is inserted. When DrawGL is used, the AndroidView SYSTEM context will wait for this synchronization point, the content of WebGL is used only after all the current WebGL commands are submitted to the GPU driver, thus ensuring the synchronization of WebGL content and DrawGL. For details about the synchronization point mechanism, see the article ChromiumGraphics: principles and implementation analysis of the synchronization mechanism between GPU clients.

AndroidWebViewShell also supports hardware rendering Mode

WebView Shell is built on the core class AwContents. AwContents can basically be equivalent to android. webkit. WebView. In the Chromium M30 code library, only WebViewShell that supports software rendering mode can be compiled. Starting from Chromium M38, WebView Shell also supports hardware accelerated rendering. The main improvement is, GLSurfaceView is used as the final rendering destination. As described above, the DrawGL callback function can be called on a non-UI thread, while GLSurfaceView also supports rendering in a separate thread.

Summary

Finally, let's summarize several important features of Android L WebView: first, the in-process mode command line buffer is used, unlike the Chromium browser, which has no IPC overhead; second, the DrawGL function of the Parent synthesizer is the only entry for the Android Render thread to enter the Chromium system. It is called in the Android Render thread. Third, the WebGL and hardware-accelerated Canvas 2D run in a separate GPU thread, and use EGLImage to implement GPU resource sharing between the GPU thread and the Android Render thread. Fourth, the UI thread still has a lot to do, including the Child synthesizer "merging" page content packaged into a CompositorFrame structure to process user input events. The fifth, the UI thread and the Android Render thread need a certain mechanism to synchronize the access to the CompositorFrame struct. Sixth, WebView Shell can support the hardware acceleration mode.
Original article. For more information, see http://blog.csdn.net/hongbomin/article/details/40896313. thank you!

How does Chromium display web pages)

Application level concept Diagram
Each layers box represents the application layer in a concept. Generally, it is possible to replace any layer and its upper layers to form a new browser. Therefore, no layer should be dependent on a higher level. WebKit: Safari, Chromium, and all other WebKit-based browsers use Webkit as the rendering engine. WebKit Port is a part of WebKit. It processes platform-specific operations, such as resource loading and graphics. Glue: converts the WebKit type to the Chromium type. This is our "WebKit embedding layer ". This is the basis for chrome and test_shell browsers (allowing us to test WebKit. Renderer/Render Host: this is the "multi-process embedding layer" of Chromium ." It is used to send messages and commands across processes. As you can imagine, other multi-process browsers can also use this layer, which is not dependent on other browser services. Tab contents: the special layer of Chrome to indicate the content displayed by the tag. It is bound to the Application Service, such as the password manager and history system. This layer should not be assumed to be embedded in the Chromium browser window (Other Chromium components such as the "HTML dialog box" to use this layer ). Browser: displays the browser window, which is embedded with multiple TabContentses. WebKit: we use the WebKit Open-Source Project to display webpages. This code is written by Apple and stored in the/third_party/WebKit directory. WebKit consists of two parts: "WebCore", which is responsible for the core layout function and "JavaScriptCore", which is used to execute JavaScript. We only use JavaScriptCore for testing purposes. We usually use a high-performance V8 JavaScript Engine to replace it. We do not actually use the software layer called "WebKit" by Apple (the content under the WebKit/Source/WebKit directory, and The WebCore and JavaScriptCore directories under the Webkit/Source directory ), this software layer is used in applications such as Safari to connect WebCore and OS X. For convenience, we usually call the code obtained from Apple "WebKit ". The WebKit Port is at The bottom layer. We have our WebKit Port ". This is the Code related to the platform we implemented. It is used to connect the platform and WebCore. These files are located in the WebKit Directory, which is usually in the Chromium directory or suffixed with Chromium. In fact, most of the Port code is not related to the operating system: You can regard it as the Chromium Port of WebCore ). Some parts, such as font rendering, must be processed separately for each operating system platform. Network Traffic is handled by our multi-process resource loading system, rather than directly calling the operating system by the rendering process. Use the Skia graphics library developed for Android. This is a cross-platform graphics library that processes all images and images except text. Skia is located in/third_party/skia. The main entry point for graphic operations is/WebKit/port/platform/graphics/GraphicsContextSkia. cpp. It uses the same directory and many other files in the/base/gfx directory. WebKit glue Chromium uses different types, encoding methods, and code structures than WebKit code. WebKit "glue" provides a more convenient WebKit embedded API... the rest of the full text>

Libwebviewchromiumso can be deleted

After deletion, you cannot view your camera from the chromium browser.