HW-Video acceleration in Chrome/chromium HTML5 video playback hardware acceleration

Introduction

Video decode (e.g. YouTube playback) and encode (e.g. video chat applications) is some of the most complex compute operat  Ions on the modern web. Moving these operations from software running on general-purpose CPUs to dedicated hardware blocks means lower power consu Mption, longer battery life, higher quality (e.g. HD instead of SD), and better interactive performance as the CPU is free Everything else it needs to do.

design

media::videodecodeaccelerator (VDA) and media::videoencodeaccelerator (VEA) (with their respective Client subclasses) is the interfaces at the center of all video HW AC Celeration in Chrome.  each consumer of HW acceleration implements the relevant Client interface and calls an object of the relevant V[de]a interface.

In general the classes this want to encode or decode video live in the renderer process (e.g. the <video> player , or WebRTC ' s video encoders & decoders) and the HW being utilized are not accessible from within the renderer process, soIPC was used to bridge the RENDERER<->GPU process gap.

implementation Details

The main consumers of the acceleration APIs is: <video> pipeline (what is plays media on the web), WebRTC (enabling PL Ugin-free Real-time video chat on the web), and Pepper APIs (offering HW acceleration to Pepper plugins such as Adobe Flash ).

The implementations of the acceleration APIs is specific to the OS (and sometimes HW platform) due to radically different Options offered by the OS and DRIVERS/HW present.

(Not Pictured:obsolete openmax-il-based

id=223194 "style=" Text-decoration:none ">Ovda, and never-launched

id=133828 "style=" Text-decoration:none ">macvda).

Current Status

New devices was released all the time so this list was likely already out of date, but as of early June, existing (pub LIC) support includes:

Decode

• Windows:starting with Windows 7, HW accelerated decode of H. is used via Dxvavda.

• Cros/intel (Everything POST-MARIO/ALEX/ZGB): HW accelerated decode of H. is used via Vavda

• CROS/ARM:HW accelerated decode of VP8 and H. Available via V4l2vda

• ANDROID:HW accelerated decode of VP8 is available on N10, N5, some S4 ' s, and a bunch of other devices. (Note that on Android this has applies to WebRTC, as there is no PPAPI and <video> uses the platform ' s player)

Encode

• CROS/ARM:HW accelerated encode of H. (Everywhere) and VP8 (Devices) is available via V4l2ve A

• ANDROID:HW accelerated encode of VP8 is available on N5.

Results

Generally speaking offloading encode or decode from CPUs to specialized HW have shown an overall battery-life extension of 1 0-25% depending on the platform, workload, etc

Turn from

https://docs.google.com/document/d/1LUXNNv1CXkuQRj_2Qg79WUsPDLKfOUboi1IWfX2dyQE/preview#

HTML5 video playback hardware acceleration, based on Chromium code analysis on KitKat. Finally, the use of Mediacodec.java to use the local native hardware resources to do video playback acceleration.

The approximate block diagram for MediaPlayer is as follows:

And pipeline will finally call through Gpuvideodecoder to Mediacodec:

The detailed timing diagram is not yet clear. With MediaPlayer as a starting point to see chromium the code is really pretty difficult

HW-Video acceleration in Chrome/chromium HTML5 video playback hardware acceleration