Hardware accelerated rendering technology based on FFMPEG+VAAPI

Reproduced from the spiritual Leap cloud: the original link

1. Write in front of hardware acceleration that uses the GPU to do graphics-related operations, and the CPU is freed up to handle other transactions, especially if the CPU is running low on performance. In video processing, DXVA is Microsoft's Custom Video acceleration specification, while on Linux platforms is the Vdpau and VAAPI acceleration specifications provided by Nvidia and Intel respectively. The LEAP Desktop Cloud leverages these APIs to effectively accelerate video decoding and image processing, reduce system CPU usage, and improve system performance. 2. The implementation of the solution for the Intel CPU X86 architecture platform, can be used ffmpeg+vaapi hard solution. The ffmpeg is used to handle video decoding in various formats, while VAAPI provides access to hardware-accelerated video processing, with the following flowchart.

2.1 Related environments Deployment (i) Install dependent packages

# apt-get Install autoconf automake debhelper ladspa-sdk libsoundio-dev libbluray-dev libbs2b-dev libcaca-dev libcdio-par Anoia-dev libdvdnav-dev libdvdread-dev libegl1-mesa-dev libenca-dev libfontconfig1-dev Libfreetype6-dev Libfribidi-dev libgl1-mesa-dev libgbm-dev libgme-dev libgnutls28-dev libgsm1-dev libguess-dev Libharfbuzz-dev Libjack-jackd2-dev libjpeg-dev liblcms2-dev liblircclient-dev liblua5.3-dev libmodplug-dev Libmp3lame-dev Libnetcdf-dev libopenal-dev libopus-dev libopencore-amrnb-dev libopencore-amrwb-dev libpulse-dev Librtmp-dev Librubberband-dev libschroedinger-dev libsmbclient-dev libssh-dev libsoxr-dev libspeex-dev libtheora-dev libtool Libtwolame-dev libuchardet-dev libv4l-dev libva-dev libvorbis-dev libvo-amrwbenc-dev libvpx-dev Libwavpack-dev Libx264-dev libx265-dev libx11-dev libxext-dev libxinerama-dev libxrandr-dev libxss-dev libxv-dev Libxvidcore-dev Pkg-config python python-docutils x11proto-core-dev yasm Zlib1g-dev

(ii) Installation of LIBVA

Enter the current user directory and download the LIBVA source package:

# CD ~

#git Clone Https://github.com/01org/libva.git

Go to the Libva folder,

#cd LIBVA

Switch to 1.7.3 Version:

# git checkout libva-1.7.3

To perform a compilation installation:

./autogen.sh &&./configure && make && make install

Run Vainfo View Execution results

The video compression format for the currently supported hard solutions is shown in the red box.

(iii) installation of FFmpeg

Enter the current user directory of the desktop cloud and download the FFmpeg source package

# CD ~

#git Clone Https://git.ffmpeg.org/ffmpeg.git FFmpeg

Go to the FFmpeg folder,

#cd FFmpeg

Perform a compilation installation

./configure--disable-static--enable-shared--enable-gpl--enable-version3--disable-debug--enable-vaapi && Make && make install

Last execution FFmpeg view the execution results.

2.2 Code implementation (a) The principle of video playback

To put it simply, the video player needs to go through the following steps when playing a local video file: Unpacking, decoding, and audio-video frame synchronization, which will eventually deliver the data to the corresponding drive device, as shown in the process:

(ii) VAAPI decoding process

When the program passes the compressed data of the video to the VAAPI decoding module, the corresponding Vadecoder is created according to the compressed format of the video (such as h.264.mpeg-2, etc.), and then the data is sent to Vadecoder for decoding. Finally, the vaputsurface to deal with the video decoding after the synthesis and rendering, the following process:

(iii) Concrete implementation

The decoding flowchart based on Vaapi and FFmpeg is as follows:

Since the FFmpeg version of the desktop cloud is version 3.3, which integrates the details of the video in the Vaapi, it is only necessary to specify the hard solution parameters in the Avcodeccontext Get_format callback function, and the decoded video data is stored in the GPU memory. Process diagrams for window creation and image processing using the Vaputsruface function to handle post-production compositing and image rendering:

3. Test results

This test is a comparison of CPU utilization using GPU hard and CPU soft solution, video parameters: Resolution 1920*1080, Frame rate 25FPS, encoding format H. Desktop cloud based on the test results, it can be seen that the use of GPU hard solution can greatly reduce CPU utilization, while the output of the picture is clearer and smoother.

Decoding type	Percent CPU consumption
GPU Hard solution based on VAAPI	4%
Based on CPU soft solution	50%

4. Written in the end, the Spirit Leap desktop cloud based on VAAPI hardware acceleration technology is complete.

Hardware accelerated rendering technology based on FFMPEG+VAAPI