The analysis of HEVC coding process

This is the HEVC coded frame diagram, and I'll write down my own understanding of the HEVC coding process for this diagram. (A lot of mistakes, in the rookie stage, look at the peer of this article can give some advice, I will be greatly appreciated)

In the diagram, you can see that there are several clearly interconnected modules. Coding control, transformation quantization, inter-frame prediction (motion estimation, motion Compensation), intra-frame prediction, filter control analysis, loop filtering (DE-block filtering, sampling point adaptive offset filtering), entropy coding. These coding modules are interconnected, and one module affects the other.

(1) Video sequence Input encoder, is divided into several DOP (image group), in the image layer has three coding structure, that is, intra-frame coding, low-latency encoding and random access coding. These three encoding structures define the encoding that follows.

(2) Each frame of the image is divided into non-overlapping coding tree units, these coding tree units can continue to decompose into a coding unit, the coding unit can continue to be decomposed into a predictive unit and a transformation unit.

(3) The first frame of a video sequence (or the first frame of each blank random access point of a video sequence (CRA, clean random access points) only uses intra-frame prediction (that is, only the spatial information between adjacent regions of the same frame image is used for prediction, But the frame is not independent of each other frame). Other frames of the video sequence, or other frames between the two CRA, are mostly predicted by the time-domain inter-frame prediction method. The encoding process for inter-frame prediction consists of motion data, which contains the reference frame and motion vector (MV, motion vector) for each block sample point prediction. Encoders and decoders use motion compensation (MC, motion Compensation) that is composed of MV and mode selection data transmitted as edge information to generate the same inter-frame predictive signal.

(4) According to the coding structure chosen by the GOP layer, the I,P,B frame is specified for each frame in the GOP, and if the sequence is I b b p p p, the frame prediction is made for the first frame in the input GOP, and the frame is predicted by P B frame. The predictive data is subtracted from the original video frame, and the predictive residual input transform quantization module is obtained. The predicted residuals are transferred to the Entropy encoder after the change quantization, and the entropy code is encoded to form a binary code stream. Another way is to reverse the transformation, and then add the predictive data to the block filter and the Sao filter, to form a reconstruction reference frame for inter-frame prediction reference frame.

The role of each module:

(1) Transform quantization: The image is subtracted from the original video frame after intra-frame prediction and real prediction, and the predicted residuals are formed. By compressing the predictive residual information, the transmission code rate can be reduced effectively, and the process is carried out by using the transform coding and quantization technique in HEVC, which can remove the correlation disease of image signal and reduce the dynamic range of image coding.

The transform code transforms the time domain signal into the frequency domain signal, and the energy of the signal in the frequency domain is concentrated in the low frequency region.

The quantization process, without reducing the visual effect, preserves the details of the image, determines the quantization parameters, and reduces the encoding length of the image.

(2) Loop filter: including de-block filtering, sampling point Adaptive offset filter.

GOP meaning: An image group (group of picture) that encodes an encoded video sequence into a set of ordered frames. Each GOP must start with an I-frame, but it does not necessarily refer to the distance between two I frames. Because a GOP may contain several I frames, only the first I-frame (that is, the first frame) is a keyframe. In the program CFG, the length of the GOP and the distance of two I frames are also specified by two different parameters (such as intraperiod and GOP size or similar parameters). Therefore, the spacing of two I frames cannot be greater than the length of the GOP, and the general situation is smaller.

Coding structure and configuration: In order to adapt to the use of different occasions, HEVC has set up three kinds of GOP coding structure, that is, intra-frame coding, low-latency encoding and random access coding.

(1) in-frame coding: Each frame of the image is in accordance with the intra-frame spatial prediction code, not applicable time frame of reference.

(2) Low-latency encoding: Only the first frame image is encoded in the frame, and becomes an IDR frame, and subsequent frames are encoded as normal p-frames and B-frames

(3) Random access Encoding: (random access) The coding structure consists of a batch of B frames, periodically inserting random access frames (clean random to access). The back CRA frames will become random access points in the encoded video stream (random access points: The decoding of these frames can be performed independently, without reference to the image frames previously decoded in the Bitstream).