H.264 Study Notes 4 -- transform Quantization

A. General Introduction to the transformation quantification process

Frame (16x16 and 4x4 brightness, 8x8 color) and frame (4x4 ~ 16x16 brightness, 4x4 ~ 8x8 color) after pixel block prediction, the residual of the prediction block is obtained. To reduce the statistical redundancy of the residual information, the residual data must be transformed and quantified. The overall operation process of transformation and quantification is as follows:

　　　　　　　　

For the brightness block of intra_16x16, perform 16 (4x4) 4x4 forward DCT transformations, and then perform 4x4 Hadamard transformations on the obtained 16 DC coefficients, then, 16 DC coefficients and 240 AC coefficients are quantified (the quantitative formula of DC and AC is slightly different. To control the quantitative dead zone, see the quantizer design ).

For 8x8 color blocks (within and between frames), after 4x4 DCT transformation, four DC and 60 AC coefficients are obtained, similarly, the DC and AC coefficients are quantified after 2x2 Hadamard transformation.

The larger the transform block, the higher the encoding efficiency, and the more detailed information of the image can be retained. So h. 264 in the HD class, 8x8 DCT transformation is supported (set the flag transform_size_8x8_flag = 1), and the Hadamard transformation of the DC coefficient is not required, after DCT transformation, the DC and AC coefficients are quantified in a unified manner.

Then, for the conversion blocks of other dimensions (not intra_16x16, 8x8 color, without the transform_size_8x8_flag = 1 sign), a 4x4 DCT transform is used, and then the DC and AC coefficients are uniformly quantified.

B. Quantitative Introduction

Quantization reduces the bit rate through multi-to-one ing. They mainly include uniform quantization, non-uniform quantization, and adaptive quantization. According to the optimal quantizer design principle: the boundary value of the optimal quantization interval is the average of the two adjacent optimal quantization values, and the optimal quantization value is the mean of the quantization interval.

1. A simple formula for fixed-length scalar quantizer is as follows:

Z = int (| w |/s) * SNG (W ).

W indicates the input value, s indicates the quantization step, INT (x) indicates the maximum integer smaller than or equal to X, and SNG (x) indicates the symbol of X, values:-1, 0, and + 1.

Inverse quantization formula:

W = S * z;

The smaller the quantization duration, the higher the quantization efficiency, but the higher the distortion of the corresponding image. However, for the quantizer above, the residual difference between the range [0, S) is quantified to 0, which obviously does not conform to the design criterion of the optimal quantizer. Therefore, the quantization offset F is introduced.

2. deformation quantizer: introducing quantitative offset F, the formula is as follows:

Z = int (| w | + F)/s) * SNG (w );

The inverse quantization formula is as follows:

W = S * z;

Since the residual Quantization of intervals (F-S, S-F) is 0, the residual Quantization of [S-F, 2s-f) is S, therefore, the quantizer can adjust the value of F to adjust the optimal quantization value of the quantization interval, so that the optimal quantization value meets the average value of the range (W can also be adjusted to meet the requirements of the optimal quantizer when the distribution is uneven), so f can control the offset of the quantization range. In H.264 reference model, it is recommended that f = s/6 be used for inter-frame prediction, and f = S/2 be used for intra-frame prediction.

When the residual difference is (F-S, S-F), the quantization value is 0, so the interval (F-S, S-F) is called the quantization dead zone, F can be used to control the quantitative dead zone size. After DCT transformation, a large number of residual values are in the quantified dead zone, and the residual values of the quantified dead zone often correspond to high-frequency details. In practical application, it is often necessary to deal with the high frequency or a large amount of residual information of the quantum dead zone, so the actual size of the quantum dead zone needs to be controlled. However, F simultaneously controls the quantitative offset and the quantitative dead zone, so in order to understand the coupling, a kind of decoupled quantizer is proposed in the JVT-K026.

3. deformation quantizer: The parameter T is introduced to control the quantization dead zone. The formula is as follows:

Z = int (| w | + F + T)/s) * SNG (w );

Inverse quantization formula:

W = (S * | z |-t) * SNG (z ).

The quantizer uses F to control the quantization value (that is, the offset of the quantization value relative to the breakpoint of the quantization interval), determines that the quantization offset satisfies the optimum quantizer, and then controls the quantization dead zone size through T.

H.264 Study Notes 4 -- transform Quantization