H.264 Structure

1. H.264

H.264, also known as H.264/AVC (Advanced Video Encoding), is a video encoding standard jointly developed by MPEG and ITU. In principle, H. 264 still adheres to the traditional hybrid encoding architecture, but due to the introduction of coding details and many improvements, many technologies that have been stuck on paper, such as motion compensation for Variable Size Blocks and multi-reference frame prediction, have been put into practice, significantly improving the coding efficiency, it also greatly increases the complexity and computing workload of the decoder. JVT, Joint Video Team) the proposed high compression digital video codecs standard. The corresponding standard document is ISO 14496-10. this standard is generally called H. 264/AVC (or AVC/h. 264 or H. 264/MPEG-4 AVC or MPEG-4/h. 264 AVC.

2. Structure

H.264 is divided into two layers:

Video Encoding layer (VCL: Video Coding layer): implements video encoding and decoding, including motion compensation prediction, transform encoding, and entropy encoding;

NAL: Network extract action layer: Used to encapsulate and package VCL video data in an appropriate format.

VCL needs to be packaged into nal for transmission or storage.

Layered objective:

(1) You can define the interface between VCL video compression processing and NAL network transmission mechanism. This allows the design of VCL in the video encoding layer to be transplanted on different processor platforms, it has nothing to do with the data Encapsulation Format of the nal layer;
(2) Both VCL and NAL are designed to work in different transmission environments. In heterogeneous network environments, VCL bit streams do not need to be reconstructed and reencoded.

3. nal3.1 nal Unit)

The Nal unit (NALU) is the basic syntax structure of Nal. It consists of a 1-byte header, three fixed-length fields, and an uncertain number of encoding segments.

NALU header (8bit) = prohibited bit (1bit) + importance indication bit (2bit) + NALU type (5bit)

NALU type: 1 ~ 12 used by H.264, 24 ~ 31 is used by applications other than H.264.

Importance indication: indicates the importance of the nal unit for reconstruction. The greater the value, the more important it is.

Bit prohibited: when the network discovers that the nal unit has a bit error, you can set this bit to 1 so that the receiver can discard this unit.

Note: NALU type

• 0: Not specified
• 1: Non-IDR images do not use data segments
• 2: segments of Class A data in non-IDR Images
• 3. segment division of Class B data in non-IDR Images
• 4: division of Class C data segments in non-IDR Images
• 5: IDR image Fragment
• 6: Supplemental enhancement information (SEI)
• 7. sequence parameter set (SPS)
• 8: Image parameter set (PPS)
• 9: delimiter
• 10: sequence Terminator
• 11: stream Terminator
• 12: Fill in Data
• 13. sequence parameter set Extension
• 14: nal unit with a prefix
• 15: subsequence parameter set
• 16-18: Reserved
• 19: No data division is used to encode image segments.
• 20: encoding segment Extension
• 21-23: Reserved
• 24-31: Not specified

If the slice corresponding to NALU is the beginning of a frame, it is expressed in 4 bytes, that is, 0x00000001; otherwise, it is expressed in 3 bytes, 0x000001

3.2 nal Parsing

After being split with 0x000001, It is the nal header. Format: 0aab BBBB

Resolution:

(1) A 1st-bit prohibition bit. A value of 1 indicates a syntax error.
(2) 2nd ~ The third digit is the reference level.
(3) 4th ~ 8 is the nal unit type (see 3.1 for details)

AA indicates whether the nal can be discarded (whether there is any nal reference after it is discarded ):

0 -- indicates that it has no reference function and can be discarded, such as B slice and SEI,

Non-zero -- including 01b, 10B, and 11B -- indicates that the nal cannot be discarded, such as SPS, PPS, I slice, and P slice.

The values of commonly used nal headers are as follows:

0x67: SPS 0x68: PPS 0x65: IDR   0x61: non-IDR Slice0x01: B Slice0x06: SEI0x09: AU Delimiter

Note: The method for determining whether an I frame is used is as follows:

(NALU type & 0001 1111) = 5 that is, NALU type & 31 = 5

Since the nal syntax does not provide length information, the actual transmission and storage system needs to add additional headers to implement the division of each nal unit.

The AVI file and the mpeg ts broadcast stream adopt the syntax format of the byte stream, that is, the synchronization code 0x00000001 is added before the nal unit, then an H is read from the AVI file or the mpeg ts PES package. 264 The following video frames exist:

00 00 00 01 06... 00 00 00 01 67... 00 00 01 68... 00 00 00 00 00 01 65... sei information sps pps idr slice

For MP4 files, there is no synchronization code before the nal unit, but there are several bytes of length code to indicate the length of the nal unit. The number of bytes occupied by this length code is given by the MP4 File Header; in addition, Video Frames read from MP4 do not contain PPs and SPS. The information is located in the MP4 file header and must be obtained by the parser when files are opened. An H.264 frame read from an MP4 file is usually in the following format (assuming the length code is 2 bytes ):

00 19 06 [... 25 bytes...] 24 AA 65 [... 9386 bytes...] sei information IDR slice

4. References

[1] h264 nal Header Parsing http://blog.csdn.net/occupy8/article/details/9042139

[2] h264 (Nal introduction and I frame judgment) http://blog.csdn.net/jefry_xdz/article/details/8461343

H.264 Structure