Analysis of h264 code stream Packaging

 

Analysis of h264 code stream Packaging

Sodb data Bit String --> the original encoding data

Rbsp original byte sequence load --> after sodb, add the ending bit (rbsp trailing bits is a bit "1") to several BITs "0" to facilitate byte alignment.

Ebsp extended byte sequence load --> added the imitation validation byte (0x03) based on rbsp. The reason is: When NALU is added to limit B, you need to add the start code startcodeprefix before each set of NALU. If the slice corresponding to the NALU is the start of a frame, it is represented by four bytes, ox00000001; otherwise, it is represented by three bytes.
Ox000001. in order to make the NALU subject excluded from the conflict with the start code, the 0x03 of a byte is inserted every time two bytes are consecutively 0 during encoding. Remove 0x03 from decoding. Also known as shell removal.

H264 functions are divided into two layers: Video Encoding layer (VCL) and network extraction layer (NAL)

VCL data is the video data sequence after compression and encoding. VCL data can be transmitted or stored only after it is encapsulated in the nal unit. The Nal unit format is as follows:

NAL Header

Ebsp

NAL Header

Ebsp

NAL Header

Ebsp

NAL Unit
Each nal unit is a variable-length byte string of a certain syntax element, including a byte header information (used to represent the data type), and load data of several integer bytes. A nal unit can carry an encoding piece, A/B/C data segment, or a sequence or image parameter set.

The Nal unit is transmitted in sequence according to the RTP serial number. Among them, T is the load data type, accounting for 5 bits; r is the importance indicator bit, accounting for 2 bits; the last F is the prohibited bit, accounting for 1 bit. The details are as follows:

(1) NALU type bit

It can represent 32 different types of features of nalu, type 1 ~ 12 is defined by H.264. The type is 24 ~ 31 is used outside H.264. the RTP load specification uses some of these values to define package aggregation and splitting. Other values are reserved for H.264.

(2) Importance indicator

It is used to mark the importance of a nal unit during refactoring. The larger the value, the more important it is. If the value is 0, this nal unit is not used for prediction. Therefore, it can be discarded by the decoder without error diffusion. If the value is greater than 0, this nal unit is used for non-drift reconstruction, and the value is higher, the loss of this nal unit has a greater impact.

(3) Bit prohibited

The default value in encoding is 0. When the Network identifies a bit error in this unit, it can be set to 1 so that the receiver can discard this unit, it is mainly used to adapt to different types of network environments (such as wired and wireless environments ). For example, for a gateway from wireless to wired, one side is a wireless non-IP environment, and the other side is a wired network without bits errors. When a false nal unit arrives at the wireless side, the verification and detection fails. The Gateway can remove the nal unit from the nal stream or forward the known nal unit to the receiving end. In this case, the smart decoder will try to refactor The Nal unit (it is known that it may contain bit errors ). Instead of intelligent decoders, this nal unit will be simply discarded. The Nal unit structure specifies
The common format of the group or the transmission subsystem used for stream. In H.320 and MPEG-2 systems, the stream of the nal unit should be within the nal unit boundary, with a 3-byte starting prefix before each nal unit. In a packet transmission system, The Nal unit is determined by the system's transmission procedure. Therefore, the above starting prefix is not required. A set of NAL units is called an access unit, and the time information (SEI) is added after the demarcation to form a basic encoded image. The basic encoded image (PCP) is composed of a group of encoded NAL units, followed by a redundant encoded image (RCP), which is a redundant representation of the same video image of PCP, it is used to recover information when the PCP program is lost in decoding. If the encoded video image is the last image of the encoded video sequence, the end of the sequence nal unit should appear, indicating that the sequence ends. One image sequence
The column has only one sequence parameter group and is decoded independently. If the encoded image is the last image of the entire nal unit stream, the end of the stream should appear.

H.264 adopts the strict access unit mentioned above, which not only makes H.264 adaptive to multiple networks, but also further improves its anti-error capability. When the serial number is set, you can find out which VCL unit is lost. The redundant encoded image can still get a rough image even if the basic encoded image is lost.