Multimedia Development --- h264 images, frames, slices, NALU

Images, frames, slices, and NALU are learning H. 264 of people are often confused about some concepts. I will elaborate on my understanding here. You are welcome to discuss them: H. 264 is a conceptual innovation, which breaks the regular, There is no concept of I frame, P frame, B frame, or IDR frame.. Some concepts in H.264 are sorted in ascending order: Sequence, image, Film Group, film, NALU, Macro Block, sub-macro block, block, pixel. Here are some notes:
(1) In H.264 Protocol Image is a Set Concept, Top Fields, bottom fields, and frames can all be called images.(Image concepts in this article are both set concepts ). Therefore, we can understand the terms that we are familiar with for H.264 protocols, such: I frame, P frame, B frame, etc. In fact, we have made the concept of image more specific and smaller.. We The frame mentioned in H.264 usually refers to an image with no field division;
(2) If Do not use FMO (flexible macro block sorting)Mechanism, then An image has only one image group.;
(3) If If multiple slices are not used, only one slice is in a group.;
(4) If Not usedDP ( Data Segmentation) Mechanism, then A piece is a nalu, A nalu is also a piece. Otherwise, A piece consists of three nalu(That is, the nal_unit_type value in the standard "Table 7-1" is 2, 3, and 4 ThreeNALU is a piece ); 2 encoding strip data split Block A slice_data_partition_a_layer_rbsp ()

3 encoding strip data split block bslice_data_partition_ B _layer_rbsp ()

4. Encoding band data split block cslice_data_partition_c_layer_rbsp ()(5) The relationship between slices and NALU is not an abstract concept affiliation.. Concept SubordinationLet's see, NaluActually It is also a collection concept of slices.For example, the NALU with the nal_unit_type value 5 in the standard "Table 7-1" includes I or Si.

An image is divided according to the image type, Can be divided into standard 8 types in Table 7-5. The most common Actually these types. For example: The so-called "I frame" and "IDR frame ",Actually Image with primary_pic_type 0What we call P frame "is actually a special case of an image with a primary_pic_type of 1., The so-called "B frame" is actually a special case of an image with a primary_pic_type of 2.

An image can be divided into two types by Concept: IDR images and non-IDR Images. Whether an image is IDR ImageIs composed of Determined by the NALU of the image, If the group The NALU of the image is the NALU with the nal_unit_type value 5 in the standard "Table 7-1 "., Then IDR ImageOtherwise Non-IDR Image. It is worth noting that: (1) The NALU with the nal_unit_type value of 5 only appears in the IDR image, and all NALU in the IDR image is the NALU with the nal_unit_type value of 5;
(2) We make up Image typesTo distinguish this Whether the image is an IDR image is incorrect. For example: All images in an image are I-slices, which does not mean that this image is an IDR image.. Because I-slices can also be included in all I-slices from NALU that belongs to nal_unit_type with a value of 1, that is, non-IDR images. However, the most common form is that all images are I images and IDR images. In fact, this time IDR ImageThe concept is also embodied and refined. However, IDR images must all contain I or Si images, Only the NALU type can be used to determine whether the image is an IDR image.

An image consists 1 ~ N groupsAnd Each groupBy one or Several slicesA piece consists of one NALU or three NALU (if there is data segmentation. Image decoding is always performed based on photos.And then combine the decoded macro block into the image by the photo group. In this sense, The chip is actually the largest decoding Unit. What types of macro blocks does a piece contain? The standard table 7-10 provides the best description.  What are the types of an I Macro Block? The standard table 7-11 provides the best description.  What are the types of P macro blocks? The standard table 7-13 provides the best description.  What are the types of a B Macro Block? The standard table 7-14 provides the best description.  What are the types of sub-macro blocks of a p Macro Block? The standard table 7-17 provides the best description.  What are the types of sub-macro blocks of a B Macro Block? The standard table 7-18 provides the best description.



How does one differentiate I frame and pframe of H.264 video streams? Favorites

How does one differentiate I frame and P frame of H.264 video streams?

 
I am a newbie. When I read the H.264 standard documentation and other materials to find the answer a few days ago,
The relationship between several concepts has not yet been clearly understood. I hope the experts can give some advice:
NAL, slice and frame meanings and relationships

1 (encoding band of non-IDR image), 2 (encoding Band Data Segment A), and 3 (encoding band data segment B) in nalnalnal_unit_type), 4 (encoding Band Data Segment c), 5 (IDR image encoding band) Types

And
Three encoding modes of Slice: I _slice, p_slice, and B _slice

There are three types of frames: I frame, P frame, and B frame. Is there any ing relationship between them?


Finally, what frames are 6 (SEI), 7 (SPS), and 8 (PPS) in nal nal_unit_type?

---------------------------------------------------------------------------
1 frame data can be divided into multiple slice.
Data in each slice only uses the data of its own slice for intra-frame prediction, and has no dependency with other slice data.
NAL is used to pack the encoded data. For example, each slice data can be stored in the nal package.
I frame is independently encoded and independent from other frame data.
P frame depends on I frame data.
B frame depends on I frame, P frame, or other B frame data.

----------------------------------------------------------------------------
Which of the following types of NAL nal_unit_type are IFRAME? Now you can only determine that nal_unit_type = 5 (IDR image encoding band) is I frame

Is SPS, PPS, and SEI an I frame? What about the DPA, DPB, and DPC that are separated by encoding bands?

Can I extract I frame and P frame from a video stream?


Bytes -----------------------------------------------------------------------------------------------

A frame can be divided into multiple slice codes, while a slice code is packaged into a nal unit. However, in addition to the slice code stream, the nal Unit, it can also accommodate other data, such as the sequence parameter set SPS

Bytes ------------------------------------------------------------------------------------------------

// H.264 nal type
Enum hsf-naltype
{
Hsf-nt_nal = 0,
Hsf-nt_slice, // P Frame
Hsf-nt_slice_dpa,
Hsf-nt_slice_dpb,
Hsf-nt_slice_dpc,
Hsf-nt_slice_idr, // I frame
Hsf-nt_sei,
Hsf-nt_sps,
Hsf-nt_pps,
};

// 0x00 0x00 0x00 0x01 0x65 (0x45) the first four bytes are the frame header, and 0x65 is the key frame.

// 0x00 0x00 0x01 0x65 (0x45) is also a key frame

Hsf-getnaltype (unsigned char * pbsbuf, const int nbslen)
{
If (nbslen <5) // incomplete nal Unit
Return hsf-nt_nal;


Unsigned char * PBS = (unsigned char *) pbsbuf;

Int ntype = PBS [4] & 0x1f; // The Nal type is in a fixed position.
If (ntype <= hsf-nt_pps)
Return ntype; // If ntype is 5, it indicates the key frame.


Return 0;
}

Bytes ------------------------------------------------------------------------------------------------

Hsf-nt_slice_idr is the key frame and hsf-nt_slice is the P frame.

Bytes ------------------------------------------------------------------------------------------------

A frame can be divided into multiple slice codes, while a slice code is packaged into a nal unit. However, in addition to the slice code stream, the nal Unit, it can also accommodate other data, such as the sequence parameter set SPs.

Bytes ------------------------------------------------------------------------------------------------

1. NAL, slice and frame meanings and Their Relationships

NAL refers to the network extraction layer, which contains some network-related information.
Slice means slice. In 264, the image is divided into one frame or two fields, and the frame can be divided into one or several slilce ); A piece consists of a macro block (MB. Macro Block is the basic unit for encoding.

2. 1 (non-IDR image encoding band), 2 (encoding Band Data Segment A), and 3 (encoding band data segment B) in nalnalnal_unit_type), 4 (encoding Band Data Segment c), 5 (IDR image encoding band) Types
Three encoding modes: I _slice, p_slice, and B _slice
The five types in nal nal_unit_type indicate what information is next and how to block specific data.
I _slice, p_slice, and B _slice indicate I-type slice, p-type slice, and B-type slice. here, I _slice is the intra-frame prediction mode encoding; p_slice is the unidirectional prediction encoding or intra-frame mode; B _slice is the bidirectional prediction or intra-frame mode.

3. Are there three types of frames: I frame, P frame, and bframe?
The differences between I frame, P frame, and B frame are as follows: I _slice, p_slice, B _slice, slice, and frame, which are already known in question 1.

4. At last, what frames are 6 (SEI), 7 (SPS), and 8 (PPS) in nalnal_unit_type?
NAL nal_unit_type is a sequence parameter set (SPS), image parameter set (PPS), and enhancement information (SEI. It indicates that the subsequent data information includes the sequence parameter set (SPS), image parameter set (PPS), and enhancement information (SEI ).

In H.264 decoding, the parameter set is obtained by filtering the bitstream. It is a new concept of H.264 standard and a method to improve the error recovery capability by improving the video bitstream structure. As we all know, the loss of some key information bits (such as the sequence and image header information) will cause serious negative effects of decoding, while H. 264 separate the key information and ensure correct transmission in an error-prone environment based on the design of the parameter set. The design of this code stream structure undoubtedly enhances the error recovery capability of the code stream transmission.

The H.264 parameter set is further divided into sequence parameter set and pictureparameterset ). The sequence parameter set includes all the information of an image sequence, that is, all the image information between two IDR images. The image parameter set includes all the relevant information of all parts of an image, including the image type and serial number. The loss of some serial numbers during decoding can be used to check whether the information package is lost or not. Multiple different sequences and image parameter sets are stored in the decoder. The encoder selects an appropriate parameter set based on the storage location of the header of each encoding shard, the image parameter set itself also contains reference information about the sequence parameter set used.

The specific implementation methods of the parameter set are also diverse: (1) through out-of-band transmission, This method requires the parameter set to be transmitted to the decoder before the first encoding arrives through a reliable protocol; (2) Pass in-band transmission, which requires more advanced protection for the parameter set. For example, send a copy package to ensure at least one target is reached. (3) hardware processing parameter sets are used in encoder and decoder.

The sequence parameter set and image parameter set must be transmitted before decoding and activated during decoding. Once activated, the previous sequence parameter set or image parameter set becomes invalid. The image parameter set is activated by using its slice data or using its a-separated NALU. The sequence parameter set is activated by the image parameter set that uses it or the seinalu that includes the message in the buffer period. The sequence parameter set of the same IDR image has the same seq_parameter_set_id until the last accessunit or sei NALU of the image contains the buffer period message. In this case, the sequence parameter set of the next image needs to appear. Seinalu activates the sequence parameter set of the next image. If the sequence parameter set and image parameter set are sent through other transmission pipelines, ensure the above Transmission sequence.

Bytes ------------------------------------------------------------------------------------------------

H.264 video streams are transmitted in NAL units... However, in a nal unit, I-slice (p-slice or B-slice) may be stored, and colleagues may also store other information about the image.
Does it mean that I frame, P frame, and B frame extract the VCL information in the received nal unit first, and then classify I, P, and bframe according to the content?

However, we can only identify the data type in the nal unit through nal nal_unit_type ~~~

Sorry, I haven't fully understood it yet ~~


The Nal unit first has an H.264 nal type, which can be used to determine the information. If yes
Listen, hsf-nt_slice_dpb, hsf-nt_slice_dpc, hsf-nt_slice_idr video data, which also contains the slicehead header information. Based on this header information, you can determine whether it is an I-slice (p-slice or B ), the MB head information is available for each macro block. The block mode can be determined based on the macro block header information.

H264 organizes information in layers. I don't know if you understand it.

Http://blog.csdn.net/zqnihao917/article/details/7760170

Multimedia Development --- h264 images, frames, slices, NALU