H264 code stream parsing and Nalu

FFmpeg extract H264 Nalu from mp4

http://blog.csdn.net/gavinr/article/details/7183499

639/* bitstream filters */
640 REGISTER_BSF (AAC_ADTSTOASC, AAC_ADTSTOASC);
641 REGISTER_BSF (CHOMP, CHOMP);
642 REGISTER_BSF (Dump_extradata, dump_extradata);
643 REGISTER_BSF (H264_MP4TOANNEXB, H264_MP4TOANNEXB);
644 REGISTER_BSF (HEVC_MP4TOANNEXB, HEVC_MP4TOANNEXB);
645 REGISTER_BSF (Imx_dump_header, Imx_dump_header);
646 REGISTER_BSF (Mjpeg2jpeg, mjpeg2jpeg);
647 REGISTER_BSF (Mjpega_dump_header, Mjpega_dump_header);
648 REGISTER_BSF (mp3_header_decompress, mp3_header_decompress);
649 REGISTER_BSF (Mpeg4_unpack_bframes, mpeg4_unpack_bframes);
650 REGISTER_BSF (Mov2textsub, mov2textsub);
651 REGISTER_BSF (NOISE, NOISE);
652 REGISTER_BSF (Remove_extradata, remove_extradata);
653 REGISTER_BSF (Text2movsub, text2movsub);
annex-b Format
analysis of NAL starting byte of http://blog.sina.com.cn/s/blog_5f435c1301015oxp.html H264 bitstream

This is a H264 stream, 00 00 00 01 This is the corresponding forbidden_zero_bit F (1), followed by the NAL_REF_IDC U (2), this How to solve ah, a little dizzy. Shortly after contact, Nal_unit_type's U (5), the back of the PROFILE_IDC U (8) refers to the few, is how much.

Supplementary questions:

The NAL_REF_IDC u (2) corresponds to 67 to the first two bits 0 1 that is 1,nal_unit_type 64 followed by 5 bits 10 01 1 is combined into a, PROFILE_IDC U (8) is the back 64 is converted to decimal is 100,level_ IDC is the 0D is 13,seq_parameter_set_id UE (v), then point to AC, this is Columbus code, the answer is 0, how this is calculated

2010-9-17 18:01

Satisfactory answer

00 00 00 01 is the ox67 after start code is 0110 0111

Forbidden_zero_bit is a forbidden bit and should be the first digit i.e. F (1) =0,1 for syntax error

NAL_REF_IDC is the reference level, which is represented by other frame reference cases, U (2) = 11 = 3 Most (0 for no reference, see specification)

Nal_unit_type is the type of the frame, for the remaining 5 bits, U (5) = 0 0111 = 7

The current types are:

H264 defined type values for Nal_unit_type

typedef enum {

Nalu_type_slice = 1,

NALU_TYPE_DPA = 2,

NALU_TYPE_DPB = 3,

NALU_TYPE_DPC = 4,

NALU_TYPE_IDR = 5,

Nalu_type_sei = 6,

Nalu_type_sps = 7,

Nalu_type_pps = 8,

Nalu_type_aud = 9,

Nalu_type_eoseq = 10,

Nalu_type_eostream = 11,

Nalu_type_fill = 12,

#if (mvc_extension_enable)

Nalu_type_prefix = 14,

Nalu_type_sub_sps = 15,

Nalu_type_slc_ext = 20,

NALU_TYPE_VDRD =//View and Dependency representation Delimiter NAL Unit

#endif

} Nalutype;

It can be seen that Nalu_type_sps is sequence parameter sets

The U (8) of the PROFILE_IDC is the 64 converted to a decimal, then 100,

Baseline

Main

Extended

(FRExt)

(FRExt)

122 High 4:2:2 (FRExt)

144 High 4:4:4 (FRExt)

100 is high (FRExt)

"LEVEL_IDC is 0D is 13,seq_parameter_set_id UE (v), then refers to AC, this is Columbus code, the answer is 0, how this is calculated." “

I don't know much about it. Help each other.

Agree

8

| Comments (1)

Ask Ta for help

Respondents: Sindana level Four

Areas of expertise: not yet customized

Activities to attend: Events not currently in attendance

The questioner evaluates the answer:

Thank you very much, very detailed. The above content is from link http://zhidao.baidu.com/question/184301276.html

Recently learning the H264 video stream in. flv file format.
When you receive each NAL data (buffer pointer) of the H264 stream, the following code is understood:
if ((* (Buffer) = = 0) && (* (buffer+1) = = 0) && (* (buffer+2) = = 0) && (* (buffer+3) = = 1))//nal head 0x00 00 00 01 Start Code
{
if (* (buffer+4) = = Sps_frame)
{//ox67 is 0110 0111 (Nal_unit_type is low 5 bits, u (5) = 0 0111 = 7)
Frame_type = Sps_frame;
}
else if (* (buffer+4) = = Pps_frame)
{//ox68 is 0110 (Nal_unit_type is low 5 bits, u (5) = 0 1000 = 8)
Frame_type = Pps_frame;
}
else if (* (buffer+4) = = I_frame)
{//ox65 is 0110 0101 (Nal_unit_type is low 5 bits, u (5) = 0 0101 = 5)
Frame_type = I_frame;
}
Else
{//0x41 is 0100 00001 (NAL_REF_IDC is the reference level, which is referred to by other frames, u (2) = ten = 2; Nal_unit_type is low 5 bits, u (5) = 0 0001 = 1)
Frame_type = P_frame;
}
if ((* (buffer+5) & 0x80) = = 0x80)
{
Start_frame = 1;
}
}

h264 images, frames, slices, nalu

Http://blog.csdn.net/zqnihao917/article/details/7760170 is a concept of innovation, it breaks the routine, completely without the concept of I-frame, p-frame, B-frame, nor the concept of IDR frames. Some of the concepts appearing in H. s are from large to small: sequences, images, slices, slices, NALU, macro blocks, yahong blocks, blocks, and pixels. Here are a few points worth explaining:
(1), the image is a set concept in the H. S protocol, the top field, the bottom field and the frame can all be called images (the concept of this image is a set concept). So we can know that for the H. C protocol, the names that we are familiar with, such as I-frames, P-frames, B-frames, and so on, are actually the concept of the image that we have materialized and minutely. The "frame" we mentioned in H. S is usually the image of the non-divided field;
(2), if do not adopt FMO (Flexible macro block sort) mechanism, then an image has only one slice group;
(3), if you do not use more than one piece, then a slice group has only one slice;
(4), if not using the DP (data segmentation) mechanism, then a piece is a Nalu, a nalu is a piece. Otherwise, a piece consists of three nalu (that is, the standard "table 7-1" Nal_unit_type value 2, 3, 4 of the three Nalu belong to a slice); 2 coded stripe data split block a Slice_data_partition_a_layer_rbs P ()

3 coded stripe data split block bslice_data_partition_b_layer_rbsp ()

4 coded stripe data partition block cslice_data_partition_c_layer_rbsp () (5), the size relationship of the slices and Nalu described above is not an abstract conceptual dependency. From the concept of subordination, Nalu is actually a set of pieces of concept, for example: the standard "Table 7-1" Nal_unit_type value of 5 nalu including I or si pieces.

An image is divided according to the type of film it consists of, and can be classified into 8 types in the standard "table 7-5". One of the most common examples of our usual applications is these types of exceptions. For example: our usual so-called "I-Frame" and "IDR-frame", in fact, is a primary_pic_type value of 0 of the image, our usual so-called "P-frame" is actually a primary_pic_type value of 1 of the image of the special case, our usual so-called "B-Frame" is actually primary_ Pic_type A special case of an image with a value of 2.

An image can be divided into two types according to concept: IDR image and non-IDR image. Whether an image is an IDR image is determined by the Nalu that makes up the image, and if the Nalu that makes up the image is a nalu with a nal_unit_type value of 5 in the standard table 7-1, the image is an IDR image, otherwise a non-IDR image. Here are a few things to note:
(1), Nal_unit_type value of 5 Nalu will only appear in the IDR image, and all the Nalu in the IDR image are Nalu with a nal_unit_type value of 5;
(2), we are wrong to distinguish whether the image is an IDR image by the type of slice that makes up an image. For example, all slices in an image are an I-slice and do not mean that the image is an IDR image. Because I can also be from a nal_unit_type value of 1 Nalu is also non-IDR image may contain all I tablets. But the most common form that we usually get is that all the slices are the image of I and the IDR image. In fact, this time the concept of IDR image also by our specific and small. However, IDR images must all contain I or SI slices, but only the type of Nalu can be used to determine if the IDR image

An image consists of a 1~n, and each piece is made up of one or more slices consisting of one nalu or three nalu (if data is divided). In the process of image decoding, decoding is always done according to slices, then the decoding macro block is reconstructed into an image according to the slice group.   In this sense, the film is actually the largest decoding unit. What kind of macro blocks does a piece contain?   The standard "Table 7-10" is the best description. What are the types of an I-macro block?   The standard "Table 7-11" is the best description. What are the types of a P-macro block?   The standard "Table 7-13" is the best description. What are the types of a B-macro block?   The standard "Table 7-14" is the best description. What types of sub-macro blocks of a P-macro block are divided into?   The standard "Table 7-17" is the best description. What types of sub-macro blocks of a B macro block are divided into?

The standard "Table 7-18" is the best description.


How to differentiate the I frame and the pframe of the video stream of H. Collection

How to distinguish between the I frame and P frame of the H. Video stream.


I am a novice, I saw the last few days of the specification documents and other materials to find answers,
There are several concepts of the relationship has not been able to understand clearly, look for the talent pointing twos:
The meaning of NAL, slice and frame and their relationship with each other

Nalnal_unit_type 1 (encoding strip for non-IDR images), 2 (coded stripe data split block a), 3 (coded stripe data split block B), 4 (coded stripe data block C), 5 (encoded stripe of IDR image) type

And
Three coding modes for slice species: I_slice, P_slice, B_slice

There are 3 types of frame: what is the mapping between I-frame, P-frame, and B-frame?


Finally, what is the frame of 6 (SEI), 7 (SPS), 8 (PPS) in NAL Nal_unit_type?

---------------------------------------------------------------------------
1 frame data can be divided into multiple slice.
The data in each slice, in-frame prediction uses only the data of its own slice, and no dependency on other slice data.
The NAL is used to carry the encoded data into large packets. For example, each slice data can be placed in a NAL package.
I-frame is self-coded and does not depend on other frame data.
P frame relies on the I frame data.
b frame relies on I frame, P frame, or other B frame data.

----------------------------------------------------------------------------
Which types of nal Nal_unit_type are iframe, now only OK nal_unit_type==5 (IDR image encoding stripe) is I frame

SPS, pps, SEI don't count as I frame. There are DPA, DPB, DPC which belong to coded stripe division.

Can you give me a way to extract the I frame and p frame from the video stream?


-----------------------------------------------------------------------------------------------

A frame can be divided into multiple slice to encode, and a slice code is packaged into a nal unit, but the NAL unit can accommodate other data, such as the sequence parameter set SPS, in addition to the slice encoded stream.

------------------------------------------------------------------------------------------------

H. NAL type
Enum H264naltype
{
h264nt_nal = 0,
H264nt_slice,//p Frame
H264NT_SLICE_DPA,
H264NT_SLICE_DPB,
H264NT_SLICE_DPC,
H264NT_SLICE_IDR,//I-Frame
H264nt_sei,
H264nt_sps,
H264nt_pps,
};

0x00 0x00 0x00 0x01 0x65 (0x45) The first four bytes are the frame header, 0x65 is the keyframe

0x00 0x00 0x01 0x65 (0x45) is also a keyframe

H264getnaltype (unsigned char * pbsbuf, const int Nbslen)
{
if (Nbslen < 5)//incomplete NAL unit
return h264nt_nal;


unsigned char * PBS = (unsigned char *) pbsbuf;

int nType = pbs[4] & 0x1F; NAL type in fixed position
if (NType <= H264nt_pps)
Return ntype;//NType to 5 for keyframes


return 0;
}

------------------------------------------------------------------------------------------------

Where H264nt_slice_idr is a keyframe, H264nt_slice is a P-frame

------------------------------------------------------------------------------------------------

A frame can be divided into multiple slice to encode, and a slice code is packaged into a nal unit, but the NAL unit can accommodate other data, such as the sequence parameter set SPS, in addition to the slice encoded stream.

------------------------------------------------------------------------------------------------

1, NAL, slice and frame meaning and mutual relations

NAL refers to the network extraction layer, which contains some network-related information
Slice is the meaning of the film, 264 in the image into a frame (frame) or two fields (field), and the frame can be divided into one or several slices (slilce), the slices are composed of macro blocks (MB). A macro block is a basic unit of encoding processing.

2, Nalnal_unit_type in 1 (non-IDR image coding strip), 2 (coded stripe data Block a), 3 (coded stripe data partition block B), 4 (coded stripe data block C), 5 (IDR image encoding strip) type
Three coding modes with slice species: I_slice, P_slice, B_slice
The five types of NAL Nal_unit_type represent what information is then and how it is divided.
I_slice, P_slice, b_slice represent I-type slices, P-type slices, and B-type slices. I_slice is the intra-frame predictive mode encoding, P_slice is one-way predictive encoding or intra-frame mode, and B_slice is bidirectional prediction or intra-frame mode.

3, there are 3 types of frame: I frame, P frame, bframe What is the mapping relationship between.
I-Frame, P-frame, B-frame relationship with I_slice, P_slice, B_slice,slice, and frame differences are already understood in question 1.

4. Finally, what frame does 6 (SEI), 7 (SPS), 8 (PPS) belong to in Nalnal_unit_type?
NAL Nal_unit_type is the concept of a sequence parameter set (SPS), image parameter set (PPS), and enhanced information (SEI) that does not belong to a frame. Indicates that the following data information is a sequence parameter set (SPS), image parameter set (PPS), enhanced information (SEI).

In the decoding process, the parameter set is first filtered, and the parameter set is a new concept of H. s standard, and it is a method to improve the error recovery ability by improving the video bitstream structure. It is well known that the loss of some key bits of information (such as the header information of sequences and images) can cause serious negative effects of decoding, while the key information is separated out by H-series, and the parameter set is designed to ensure proper transmission in error-prone environments. The design of the code stream structure undoubtedly enhances the error recovery ability of the code stream transmission.

The parameter set of H. Sequence is divided into sequence parameter set (parameter set) and image parameter set (Pictureparameterset). The sequence parameter set includes all the information of an image sequence, that is, all the image information between the two IDR images. The image parameter set includes all the relevant information of all the shards of an image, including the image type, serial number, etc., and the loss of some serial number can be used to verify the loss of the packet when decoding. Multiple sets of sequence and image parameters are stored in the decoder, and the encoder selects the appropriate set of parameters based on where the head of each encoded shard is stored, and the image parameter set itself includes reference information for the sequence parameter set used.

The method of parameter set implementation is also diversified: (1) through out-of-band transmission, this method requires the parameter set through a reliable protocol, before the first chip encoding arrives to the decoder, (2) through in-band transmission, which needs to provide a higher level of protection for the parameter set, such as sending a replication packet to ensure that at least one destination ; (3) A hardware processing parameter set is used in the encoder and decoder.

The sequence parameter set and the set of image parameters are transmitted before decoding and are activated during decoding. Once activated, the previous sequence parameter