The meaning and relationship of nal, slice and frame in H.

The meaning and relationship of nal, slice and frame in H.

NAL Nal_unit_type 1 (encoding strip for non-IDR images), 2 (coded stripe data split block a), 3 (coded stripe data split block B), 4 (coded stripe data block C), 5 (encoded stripe of IDR image) types

And
Three coding modes for slice species: I_slice, P_slice, B_slice

There are 3 types of frame: what is the mapping between I-frame, P-frame, and B-frame?

Finally, what frame does the 6 (SEI), 7 (SPS), 8 (PPS) in NAL nal_unit_type belong to?

Sorry, the document looks dizzy dizzy, the problem is more ~ ~ ~
PS: I do not have many points, if the person who can help me, I will give all the points, as if only more than 10 points

1 frame data can be divided into multiple slice.
The data in each slice, in-frame prediction uses only the data of its own slice, and no dependency on other slice data.
The NAL is used to carry the encoded data into large packets. For example, each slice data can be placed in a NAL package.
I-frame is self-coded and does not depend on other frame data.
P frame relies on the I frame data.
b frame relies on I frame, P frame, or other B frame data.

Suggest the landlord to see a bit of video coding book Bar, see the standard or difficult to understand.

Which of the nal nal_unit_type types is I frame, now can only determine nal_unit_type==5 (IDR image encoding stripe) is I frame

SPS, pps, SEI don't count as I frame? And the DPA, DPB, DPC that belong to coded stripe division?

Can you give me a way to extract the I frame and p frame from the video stream?

Thank you for the reply upstairs, I also turned over two or three videos of the books, feeling are a kind of, are rarely said to point. Landlord can recommend one or two good points of video books?

No one answered??

Give you the code directly:)

//////////////////////////////////////////////////////////////////////////
H. NAL type
Enum h264naltype{
h264nt_nal = 0,
H264nt_slice,
H264NT_SLICE_DPA,
H264NT_SLICE_DPB,
H264NT_SLICE_DPC,
H264NT_SLICE_IDR,
H264nt_sei,
H264nt_sps,
H264nt_pps,
};
int H264getnaltype (LPVOID pbsbuf, const LONG Nbslen)
{
if (Nbslen < 5)//incomplete NAL unit
return h264nt_nal;

uint8* PBS = (UINT8 *) pbsbuf;
ULONG NType = pbs[4] & 0x1f; NAL type in fixed position
if (NType <= H264nt_pps)
return nType;

return 0;
}

Where H264nt_slice_idr is a keyframe, H264nt_slice is a P-frame

A frame can be divided into multiple slice to encode, and a slice code is packaged into a nal unit, but the NAL unit can accommodate other data, such as the sequence parameter set SPS, in addition to the slice encoded stream.

1, NAL, slice and frame meaning and mutual relations

NAL refers to the network extraction layer, which contains some network-related information
Slice is the meaning of the film, 264 in the image into a frame (frame) or two fields (field), and the frame can be divided into one or several slices (slilce), the slices are composed of macro blocks (MB). A macro block is a basic unit of encoding processing.

2, NAL Nal_unit_type 1 (non-IDR image encoding strip), 2 (coded stripe data Block a), 3 (coded stripe data partition block B), 4 (coded stripe data block C), 5 (IDR image encoding stripe) type
Three coding modes with slice species: I_slice, P_slice, B_slice
The five types of NAL Nal_unit_type represent what information is then and how it is divided.
I_slice, P_slice, b_slice represent I-type slices, P-type slices, and B-type slices. I_slice is the intra-frame predictive mode encoding, P_slice is one-way predictive encoding or intra-frame mode, and B_slice is bidirectional prediction or intra-frame mode.

3, there are 3 types of frame: I frame, P frame, B frame what is the mapping relationship between?
I-Frame, P-frame, B-frame relationship with I_slice, P_slice, B_slice,slice, and frame differences are already understood in question 1.

4. Finally, what frame does 6 (SEI), 7 (SPS), 8 (PPS) in NAL nal_unit_type belong to?
NAL Nal_unit_type is the concept of a sequence parameter set (SPS), image parameter set (PPS), and enhanced information (SEI) that does not belong to a frame. Indicates that the following data information is a sequence parameter set (SPS), image parameter set (PPS), enhanced information (SEI).

Can you give me a way to extract the I frame and p frame from the video stream?
You can see the header information in the slice.

Look for the NAL start code, and then read the NAL type.

PBS[4] & 0x1f

What is the fifth bit of the 5th byte? What is the preceding 4 bytes (value)?

The 1th byte of the NAL unit is five bits lower?

Then ask a question, how to detect the beginning and end of nal in a video stream?

Reference 5 Floor Hugeice's reply:
Give you the code directly:)

//////////////////////////////////////////////////////////////////////////
H. NAL type
Enum h264naltype{
h264nt_nal = 0,
H264nt_slice,
H264NT_SLICE_DPA,
H264NT_SLICE_DPB,
H264NT_SLICE_DPC,
H264NT_SLICE_IDR,
H264nt_sei,
H264nt_sps,
H264nt_pps,
};
int H264getnaltype (LPVOID pbsbuf, const LONG Nbslen)
{
if (Nbslen < 5)//incomplete NAL unit ...

The video stream is transmitted in the NAL unit ... In a nal unit, however, it is possible to store I-slice (P-slice or B-slice), and colleagues may also store additional information about the image
So is it said I frame, p frame,b frame is to receive the NAL unit in the VCL information extracted first, and then by the content of I, p, B frame classification?

And we can only judge the type of data in the NAL unit by nal Nal_unit_type, ha ~ ~

====================================================================

"How to combine the standard of reading JM code" This Web file, everyone should have it. However, the Web document is the "Paradise" group chat content, so it is not very convenient to see ... I am in the school when the time to see, do some summary, I hope to share with you! The wrong place please correct me!

1. After an SPS, is there a number of PPS?
This is mostly the encoder decision, but the JM code has only one

2. What does C mean in the second column of the standard?
Please see the standard 7.2--classification (marked with C in the table) indicates that the slice data is divided into three types of slice data segmentation. The slice data Class A split contains all of the 2 class syntax elements. The Slice data class B segment contains all of the 3 class syntax elements. The Slice data class C segment contains all of the 4 class syntax elements. The meaning of the value of other class syntax elements is not specified. For some syntax elements, a vertical bar is used to indicate that it contains two types of syntax elements. In this case, the classification values that the syntax element will use will be further determined in the text.

3, a Nalu corresponding to a film?
This is not a very accurate statement, Nalu includes a film, SPS, PPS, Sei and so on

4, Decode_one_frame () including I, P, B

5. Case Nalu_type_slice:
Case NALU_TYPE_IDR:
Case NALU_TYPE_DPA
Case NALU_TYPE_DPB:
Case NALU_TYPE_DPC
Case Nalu_type_sei:
Case Nalu_type_pps
Case Nalu_type_sps
Case Nalu_type_aud:
Case NALU_TYPE_EOSEQ:
Case Nalu_type_eostream:
Case Nalu_type_fill
Question: When to enter which, what is the description of the article or book?
A: Which case to enter is determined by the Nalu_type decoded from Nalu.

6, the decoder in the hidden error is only useful for the packet loss, after the packet loss, the package sequence number is not continuous, the decoder once detected that the packet sequence will be discontinuous place of the Ei_flag 1

7, the byte stream format and the RTP format stream, what are the specific different points? Where are the relevant information?
The byte stream format is primarily used for file storage, so there is only one start prefix Nalu in the format stream, and the RTP format stream is required for network transmission, so there is a lot of auxiliary information in front of Nalu.

8, RTP format is in the word stream before adding Baotou?
No, byte stream = start prefix +nalu, and RTP does not have a start prefix

9, RTP does not start the prefix, why still want to plug 03?
To prevent the pseudo-starting code, RTP can not completely use the starting code, perhaps in order to unify with the byte stream format

10, Nalu is the encapsulation of rbsp. And RTP is the encapsulation of Nalu.

11, why should be divided into ABC film?
--The main purpose of the ABC film is to protect the data with different degrees of importance.

12, Baseline no data segmentation it?
Baseline just how to produce rbsp, how to encapsulate Nalu. Specifically how to transfer, RTP and so on is just one way, file copy is also a way, that general baseline how many reference frames? any one.

13, decoding PROFILE_IDC after the decoder to do what work? For example baseline does not support CABAC then the corresponding bit entropy_coding_mode_flag is not there, if there is, what to do?
Of course, the code is not executed CABAC, if the encoder is encoded baseline, then there is naturally no CABAC in the code-related syntax elements, such as Entropy_coding_mode_flag, decoder decoding SPS, learned that the bitstream is baseline, since However, you will not be able to call the CABAC-related decoding program, there will be no error. PROFILE_IDC for baseline, Active_pps->entropy_coding_mode_flag will not be CABAC, whether the stream is baseline is not determined by how many reference frames

14, JM for CAVLC encoding, for level = 8 is the case of the escape suffix processing, I modified the code to the level = 8 in the case of unsigned number, the result encoded code stream and unmodified exactly the same

Attached: rbsp, Sodb, ebsp the difference and contact!
Sodb: The most primitive encoding data, no additional data
RBSP: Added rbsp_stop_ont_bit on sodb basis (bit value 1) and 0-byte complement alignment
EBSP: Added protection against pseudo-starting code byte (0X03) on rbsp basis

1, 1 frame data can be divided into multiple slice.
2, the data in each slice, in the frame prediction only uses the own slice data, and other slice data does not have the dependency relationship.
3, NAL is used to encode the data to the large package. For example, each slice data can be placed in a NAL package.
4, I frame. It is self-coded and does not depend on other frame. Data.
P frame. Depend on I frame. Data.
B frame. Depend on I frame, P frame. or other B-frame. Data.

A frame can be divided into multiple slice to encode, and a slice code is packaged into a nal unit, but the NAL unit can accommodate other data, such as the sequence parameter set SPS, in addition to the slice encoded stream.

15, NAL, slice and frame meaning and mutual relations

NAL refers to the network extraction layer, which contains some network-related information
Slice is the meaning of the film, 264 in the image into a frame (frame) or two fields (field), and the frame can be divided into one or several slices (slilce), the slices are composed of macro blocks (MB). A macro block is a basic unit of encoding processing.

16, NAL Nal_unit_type in 1 (non-IDR image encoding strip), 2 (coded stripe data Block a), 3 (coded stripe data partition block B), 4 (coded stripe data partition block C), 5 (IDR image encoding stripe) type and slice species of three encoding mode: I The five types of _slice, P_slice, B_slice NAL Nal_unit_type represent what the next data is about and how to block it in detail. I_slice, P_slice, b_slice represent I-type slices, P-type slices, and B-type slices. I_slice is the intra-frame predictive mode encoding, P_slice is one-way predictive encoding or intra-frame mode, and B_slice is bidirectional prediction or intra-frame mode.

17, there are 3 types of frame: I frame, P frame, B frame what is the mapping relationship between?
I-Frame, P-frame, B-frame relationship with I_slice, P_slice, B_slice,slice, and frame differences are already understood in question 1.

18. Finally, what frame does 6 (SEI), 7 (SPS), 8 (PPS) in NAL nal_unit_type belong to?
NAL Nal_unit_type is the concept of a sequence parameter set (SPS), image parameter set (PPS), and enhanced information (SEI) that does not belong to a frame. Indicates that the following data information is a sequence parameter set (SPS), image parameter set (PPS), enhanced information (SEI).

======================================================== first I think to find relevant books to understand the fundamentals, do not rush to see the standard and source code. To know what is the sampling format, what is the I, P, B, what their principle is, understand the CAVLC, CABAC entropy coding implementation process, must be serious. After the main modules have been conquered, you will be able to combine the standard and source code step-by-step look down.

The following is a decoding process as an example of the specific process:

1, Process: Code flow →nalu→rbsp.

If it is a stream of bytes of course is the first to parse the byte stream, this depends on Appendix B, how to find the starting code and remove the pseudo-starting code (why is there a pseudo-starting code?). If it is the RTP format of the code stream, it will be according to RFC3984 to parse (the standard does not specify the RTP format stream parsing process), the word stream after parsing is extracted is nalu, the analysis of Nalu will see the seventh chapter. Black bold characters are the syntax elements that may appear in the stream, and the first task of the decoder is to parse the syntax elements. For the syntax elements in these streams we need to know three questions to parse:

(1), when existed in the stream? So that we can know which syntax element is currently parsed;

(2), what kind of entropy coding method is used? So that we can know how to parse;
(3), what is the meaning? So that we know what to do after parsing it out.

Nalu the first three syntax elements of a byte we call the Nalu header, the remainder (that is, the rest of the syntax table 7.3.1) We call the Nalu body. The analysis of the Nalu body depends on the 7.3.2 section. Because there are many types of nalu, it is necessary to parse the Nalu body for different types of nalu (table 7-1 illustrates the syntax tables for different Nalu). For example, if the current Nalu is an SPS, then of course it is necessary to look at the 7.3.1 subsection; if the current Nalu is DPA, then of course the 7.3.2.9.1 bar will be seen;
2, for the Nalu belongs to the VCL (which Nalu is the VCL Nalu? If you look at the semantics of Nal_unit_type, you should know that, for example, table 7-1, type 5 Nalu, according to table 7-1 we know that the Nalu Body syntax table is 7.3.2.8. And from the 7.3.2.8 we can see that this nalu of the Nalu body is the actual parsing of the chip-level syntax. Syntax Table 7.3.2.8 Show slice-level parsing first to parse Slice_header () (This parenthesized representation is another syntax structure), then how does Slice_header () parse it? Looking down, all the contents of 7.3.3 are included in the first line of Slice_header (), so 7.3.3 is the code flow for the Slice_header () syntax layer;
3, according to the grammar table 7.3.2.8 parsing Slice_header () on the resolution Slice_data (). The following is the most common I-frame (CAVLC entropy Coding, non-MBAFF) parsing process as an example to briefly describe how to continue reading standards. At this time in the code flow in the first slice_data () layer of the syntax element is the syntax table 7.3.4 in the Macroblock_layer (), that is, directly to the macro block layer of the syntax parsing, it is necessary to see the 7.3.5 bar again;
4, based on our understanding of the codec process, we know that decoding is a predictive value and residual error to get the reconstructed image process, then we will be divided into two steps of the following decoding process: First, get the predicted value; second, get the residuals. Based on our understanding of the key technologies of H. S, we know that the intra macro block (reminder: the example we cite is I-frames, so the predicted values of the parsed intra macro blocks) need to be used in predictive mode, so we need to parse the mb_pred (mb_type) syntax layer in the syntax table 7.3.5 , then go to see the 7.3.5.1 section. How do we calculate the predicted value after parsing the macro block or block prediction in the 7.3.5.1 section? Go to the standard 8.3 bar; After we get the predicted values, we continue to parse the syntax elements into the syntax table 7.3.5 to the residual () syntax layer, and this goes to the 7.3.5.3 bar; how do we revert to the actual residuals after parsing the residuals coefficients in the 7.3.5.3 section? To see the standard 8.5 bar;
5, the predicted value and residuals have been, add up is to decode the image. Decoding of the main work to this is also basically completed. Of course, the above process will also use the relevant content of the standard other chapters (for example, the 8.5 section will be used in the 5.7 section of the definition of Inverserasterscan) the overall process is so, the details of everyone to seriously learn

The meaning and relationship of nal, slice and frame in H.