The extraction and function of SPS and PPS in H264

Awesome video conferencing website: http://wmnmtm.blog.163.com/blog/#m =0

++++++++++++++++++++++++++++++++++++++++++++++++++++

http://wmnmtm.blog.163.com/blog/static/38245714201192491746701/

When using RTP to transmit H264, it is necessary to use the SDP protocol description, two of which are: Sequence Parameter sets (SPS) and picture Parameter Set (PPS) need to be used, then where are these two items obtained from? The answer is from the H264 stream. In the H264 stream, both "0x00 0x00 0x01" or "0x00 0x00 0x00 0x01" is the starting code, after the start code is found, the lower 5 bits of the first byte after the start code are used to determine whether it is 7 (SPS) or 8 (PPS) , and data[4] & 0x1f = = 7 | | DATA[4] & 0x1f = = 8. Then base64 encode the obtained nal after the start code is removed, and the resulting information can be used for Sdp.sps and PPS to be separated by commas.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

How to parse the SPS and PPS strings contained in the SDP

Http://www.pernet.tv.sixxs.org/thread-109-1-1.html

The SPS and PPS strings in the SDP contain the information parameters required to initialize the H. A decoder, including the profile,level used for encoding, the width and height of the image, and the Deblock filter.
Since SPS and PPS in the SDP are BASE64 encoded, they are not easy to understand, and an accessory tool software can parse SPS and PPS in the SDP.
The usage is entered on the command line:
Spsparser sps.txt pps.txt output.txt

For example, the content in Sps.txt is:
z0lgfnofgle=
The content in Pps.txt is:
Am4wpia=

The result of the final parsing is:

Start Dumping SPS:
PROFILE_IDC = 66
Constrained_set0_flag = 1
Constrained_set1_flag = 1
Constrained_set2_flag = 1
Constrained_set3_flag = 0
LEVEL_IDC = 20
seq_parameter_set_id = 0
CHROMA_FORMAT_IDC = 1
BIT_DEPTH_LUMA_MINUS8 = 0
BIT_DEPTH_CHROMA_MINUS8 = 0
Seq_scaling_matrix_present_flag = 0
LOG2_MAX_FRAME_NUM_MINUS4 = 0
Pic_order_cnt_type = 2
LOG2_MAX_PIC_ORDER_CNT_LSB_MINUS4 = 0
Delta_pic_order_always_zero_flag = 0
Offset_for_non_ref_pic = 0
Offset_for_top_to_bottom_field = 0
num_ref_frames_in_pic_order_cnt_cycle = 0
Num_ref_frames = 1
Gaps_in_frame_num_value_allowed_flag = 0
PIC_WIDTH_IN_MBS_MINUS1 = 21
Pic_height_in_mbs_minus1 = 17
Frame_mbs_only_flag = 1
Mb_adaptive_frame_field_flag = 0
Direct_8x8_interence_flag = 0
Frame_cropping_flag = 0
Frame_cropping_rect_left_offset = 0
Frame_cropping_rect_right_offset = 0
Frame_cropping_rect_top_offset = 0
Frame_cropping_rect_bottom_offset = 0
Vui_parameters_present_flag = 0

Start Dumping PPS:
pic_parameter_set_id = 0
seq_parameter_set_id = 0
Entropy_coding_mode_flag = 0
Pic_order_present_flag = 0
NUM_SLICE_GROUPS_MINUS1 = 0
Slice_group_map_type = 0
NUM_REF_IDX_L0_ACTIVE_MINUS1 = 0
NUM_REF_IDX_L1_ACTIVE_MINUS1 = 0
Weighted_pref_flag = 0
WEIGHTED_BIPRED_IDC = 0
PIC_INIT_QP_MINUS26 = 0
PIC_INIT_QS_MINUS26 = 0
Chroma_qp_index_offset = 10
Deblocking_filter_control_present_flag = 1
Constrained_intra_pred_flag = 0
Redundant_pic_cnt_present_flag = 0
Transform_8x8_mode_flag = 0
Pic_scaling_matrix_present_flag = 0
Second_chroma_qp_index_offset = 10

/////////////////////////////////////////////////////////////////////////////////////////////////
These two parameters need to be specifically mentioned here
PIC_WIDTH_IN_MBS_MINUS1 = 21
Pic_height_in_mbs_minus1 = 17
Represents the width and height of the image, minus 1 in the macro block (16x16) value, respectively
Therefore, the actual width is (21+1) *16 = 352
Spsparser.rar

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Http://krdai.info.sixxs.org/blog/mp4-sps-pps-data.html

Recently doing research related to H264 Encode/decode, the goal is to be able to extract h264 information from Android Mediarecorder. The question now is how to get the SPS and PPS. Since Mediarecorder was written into the MP4 file, it was necessary to analyze the file format of MP4 and discover that there was no imaginary difficulty. Mainly in the ISO/IEC 14496-15 section. In MP4 's file, find the AvcC string, and then the Avcdecoderconfigurationrecord. The format of the Avcdecoderconfigurationrecord is as follows:

[CPP]View Plaincopy

2. Aligned (8) class Avcdecoderconfigurationrecord {
4. unsigned int (8) configurationversion = 1;
6. unsigned int (8) avcprofileindication;
8. unsigned int (8) profile_compatibility;
10. unsigned int (8) avclevelindication;
14. Bit (6) reserved = ' 111111 ' B;
16. unsigned int (2) Lengthsizeminusone;
20. Bit (3) reserved = ' 111 ' B;
22. unsigned int (5) numofsequenceparametersets;
26. for (i=0; i< numofsequenceparametersets; i++) {
28. unsigned int (sequenceparametersetlength);
30. Bit (8*sequenceparametersetlength) sequenceparametersetnalunit;
32. }
34. unsigned int (8) numofpictureparametersets;
36. For (i=0; i< numofpictureparametersets; i++) {
38. unsigned int (pictureparametersetlength);
40. Bit (8*pictureparametersetlength) pictureparametersetnalunit;
42. }
44. }

You can find SPS and PPS in this way.

+++++++++++++++++++++++++++++++++++++++++++++ VLC did not receive PPS and sps2010-10-08 16:16

Question packetizer_h264 Packetizer warning:waiting for Sps/pps Is that the decoder only encodes SPS, PPS, and I_frame when the code is first executed; H264 Packetizer have set so, which it sends Sps/pps only first keyframe, I ' m trying to the figure what breaks if the is changed So Sps/pps are written in every keyframe. [from | http://trac.videolan.org/vlc/ticket/1384] Workaround: 1. The encoder encodes each keyframe with SPS and PPS, which is said to be the same as the SPS and PPS that are usually programmed by the encoder, so this method consumes resources. 2. When the server receives the client request, it sends the first package plus SPS, PPS. Specific as follows: 1, add a variable isfirstframe in Videoopenfilesource; 2. Initialize Isfirstframe = True at the time of construction; 3, the int videoopenfilesource::readfrombufferchain () is modified as follows: 1IfIsfirstframe = =True2 {3memcpyFTo,H264_header,sizeofH264_header));/* H264_header = PPS +sps*/4offset =sizeofH264_header);5Framesize =Bufferchain_get (Finput.Video_bufs,FTo +Offset);6Offset + =Framesize;7Isfirstframe =False8printf"this is the first fime\n 9 sleep (1); 10} one else { framesize = bufferchain_get (finput. Video_bufs, FTO + offset); offset + = framesize; 15} 1 [http://topic.csdn.net/u/20100801/17/ef35e664-92ff-4144-a35f-3984dcf11da3.html| Reference] ============================ ============================================SDP questions about PPS and SPS: Packetization-mode is primarily a pattern for defining packages, a single Nalu unit mode (0), non-interlaced ( non-interleaved) packet mode (1), Interleaved (interleaved) packet mode (2) sprop-parameter-sets equals the sequence parameter set and the image parameter Nal unit, base64 conversion (= sps+ PPS) Profile-level-id This parameter is used to indicate the profile type and level of the H. This is what this is. Reference to the Dark elder Www.cppblog.com/czanyou/ffmpeg decode about PPS SPS issue: stackoverflow.com/questions/3493742/ problem-to-decode-h264-video-over-rtp-with-ffmpeg-libavcodec/3500432#3500432

How to remove nal (Sps,pps) information from the h.264es file in C language. Like width, height, profile, etc.

Please expert guidance ... http://www.oschina.net/question/225813_35707

Parsing Sps,pps code in the FFmpeg inside, copied out on the line, I have written before ...
FFmpeg's libavcodec/h264_parser.c,
H264_ps.c
Function
Ff_h264_decode_seq_parameter_set
Ff_h264_decode_picture_parameter_set
You can see the code yourself.

H264 parameter Syntax Documentation: SPS, PPS, IDR http://blog.csdn.net/heanyu/article/details/6205390

The first Nalu is the SPS (sequence parameter set sequence Parameter set)
Syntax for 7.3.2.1 sequence parameter set corresponding to H264 standard document parsing

about H264 Packaging by RTP transmission

| Font size subscription

Q: Now my younger brother's first attempt to H264 encoding through RTP mode transmission, the specific experimental environment is as follows:
Environment:
Server-side, H264 frame data (possibly more than 64k), divided into N 1460-byte packets, then add RTP hair to send.
The client, VLC Player, establishes the connection via the RTSP protocol and then receives the data decoding playback.
Results:
VLC can not decode the received data, decoding error, VLC information display can not decode frame data.
I have read the rfc3984 document, on the inside of how to package and use RTP transmission is not very understanding, I hope you can help my brother shrimp, tell the younger brother these and H264 frame how to send, how to sub-package, how to add header information and so on.
(where the way to see Fus seems to be suitable for sub-packet transmission, because the younger brother's data frame may exceed 64k, so forget the prawns can be explained to the younger brother in this case of RTP transmission)

A: I think all the questions have been made very clear in the RFC3984. I do not know what you do not understand, please make specific.

Q: Good, my side is to use VLC and server-side communication, they are using the RTSP protocol to establish the connection at the beginning, the server returned discribers request the SDP and the following description of the same, I use the packetization-mode= 1, that is, Fu-as way to package, because my side of the data frame may exceed 64k data. Can you trouble the owner to see if my side of the SDP is written correctly.
SDP:
v=0
o=-1 1 in IP4 127.0.0.1
S=vstream Live
A=type:broadcast
t=0 0
c=in   IP4 0.0.0.0
M=video 49170 RTP/AVP
a=rtpmap:99 h264/90000
a=fmtp:99 profile-level-id=42a01e; packetization-mode=1; Sprop-parameter-ets=z0iacpztbymi, amljia==
a=control:trackid=0

And there is the way in which the RTP is sent, I have the data in the package as follows:
The frame data comes up as follows: Nalu header +ebsp Data
because the frame data is greater than 1460 bytes, I divide the data into n packets of not more than 1460 bytes, each preceded by a packet of RTP hair.
where the Nalu header of the value I frame is 0x65, the parameter set is 0x67 and 0x68, this value is not a bit wrong, I see RFC3984 above said like I am now a little different, RFC3984 said Fu-as way Package Type value is 28, I do not know whether the decimal , if according to RFC3984 said Nalu head should be how much? Or use the Fu-as way of FU indicator instead of the original Nalu head.
and this fu-as way head seems to have two values, one is FU indicator, the other is the FU header, what should I fill in these two values?

According to what I now fill in, VLC will appear to solve the situation of the code, I hope the owner can help me to answer the detailed point. Thank you, sir.

A: I think RFC3984 said it very clearly.
First you split a Nalu EBSP into multiple packages, such as 3, according to the requirements:

The FU indicator of the first fu-a package should be: F = F;nri in Nalu head = Nalu in Nri;type head = 28. The FU header should be: S = 1;e = 0;r = 0;type = The Type in the Nalu header.

The FU indicator of the second fu-a package should be: F = F;nri in Nalu head = Nalu in Nri;type head = 28. The FU header should be: S = 0;e = 0;r = 0;type = The Type in the Nalu header.

The FU indicator of the third FU-A package should be: F = F;nri in Nalu head = Nalu in Nri;type head = 28. The FU header should be: S = 0;e = 1;r = 0;type = The Type in the Nalu header.

Q: Moderator, I follow your way to divide the package sent, found that VLC will not be able to solve the situation of the frame, but, still do not come out of the image. I think it may be because the method of sending sequence parameter set and image parameter set is not correct, both of them are small in length, as long as a package is available, I will now send them according to Singal Nalu, which is to add a RTP head directly to the NALU package and send it out.
is not that I have a problem with the parameter set, anyway, my side of VLC is not able to solve this parameter set, because the parameter set can not be solved, so the following frame must not be able to solve the image.
Trouble moderator again explain how to set the parameter.

A: I just received the training on streaming media today. Know how to see your SDP.

For your question, do not know whether the SPS, PPS packaging problems. According to RFC3984, and the way you play a single package is also wrong. I hope you can learn the way to make this problem clear, because RFC3984 inside said very clearly, please yourself learn to learn RFC3984 it. Since you are doing this work, you should study RFC3984 carefully.

In addition, the Sprop-parameter-ets=z0iacpztbymi in the SDP is actually the BASE64 transcoding of SPS and PPS, and you do not have to transfer the Sps/pps in the bitstream directly from the SDP.

The Sps&pps is already included in the A2:1.SDP and is completely free of transmission in the bitstream Sps&pps
2. profile-level-id=42a01e, this is the beginning of the SPS several bytes, the remainder in Sprop-parameter-ets=z0iacpztbymi, amljia==, BASE64 encoded, put "Z0iacpztbymi, amljia== "anti-BASE64 conversion back, should happen to be sps&pps content
3. Packaging note, requires that H. T stream is not a byte stream format, that is, no 0x000001 delimited, no insert 0x03, how to generate, check your encoder options.
4. Packetization-mode=1 mode, requires only one NAL unit per RTP, or a FU, non-segmented nal without any modification, directly as the RTP load; nal Note that the NAL header does not transmit, the payload starts after the NAL head, The first two bytes of the FU are generated according to the information of the NAL header (equivalent to the NAL head split into two parts), and the specific generation method moderator has made it very clear.
5. RTP payload type to be consistent with the SDP, otherwise the solution is strange

The extraction and function of SPS and PPS in H264