H.264 RTP payload format

Http://www.cppblog.com/czanyou/archive/2008/11/26/67940.html

H.264 video RTP load format

1. Network abstraction layer unit type (NALU)

The NALU header consists of one byte. Its syntax is as follows:

+ --------------- +
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
+-+
| F | NRI | type |
+ --------------- +

F: 1 bit.
Forbidden_zero_bit. The H.264 specification specifies that this digit must be 0.

NRI: 2 bits.
Nal_ref_idc. 00 ~ 11. It seems to indicate the importance of this NALU. For example, the NALU decoder of 00 can discard it without affecting the playback of the image.

This attribute.

Type: 5 bits.
Nal_unit_type. Type of the NALU unit. The description is as follows:

0 is not defined
1-23 nal unit Single nal unit package.
24 STAP-A single time combo pack
25 STAP-B single time combo pack
26 mtap16 time Combination Package
27 mtap24 time Combination Package
28 unit of FU-A Fragment
29 unit of FU-B Fragment
30-31 Not Defined

2. Packaging Mode

The following is the structure of the RTP Header specified in RFC 3550.

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 4 5 6 7 8 9 0 1
+- +-+
| V = 2 | p | x | CC | M | Pt | sequence number |
+- +-+
| Timestamp |
+- +-+
| Synchronization source (SSRC) identifier |
+ = + + = +
| Contributing source (CSRC) identifiers |
|... |
+- +-+

Payload type (PT): 7 bits
Serial number sequence number (SN): 16 bits
Timestamp: 32 bits

H.264 the payload format defines three basic payload structures. The receiving end may use RTP payload
The first byte to recognize them. This byte is similar to the format of the NALU header, and The Nal unit type field of this header Structure
Which structure is represented,

The structure of this byte is as follows. It can be seen that it has the same structure as the NALU header of H.264.
+ --------------- +
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
+-+
| F | NRI | type |
+ --------------- +
Field type: the type of the nal unit in the RTP payload. The difference between this field and the type field in H.264 is that when type
Is 24 ~ 31 indicates that this is a nal unit in special format, while in H.264, only 1 ~ 23 is a valid value.

24 STAP-A single time combo pack
25 STAP-B single time combo pack
26 mtap16 time Combination Package
27 mtap24 time Combination Package
28 unit of FU-A Fragment
29 unit of FU-B Fragment
30-31 Not Defined

Possible structure types are:

1. Single nal unit Mode
A rtp packet is composed of only one complete NALU. In this case, the RTP nal header type field and the original H.264
The NALU header type fields are the same.

2. Combined Packet Mode
A rtp package may be composed of multiple NAL units. There are four combination methods: STAP-A, STAP-B, mtap16, mtap24.
The type values here are 24, 25, 26, and 27.

3. multipart Packet Mode
It is used to encapsulate a NALU unit into multiple RTP packets. Two types of FU-A and FU-B exist. The type values are 28 and 29, respectively.

2.1 single nal unit Mode

For packages whose NALU length is smaller than MTU size, the single nal unit mode is generally used.
For an original H.264 NALU unit, it is usually composed of three parts: [start code] [NALU header] [NALU payload]. The start code indicates that this is

The start of the NALU unit must be "00 00 00 01" or "00 00 01". The NALU header is only one byte, followed by the content of the NALU unit.
Remove the start code of "00 00 01" or "00 00 01" during packaging, and use the RTP package of other data packets.

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 4 5 6 7 8 9 0 1
+- +-+
| F | NRI | type |
+-+ |
|
| Bytes 2.. n of a single nal unit |
|
| +-+
|:... Optional RTP padding |
+- +-+

If a NALU of H.264 is like this:

[00 00 00 01 67 42 A0 1E 23 56 0e 2f...]

This is a serial parameter set nal unit. [00 00 01] is the start code of four bytes, 67 is the NALU header, and the data starting from 42 is the content of NALU.

The RTP package is encapsulated as follows:

[RTP Header] [67 42 A0 1E 23 56 0e 2f]

That is, you only need to remove the start code of 4 bytes.

2.2 combined Packet Mode

Secondly, when the length of NALU is particularly small, several NALU units can be enclosed in a RTP package.

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 4 5 6 7 8 9 0 1
+- +-+
| RTP Header |
+- +-+
| STAP-A nal HDR | NALU 1 size | NALU 1 HDR |
+- +-+
| NALU 1 Data |
::
++-+ -+
| NALU 2 size | NALU 2 HDR |
+- +-+
| NALU 2 Data |
::
| +-+
|:... Optional RTP padding |
+- +-+

2.3 fragmentation units (FUS ).

When the length of NALU exceeds MTU, The NALU unit must be split into packets, also known as fragmentation units (FUS ).

0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 4 5 6 7 8 9 0 1
+- +-+
| Fu indicator | fu header |
+-+ |
|
| Fu payload |
|
| +-+
|:... Optional RTP padding |
+- +-+

Figure 14. RTP payload format for FU-A

The fu indicator octet has the following format:

+ --------------- +
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
+-+
| F | NRI | type |
+ --------------- +

The fu header has the following format:

+ --------------- +
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
+-+
| S | E | r | type |
+ --------------- +

3. SDP parameters

The following describes how to represent an H.264 stream in SDP:

. "M =" the media name in the row must be "video"
. The encoding name in the "A = rtpmap" line must be "h264 ".
. The clock frequency in the "A = rtpmap" row must be 90000.
. Other parameters are included in the "A = fmtp" line.

For example:

M = video 49170 RTP/AVP 98
A = rtpmap: 98 h264/ 90000
A = fmtp: 98 profile-level-id = 42a01e; sprop-parameter-sets = z0iacpztbymi, amljia =

The following describes some common parameters.

3.1 packetization-mode:
Indicates the supported encapsulation mode.
When the value of packetization-mode is 0 or does not exist, the single NALU unit mode must be used.
When the value of packetization-mode is 1, the non-interleaved packet mode must be used.
When the value of packetization-mode is 2, the interleaved packet mode must be used.
This parameter cannot be set to another value.

3.2 sprop-parameter-sets:
This parameter can be used to transmit the sequence parameter set of H.264 and The Nal unit of the image parameter. The value of this parameter is base64 encoded. Different parameter sets are separated.

3.3 profile-level-ID:
This parameter indicates the profile type and level of the H.264 stream. Three bytes represented by base16 (hexadecimal). The first byte indicates the profile type of H.264.

Three bytes indicate the profile level of H.264:

3.4 max-Mbps:
The value of this parameter is an integer that indicates the maximum macro block processing speed per second.

Posted on
Reading by elders of Diablo (26046)
Comment (17) EDIT favorites
Reference

Feedback

# Re: H.264 RTP payload Format 264 17: 17 Heshui

Hard work.
When combining packets, NALU 1 size is not mentioned before. It belongs to NALU payload. That is to say, the NALU payload of h264 = NALU payload size + NALU payload header + NALU payload data, right. If my understanding is incorrect, please correct me. Thank you. Reply to more comments

# Re: H.264 RTP payload Format 264 dark elders

@ Heshui

This is basically the case.
If a NALU of H.264 is like this:

[00 00 00 01 67 42 A0 1E 23 56 0e 2f...]
[00 00 00 01 67 42 A0 1E 23 56 0e 2f...]

This is a serial parameter set nal unit. [00 00 01] is the start code of four bytes, 67 is the NALU header, and the data starting from 42 is the content of NALU.

The RTP package may be encapsulated as follows:

[RTP Header] [78, STAP-A nal HDR, one byte] [length, two bytes] [67 42 A0 1E 23 56 0e 2f...] [length, two bytes] [67 42 A0 1E 23 56 0e 2f...]

Reply to more comments

# What is the actual example of multipart packets? 2010-01-04 09: 36 Heshui

@ Elders Diablo
Can you give me another actual example of multipart packets? That is to say, if the NALU length exceeds MTU, what is the RTP packet format. Thank you. Reply to more comments

# Re: H.264 RTP payload format [not logged on] 2010-01-06 dark elders

@ Heshui
/** Send the specified NALU unit .*/
Int geplayback: sendnalupacket (byte * slicedata, int slicesize, bool isend,
Bool isvideo, int type, time_t pts, int64 timestamp)
{
// When NALU is smaller than the maximum RTP Package Size
If (slicesize <1350 ){
Return sendpacket (slicedata, slicesize, isend, true, type, PTS, timestamp );
}

// If a NALU is larger than the maximum RTP package size, you need to split it and package it for sending.
Byte buffer [1500];

Byte nalheader = slicedata [0]; // NALU Header
Byte * Data = slicedata + 1;
Int leftover = slicesize-1;
Bool isstart = true;

While (leftover> 0 ){
Int size = min (1350, leftover );
Isend = (size = leftover );

// Construct the Fu Header
Buffer [0] = (nalheader & 0x60) | 28; // Fu indicator
Buffer [1] = (nalheader & 0x1f); // Fu Header
If (isstart ){
Buffer [1] | = 0x80;
}
If (isend ){
Buffer [1] | = 0x40;
}

Memcpy (buffer + 2, data, size );
Sendpacket (buffer, size + 2, isend, true, type, PTS, timestamp );

Leftover-= size;
Data + = size;
Isstart = false;
}

Return slicesize;
}
Reply to more comments

# Re: H.264 RTP payload Format 264 17: 27 Heshui

A few days ago, I accidentally spilled water on the current book. I failed to immediately check the reply from the landlord. I changed the motherboard.
The code is very detailed and the example is clear.
Thank you for your repeated replies. :) For more comments.

# Re: H.264 RTP payload format 2010-01-29 09: 11asd

Hao replies to more comments

# Re: H.264 RTP payload format 2010-02-09 10: 49 Marc

I have found many articles
This article is the clearest
Thank you for replying to more comments

# Re: H.264 RTP payload

Thank you very much. You are studying streaming media. It is better to learn this protocol, as the old ones say. Reply to more comments

# Re: H.264 RTP payload Format 264 14: 33 Winston

In which city are you located, we have a streaming media-related project to seek cooperation from engineers. My mailbox is: message.winston@gmail.com reply more comments

# Re: H.264 RTP payload format 2010-07-14 10: 21sc

Ask a question, H. in 264, what are the differences between the three modes of RTP transmission (single nal unit mode, non-interactive mode, and interactive mode) and the three load structures (single nal unit package, aggregation package, and multipart package, how do I feel the same? Thank you... for replying to more comments

# Re: H.264 RTP payload Format 264 dark elders

@ SC
The table clearly shows their relationships:

Able 3. Summary of allowed nal unit types for each packetization
Mode (Yes = allowed, no = disallowed, Ig = ignore)

Type Packet single nal non-interleaved
Unit mode Mode
-------------------------------------------------------------

0 undefined ig
1-23 nal unit Yes No
STAP-A No Yes No
STAP-B No Yes
26 mtap16 No Yes
27 mtap24 No Yes
28 FU-A no yes
29 FU-B No Yes
30-31 undefined ig
Reply to more comments

# Re: H.264 RTP payload format 2010-07-20 10: 21sc

Thank you very much for your answers and comments.

# Re: H.264 RTP payload format 2010-07-20 11: 13sc

// If a NALU is larger than the maximum RTP package size, you need to split it and package it for sending.
Byte buffer [1500];

Byte nalheader = slicedata [0]; // NALU Header
Byte * Data = slicedata + 1;
Int leftover = slicesize-1;
Bool isstart = true;

While (leftover> 0 ){
Int size = min (1350, leftover );
Isend = (size = leftover );
Ask a question. Int size = min (1350, leftover) in the upstairs program; it should be
Int size = max (1350, leftover); or min. I think it's Max. Thank you.
Reply to more comments

# Re: H.264 RTP payload format [not logged on] 264

@ Elders Diablo
How to Set PTS and timestamp ?? Reply to more comments

# Re: H.264 RTP payload Format 264

It must be Min.
@ SC
Reply to more comments

# Re: H.264 RTP payload format 2013-02-25 17: 34 Hiber

No source code is displayed. Can I share it? Reply to more comments

# Re: H.264 RTP payload Format 264 11: 26kaly_liu

This article !!!!
Thieves are clear ~~~ Reply to more comments