Analysis of h264 RTP payload using instances

There are three different basic loads (single Nal, non-interleaved, interleaved) in RTP of h264)

The application can use the first byte for recognition.

 

The properties of this session are also described in SDP.

SDP parameters

The following describes how to represent an H.264 stream in SDP:

. "M =" the media name in the row must be "video"

. The encoding name in the "A = rtpmap" line must be "h264 ".

. The clock frequency in the "A = rtpmap" row must be 90000.

. Other parameters are included in the "A = fmtp" line.

For example:
M = video 49170 RTP/AVP 98

A = rtpmap: 98 h264/ 90000

A = fmtp: 98 profile-level-id = 42a01e; packetization-mode = 1; sprop-parameter-sets = z0iacpztbymi, amljia =

The following describes some common parameters.
3.1 packetization-mode:

Indicates the supported encapsulation mode.

When the value of packetization-mode is 0 or does not exist, the single NALU unit mode must be used.

When the value of packetization-mode is 1, the non-interleaved packet mode must be used.

When the value of packetization-mode is 2, the interleaved packet mode must be used.

 

 

Summary of the nal unit types allowed in each packaging method (Yes = allowed, no = not allowed, Ig = ignored)

Type Packet single nal non-interleaved

Unit mode Mode

-------------------------------------------------------------

0 undefined ig

1-23 nal unit Yes No

STAP-A No Yes No

STAP-B No Yes

26 mtap16 No Yes

27 mtap24 No Yes

28 FU-A no yes

29 FU-B No Yes

30-31 undefined ig

This parameter cannot be set to another value.

3.2 sprop-parameter-sets:

SPS, pps

This parameter can be used to transmit the sequence parameter set of H.264 and The Nal unit of the image parameter. The value of this parameter is base64 encoded. Different parameter sets are separated.

3.3 profile-level-ID:

This parameter is used to indicate H. 264 the profile type and level of the stream. three bytes represented by base16 (hexadecimal. the first byte represents H. the profile type of 264. The third byte represents H. profile level of 264:

3.4 max-Mbps:

The value of this parameter is an integer that indicates the maximum macro block processing speed per second.

 

The first byte of RTP payload is similar to that of 264 NALU.

 

+ --------------- +

| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |

+-+

| F | NRI | type |

+ --------------- +

 

F: 1 bit.

Forbidden_zero_bit. The H.264 specification specifies that this digit must be 0.

NRI: 2 bits.

Nal_ref_idc. 00 ~ 11. It seems to indicate the importance of this NALU. For example, the NALU decoder of 00 can discard it without affecting the playback of the image. However, this attribute is generally not very concerned.

Type: 5 bits.

Nal_unit_type. Type of the NALU unit. The description is as follows:

0 is not defined

1-23 nal unit Single nal unit package.

24 STAP-A single time combo pack

24 STAP-B single time combo pack

26 mtap16 time Combination Package

27 mtap24 time Combination Package

28 unit of FU-A Fragment

29 unit of FU-B Fragment

30-31 Not Defined

Example:

0x5c = 01011100 (F: 0 NRI: 10 type: 28) FU-A

0x41 = 01000001 (F: 0 NRI: 10 type: 01) single nal

0*68 = 01000100 (F: 0 NRI: 10 type: 08) single nal

 

Single nal unit Mode

: Type [1-23]

Packetization-mode = 0

For packages whose NALU length is smaller than MTU size, the single nal unit mode is generally used.

For an original H.264 NALU unit, the [start code] [NALU header] [NALU payload]

The start code indicates the start of a NALU unit, which must be "00 00 00 01" or "00 00 01". The NALU header is only one byte, the subsequent content is the content of the NALU unit.

Remove the start code of "00 00 01" or "00 00 01" during packaging, and use the RTP package of other data packets.

 

 

 

 

Non-interleaved mode: Type [1-23,24, 28]

Packetization-mode = 1

For details about type = [1-23], see packetization-mode = 0.

Type = 28 FU-A

+ --------------- +

| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |

+-+

| F | NRI | type: 28 | S | E | r | type |

+ --------------- +

 

S: Start flag

E: End mark (same as mark)

R: The value must be 0.

 

Type: NALU type of h264

 

Example:

 

0x7c85 = 01111100 10000101 (start package)

0x7c05 = 01111100 00000101 (medium density)

0x7c45 = 01111100 01000101 (end package)

STAP-A type = 23

0 1 2 3

| 0 1 2 3 4 5 6 7 | 8 9 0 1 2 3 4 | 5 6 7 8 9 0 1 2 3 | 4 5 6 7 8 9 0 1

+- +-+

| RTP Header |

+- +-+

| STAP-A nal HDR | NALU 1 size | NALU 1 HDR |

+- +-+

| NALU 1 Data |

::

++-+ -+

| NALU 2 size | NALU 2 HDR |

+- +-+

| NALU 2 Data |

::

| +-+

|:... Optional RTP padding |

+- +-+

 

Class hsf-naluparser <br/>{< br/> Public: <br/> hsf-naluparser (INT width, int height); <br/> hsf-naluparser (); <br/> virtual ~ Hsf-naluparser (); <br/> void setbuffer (unsigned char * buffer, int Len, int F, int NRI, int type ); <br/> bool readonepacket (unsigned char * buffer, Int & Len); <br/> bool ispacketoutstanding (); <br/> PRIVATE: <br/> unsigned char * m_pnalubuffer; // pointer of the buffer to which NALU data points <br/> unsigned int m_nnalusize; // NALU data buffer size <br/> unsigned char * m_pcurnalupos; // pointer to the buffer to be read by the next data packet <br/> int m_nframewidth; <br/> int m_nframe Height; <br/> int m_npacketcounts; <br/> int m_npacketseqnum; <br/> int m_nf; <br/> int m_nnri; <br/> int m_ntype; <br/> Enum {<br/> stap_a = 24, <br/> stap_ B = 25, <br/> mtap16 = 26, <br/> mtap24 = 27, <br/> fu_a = 28, <br/> fu_ B = 29 <br/>}; <br/> }; </P> <p> // class hsf-naluparser ///////////// ////////////// <br/> hsf-naluparser:: hsf-naluparser (INT width, int height) <br/>{< br/> m_nfr Amewidth = width; <br/> m_nframeheight = height; <br/> m_pnalubuffer = NULL; <br/> m_nnalusize = 0; <br/> m_npacketcounts = 0; <br/> m_npacketseqnum = 0; <br/> m_nf = 0; <br/> m_nnri = 0; <br/> m_ntype = 0; <br/>}< br/> hsf-naluparser: hsf-naluparser () <br/>{< br/> m_pnalubuffer = NULL; <br/> m_nnalusize = 0; <br/> m_npacketcounts = 0; <br/> m_npacketseqnum = 0; <br/> m_nf = 0; <br/> m_nnri = 0; <br/> m_ntype = 0; <br/>}< br/> H2 64 naluparser ::~ Hsf-naluparser () <br/>{< br/>}< br/> void hsf-naluparser: setbuffer (unsigned char * buffer, int Len, int F, int NRI, int type) <br/>{< br/> m_pnalubuffer = buffer; <br/> m_nnalusize = Len; <br/> m_nf = f; <br/> m_nnri = NRI; <br/> m_ntype = type; <br/> m_pcurnalupos = m_pnalubuffer; <br/> m_npacketcounts = (m_nnalusize + hsf-_mtu-1)/hsf-_mtu; <br/> Limit = 0; <br/>}< br/> bool hsf-naluparser: readonepacket (unsigned char * buffer, Int & Len) <br/>{< br/> If (m_pcurnalupos> = m_pnalubuffer + m_nnalusize) <br/>{< br/> return false; <br/>}< br/> struct hsf-_rtp_hdr header; <br/> int headersize; <br/> unsigned char * pcurbuf = buffer; <br/> If (m_nnalusize <= hsf-_mtu) // single NALU <br/> {<br/> header. singlenalu. F = m_nf; <br/> header. singlenalu. NRI = m_nnri; <br/> header. singlenalu. type = m_ntype; <br/> headersize = sizeof (header. singlenalu); <br/> memcpy (pcurbuf, & (header. singlenalu), headersize); <br/> pcurbuf + = headersize; <br/>}< br/> else // FU-A <br/>{< br/> header. fu_a.f = m_nf; <br/> header. fu_a.nri = m_nnri; <br/> header. fu_a.type_indicator = fu_a; <br/> If (0 = m_npacketseqnum) <br/>{< br/> header. fu_a.s = 1; <br/>}< br/> else <br/>{< br/> header. fu_a.s = 0; <br/>}< br/> If (m_npacketseqnum = m_npacketcounts-1) <br/>{< br/> header. fu_a.e = 1; <br/>}< br/> else <br/>{< br/> header. fu_a.e = 0; <br/>}< br/> header. fu_a.r = 0; <br/> header. fu_a.type_header = m_ntype; <br/> // <br/> headersize = sizeof (header. fu_a); <br/> memcpy (pcurbuf, & (header. fu_a), headersize); <br/> pcurbuf + = headersize; <br/>}< br/> If (m_npacketseqnum <m_npacketcounts-1) <br/>{< br/> memcpy (pcurbuf, m_pcurnalupos, hsf-_mtu); <br/> m_pcurnalupos + = hsf-_mtu; <br/> Len = headersize + hsf-_mtu; <br/>}< br/> else <br/>{< br/> int remainlen = m_nnalusize % h1__mtu; <br/> If (0 = remainlen) <br/>{< br/> remainlen = hsf-_mtu; <br/>}< br/> memcpy (pcurbuf, m_pcurnalupos, remainlen); <br/> m_pcurnalupos + = remainlen; <br/> Len = headersize + remainlen; <br/>}< br/> m_npacketseqnum ++; <br/> return true; <br/>}< br/> bool hsf-naluparser: ispacketoutstanding () <br/>{< br/> return (m_npacketseqnum <m_npacketcounts ); <br/>}</P> <p>

Interleaved mode: Type [26-29]

Packetization-mode = 2

 

To be continued

 

STAP-B

Mtap16

Mtap24

FU-B