Rtmp protocol analysis

 

I. rtmp Headers

For details about rtmp protocol packets, refer to red5.
The rtmp Protocol packet consists of a packet header and a packet body. The packet header can be of any of the four lengths: 12, 8, 4, 1 byte (s ). the complete rtmp header should be 12 bytes, containing the timestamp, head_type, amfsize, amftype, streamid information. The 8-byte header only records the timestamp, head_type, amfsize, amftype, the timestamp head_type is recorded in the header of four bytes. The header of one byte only records head_type. The maximum length of a package is 128 bytes by default. The chunksize can be used to change the maximum length of the package body. Generally, when the length of an atom file exceeds 128 bytes, more than 128 of the packets are placed in other rtmp packets. The packet header is a byte.
The complete rtmp header consists of 12 bytes, which are composed of the following five parts::

Purpose	Size (byte)	Description
Head_type	1	Baotou
Timer	3	Timestamp
Amfsize	3	Data size
Amftype	1	Data Type
Streamid	4	Stream ID

1) head_type-header type
Head_type occupies the first byte of the rtmp package, which records the package type and package channelid. The first two bits of head_type determine the header length. It can be calculated using the mask 0xc0 "and:

The first two bits of head_type correspond to the length:

Bits	Header Length
00	12 bytes
01	8 bytes
10	4 bytes
11	1 byte

The last six bits and streamid of head_type determine the channelid. Streamid and channelid correspondence: streamid = (ChannelID-4)/5 + 1 Reference red5

Channelid	Purpose
02	Ping and byteread Channels
03	Invoke channel our connect () publish () and Self-written netconnection. Call () data are in this channel
04	Audio and vidio Channels
05 06 07	The server is retained. After observation, fms2 uses these channels to send audio or video data.

For example, in the rtmp package data, a 0xc2 packet is inserted, which is a one-byte packet header and channelid = 2.

2) Timmer-Timestamp

The timestamp occupies 2nd, 3, and 4 bytes of the rtmp header. Rtmp timestamps can be divided into absolute timestamps and relative timestamps, recording the time information of audio and video. The relative timestamp refers to the interval between two rtmp packets, in milliseconds. The absolute timestamp refers to the time when the current packet is sent, in milliseconds. For audio and video playback, the timestamp is critical, because the synchronization of audio and video playback is controlled by the timestamp. If your video gets choppy, the audio and video are not synchronized, And the latency increases, it is probably caused by incorrect timestamps.
For the same stream, the publish timestamp differs from the play timestamp.
Publish timestamp, with a relative timestamp. The timestamp value is equal to the gap between the absolute timestamp of the current media package and the absolute timestamp of the previous media package, that is to say, the time stamp of the audio and video is on a timeline. unit: milliseconds.
The play timestamp is also a relative timestamp. The timestamp value is equal to the gap between the absolute timestamp of the current media package and the absolute timestamp of the previous media package of the same type, note that the difference here is that it emphasizes "media packages of the same type ". That is to say, the audio and video timestamps adopt separate timelines, in milliseconds.
FLV Format File timestamp, absolute timestamp, timestamp length 3 bytes. The timestamp value after 0xffffff is greater than timestamp & 0 xffffff.
The total time length of a FLV file is stored in the Duration Attribute of onmetadata. The length is 8 bytes and is a double type.

3) amfsize-data size
Amfsize occupies three bytes. The length is AMF and can exceed the maximum length of the rtmp package by 128 bytes. If it exceeds 128 bytes, multiple subsequent rtmp packets are combined. Each subsequent rtmp packet header occupies only one byte. Generally, it is 0xc? . The header of one byte indicates that the packet's timestamp, data size, data type, and stream ID are exactly the same as the rtmp package with the same channelid.

4) amftype-Data Type
Amftype is the data type in the rtmp package and occupies 1 byte. For example, the audio package type is 8, and the video package type is 9. Common data types are listed below:

0 × 01	Chunk Size	Changes the chunk size for packets
0 × 02	Unknown
0 × 03	Bytes read	Send every X bytes read by both sides
0 × 04	Ping	Ping is a stream control message, has subtypes
0 × 05	Server BW	The servers downstream BW
0 × 06	Client BW	The clients upstream BW
0 × 07	Unknown
0 × 08	Audio Data	Packet containing audio
0 × 09	Video data	Packet containing video data
0x0a-0x0e	Unknown
0x0f	Flex_stream_send	Type_flex_stream_send
0x10	Flex_shared_object	Type_flex_shared_object
0x11	Flex_message	Type_flex_message
0 × 12	Notify	An invoke which does not found CT a reply
0 × 13	Shared object	Has subtypes
0 × 14	Invoke	Like remoting call, used for stream actions too.
0 × 16	Streamdata	This is the new data type after fms3 is released. This type of data includes audiodata and videodata.

5) streamid-Stream ID
Occupies the last four bytes of the rtmp header. It is an int-type data of big-Endian. In the memory of our x86 computer, data is stored in a small tail mode: Little-Endian, while network data streams are generally in the big-Endian mode. Streamid is the unique ID of an audio/video stream. If both an audio package and a video package exist for a single stream, the streamid of this audio package is the same as the streamid of its video package, but the channelid is different.
Calculation formula between channelid and streamid: streamid = (ChannelID-4)/5 + 1 Reference red5. If the package is neither an audio package nor a video package, its streamid is 0.
For example, when the channel ID of an audio/video package is 2, 3, or 4, streamid is 1. When the channel ID of the audio/video package is 9, streamid is 2.

6) packet analysis
For example, there is an rtmp packet whose data volume is 0300 00
00 01 02 1400 00 00 00
0200 07
63 6f 6e 6e 65 63 74 003f F0 00 00 00 00 00
08 ,,,
Meaning of data parsing in sequence
03 indicates the 12-byte header, channelid = 3
000000 indicates the timestamp timer = 0
000102 indicates amfsize = 18
14 indicates amftype = invoke method call
00 00 00 00 indicates streamid = 0
// At this point, the 12-byte rtmp header ends with AMF data analysis. For specific amf0 data formats, see rtmp Protocol 2 and AMF data
02 indicates string
0007 indicates the length of string 7
63 6f 6e 6e 65 63 74 is the string's ascall value "Connect"
00 indicates double
3f F0 00 00 00 00 00 represents 0.0 of double
08 indicates that the map data starts

 

Ii. AMF data

The default maximum length of an rtmp package is 128 bytes (or the maximum length of an rtmp package is changed through chunksize). When AMF data exceeds bytes, multiple rtmp Packages may exist, if the rtmp package to be decoded is too long, it is divided into multiple TCP packets by the TCP protocol. the TCP packet containing the rtmp package must be merged before decoding. After decoding, the merged data can be obtained in AMF format, extract the AMF data to decode the AMF data.
The rtmp package includes two parts: the header and AMF data. The AMF data can contain commands or audio and video data. All data between the server and the flash client is transmitted in AMF format, such as connect () publish () and other commands. AMF data consists of two parts: objtype and objvalue. The size of the objtype is one byte. The objvalue size is not fixed and is related to the objtype. The commonly used objtype and corresponding objvalue are sorted as follows. Detailed objtype data is listed at the bottom of this article:

Type description (objtype)	Data	Datasize
Core_string	0x02	2 bytes (2 bytes of data records the actual length of the string)
Core_object	0x03	0 bytes (start nesting 0x00000009 indicates nesting ends)
Null	0x05	0 bytes NULL bytes meaningless
Core_number	0x00	8 bytes
Core_map	0x08	4 bytes (start nesting)
Core_boolean	0x01	1 byte

Objvalue is not necessarily a fixed size. It can contain another AMF data. The other AMF data contains objtype and objvalue, that is, the nesting relationship of AMF data.
The nesting relationship of amf0 data is as follows:
Object = {objtype + objvalue}
Core_boolean = {value (1 byte )}
Core_number = {value (8 byte )}
Core_string = {stringlen (2 byte) + stringvalue (stringlen byte )}
Core_date = {value (10 byte )}
Core_array = {arraylen (4 byte) + object}
Core_map = {mapnum (4 byte) + core_object}
Core_object = {core_string + object}

It seems a little complicated, so I will illustrate it here. For example, after handshaking, the first rtmp data sent by flash to the FMS is as follows:


Blue indicates the header, red indicates the objtype, and Green indicates the data. The preceding data segment consists of two rtmp packets. The two rtmp headers are displayed in blue. The first one is a 12-byte packet header, and the next one is a byte packet header, the green part is the data and the red part is the AMF data type. The entire rtmp decoding process is as follows:
[16:59:20] decodeinvoke:
[16:59:20] invokename: String: connect
[16:59:20] invokeid: Double: 0
[16:59:20] map: mapnum: 0
[16:59:20] Params :{
[16:59:20] key: String: objectencoding
[16:59:20] value: Double: 0
[16:59:20] key: String: app
[16:59:20] value: String: mediaserver
[16:59:20] key: String: fpda
[16:59:20] value: bool: 0
[16:59:20] key: String: tcurl
[16:59:20] value: String: rtmp: // 127.0.0.1/mediaserver
[16:59:20] key: String: audiocodecs
[16:59:20] value: Double: 615
[16:59:20] key: String: videocodecs
[16:59:20] value: Double: 76
[16:59:20]} end Params
[16:59:20] invokeparams: String: publisher
[16:59:20] invokeparams: String: streamrecode

In fms3, a data type is added to realize the live streaming of H.264 data. The value of this type is 0x16. This type is about the packets whose data type is 0x16 in the rtmp packet.
When you use the rtmp protocol to pull audio and video data from fms3, you will receive amftype = 0x16 packets, which have never appeared in fms2.
The 8th bytes of the rtmp header are amftype, that is, the data type. For example, amftype = 0x08 indicates the audio package, amftype = 0x04 indicates the ping package, and so on. In fms3, a data type is added to implement live streaming of H.264 data. The value of this type is 0x16. The amftype = 0x16 package contains both audio frames and video frames. Audio and Video Frames are stored in a new format, similar to the FLV file storage format. Each audio and video package is used as a tag, many tags constitute the amftype = 0x16 data type. The tag format is as follows:

Purpose	Size (byte)	Data Description
Streamtype	1	Stream type (0x08 = audio, 0x09 = video)
Mediasize	3	Media data area size
Timmer	3	Absolute timestamp, in milliseconds
Reserve	4	Reserved; Value: 0
Mediadata	Mediasize	Media data, audio or video
Taglen	4	Frame size. The value is the size of the media data area + parameter length (mediasize + 1 + 3 + 3 + 4)

 

The detailed objtype data types are as follows:, refer to red5
Enum AMF
{
/**
* Boolean value marker constant
*/
Type_boolean = 0x01,

/**
* String marker constant
*/
Type_string = 0x02,

/**
* Object marker constant
*/
Type_object = 0x03,

/**
* Movieclip marker constant
*/
Type_movieclip = 0x04,

/**
* Null marker constant
*/
Type_null = 0x05,

/**
* Undefined marker constant
*/
Type_undefined = 0x06,

/**
* Object reference marker constant
*/
Type_reference = 0x07,

/**
* Mixed Array marker constant
*/
Type_mixed_array = 0x08,

/**
* End of object marker constant
*/
Type_end_of_object = 0x09,

/**
* Array marker constant
*/
Type_array = 0x0a,

/**
* Date marker constant
*/
Type_date = 0x0b,

/**
* Long String marker constant
*/
Type_long_string = 0x0c,

/**
* Unsupported type marker constant
*/
Type_unsupported = 0x0d,

/**
* Recordset marker constant
*/
Type_recordset = 0x0e,

/**
* XML marker constant
*/
Type_xml = 0x0f,

/**
* Class marker constant
*/
Type_class_object = 0x10,

/**
* Object marker constant (for amf3)
*/
Type_amf3_object = 0x11,

/**
* True marker constant
*/
Value_true = 0x01,
/**
* False marker constant
*/
Value_false = 0x00
};