Standard Specification Learning: RTMP message structure, including several parts: timestamp: 4 byte, in milliseconds. is flipped when the maximum value is exceeded. Length: The length of the message payload. Type Id:type ID part of the ID range used for rtmp control signaling. There is also a part to use for the upper layer, rtmp is just a pass. This makes it easy to expand on rtmp. The message Flow id:message stream ID, which is used to differentiate messages from different streams.
the difference between two IDs:
message stream: The logical channel that transmits the message.
message stream ID: Each message has a stream ID that indicates which stream belongs to.
Chunk: It is a concept that is more underlying. The message may be too large to be split into fragments, and chunk is a partial fragment of a piece of information.
Chunk Stream: Transmits the logical channel of the Chunk.
Chunk stream ID: Used to indicate which logical channel the Chunk belongs to. Message
stream and chunk stream should belong to different levels, message belongs to application hierarchy, and chunk belongs to the lower rtmp protocol level. A chunk stream can run on more than one message stream. The message stream ID is not equal under a chunk stream. Messages are cut into small chunks for transmission. The time stamp unit for rtmp is milliseconds. Connection handshake Flowchart: The handshake process: the server and the client each send three packets: C0,C1,C2,S0,S1,S2, the server must be received c0 after sending S0 and S1, also can wait until C1 to send, the service side must receive C1 to be able to send S2, the client must receive S2 to be able to send C2
C0 and S0 are just one byte:Represents the protocol version number. It is now 03. 01 234567+-+-+-+-+-+-+-+-+| Version |+-+-+-+-+-+-+-+-+ C0 and S0 bits
C1 and s1 length 1536,Scratch discovery, the zero field is not necessarily 0.
the lengths of C2 and S2 are also 1536The discovery and specification of packets caught through VLC and nginx rtmp do not correspond. Installation specifications, time and time2 should be designed to calculate bandwidth and latency.
Interactive Process:Common clients occur together with C0 and C1, and then the server sends the s0,1,2 directly. After the client receives the Send C2, the handshake is complete. Three interactions are possible.
Block Flow: A connection can be worn across multiple block streams. Each block stream has a different ID to differentiate. Block transport runs a large message packet that is cut into a small message packet. Prevent large data such as video blocking connections, resulting in audio and signaling not being delivered. Small messages can also be transferred at lower cost, and Baotou will compress. The block stream size can be configured. There is a message: Set chunk size can be negotiated. Large blocks can reduce CPU load, but may block important messages; small blocks are not conducive to transmission, especially in low-bandwidth situations. Blocks allow the layer protocol to take large messages to smaller messages, such as large, but small-priority messages (such as video) that block small but high-priority messages (such as audio or control life)
--the transfer process, audio and control priority is higher. The size of the block is configured to use a set block Size control message line setting (refer to 5.4.1) a larger block size drops CPU overhead, but in the wide connection because of its large number of writes will also delay the delivery of his content smaller blocks are not conducive to high bit rate of fluidization block size set in the case of the body 。
--This is an issue that must be considered when using TCP. Block over the assembly affects the transmission of audio and control. Too small can affect efficiency and bit rate.
Block Type:
Block Base header: This is true of the first byte. Where FMT is used to indicate the message Header.
CS ID:The CS ID is an abbreviation for the chunk stream ID. The range is 3--65599. Variable bytes, followed by a byte, or two bytes. A few bytes are determined by the value of the next 6 bits with the first byte. This is not a complete Chinese translation.if the last 6-bit value is 0, it means that there is only one byte later. ID range is 64--319, is actually 0--255, maximum and minimum plus 64. if the last 6-bit value is 1, then only two bytes are followed. the ID range is64-65599, in fact, is 0--65535,Maximum and minimum plus 64. JThe calculation method is special: (the third section) * 256 + the second section + 64 block stream ID should be differentiated for each stream, for example, a video call, including two streams of video and audio. The meaning of this should be that a connection can pass multiple streams, which are differentiated by the block stream ID.
Block message Header The block message header format is determined by the FMT field of the block base header. A total of four kinds.
FMT:Four types: 0 1 2 3 Baotou must compress as much as possible.
Type 0:Used at the beginning of the stream, or when the timestamp is reset. Message header headers Total 11byte. Three-byte timestamp: Indicates a range of 0--16777215 (0XFFFFFF), and if greater than or equal to 16777215, indicates that there is also an extended timestamp for one byte. The third part of the chunk header is extend timestamp. Whether this part exists is determined by a value of three bytes.
Note that this is an absolute time stamp, while the following types of trunks are increments of timestamp, and the message cutoff is based on the length of the message. What about the input processing of the active discard package?
Type 1: The message header is 7byte, with a message stream ID of 4 bytes less than type 0, using the ID of the previous block. Variable-sized messages, such as each frame of the video, tend to use type1 in the first chunk of the frame after the first frame (the first chunk of the first frame needs to be TYPE0). Why? Because of variable data, length information must be logged.
type2: only three bytes. There is no message stream ID and message length. All use the previous chunk of the same chunk stream. Suitable for the length of the constant pace of transmission.
Type 3: A 0-byte message header. Timestamp, length, message flow ID all use the previous chunk of the same chunk stream. When a message is segmented into multiple chunk, all other chunk should use type3 except with a chunk. --How to determine the end of the package? Depending on the length of the message, the length in the first chunk should be the length of a message completion.
use in four formats:The first chunk of the first message uses TYPE0 to carry all the information; the next chunk uses type3; the second message, if it is a video message, has a variable length, the first chunk uses type1, and the value discards the message stream ID. The second message of the other chunk uses type3. If the audio is the same length, the first message is TYPE0+TYPE3 and the second message is type2+type3. If the audio message, length, and the time stamp increment (delta) between messages are the same, then the first message can be type0+type3 and all subsequent messages will be type3. This also requires that the time stamp of the first message must be the same as the timestamp increment between messages. Note: In addition to TYPE0, all others are increments of timestamp. Instead of the time stamp absolute value.
from the above, the chunk ID of the audio and video should be different. Otherwise it cannot be distinguished. Because some blocks are not the message stream ID, only the chunk stream ID.
common field Definitions:
1, timestamp delta: Represents the time stamp difference between the previous chunk and the current chunk. is not the time difference of the message. If it is greater than or equal to 16777215, it indicates that there is also an extended timestamp for one byte. The third part of the chunk header is extend timestamp.
2. Message length: The length of the message. The size of his and chunk's payload is different. Chunk size length is all chunk and the last size. --The message length is the actual size of the message. 3. Message type ID: Indicates the type of message, such as audio, video, control, etc. 4. Message stream ID: stored in small-endian order. Extended Timestamp: The time stamp is greater than play2: The bitrate can be switched.
Message length,chunk size,tcp subcontracting and message cutting Finally figured out the principle. Originally the concepts and methods described in the specification is not very clear, the combination of grasping bag finally clarified.
Message Length: The size of the timely message itself. This message is to be transmitted in the chunk.
the size of the chunk size:chunk. The size of the chunk is 128 by default. The maximum value of chunk size can be set by message. The maximum size of each chunk can not exceed max chunk size. Understand the above two points after analysis: 1, if the message length is less than or equal to max chunk size, then a chunk contains a message. 2. If the message length is greater than max chunk size, you need to divide the message into multiple chunk. The first few chunk size must be max size, and the last one is the remaining size. TCP subcontracting Process: 1, received chunk, clear Chunk stream ID, a chunk stream of data processing. 2. Decide the type of FMT and determine the length of the message. 3, if the length is less than or equal to max chunk size, then the size of this chunk body is the message length. 4, after processing this chunk, skip chunk header and chunk body, is the next chunk. 5, if the length is greater than max chunk size, then the size of chunk body is max chunk size. Process this part of the data, and then skip Max chunk size to find the next chunk header. The message length minus max Chunk size calculates the remaining length of the data, determines whether the remaining lengths are greater than max chunk size, or, if greater than, the chunk size or max Chunk. And so on, until the last length is less than or equal to max chunk size, then this length is the actual length of the chunk and is the last message slice of the message. Then all the useful slices are stitched together, which is the complete message. What you receive later will be another message.
protocol control messages: RTMP uses 1,2,3,4,5,6 's message type ID as the control message. The message stream ID of the control protocol must be 0, and the chunk stream ID must be 2 (2). 0,1 is used to represent the number of bytes, 2 is signaling, and 3-6xxxx is practical. ）。 Control signaling comes into effect as soon as it is received-there is no negotiation process ... --time stamp can be ignored.
set Chunk size (1):Set the maximum size of the chunk Message type ID is 1. A total of four bytes: the first bit must be 0. The default value is 128, and both the server and client can set chunk size. Note that an interaction in chunk size is independent of each other, that is, the client can set him to send the past chunk size, and the server can also set him to send the past chunk size. The two are not affected. Chunk size Minimum recommendation 128, must be greater than or equal to 1. The range of chunk size is 1 to 0x7fffffff (2147483647), but because the message header has a message length of only three bytes, the maximum value for chunk size is 16777215 if it is greater than this, go to him.
Abort Message (2) The message type ID is 2, four bytes, and the content that is carried is the chunk stream ID. For the notification peer, if the chunk stream has a message waiting to receive the unreachable message content, it stops and discards the previously accepted content. Can be used in bandwidth is limited is the priority to send audio and signaling, discard video.
Acknowledgement (3)Window Acknowledgement Size (5) Window acknowledgement size is used to set the Windows confirmation acknowledgement, which is the window confirmation message. At the beginning of the session, both sides send the window acknowledgement size to indicate the size expected to be confirmed. When one side receives a content larger than window acknowledgement size, it is like sending acknowledgement. 1, the session begins to calculate the number of bytes received when the time is received window acknowledgement size message starts. 2. Byte size does not include the TCP header, which should be the size of the chunk, which is the size of the content obtained from the TCP recv function. 3. Both sides should send window acknowledgement size and acknowledgement to each other. 4, the sending end sends the window acknowledgement size message, did not receive the acknowledgement is no longer sends the further message-this can cause the error easily, causes the message to send again.
Set Peer Bandwidth (6) Sets the peer-to-peer output bandwidth. The peer is controlled by setting the window acknowledgement size to achieve traffic control. After window acknowledgement size is not confirmed, the sending end will no longer send messages. So after receiving set peer bandwidth on the peer, if the previously sent window acknowledgement size is different from the window acknowledgement size written here, a window is typically sent Acknowledgement Size.
Message Format The type ID of the above protocol Control message is 1,2,3,5,6 (these are the controls of the chunk stream), with no 4. 4 is another type:
User Control Messages (4): User Control Protocol, which is the control protocol for RTMP stream.
Similarly, chunk stream ID to be set to 2,message stream ID to be set to 0. Message physique: Two-byte event type, followed by variable-length event data,
RTMP Command Messages Messages that can be sent between the client and the server include: Audio messages, video messages, data messages, command messages. The command message is AMF encoded.
command message type 20,17Represents a command message. 20 means using AMF0 encoding, 17 means using AMF3 encoding. command messages include Connect, CreateStream, publish, play, pause, and response messages using Onstatus,result.
Data Message (at: A data message for transmitting metadata or user data. 18 indicates that amf0,15 is AMF3.
Shared Object Message(19, 16): 19 means amf0,16 is AMF3. Audio Message (8): Voice Messages Video MSG (9): Video message Aggregate message (22): One message contains multiple messages.
Types of Commands (20,17)
netconnetion Command:Connect: Request an application instance to connect to the server. Call: Requests a remote invocation. Creatstream: Create a logical channel for the client to send audio, video, and descriptive data. Netconnection is the default channel, and the message stream ID is 0,creatstream using 0 as the message stream ID.
NetStream Commands:Multiple NetStream can share a single netconnection. But when was the ID of the stream determined??? ——
QuestionsPlay2: Switch to a different bit rate stream without changing the playback time. This is a useful feature. The message is sent after the paly. Deletestream: Deletes the stream. Receiveaudio: whether to accept audio. If False, the server does not reply. If true, the server replies to two states: NetStream.Seek.Notify and NetStream.Play.Start Receivevideo: Same. Publish: Publish a stream to server. A onstatus response is required. Seek: Offset. The millisecond is the unit. Pause: Paused.Example:First, connect is a connected server-side application (Nginx rtmp has multiple applications configured.) ）
How is video bare data packaged into a rtmp package? FFmpeg the conversion of FLV and H264 files is problematic. Video character packet format and FLV format very much like, basically is the FLV format Variant. Refer to the FLV file format for an official agreement. Video-related packages include: onmetadata,avcdecoderconfigurationrecord,h264 data. And the FLV file is the difference between the FLV file with the tag header, rtmp in the relevant information is placed in the rtmp header. As for the internal content is the same.
AMF0 encoding Format
- *AMF Data type:
- *type Byte Code
- *number 0x00
- *boolean 0x01
- *string 0x02
- *object 0x03
- *movieclip 0x04
- *null 0x05
- *undefined 0x06
- *reference 0x07
- *mixedarray 0x08
- *endofobject 0x09
- *array 0x0a
- *date 0x0b
- *longstring 0x0c
- *unsupported 0x0d
- *recordset 0x0e
- *xml 0x0f
- *typedobject (Class instance) 0x10
- *AMF3 Data 0x11
AMF Coding: The collection of data, the boundaries of the data are differentiated by different types, and by different types of lengths. The first byte is the data type, and then the length of the data is determined based on the data type. Where string is a variable-length, two-byte-length value. If it is an object, then object inside is a dictionary, key is a string, value is the data, here also to the data according to the type of data parsing, ibid.
My public number.
Detailed analysis of RTMP official standard specification
