Real-time transmission protocol Introduction: RTP protocol introduction, RTCP protocol introduction, RTSP Protocol Introduction

RTP (Real Time Transport Protocol)

RTP is a transmission protocol for multimedia data streams over the Internet.Rfc1889Release. RTP is defined to work during one-to-one or one-to-many transmission. It aims to provide time information and implement stream synchronization. Typical RTP applications are built on UDP, but can also work on TCP, ATM, and other protocols. RTP itself only guarantees the transmission of real-time data. It does not provide a reliable transmission mechanism for transmitting data packets in sequence, nor provides traffic control or congestion control. It relies on RTCP to provide these services.

RTP Mechanism
An acute problem that threatens multimedia data transmission is the unpredictable data arrival time. However, streaming media transmission requires timely arrival of data for playback and playback. The RTP protocol provides time tags, serial numbers, and other structures for controlling the exile of timely data. In the concept of stream, "Time tag" is the most important information. The sender sets the time tag in the data packet according to the real-time sampling. After receiving the data packet, the receiver restores the original and timely data according to the time tag at the correct rate. Different media formats have different attributes. However, RTP itself is not responsible for synchronization. RTP is only a Transport Layer Protocol. In order to simplify the transport layer processing and improve the efficiency of this layer. Move some transport layer protocol functions (such as traffic control) to the application layer. Synchronization is completed by the application layer protocol. It does not have the complete functions of the transport layer protocol, and does not provide any mechanism to ensure real-time data transmission. It does not support resource reservation or service quality. RTP packets do not even contain the length and packet boundary descriptions. At the same time, the RTP protocol datagram text and control packets use different adjacent ports, which greatly improves the Protocol flexibility and simplicity of processing.
The RTP protocol and UDP protocol work together to complete the transport layer protocol function. UDP only transmits data packets regardless of the time sequence of data packets. RTP protocol data units are carried by UDP groups. When carrying RTP data packets, sometimes a frame of data is divided into several packets with the same time tag, you can know that the time tag is not necessary. The multiplexing of UDP allows the RTP protocol to meet the needs of multimedia sessions by supporting explicit multi-point delivery.
Although the RTP protocol is a transport layer protocol, it is not implemented as a separate layer in the OSI architecture. RTP provides services based on a specific application. RTP only provides the Protocol framework. developers can fully expand the protocol according to the specific requirements of the application.
RTP packet structure
RTP Header Format:

The first 12 octal labels appear in each RTP packet, and the csrc id list only appears when the mixer is inserted. The meanings of each section are as follows:
Version 1 (v)
Version (V): 2 bits 2-bit. It indicates the RTP version. The initial protocol version is 0 and the rfc3550 version is 2 ..
② Fill Mark (P)
Padding (P): 1 bit 1 bit. For example, if a fill bit is set, additional information is included at the end of the package, which is not a payload. The last byte of the additional information indicates the length of the additional information (including the byte itself ). This field exists because some encryption algorithms require fixed-size padding characters or several RTP packets carried in the underlying protocol data unit.
③ Extension (X)
Extension (x): 1 bit 1 bit. If this bit is set, an extension header exists behind the Fixed Header. The format is defined in rfc3550 5.3.1.
④ CSRC count (cc)
CSRC count (CC): 4 bits 4 bits. The CSRC count includes the number of CSRC IDs that follow the Fixed Header.
⑤ Mark (m)
Marker (m): 1-bit, which is defined by the setting. The purpose is to allow important events to be marked out in the package stream. Another flag bit can be defined, or no flag bit can be specified by changing the number of bits. The function of this bit depends on the definition of profile. Profile can change the length of this bit, but keep the total length of marker and payload type unchanged (a total of 8 bits )..
⑥ Load type (PT)
Payload type (PT): 7 bits 7 bits, which records the code C used in the subsequent data, and the decoder er finds the corresponding decoder to extract the code, this bit indicates the type of information carried by RTP packet. The standard type is listed in rfc3551. If the receiver cannot recognize this type, the packet must be ignored.
Series 7
Sequence Number: 16 bits 16-bit. The serial number increases by 1 after each RTP packet is sent. The receiver can rearrange the packet sequence based on the serial number or detect packet loss. The initial values of serial numbers are random, making it more difficult to attack encrypted text.
Time Scale
Timestamp: 32 bits 32-bit. The time scale reflects the sampling time of the first octal number in RTP data packets. The sampling time must be exported from a monotonic and linearly increased clock to allow synchronization and jitter calculation. The time scale allows the timer er to know how to play the data at the right time.

It can be seen that if there is only a series number, it cannot completely play the data in order, because if there is a section in the data that has no data, there will be errors if there is only a series number, you need to make it know at which time the data is correctly played, so that we can play the correct information.
⑨ SSRC
SSRC: 32 bits 32-bit. The SSRC segment identifies the synchronization source. This identifier is not randomly selected, so that no two synchronization sources in the same RTP packet connection have the same SSRC identifier, that is, each data stream in an RTP session should have a different SSRC. Although the probability of selecting the same identifier for multiple sources is low, all RTP implementations must detect and resolve conflicts. If the source changes the source transmission address, you must also select a new ssrc id to avoid inserting it into the loop source.
⑩ CSRC list
CSRC list: 0 to 15 items bits0 to 15 items, each item 32-bit. The CSRC list indicates the load source in the package. The number of IDS is given by the CC segment. If more than 15 sources are used, only 15 are identified. The CSRC identifier is inserted by the mixer and is identified by the SSRC identifier of the active source. It is valid only when mixer exists. For example, if a multi-channel speech stream is merged into a single-channel speech stream, the SSRC of each original audio channel is listed here.

RTCP (real time contorl Protocol)

RTCP is responsible for managing the communication quality exchange control information between the current application processes. During the RTP session, each participant periodically transmits the RTCP package, which contains statistics such as the number of sent packets and the number of lost packets. Therefore, the server can dynamically change the transmission rate or even the type of the payload. RTP and RTCP can be used together to optimize transmission efficiency with effective feedback and minimal overhead. Therefore, RTP is especially suitable for transmitting real-time online data.
RTCP Working Mechanism
When an application starts a RTP session, two ports are used: one for RTP and the other for RTCP. RTP itself does not provide a reliable transmission mechanism for transmitting data packets in sequence, nor does it provide traffic control or congestion control. It relies on RTCP to provide these services. Some RTCP packets are issued between RTP sessions in a periodic manner to transmit functions such as listening service quality and exchanging session user information. The RTCP package contains statistics such as the number of sent data packets and the number of lost data packets. Therefore, the server can dynamically change the transmission rate or even the type of the payload. RTP and RTCP can be used together to optimize transmission efficiency with effective feedback and minimal overhead. Therefore, RTP is particularly suitable for transmitting real-time online data. Based on the data transmission feedback between users, you can develop a traffic control policy, while the interaction of session user information can develop a session control policy.
RTCP control packagePeriodical TransmissionThe same distribution mechanism of applications and data packets is applied to all connectors. The low-Layer Protocol provides the reuse of data and control packets, such as using a separate UDP port number. RTCP performs the following four functions:
It mainly provides quality feedback on data publishing. It is part of the RTP transmission protocol and is related to the stream and blocking control of other transmission protocols. Feedback directly applies to adaptive encoding control, but the experience of IP multicast shows that it is important to diagnose sending errors when feedback is received from the sender. Send a receiving feedback report to all participants to allow the problem observer to estimate whether the problem is local or global. Publishing mechanisms such as IP multicast allow network service providers and groups to receive feedback and act as third-party monitors to diagnose network problems. The feedback function is reported by the RTCP sender and receiver.
RTCP carries the RTP source persistent transport layer identifier known as the canonical name (cname. In the event of a conflict or Program Restart, since the SSRC identifier can be changed, the recipient needs a cname to track the participants. The receiver also needs to connect the cname with several data streams specified in the relevant RTP connection.
The first two features require all participants to send RTCP packets. Therefore, the rate must be controlled to scale RTP To a large number. Each participant is allowed to send a control package to other participants to observe the number of participants independently. This number of terms calculate the packet sending rate.
The fourth optional function is to transmit the minimum connection control information, such as participant identification. It is most likely to be used in a "loose control" connection where participants enter or leave freely without member control or parameter coordination. RTCP acts as a convenient channel to all participants, however, it does not need to support all control communication requirements of applications. When using RTP in IP multicast scenarios, the first three features are required. We recommend that you use them in all scenarios. RTP Application designers must avoid using a mechanism that only works in Unicast mode, which will lead to scalability.

RTCP Datagram
In the RTCP communication control, the functions of the RTCP protocol are implemented through different RTCP datagram, mainly including the following types:
① SR: Sending end report. The sending end refers to the application or terminal that sends the RTP datagram, And the sending end can also be the receiving end.
② Rr: Receiving end report. The so-called receiving end refers to an application or terminal that only receives but does not send RTP datagram.
③ Sdes: Source description. It serves as a carrier of the identity information of a session Member, such as the user name, email address, and phone number. In addition, it can send session control information to session members.
④ Bye: Indicates that one or more sources are no longer valid, that is, other members in the notification session will quit the session.
⑤ App: Defined by applications, this solution solves the scalability of RTCP and provides great flexibility for the Protocol Implementers.

Sdes: source description RTCP package
The sdes package is a three-tier structure consisting of a header and a data block. There can be no or multiple data blocks. The component describes the source indicated by the block. Description:
Version (V), fill (p), and length: As described in the SR package.
Packet type (PT): 8-bit, containing a constant of 202, identifying RTCP sdes packets.
Source count (SC): The number of SSRC/CSRC blocks contained in the sdes package. The value 0 is valid, but it makes no sense.
The source description is as follows:
Cname: standard terminal identity sdes item cname identity attributes are as follows:
In the event of a conflict or restart of the program, because the randomly assigned ssrc id may change, the cname item must be provided to bind the source ID from the ssrc id to the constant. Like the SSRC identifier, the cname identifier must be unique among all the participants of the RTP connection. To provide a set of cross-multimedia tools used by a participant in the RTP connection, the cname should be fixed as the participant. To facilitate third-party monitoring, cname should be suitable for programs or personnel to locate the source.
Name: User Name sdes item
Bye: disconnect the RTCP package
If the mixer receives a bye packet, the mixer forwards the bye packet without changing the SSRC/CSRC identifier. If the mixer is off, it should also issue a bye package to list all the sources it handles, not just its own SSRC identifier. As an option, the bye package can contain an 8-bit octal counter followed by many octal texts, indicating the reason for leaving, such as "camera malfunction" or "RTP loop detected ". The string has the same encoding, as described in sdes. If the string padding package reaches the bottom 32-bit boundary, the string will not end with an empty string; otherwise, the bye package will be filled with an empty octal array.
APP: Define the RTCP package of the application
The app package is used to develop new applications and new features. You do not need to register the package type value. Apps with unrecognized names should be ignored. After the test, if it is determined that the application is widely used, we recommend that you redefine each app package without registering the subtype and name segment with IANA.

RTSP (Real Time Streaming Protocol)
Protocol features:

● Scalability: new methods and parameters can be easily added to RTSP.
● Easy resolution: RTSP can be parsed by a standard HTTP or mime parser.
● Security: RTSP uses the webpage security mechanism.
● Independent Transmission: the RTSP transmission channel can use unreliable packet protocol (UDP) or reliable packet Protocol (RDP). To achieve application-level reliability, reliable stream protocols such as TCP can be used.
● Recording device control: the Protocol can control recording and playback devices.
● Suitable for professional applications: Using SMPTE time scales, RTSP supports frame-level precision and remote digital editing.
● Neutral demo Description: The Protocol does not impose a special demo or Metafile and can be transferred in the format type. However, the demo description must contain at least one RTSP Uri.
● Proxy and firewall friendliness: protocols can be processed by application and transport layer firewalls. The firewall must understand the setup method and open a "gap" for UDP media streams ".
● Proper Server Control: if you start a stream, you can also stop a stream.
● Transmission coordination: the user can coordinate the transmission method before handling continuous media streams.
● Performance coordination: if the basic features are invalid, there must be some cleanup mechanisms for the user to decide which method does not take effect. This allows users to propose a suitable interface.

Relationship with other protocols:

The RTSP function overlaps with HTTP, and the most obvious Cross is in the publishing of streaming media content-mostly through web pages. The current protocol specifications allow both web servers and streaming media servers to support RTSP. For example, the demo description can be obtained through HTTP or RTSP, which reduces the round-trip transmission time based on the browser, and also supports independent RTSP servers to communicate with clients that do not rely on HTTP.
However, the essential difference between RTSP and HTTP lies in the following five aspects:
● RTSP and HTTP are two different protocols.Use different methods and protocol identifiers.
● RTSPProtocol data transmission does not occupy protocol bandwidth and is sent using different protocols.
● HTTP isAsymmetric protocolsThe client sends a request and the server responds. In RTSP, both the client and server can send requests in a stateful manner.
● HTTP isStateless protocolRTSP must be in a certain state under any circumstances, so that parameters can still be set to control the media stream for a long time after the request is confirmed.
● RTSP uses the ISO 10646 (UTF-8) definition instead of the ISO 8859-1 definition to stay consistent with the current HTML.
Although most real-time media use RTP as the transmission protocol, RTSP is not bound to RTP. The reuse of HTTP functions has at least two advantages: security and proxy. Because the requirements are very similar, it is valuable to use HTTP functions in cache, proxy, and authorization.

1. RTSP

RTSP [3] working as a client server, RTSP is a multimedia playback control protocol used to enable users to play real-time data (audio and video streams) downloaded from the Internet) can be controlled, such as: Pause/resume, back, forward, etc. Therefore, RTSP is also called "Internet video recorder Remote Control Protocol ".

1.1. Introduction to RTSP

To implement the RTSP control function, not only the protocol, but also the media player and Media Server ). The relationship between media servers and media players is the relationship between servers and customers. The biggest difference between a media server and a common web server is that a Media Server supports stream audio and video transmission, therefore, you can download and play videos on the client's Media Player (You Need To Cache a short segment ). However, when downloading a multimedia program from a common web server, the entire file is downloaded and then played. RTSP only enables the media player to control the transmission of Multimedia Streams. Therefore, RTSP is also called the out-of-band protocol, while Multimedia Streams are transmitted in the band using RTP.

Figure 1 Relationship Between RTSP and RTP and RTCP

1.2. RTSP message structure

RTSP has two types of packets:Request MessageAndResponse Message.
Request Message refers to the request message sent from the customer to the server, and the Response Message refers to the response from the server to the customer. Because RTSP is text-oriented, every field in the message is an ASCII string, so the length of each field is uncertain. The RTSP packet consists of three parts:Start line, first line, and entity body. In the request message, the start line is the request line, and the structure 2 of the RTSP request message is shown.

Figure 2 Structure of the RTSP request message

The methods of RTSP request packets include:Options(Obtain available methods provided by the server ),Describe(Obtain the Session Description ),Setup(The client reminds the server to establish a session and determine the transmission mode ),Teardown(The client initiates a close request ),Announce(Update Session Description), Play(The client sends a playback request ),Pause (Temporarily stop the stream without releasing server resources),Get_parameter (Obtain flow control parameters, which may not be supported by some servers)AndSet_parameter (Set traffic control parameters, which may not be supported by some servers). The starting line of the response packet is the status line, as shown in figure 3. Figure 3 structure of RTSP Response Message

1.3. RTSP interaction process
C Indicates the RTSP client, and s indicates the RTSP server.
① C-> S: Option request // ask s which methods are available
S-> C: Option response // all available methods provided are included in the Response Information
② C-> S: DESCRIBE request // obtain the media initialization description provided by S.
S-> C: Describe response // s response media initialization description, mainly SDP
③ C-> S: SETUP Request // sets session attributes and transmission mode to remind s to establish a session
S-> C: setup response // s creates a session and returns the session identifier and session-related information.
④ C-> S: Play request // C request playback
S-> C: Play response // s Response Request Information
S-> C: Send streaming media data
⑤ C-> S: teardown request // C. Close the session.
S-> C: teardown response // s response request
The above process is a standard RTSP process, of which steps 1 and 2 are required.

Refer to http: similar to RTSP in terms of functionality. RTSP adopts HTTP/1.1 and adds a specific RTSP status code.
The HTTP protocol defines eight possible request methods:
------------
Get retrieves a simple request that identifies a resource in a URI.
The head method is the same as the get method. The server only returns the status line and header label, and does not return the request document.
The Post Server accepts requests for data written into the output stream of the client.
The put server saves the request data as a request to specify the new content of the URI.
Delete server request to delete the resource named in URI
Options requests for server-supported request methods
The trace web server provides feedback on HTTP requests and headers.
Connect is a documented but not implemented method. It is reserved for tunnel processing.
------------
The RTSP and HTTP protocol specifications are described in detail in rfc2326 and rfc2616 respectively.
The MMS protocol is a private protocol of Microsoft and is not published. Use a private custom control struct to send commands, rather than http. RTSP uses the send text Command Control