LIVE555 Study ii rtsp, RTP/RTCP protocol Introduction

LIVE555 Study ii rtsp, RTP/RTCP protocol Introduction

LIVE555 Study ii rtsp, RTP/RTCP protocol Introduction

I. Overview

RTSP (Real-Time Stream Protocol) is a text-based application layer Protocol. RTSP is similar to HTTP in terms of syntax and message parameters.

RTSP is used to control the transmission of media streams. It plays the "Network Remote Control" role for multimedia services. RTSP itself is not used to transmit media stream data. Media data can be transmitted through RTP, RTCP, and other protocols.

Basic RTSP operation process

First, the client connects to the streaming server and sends an OPTIONS command to query the method provided by the server. After receiving the response from the server, the client sends the DESCRIBE command to query the SDP information of a media file. The stream server responds with an SDP description. The response information includes the stream quantity, media type, and other information. The client analyzes the SDP description and sends a SETUP command for each stream in the session. The SETUP Command tells the Server Client the port for receiving media data. After the streaming media connection is established, the client sends a PLAY command, and the server starts to transmit the media stream data. During playback, the client can also send PAUSE and other command control flow playback commands to the server. After the communication is completed, the client can send the TERADOWN command to end the streaming media session.

It is the RTSP interaction between the complete client and the server obtained through wireshark packet capture. The black font indicates the client request, and the red font indicates the server response.

OPTIONS rtsp: // 10.34.3.80/D:/a.264 RTSP/1.0 CSeq: 2User-Agent: LibVLC/2.0.7 (LIVE555 Streaming Media v2012.12.18) RTSP/1.0 200 OKCSeq: 2 Date: Tue, jul 22 2014 02:41:21 GMTPublic: OPTIONS, DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE, GET_PARAMETER, SET_PARAMETERDESCRIBE rtsp: // 10.34.3.80/D:/a.264 RTSP/1.0 CSeq: 3User-Agent: libVLC/2.0.7 (LIVE555 Streaming Media v2012.12.18) Accept: application/sdpRTSP/1.0 200 OKCSeq: 3 Date: Tue, Jul 22 2014 02:41:21 GMTContent-Base: rtsp: // 10.34.3.80/D: /a.264/Content-Type: application/sdpContent-Length: 494 v = 0o =-1405995833260880 1 IN IP4 10.34.3.80s = H. 264 Video, streamed by the LIVE555 Media Serveri = D:/a.t T = 0 0a = tool: LIVE555 Streaming Media v2014.07.04a = type: broadcasta = control: * a = range: treaty = 0-a = x-qt-text-nam: H. 264 Video, streamed by the LIVE555 Media Servera = x-qt-text-inf: D:/a.20.m = video 0 RTP/AVP 96c = IN IP4 0.0.0.0b = AS: 500a = rtpmap: 96 H264/90000a = fmtp: 96 packetization-mode = 1; profile-level-id = 42001E; sprop-parameter-sets = Z0IAHpWoLQSZ, aM48gA = a = control: track1SETUP rtsp: // 10.34.3.80/D:/a.264/track1 RTSP/1.0 CSeq: 4User-Agent: LibVLC/2.0.7 (LIVE555 Streaming Media v2012.12.18) Transport: RTP/AVP; unicast; client_port = 60094-60095RTSP/1.0 200 OKCSeq: 4 Date: Tue, Jul 22 2014 02:41:25 GMTTransport: RTP/AVP; unicast; destination = 10.34.3.80; source = 10.34.3.80; client_port = 60094-60095; server_port = Protocol: 54DAFD56; timeout = 65 PLAY rtsp: // 10.34.3.80/D:/a.264/RTSP/1.0 CSeq: 5User-Agent: LibVLC/2.0.7 (LIVE555 Streaming Media v2012.12.18) Session: 54DAFD56Range: Treaty = 0.000-RTSP/1.0 200 OKCSeq: 5 Date: Tue, Jul 22 2014 02:41:25 GMTRange: Treaty = 0.000-Session: 54DAFD56RTP-Info: url = rtsp: // 10.34.3.80/D:/a.264/track1; seq = 10244; rtptime = 2423329550GET_PARAMETER rtsp: // 10.34.3.80/D:/a.264/RTSP/1.0 CSeq: 6User-Agent: libVLC/2.0.7 (LIVE555 Streaming Media v2012.12.18) Session: 54DAFD56RTSP/1.0 200 OKCSeq: 6 Date: Tue, Jul 22 2014 02:41:25 GMTSession: 54DAFD56Content-Length: 10 // terminate TEARDOWN rtsp: // 10.34.3.80/D:/a.264/RTSP/1.0 CSeq: 7User-Agent: LibVLC/2.0.7 (LIVE555 Streaming Media v2012.12.18) Session: 54DAFD56RTSP/1.0 200 OKCSeq: 7 Date: Wed, jul 30 2014 07:13:35 GMT

It can be found that the RTSP protocol is similar to the http protocol, both of which are text-based and have the same syntax. But they are not the same and there are the following main differences:

First, the method names are different. The DESCRIBE, SETUP, and PLAY methods are added to RTSP.

Second, HTTP is a stateless protocol, and there is no obvious order between sending methods. RTSP is a stateful protocol, and each method has a sequential relationship.

The HTTP protocol can transmit data, such as webpage data, in the form of load data. RTSP only provides stream playback control and does not transmit streaming media data. Streaming media data can be transmitted through RTP/RTCP.

Ii. RTSP messages

1. RTSP request message format

Method name url rtsp version carriage return line feed

Carriage Return, newline, carriage return, newline

Carriage Return newline

The method names include OPTIONS, DESCRIBE, SETUP, PLAY, and TEARDOWN.

The URL is the recipient's address, such as rtsp: // 192.168.0.1/video1.3gp.

The RTSP version is generally RTSP/1.0

Each line of a message ends with a line break. To facilitate identification, the last line of the message header has two line breaks.

The message body is optional.

2. Response Message format

RTSP version status code corresponding text interpretation carriage return line feed

Carriage Return, newline, carriage return, newline

Carriage Return newline

The RTSP version is generally RTSP/1.0.

The status code indicates the execution result of the corresponding message.

Some status codes correspond to text interpretations as follows:

Status Code text description

"200" OK: Execution successful

"400" Bad Request error Request

"404" Not Found

"500" Internal Server Error

3. Detailed introduction of each method

(1) OPTIONS

The client uses OPTION to query the methods provided by the server. The server will provide its own set of methods in the public field. From the above packet capture, we can see that the server provides OPTIONS, DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE, GET_PARAMETER, SET_PARAMETER, and other methods.

The OPTIONS method is not required. The client can bypass OPTIONS and directly send other messages to the server.

The CSeq field indicates the request sequence number. Each request of the client is assigned a serial number. Each request message corresponds to a response message with the same serial number.

The OPTIONS message can be sent at any time. Some clients regularly send OPTION messages to the server. The server can also determine whether the client is online by regularly receiving the OPTIONS message. But not all clients and servers do this.

User Agent

This domain is used for user identification. Different companies or different clients. The content of the domain in messages sent from different clients is not the same. The version number and model of the client are sometimes specified.

In the following dialog, this field specifies that VLC is used as the client, and the version number and the version of LIVE555 library are provided.

OPTIONS rtsp: // 10.34.3.80/D:/a.264 RTSP/1.0 CSeq: 2 // Request No. User-Agent: LibVLC/2.0.7 (LIVE555 Streaming Media v2012.12.18) RTSP/1.0 200 OKCSeq: 2 // reply No. Date: Tue, Jul 22 2014 02:41:21 GMTPublic: OPTIONS, DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE, GET_PARAMETER, SET_PARAMETER

(2) DESCRIBE

A describe message is sent from the client to the server. It is used by the client to obtain the description of the media file specified in the request link, generally SDP information. Session Description Protocol contains the Session Description, Media Encoding type, media bit rate, and other information. For streaming media services, the following domains must be included in SDP.

"A = control :"

"A = range :"

"A = rtpmap :"

"A = fmtp :"

When a video contains both audio and video, multiple of the above structures are available. Each media description starts with m. The following green and yellow backgrounds indicate the descriptions of the video and audio media respectively. The Accept field in the request is used to specify the type of media description that the client can Accept. Here, it is the SDP information.

DESCRIBE rtsp: // 10.34.3.80/D:/a.264 RTSP/1.0 CSeq: 3User-Agent: LibVLC/2.0.7 (LIVE555 Streaming Media v2012.12.18) Accept: application/sdp // request to obtain the SDP information RTSP/1.0 200 OKCSeq: 3 Date: Tue, Jul 22 2014 02:41:21 GMTContent-Base: rtsp: // 10.34.3.80/D: /a.264 // specify the Content-Type: application/sdp // request Type Content-Length: 494 // SDP length v = 0 // Version SDP protocol Version o =-1405995833260880 1 IN IP4 10.34.3.80 // initiation of the Origion session Registrant information s = H. 264 Video, streamed by the LIVE555 Media Server // session name I = D: /a.264 // Session Description t = 0 0 // start and end time of the session a = tool: LIVE555 Streaming Media v2014.07.04 // Attributea = type: broadcasta = control: * // control information a = range: NLE = 0-a = x-qt-text-nam: H. 264 Video, streamed by the LIVE555 Media Servera = x-qt-text-inf: D:/a.20.m = video 0 RTP/AVP 96 // Media type supported by the sender (Video) information such as c = IN IP4 0.0.0.0 // session connection information, spending the IP address used by the real media stream. B = AS: 500 // video bandwidth a = rtpmap: 96 H264/90000 // media attribute line, video (H264 is video format, 90000 is sampling rate) a = fmtp: 96 packetization-mode = 1; profile-level-id = 42001E; sprop-parameter-sets = Z0IAHpWoLQSZ, aM48gA = a = control: track1 // The track used for the video is 1 m = audio 0 RTP/AVP 97 // media type (audio) The following is the description of the audio B =: 19 // audio bandwidth a = rtpmap: 97 MP4A-LATM/11025/1 // video format (MP4A-LATM for video format, 11025 for sampling rate) a = fmtp: 97 profile-level-id = 15; object = 2; cpresent = 0; config = 40002A103FC0a = mpeg4-esid: 101 = x-envivio-verid: 000118a = control: trackID = 2/the track used for this audio is 2

MB is also called a media row. It describes the types of media supported by the sender.

M = audio 3458 RTP/AVP 0 96 97

The first parameter audio is the media name, indicating that the audio type is supported.

The second parameter 3488 is the port number, indicating that the UE sends audio streams on the local port 3458.

The third parameter RTP/AVP is the transmission protocol, indicating the UDP-based RTP protocol.

The fourth to seven parameters are supported by the four Net Load types.

A = rtpmap: 0 PCMU

A = rtpmap: 96 G726-32/8000

A = rtpmap: 97 AMR-WB

The row of a behavior media is represented by the attribute name: attribute value.

Format: a = rtpmap: <net load type> <encoding name>

The Net Load Type 0 is allocated to PCMU,

The corresponding encoding scheme for the net load type 96 is G.726, Which is dynamically allocated.

Net Load Type 97 corresponds to the adaptive multi-rate broadband coding (AMR-WB) for dynamic allocation.

For Videos

M = video 3400 RTP/AVP 98 99

The first parameter "video" is the media name, indicating that the video type is supported.

The second parameter 3400 is the port number, indicating that the UE sends video streams on the local port 3400.

The third parameter RTP/AVP is the transmission protocol, indicating RTP over UDP.

The fourth and fifth parameters provide two types of Net Load numbers.

A = rtpmap: 98 MPV

A = rtpmap: 99 H.261

The corresponding encoding scheme of the Net Load Type 98 is MPV, Which is dynamically allocated.

The corresponding encoding method of the Net Load Type 97 is H.261, Which is dynamically allocated.

(3) SETUP

The SETUP message is used to determine the transfer mechanism and establish an RTSP session. The client can also send a SETUP request again after setting up RTSP to change transmission parameters for the playing media stream. The server may agree to these parameters. If you do Not agree, the system will respond to "455 Method Not Valid In This State ".

The Transport Header field in the Request specifies the data transmission parameters acceptable to the client;

The Transport Header field in Response contains the Transport parameters confirmed by the server.

If the request does not contain SessionID, the server generates a SessionID.

SETUP rtsp: // 10.34.3.80/D:/a.264/track1 RTSP/1.0 // track1 indicates to set Channel 1 CSeq: 4User-Agent: LibVLC/2.0.7 (LIVE555 Streaming Media v2012.12.18) // Client Version Information Transport: RTP/AVP; unicast; client_port = 60094-60095 // The agreed Transport parameter RTP/AVP indicates that RTP over UDP is used, and unicast indicates unicast, used to differentiate multicast. Client_port indicates that the client RTP Port is 60094, And the RTCP port is 60095 RTSP/1.0 200 OKCSeq: 4 Date: Tue, Jul 22 2014 02:41:25 GMTTransport: RTP/AVP; unicast; destination = 10.34.3.80; source = 10.34.3.80; client_port = 60094-60095; server_port = 6970-6971 // The transmission parameter Session specified by the server: 54DAFD56; timeout = 65 // SessionID

From the SETUP session above, we can see that the RTP Port is represented by an even number, and RTCP represents an odd number of adjacent tcp ports.

The above shows the RTP over UDP mode. The SETUP dialog using RTP over TCP is as follows.

SETUP rtsp: // 10.34.3.80/D:/a.264/track1 RTSP/1.0 // track1 indicates to set Channel 1 CSeq: 4User-Agent: LibVLC/2.0.7 (LIVE555 Streaming Media v2012.12.18) transport: RTP/AVP/TCP; unicast; interleaved = 0-1RTSP/1.0 200 OKCSeq: 4 Date: Tue, Jul 22 2014 02:41:25 GMTTransport: RTP/AVP/TCP; interleaved = 0-1Session: 54DAFD56; timeout = 65 you can see that the Transport field of the SETUP command is RTP/AVP/TCP, and an interleaved = 0-1 field is added. Since RTP over TCP sends the RTP packet and RTCP packet to the same TCP port, the interleaved value is used to identify whether the RTP or RTCP packet is used. Interleaved = 0 indicates the RTP package, And interleaved = 1 indicates the RTCP package.

(4) PLAY

The PLAY method notifies the server to transmit data according to the mechanism specified in SETUP. The server transfers data from the start time of the specified range of the PLAY message until the end of the range. The server may put the PLAY request in the queue. The last PLAY request can be executed only after the previous PLAY request is complete.

Range specifies the start time of playback. If a message is received after the specified time, the playback starts immediately.

The PLAY request without the Range header is also valid. It starts from the start position of the media stream until the media stream is paused. If a media stream is paused after PAUSE, the media stream transmission continues at the PAUSE point. If the media stream is playing, the PLAY request does not work. The client can use this to test whether the server is alive.

PLAY rtsp: // 10.34.3.80/D:/a.264/RTSP/1.0 CSeq: 5User-Agent: LibVLC/2.0.7 (LIVE555 Streaming Media v2012.12.18) Session: 54DAFD56/SessionID, in the SETUP response, the returned Range: NSX = 0.000-// specifies the playing Start Time RTSP/1.0 200 OKCSeq: 5 Date: Tue, Jul 22 2014 02:41:25 GMTRange: NSX = 0.000-Session: 54DAFD56RTP-Info: url = rtsp: // 10.34.3.80/D:/a.264/track1; seq = 10244; rtptime = 2423329550/RTP Information

The Url field is the stream media link address corresponding to the RTP parameter, the seq field is the serial number of the first package of the stream media, and the rtptime is the RTP timestamp corresponding to the range field.

(5) PAUSE

The PAUSE message notifies the server to PAUSE streaming transmission. If a specific media stream is specified in the request URL, only the playback of the media stream is paused. You can specify to pause the audio only, and the subsequent playback will be muted. If a group of streams is specified in the request URL, the transmission of all streams in the group will be suspended. The server may not support PAUSE messages. For example, a real-time stream may not be paused. When a server does Not support a message, it will respond to "501 Not Implemented" to the client ".

A pause request may contain a Range header to specify the time point when the media stream is paused. The Range header must contain an exact value instead of a time Range. If the Range header specifies a time that exceeds the Range of the PLAY request, the server returns "457 Invalid Range ". If the Range header is missing, media streams are paused immediately after receiving the pause message, and the pause point is set to the current playback time.

(6) TEARDOWN

The TEARDOWN request terminates the media stream transmission of the specified URL and releases resources related to the media stream.

Iii. RTP/RTCP protocol

RTP is short for Real-Time Transport Protocol. It is a real-time transmission protocol for multimedia data streams. Generally, it is based on UDP or TCP. Some people classify it as an application layer protocol, and some people classify it as a transport layer protocol. This is all possible. The Rtp protocol provides Timestamp and serial number. The sender sets a timestamp during sampling. After receiving the timestamp, the receiver plays the video in sequence. RTP only ensures real-time data transmission. It does not provide a reliable transmission mechanism for data packets transmitted in sequence, nor provides traffic and congestion control. It relies on RTCP to provide these services.

Version (V): 0-1 2b is used to identify the RTP version used.

Fill bit (P): 2 1b if this bit is set to 1, the end of the RTP packet will be appended with the fill byte.

Extension bit (X): 3 1b if this bit is set to 1, the end of the RTP packet will be appended with the extension frame header.

CSRC counter (CC): number of CSRC followed by a Fixed Header in 4-7 4b.

Mark bit (M): 8 1b this bit is interpreted by the configuration document.

Load type (PT): 9-15 7b identifies the type of RTP load.

Serial number (SN): 16-31 16b the sender adds 1 to the value of this domain after each RTP packet is sent. The receiver can check the serial number to determine whether RTP packet loss occurs. Note: The initial values of serial numbers are random.

Timestamp: 32 32 B the sampling time of the first byte in the package. The timestamp has an initial value, which increases with the passage of time. Even if no packets are sent at this time, the timestamp will not be added. Timestamp is essential for achieving jitter removal and synchronization.

SSRC: synchronization source identifier: the source of the 32b RTP package. There cannot be two identical SSRC values in the same RTP session. This field is randomly generated based on a certain algorithm.

CSRC List: the contribution Source List contains 0-15 entries. Each 32b entry is used to identify the source of all RTP packets that contribute to the new packets produced by an RTP mixer.

RTCP protocol

RTCP is short for Real-Time Control Protocol. RTCP is usually used with RTP to manage the exchange information between the current process of transmission quality. During the RTP session, each participant periodically transmits the RTCP package, which contains statistics such as the number of sent packets and the number of lost packets. The server can use this information to dynamically change the transmission rate or even the type of the payload. The combination of RTP and RTCP can effectively achieve the optimal transmission efficiency with minimum overhead and is very suitable for transmitting real-time streams.

RTSP uses the RTP protocol to transmit real-time streams. RTP generally uses an even port, while RTCP uses an odd adjacent port, namely, the RTP Port Number + 1.

In RTCP communication control, the functions of RTCP are implemented through different types of RTCP packets. RTCP is also transmitted based on UDP packets, mainly including five types of packets:

1. SR: the sender Report, which is sent by the application that sends the RTP datagram or the middle end.

2. RR: the receiving end report, which is sent by the application that accepts but does not send the RTP datagram or the middle end.

3. SDES: source description, carrier that transmits the identity information related to session members, such as user name, email, and phone number.

4. BYE: the notification leaves, and other Members in the notification reply will exit the session.

5. APP: defined by the application itself as an extension of the RTCP protocol.

Version (V): Same as the RTP Header

Fill (P): Same as the RTP Header.

Receive report counter (RC): 5b The number of report blocks received in this SR package.

Package Type (PT): The 8bit SR package type is 200

Length: the length of the SR package is reduced by 1 in the unit of 32bit.

Synchronization source (SSRC): the synchronization source identifier sent by the SR package. It is the same as the SSRC in the RTP package.

NTP timestamp (Network Time Protocol): the absolute Time when the SR package is sent. Used to synchronize different streams.

RTP timestamp: corresponds to the NTP Timestamp and has the same initial value as the timestamp in the RTP package.

Send's Packet count: Total number of bytes of valid data sent by the sender from the start of the Packet to the time when the SR package is generated, excluding the header and fill. When the sender changes SSRC, this field must be cleared.

SSRC identifier of synchronization source n: This report contains the statistics of packets received from this source.

Loss Rate: indicates the loss rate of the RTP packet sent from the synchronization source n from the last SR or RR package.

Cumulative data loss: the total number of RTP losses sent by SSRC_n from the time when the SSRC_n packet is accepted to the time when the SR is sent.

Maximum extended serial number received: the maximum serial number of the RTP packet received from SSRC_n.

Interarrival jitter: statistical variance estimation of RTP packet receipt time.

Last SR timestamp (Last SR): takes the intermediate 32bit from the NTP timestamp in the SR package recently received from SSRC_n. If you have not received the SR package, it is 0.

Last dependent SR Delay (Delay since Last SR): the Delay from the Last time SSRC_n received the SR package to the time when the package was sent.

Audio/Video Synchronization

The transmitted audio and video streams are located in two different RTP sessions. Each RTP packet has its own timestamp, and the Network Protocol Time field in the RTCP packet is) the absolute storage time can be used to map audio and video to the same timeline for audio and video synchronization.

Positions of the preceding protocols in TCP/IP

The next article will introduce the basics of LIVE555.

2014.8.2 in Hangzhou, Zhejiang Province