Easydarwin Open Source Community streaming video course: Streaming media Transmission Control Protocol (RTSP RTP SDP) detailed RTP

Video courses and related documents code address: https://github.com/EasyDarwin/Course#course-3

RTP Protocol

The real-time Transport protocol RTP (real-time Transport Protocol) is a network transport protocol that was published by the IETF Multimedia Transmission Working Group in RFC 1889 in 1996 and later updated in RFC3550.

ITU-T also released its own RTP documentation as a h.225.0, but it was later canceled when the IETF released a standard RFC on its stability. It is described in detail as an Internet standard in [RFC 3550].

The RTP protocol details the standard packet format for transmitting audio and video over the Internet. It was first designed as a multicast protocol, but was later used in many unicast applications. RTP protocols are commonly used in streaming media systems (with RTSP protocol), videoconferencing and one-touch (Push Totalk) systems (with either a/p or SIP), making it the technical foundation of the IP telephony industry. The RTP Protocol and RTP Control protocol RTCP are used together, and it is built on the User Datagram Protocol (UDP).

The RTP standard defines two sub-protocols, RTP and RTCP

Data Transfer Protocol RTP, for real-time transmission. The protocol provides information such as timestamp (for synchronization), serial number (for packet loss and reorder detection), and payload format (the encoding format used to describe the data).

Control Protocol RTCPfor QoS feedback and synchronous media streaming. Compared to RTP, the bandwidth of RTCP is very small, usually only 5%.

Why use RTP

When it comes to streaming media, when it comes to video surveillance, video conferencing, voice telephony (VOIP), the application of RTP protocol is inseparable, but when everyone chooses the RTP protocol based on experience or someone else's application, do you ever wonder why we use RTP for streaming media? Why must we use RTP? Is TCP, UDP, or other network protocols not up to our requirements?

A reliable transport protocol such as TCP, which guarantees the correctness of every bit in the data stream through timeouts and retransmissions, can make the process of both protocol implementation and transmission very complex. Moreover, when there is data loss in the transmission process, due to the detection of data loss (timeout detection) and retransmission, the transmission of the data stream is forced to pause and delay.

Perhaps you would say that we can use the client to construct a buffer large enough to ensure that the display is normal, this method is acceptable for playing audio and video from the network, but for some situations that need real-time interaction (such as video chat, video conferencing, etc.), if the buffer exceeds 200ms, Will create an unacceptable, real-time experience.

Why RTP can solve the above delay problem

The RTP protocol is a UDP-based transport protocol, and RTP itself does not provide a reliable delivery mechanism for sequentially transmitting packets, nor does it provide traffic control or congestion control, which relies on RTCP to provide these services. Thus, for those packets that are missing, there is no delay due to time-out detection, and for those discarded packets, they can be selectively re-transmitted by the upper layer according to their importance. For example, for I-frame, p-frame, B-frame data, because of its importance in order to reduce, in the case of poor network conditions, can be considered in the case of B-frame loss or even p-frame loss without retransmission, so that, in the client side, although there may be short-term unclear picture, but to ensure the real-time experience

the protocol hierarchy of RTP Sub-layer of the Transport layer

Figure 1 shows a typical protocol architecture for streaming media applications.

As you can see, RTP is divided into the transport layer, which is built on UDP. As with the UDP protocol, RTP also has a fixed package format for its real-time transmission function. RTP is used to provide time information and stream synchronization for end-to-end real-time transmission, but does not guarantee quality of service. The quality of service is provided by RTCP.

the working mechanism of RTP is:

When an application establishes a RTP session, the application determines a pair of destination transport addresses. Destination transport address consists of a network address and a pair of ports, there are two ports: one to the RTP packet, one to the RTCP packet, so that the RTP/RTCP data can be sent correctly. RTP data is sent to even-numbered UDP ports, and the corresponding control signals RTCP data to the adjacent odd UDP ports (even UDP port +1), which makes up a UDP port pair. The sending process for RTP is as follows, and the receiving process is the opposite.

   1) RTP协议从上层接收流媒体信息码流（如H.263），封装成RTP数据包；RTCP从上层接收控制信息，封装成RTCP控制包。   2) RTP将RTP 数据包发往UDP端口对中偶数端口；RTCP将RTCP控制包发往UDP端口对中的奇数端口。

RTP packets contain only RTP data, and control is provided by the RTCP protocol. RTP selects an unused even UDP port number between 1025 and 65535, while RTCP in the same session uses the next odd UDP port number. Port numbers 5004 and 5005 are used as the default port numbers for RTP and RTCP, respectively. The first 12 bytes of the RTP packet are shown in header format 2, which is required.

part of the application layer

From the application developer's point of view, RTP should be part of the application layer. At the sending end of the application, the developer must write the program code that is grouped with the RTP package, and then give the RTP packet to the UDP socket interface. At the receiving end, the RTP packet enters the application layer through the UDP socket interface, and the application data block is extracted from the RTP packet using the program code written by the developer.

Streaming media features in the RTP header

First, let's look at the RTP header.
RTP Packet header format (see RFC3550 Page12):

Version number (V): 2 bits, used to flag the RTP version used.

Fill bit (P): 1 bits, if the position bit, the end of the RTP package contains additional padding bytes.

extension Bit (X): 1 bits, if the position is, the RTP fixed head is followed by an extension head.

CSRC counter (CC): 4 bits, containing the number of CSRC followed by a fixed head.

Mark Bit (M): 1 bits, the interpretation of this bit is assumed by the configuration document (profile).

load Type (payloadtype): 7 bits, which identifies the type of RTP payload.

serial Number (SN): 16 bits, with each RTP packet sent, the serial number is incremented by 1. The receiver can detect the packet loss and rebuild the package sequence accordingly.

Timestamp (Timestamp): 2 bits, which records the sampling time of the first byte of the data in the packet. At the beginning of a session, the timestamp is initialized to an initial value. Even when no signal is sent, the value of the timestamp increases over time (time passes). The clock frequency is dependent on the payload data format and is described in the profile.

Synchronization source Identifier (SSRC): 32 bits, which is the source of the RTP packet stream. You cannot have two identical SSRC values in the same RTP session. The identifier is randomly selected RFC1889 recommended by the MD5 random algorithm.

Contribution source list (CSRC list): 0~15 entry, 32 bits per item, used to flag the source of all RTP packets that have contributed to a new package produced by an RTP mixer. These contributed ssrc identifiers are inserted into the table by the mixer. The SSRC identifiers are listed so that the receiver can correctly identify the parties to the conversation.

RTP Expansion Header Structure

Figure 4 RTP Expansion Header

If the extended bit position in the RTP fixed Head is 1 (note: If there is a CSRC list, after the CSRC list), a variable-length header extension is added to the RTP pin. The header extension contains a 16-bit length field that indicates the number of 32-bit words in the extension, excluding 4-byte expansion headers (so 0 is a valid value).

Only one header extension is allowed after the RTP fixed head. To allow multiple interop implementations to independently generate different header extensions, or a particular implementation has several different header extensions, the first 16 bits of an extension are used to identify identifiers or parameters. The 16-bit format is defined by the implementation-specific upper layer protocol. The basic RTP description does not define any header extensions themselves.

the session process of RTP

When an application establishes a RTP session, the application determines a pair of destination transport addresses. Destination transport address consists of a network address and a pair of ports, there are two ports: one to the RTP packet, one to the RTCP packet, so that the RTP/RTCP data can be sent correctly. RTP data is sent to even-numbered UDP ports, and the corresponding control signals RTCP data to the adjacent odd UDP ports (even UDP port +1), which makes up a UDP port pair.

The sending process for RTP is as follows, and the receiving process is the opposite.

1. The RTP protocol receives streaming media information streams (such as h.263) from the upper layer, encapsulates them into RTP packets, and RTCP receives control information from the upper layer and encapsulates it as a RTCP control packet.
2. RTP sends RTP packets to an even port on a UDP port pair, and RTCP sends the RTCP control packet to the receive port in the UDP port pair.

The profile mechanism of RTP

RTP provides a great deal of flexibility for specific applications, separating the transport protocol from the specific application environment, the specific control strategy, the transmission protocol itself only provides the mechanism to complete real-time transmission, the developer can choose the appropriate configuration environment according to different application environment, and the appropriate control strategy.

The control strategy referred to here is that you can implement certain RTCP control algorithms according to your specific application requirements, such as the previously mentioned packet loss detection algorithm, packet loss retransmission strategy, some control schemes in video conferencing applications, and so on (these strategies I might describe in a later article).

For the appropriate configuration environment mentioned above, it refers to the definition of RTP's related configuration and payload format. The RTP protocol is designed to support a wide range of multimedia formats (e.g., H. MPEG-4, MJPEG, MPEG), and does not embody specific application configurations in the protocol, but is provided through profile profiles and load type format documentation. For any particular application, RTP defines a profile file and a description of the payload format associated with the file, as follows:

RTP profile for Audio and Video conferences with Minimal Control (RFC3551)

"RTP Payload Format for H. Video" (RFC3984)

"RTP Payload Format for MPEG-4 audio/visual Streams" (RFC3016)

Wait, want to know more can click here: Http://en.wikipedia.org/wiki/RTP_audio_video_profile

Description: If the application does not use a proprietary scheme to provide payload type (payload type), sequential number, or timestamp, but rather uses the standard RTP protocol, the application is more likely to work with other network applications, which is what everyone wants. For example, if two different companies are developing internet telephony software, they all incorporate RTP into their products, which makes it possible to communicate between users who use different corporate telephony software.

Package of RTCP

main functions of RTCP:
Quality of service monitoring and feedback, synchronization between media, and identification of members in a multicast group. During the RTP session, each participant periodically transmits the RTCP packet. The RTCP package contains statistics on the number of packets sent, the number of lost packets, and so on, so that each participant can use this information to dynamically change the transmission rate, or even change the payload type. RTP and RTCP are used together to optimize transmission efficiency with effective feedback and minimal overhead, making it particularly suitable for transmitting real-time data on the web.

RTCP is also transmitted with UDP, but the RTCP package is only some control information, so the grouping is very short, so you can encapsulate multiple RTCP packets in a UDP packet. RTCP has the following five types of grouping.

type	abbreviation indicates	Use
200	SR (Sender report)	Send Side Report
201	RR (Receiver report)	Receive Side reports
202	SDEs (Source Description Items)	SOURCE Point Description
203	BYE	End transfer
204	. APP	Specific applications

The package of the five groupings is similar, the following only the SR type, and other types refer to RFC3550.

The Send side reporting group SR (Sender Report) is used to enable the sending side to report the sending situation to all receivers in a multicast manner. The main contents of the SR Group are: The timestamp of the newly produced RTP packet in the SSRC,RTP stream of the corresponding RTP stream and the number of packets that the NTP,RTP stream contains, and the number of bytes that the RTP stream contains. The SR package is shown in Package 3.

Version (V): Same as RTP header domain.

Fill (P): Same as RTP header domain.

Receive report Counter (RC): 5 bits, the number of received report blocks in the SR package, can be zero.
Package Type (PT): 8 bit, SR package is 200.

Length field: 16 Bits, where the SR package is stored in a total length of 32 bits minus one.

Synchronization source (SSRC): The synchronization source identifier of the SR package sender. Same as the SSRC in the corresponding RTP packet.
The absolute time value when the NTP Timestamp (Network Time Protocol) SR package is sent. The role of NTP is to synchronize different RTP media streams.

RTP Timestamp: Corresponds to the NTP timestamp with the same unit and random initial values as the RTP timestamp in the RTP packet.

Sender's packet count: The total number of RTP packets sent by the sender from the beginning of the sending package to the time the SR package was generated. When the SSRC changes, the field is zeroed out.

Sender's octet count: The total number of bytes (excluding headers and fills) of the payload data sent by the sender from the beginning of the sending package to the time the SR package was generated. When the sender changes its ssrc, the field is zeroed out.

SSRC identifier for the synchronization source N: The report block contains statistics about the packets received from the source.

Loss rate (fraction Lost): Indicates the rate of loss of RTP packets from synchronization source N (ssrc_n) since the last SR or RR Packet was issued.

Cumulative number of packet losses: the total number of RTP packets lost from Ssrc_n, from the start of receiving SSRC_N packets to the sending of SR.

Extended Maximum serial number received: The largest sequence number in the RTP packet received from Ssrc_n,

Receive Jitter (Interarrival jitter): Statistical variance estimation of the acceptance time of RTP packets
Last SR,LSR: Takes the intermediate 32 bits of the NTP timestamp from the SR package received recently from Ssrc_n. If the SR package is not already received, the domain is zeroed.

Latency since SR (delay since last SR,DLSR): Time delay from Ssrc_n received SR package to send this report.

attached: Code description RTP Header:

/ * * RTP header * /typedef struct{#if 0 //big_endia     unsigned intVersion2;/ * Protocol version * /    unsigned intP:1;/ * Padding flag * /    unsigned intX:1;/ * Header Extension flag * /    unsigned intCc:4;/ * CSRC count * /    unsigned intM:1;/ * Marker bit * /    unsigned intPt:7;/ * Payload type * /    unsigned intSeq -;/ * Sequence number * /#else    unsigned intCc:4;/ * CSRC count * /    unsigned intX:1;/ * Header Extension flag * /    unsigned intP:1;/ * Padding flag * /    unsigned intVersion2;/ * Protocol version * /    unsigned intPt:7;/ * Payload type * /    unsigned intM:1;/ * Marker bit * /    unsigned intSeq -;/ * Sequence number * /#endifU_int32 ts;/ * Timestamp * /U_int32 SSRC;/ * Synchronization source * /U_int32 csrc[1];/ * Optional CSRC list * /} rtp_hdr_t;

RTCP Common Header:

/ * * RTCP Common header word */typedef struct{#if 0 //big_endia     unsigned intVersion2;/ * Protocol version * /    unsigned intP:1;/ * Padding flag * /    unsigned intCount5;/ * varies by packet type * /#else    unsigned intCount5;/ * varies by packet type * /    unsigned intP:1;/ * Padding flag * /    unsigned intVersion2;/ * Protocol version * /#endif    unsigned intPt:8;/ * RTCP Packet type * /    unsigned  ShortLength/ * PKT len in words, w/o this word * /} rtcp_common_t;

For more information

Email:[email protected]

WEB:www.EasyDarwin.org

Copyright? Easydarwin.org 2012-2016

Easydarwin Open Source Community streaming video lesson: Streaming Media Transmission Control Protocol (RTSP RTP SDP) detailed RTP