RTP programming in Linux (using jrtplib ))

Streaming media refers to the continuous time-based media transmitted using stream technology in the network. It features that the entire file does not need to be downloaded before playback, but is played by downloading and playing, it is the technical basis for video conferences, IP phones, and other applications. RTP is a standard protocol and Key Technology for Real-Time Streaming Media transmission. This article describes how to program Real-Time Streaming Media Using jrtplib in Linux.

 

1. Introduction to streaming media
With the increasing popularity of the Internet, the data transmitted over the network is no longer limited to text and graphics, but gradually transitioned to multimedia formats such as sound and video. Currently, when transmitting audio/video (audio/video, A/V) and other multimedia files over the network, there are basically two options: Download and stream transmission. Generally, A/V files occupy a large amount of storage space. It may take several minutes or even hours to download files in a network with limited bandwidth, therefore, this processing method has a high latency. For streaming transmission, audio, video, animation, and other multimedia files will be sent continuously and in real time by a dedicated Streaming Media Server, so that you do not have to wait until all the files have been downloaded, it takes only a few seconds to start the delay. When the multimedia data is played on the client, the remaining part of the file will continue to be downloaded from the Streaming Media Server.

Streaming is a new concept that has emerged on the Internet in recent years. It is widely defined and is a general term for multimedia data transmission over the network. Streaming media has two meanings: Broadly speaking, streaming media refers to a series of techniques, methods, and protocols that allow audio and video to form a stable and continuous transmission stream and playback stream, that is, streaming media technology. In a narrow sense, streaming media is relative to the traditional download-playback method. It refers to a new method for obtaining audio, video, and other multimedia data from the Internet, it supports real-time transmission and real-time playback of multimedia data streams. By using streaming media technology, the server can send stable and continuous multimedia data streams to the client, and the client can play back the data at a stable rate while receiving the data, you don't have to wait until all the data has been downloaded before playback.

Due to network bandwidth, computer processing capabilities, and protocol specifications, to download a large amount of audio and video data from the Internet, in terms of download time and storage space, it is unrealistic, and the emergence of streaming media technology is a good solution to this problem. Currently, there are two main methods for streaming media transmission: progressive streaming and realtime streaming. They are suitable for different application scenarios.

Sequential stream transmission

Ordered stream transmission transmits data by means of sequential download. When downloading, you can play back multimedia data online. However, you can only view the downloaded data at a given time point. You cannot skip to the undownloaded data point, the download speed cannot be adjusted based on network conditions during transmission. Because the standard HTTP server can send this form of streaming media without the support of other special protocols, it is often called HTTP streaming transmission. Ordered stream transmission is suitable for high-quality multimedia clips, such as titles, credits, and advertisements.

Real-time stream transmission

Real-time stream transmission ensures that the media signal bandwidth can match the current network conditions, so that streaming media data is always transmitted in real time. Therefore, it is particularly suitable for on-site events. Real-time stream transmission supports random access, that is, users can view the content in front or back through fast forward or backward operations. Theoretically, real-time streaming media does not pause once it is played, but in fact it is still possible to pause periodically, especially when the network condition deteriorates. Different from Sequential stream transmission, real-time stream transmission requires a specific Streaming Media Server and supports specific network protocols.

Ii. Streaming Media Protocol
Real-Time Transport Protocol (PRT) is a network protocol used to process multimedia data streams over the Internet. It can be used in one-to-one (unicast, unicast) scenarios) or you can transmit streaming media data in real time in a one-to-multiple (Multi-play) network environment. RTP usually uses UDP for multimedia data transmission, but other protocols such as TCP or ATM can be used if needed. The entire RTP protocol consists of two closely related parts: RTP data protocol and RTP control protocol. Real Time Streaming Protocol (RTSP) was first proposed by Real Networks and Netscape. It is located on RTP and RTCP, and its purpose is to transmit multimedia data effectively through an IP network.

2.1 RTP data protocol

The RTP data protocol is used to package streaming media data and implement real-time transmission of media streams. Each RTP data packet consists of header and payload, the first 12 bytes in the header are fixed, while the load can be audio or video data. The Header Format of the RTP datagram is 1:

Figure 1 RTP Header Format

The important domains and their meanings are as follows:

• CSRC count (cc)Indicates the number of CSRC identifiers. The CSRC Mark follows the fixed RTP Header to indicate the source of the RTP datagram. the RTP protocol allows multiple data sources in the same session, which can be combined into a data source through the RTP mixer. For example, a CSRC list can be generated to represent a teleconference, which combines the voice data of all speakers into a RTP data source through an RTP mixer.
• Load type (PT)Indicates the RTP load format, including the encoding algorithm, sampling frequency, and bearer channel used. For example, type 2 indicates that the RTP data packet carries voice data encoded using the ITU g.721 algorithm. The sampling frequency is 8000Hz and the single channel is used.
• Serial numberIt is used to provide a method for the receiver to detect data loss, but how to handle the lost data is the application's own business. The RTP protocol itself is not responsible for data retransmission.
• TimestampThe sampling time of the first byte in the load is recorded. the receiver can timestamp to determine whether the arrival of data is affected by latency jitter, but how to compensate for latency jitter is the application's own business.

It is not difficult to see from the RTP datagram format that it contains the type, format, serial number, timestamp, and whether there is additional data, these provide a foundation for Real-Time Streaming Media transmission. The RTP protocol is designed to provide end-to-end transmission services for real-time data (such as interactive audio and video). Therefore, there is no connection concept in RTP, it can be built on the underlying connection-oriented or non-connection-oriented transmission protocols. RTP does not depend on the special network address format, but only needs the underlying transmission protocol to support frame (framing) and segment (segmentation). In addition, RTP itself does not provide any reliability mechanisms, which must be ensured by the Transport Protocol or application itself. In typical application scenarios, RTP is generally implemented as part of an application over the transport protocol, as shown in Figure 2:

Figure 2 Relationship Between RTP and various network protocols

2.2 RTCP Control Protocol

The RTCP control protocol must be used together with the RTP data protocol. When an application starts an RTP session, both ports are used for RTP and RTCP respectively. RTP itself does not provide a reliable guarantee for data packets transmitted in sequence, nor does it provide traffic control and congestion control, which are completed by RTCP. Generally, RTCP uses the same distribution mechanism as RTP to periodically send control information to all session members. The application receives the data and obtains relevant information of the session participants, and network conditions, packet loss probability and other feedback information, so as to control service quality or diagnose network conditions.

The functions of the RTCP protocol are implemented through different RTCP datagram, mainly including the following types:

• SrThe sender report refers to the application or terminal that sends the RTP datagram. The sender can also be the receiver.
• RrThe acceptor reports that an application or terminal receives but does not send RTP data packets.
• SdesThe source description mainly serves as a carrier for the identity information of session members, such as the user name, email address, and phone number. In addition, it also provides the ability to send session control information to session members.
• ByeThe main function of the notification is to indicate that one or more sources are no longer valid, that is, other members in the notification session will quit the session.
• AppDefined by applications, this solution solves the scalability of RTCP and provides great flexibility for Protocol Implementers.

RTCP datagram carries the necessary information of service quality monitoring, which can dynamically adjust service quality and effectively control network congestion. Because RTCP datagram adopts the multicast mode, all members in the session can use the control information returned by the RTCP datagram to understand the current situation of other participants.

In a typical application, the application that sends a media stream periodically generates the sender Report SR. The RTCP datagram contains synchronization information between different media streams, as well as the sent datagram and byte count, the receiving end can estimate the actual data transmission rate based on the information. On the other hand, the receiving end sends the RR report to all known senders. The RTCP datagram contains the maximum serial number of the received datagram, the number of lost datagram, latency jitter, timestamp, and other important information, based on this information, the sender application can estimate the round-trip latency and dynamically adjust the transmission rate based on the datagram loss probability and latency Jitter to improve network congestion, or, you can smoothly adjust the service quality of your application based on the network conditions.

2.3 RTSP real-time stream Protocol

As an application layer protocol, RTSP provides a scalable framework, which makes real-time streaming media data controlled and On-Demand Streaming possible. In general, RTSP is a Streaming Media Protocol mainly used to control data transmission with real-time characteristics. However, RTSP does not transmit data, but must rely on some services provided by the lower-layer transmission protocol. RTSP can provide streaming media operations such as playing, pausing, and fast forward. It defines specific control messages, operation methods, status codes, and other operations. It also describes the interaction with RTP.

The RTSP has many references to the HTTP/1.1 protocol during the preparation, and even many descriptions are identical with HTTP/1.1. RTSP uses similar syntaxes and operations as HTTP/1.1 to be compatible with the existing web infrastructure, most HTTP/1.1 extensions can be directly introduced into RTSP.

A media stream set controlled by RTSP can be defined by the presentation description. The so-called representation refers to the set of one or more media streams provided by the Streaming Media Server to the client, the description contains information about each media stream, such as the data encoding/decoding algorithm, network address, and media stream content.

Although the RTSP server also uses identifiers to differentiate each session, the RTSP connection is not bound to a transport layer connection (such as TCP ), that is to say, during the entire RTSP connection, the RTSP user can open or close multiple reliable transmission connections to the RTSP server to send an RTSP request. In addition, RTSP connections can also be based on connectionless transmission protocols (such as UDP ).

The RTSP protocol currently supports the following operations:

• Retrieve MediaAllows users to submit a description to the Media Server through HTTP or other methods. For example, if the description is multicast, the description includes the multicast address and port number used for the media stream. If the description is unicast, to ensure security, only the target address is provided in the description.
• Invite to joinThe Media Server can be invited to an ongoing meeting, play back the media in the presentation, or record all media or its subsets in the presentation, which is very suitable for distributed teaching.
• Add MediaIt is particularly useful for on-site lectures to notify users of new available media streams. Similar to HTTP/1.1, RTSP requests can also be handled by proxy, channel, or cache.

Iii. streaming media programming
RTP is currently the best solution to the problem of Real-Time Streaming Media transmission. If you need to program Real-Time Streaming Media on the Linux platform, you can consider using open-source RTP libraries, such as librtp and jrtplib. Jrtplib is an object-oriented RTP database, which fully complies with the RFC 1889 design. In many cases, jrtplib is a very good choice. The following uses jrtplib as an example, describes how to use the RTP protocol for real-time streaming media programming on the Linux platform.

3.1 Environment Construction

Jrtplib is a RTP Library implemented in C ++. It can run on Windows, Linux, FreeBSD, Solaris, UNIX, and VxWorks operating systems. To install jrtplib for Linux, first go to the jrtplib website (http: // Lumumba. luc. AC. be/Jori/jrtplib/jrtplib.html) to download the latest source code package. Here jrtplib-2.7b.tar.bz2 is used. If the downloaded source code package is saved in the/usr/local/src directory, run the following command to decompress it:

[root@linuxgam src]# bzip2 -dc jrtplib-2.7b.tar.bz2 | tar xvf -

Next we need to configure and compile jrtplib:

[root@linuxgam src]# cd jrtplib-2.7 [root@linuxgam jrtplib-2.7b]# ./configure [root@linuxgam jrtplib-2.7b]# make

Run the following command to install jrtplib:

[root@linuxgam jrtplib-2.7b]# make install

3.2 Initialization

Before using jrtplib for Real-Time Streaming Media data transmission, you should first generate an rtpsession instance to represent the RTP session, and then call the CREATE () method to initialize it. The create () method of the rtpsession class has only one parameter to specify the port number used for this RTP session. Listing 1 provides a simple initialization framework, which completes the initialization of RTP sessions and does not have any practical functions.

Code List 1: Initial. cpp

#include "rtpsession.h" int main(void) { RTPSession sess; sess.Create(5000); return 0; }

If the RTP session fails to be created, the CREATE () method returns a negative number. Although it can easily determine whether the function call is successful or failed, however, it is difficult to understand the cause of the error. Jrtplib adopts a unified error handling mechanism. If all functions provided by jrtplib return negative numbers, it indicates that some form of errors have occurred. For specific error information, you can call rtpgterrorstring () function. The rtpgterrorstring () function passes in the error code as a parameter and returns the error message corresponding to the error code. Listing 2 provides a more complete initialization framework that can better handle errors generated during RTP Session Initialization:

Code List 2: Framework. cpp

#include <stdio.h> #include "rtpsession.h" int main(void) { RTPSession sess; int status; char* msg; sess.Create(6000); msg = RTPGetErrorString(status); printf("Error String: %s//n", msg); return 0; }

Setting an appropriate timestamp unit is another important task in the RTP Session Initialization Process. This is achieved by calling the settimestampunit () method of the rtpsession class, this method also has only one parameter, indicating the timestamp unit in seconds. For example, when the RTP session is used to transmit 8000Hz sampled audio data, the timestamp increases by 8000 per second, so the timestamp unit should be set to 1/8000 accordingly:

sess.SetTimestampUnit(1.0/8000.0);

3.3 Data Transmission

After the RTP session is successfully established, the streaming media data can be transmitted in real time. First, you need to set the destination address for data transmission. The RTP protocol allows multiple destination addresses for the same session. You can call the adddestination (), deletedestination (), and cleardestinations () of the rtpsession class () method. For example, the following statement indicates that the RTP session sends data to Port 6000 of the local host:

unsigned long addr = ntohl(inet_addr("127.0.0.1")); sess.AddDestination(addr, 6000);

After all the target addresses are specified, you can call the sendpacket () method of the rtpsession class to send streaming media data to all target addresses. Sendpacket () is an overload function provided by the rtpsession class. It has the following forms:

int SendPacket(void *data,int len) int SendPacket(void *data,int len,unsigned char pt,bool mark,unsigned long timestampinc) int SendPacket(void *data,int len,unsigned short hdrextID,void *hdrextdata,int numhdrextwords) int SendPacket(void *data,int len,unsigned char pt,bool mark,unsigned long timestampinc, unsigned short hdrextID,void *hdrextdata,int numhdrextwords)

The most typical use of sendpacket () is similar to the following statement. The first parameter indicates the data to be sent, and the second parameter indicates the length of the data to be sent, the following are the RTP load types, identifiers, and timestamp increments.

sess.SendPacket(buffer, 5, 0, false, 10);

For the same RTP Session, the load type, identifier, and timestamp increment are usually the same. jrtplib allows you to set them as the default parameters of the session. This is done by calling setdefaultpayloadtype () of the rtpsession class () setdefamark mark () and setdefatimetimestampincrement () methods. Setting these default parameters for RTP sessions can simplify data transmission. For example, if you set the default parameters for RTP sessions:

sess.SetDefaultPayloadType(0); sess.SetDefaultMark(false); sess.SetDefaultTimeStampIncrement(10);

Then, when sending data, you only need to specify the data to be sent and its length:

sess.SendPacket(buffer, 5);

3.4 Data Reception

For the receiver of streaming media data, you must first call the polldata () method of the rtpsession class to receive the RTP or RTCP datagram sent. Since multiple participants (sources) are allowed in the same RTP session, you can call the gotofirstsource () and gotonextsource () Methods of the rtpsession class to traverse all sources, you can also use the gotofirstsourcewithdata () and gotonextsourcewithdata () Methods of the rtpsession class to traverse those sources with data. After the valid data source is detected from the RTP session, you can call the getnextpacket () method of the rtpsession class to extract the RTP datagram from it. After the received RTP datagram is processed, remember to release it in time. The following code demonstrates how to process the received RTP datagram:

If (sess. gotofirstsourcewithdata ()){ Do { Rtppacket * pack; Pack = sess. getnextpacket (); // Process received data Delete pack; } While (sess. gotonextsourcewithdata ()); }

Jrtplib defines three receiving modes for RTP datagram, each of which specifies which RTP datagram will be accepted and which RTP datagram will be rejected. You can set the following receiving modes by calling the setreceivemode () method of the rtpsession class:

• Receivemode_allIn the default receiving mode, all received RTP data packets are accepted;
• Receivemode_ignoresomeExcept for some specific senders, all incoming RTP datagram data will be accepted, and the list of rejected senders can be called by addtoignorelist (), deletefromignorelist (), and clearignorelist () method;
• Receivemode_acceptsomeExcept for some specific senders, all incoming RTP datagram data is rejected, and the list of accepted senders can be set by calling the addtoacceptlist (), deletefromacceptlist, and clearacceptlist () methods.

3.5 control information

Jrtplib is a highly encapsulated RTP Library. When using it, programmers often do not care about how RTCP datagram is sent and received, because it can be done by jrtplib. As long as the polldata () or sendpacket () method is successfully called, jrtplib can automatically process the RTCP datagram and send the RTCP datagram as needed, this ensures the correctness of the entire RTP session process.

On the other hand, by calling the setlocalname (), setlocalemail (), setlocallocation (), setlocalphone (), setlocaltool (), and setlocalnote () methods provided by the rtpsession class, jrtplib allows programmers to set the control information of RTP sessions. All these methods are called with two parameters. The first parameter is a char pointer pointing to the data to be set, and the second parameter is an int value, indicates how many characters in the data will be used. For example, the following statement can be used to set the email address in the control information:

sess.SetLocalEMail(" xiaowp@linuxgam.comxiaowp@linuxgam.com ",19);

During RTP sessions, not all control information needs to be sent. by calling the enablesendname (), enablesendemail (), enablesendlocation (), and enablesendphone () provided by the rtpsession class () the enablesendtool () and enablesendnote () methods can be used to select the control information to be sent for the current RTP session.

3.6 practical application

Finally, we use a simple streaming media sending and receiving instance to introduce how to program Real-Time Streaming Media Using jrtplib. Listing 3 shows the complete code of the Data sender, which is used to send RTP packets continuously to the IP address and port specified by the user:

Code List 3: sender. cpp

# Include <stdio. h> # Include <string. h> # Include "rtpsession. H" // Error handling function Void checkerror (int err) { If (ERR <0 ){ Char * errstr = rtpgeterrorstring (ERR ); Printf ("error: % S // n", errstr ); Exit (-1 ); } } Int main (INT argc, char ** argv) { Rtpsession sess; Unsigned long destip; Int destport; Int portbase = 6000; Int status, index; Char buffer [128]; If (argc! = 3 ){ Printf ("Usage:./sender destip destport // n "); Return-1; } // Obtain the IP address and port number of the acceptor. Destip = inet_addr (argv [1]); If (destip = inaddr_none ){ Printf ("Bad IP address specified. // n "); Return-1; } Destip = ntohl (destip ); Destport = atoi (argv [2]); // Create an RTP session Status = sess. Create (portbase ); Checkerror (Status ); // Specify the RTP data receiving end Status = sess. adddestination (destip, destport ); Checkerror (Status ); // Set the default parameters of the RTP session Sess. setdefapaypayloadtype (0 ); Sess. setdefamark mark (false ); Sess. setdefatimetimestampincrement (10 ); // Send streaming media data Index = 1; Do { Sprintf (buffer, "% d: RTP packet", index ++ ); Sess. sendpacket (buffer, strlen (buffer )); Printf ("Send packet! // N "); } While (1 ); Return 0; }

Listing 4 shows the complete code of the data receiver, which is responsible for continuously reading RTP data packets from the specified port:

Code list 4: worker er. cpp

# Include <stdio. h> # Include "rtpsession. H" # Include "rtppacket. H" // Error handling function Void checkerror (int err) { If (ERR <0 ){ Char * errstr = rtpgeterrorstring (ERR ); Printf ("error: % S // n", errstr ); Exit (-1 ); } } Int main (INT argc, char ** argv) { Rtpsession sess; Int localport; Int status; If (argc! = 2 ){ Printf ("Usage:./sender localport // n "); Return-1; } // Obtain the user-specified port number Localport = atoi (argv [1]); // Create an RTP session Status = sess. Create (localport ); Checkerror (Status ); Do { // Accept RTP data Status = sess. polldata (); // Retrieve the RTP data source If (sess. gotofirstsourcewithdata ()){ Do { Rtppacket * packet; // Obtain the RTP Datagram While (packet = sess. getnextpacket ())! = NULL ){ Printf ("got packet! // N "); // Delete the RTP Datagram Delete packet; } } While (sess. gotonextsourcewithdata ()); } } While (1 ); Return 0; }