RTP programming in Linux (using jrtplib)

Streaming media refers to the continuous time-based media transmitted using stream technology in the network. It features that the entire file does not need to be downloaded before playback, but is played by downloading and playing, it is used in video conferences, IP phones, and other applications.
Technical basis. RTP is a standard protocol and Key Technology for Real-Time Streaming Media transmission. This article describes how to program Real-Time Streaming Media Using jrtplib in Linux.
 

1. Introduction to streaming media

With the increasing popularity of the Internet, the data transmitted over the network is no longer limited to text and graphics, but gradually transitioned to multimedia formats such as sound and video. Currently, audio/video is transmitted over the network.
(Audio/video, A/V for short) and other multimedia files, basically only download and streaming. Generally, A/V files occupy a large amount of storage space.
In the network environment, downloading may take several minutes or even hours, so this processing method is very delayed. If streaming is used for transmission, audio, video, animation, and other multimedia files will be played by a dedicated Streaming Media Server.
It is responsible for sending messages to the user in a continuous and real-time manner, so that the user does not have to wait until the entire file is completely downloaded, but it only takes several seconds to start the delay, when the multimedia data is played on the client
The remaining part will continue to be downloaded from the Streaming Media Server.

 

Streaming is a new concept that has emerged on the Internet in recent years. It is widely defined and is a general term for multimedia data transmission over the network. Streaming media includes both broad sense and narrow sense
In a broad sense, streaming media refers to a series of techniques, methods, and protocols that allow stable and continuous transmission and playback of audio and video streams, that is, streaming media technology; in a narrow sense, streaming media is relative to traditional
Download-Playback is a new method for obtaining audio, video, and other multimedia data from the Internet. It supports real-time transmission and playback of multimedia data streams. By Using Streaming Media
The server can send stable and continuous multimedia data streams to the client. When receiving data, the client can play back the data at a stable rate, instead of waiting until all the data is downloaded.

 

Due to network bandwidth, computer processing capabilities, and protocol specifications, to download a large amount of audio and video data from the Internet
The download time and storage space are not realistic, but the emergence of streaming media technology solves this problem well. Currently, streaming media can be transmitted in two ways: sequential streams.
Progressive streaming and realtime streaming are suitable for different applications.

 

Sequential stream transmission

 

Ordered stream transmission transmits data by means of sequential download. When downloading, you can play back multimedia data online. However, you can only view the downloaded data at a given time point. You cannot skip to the undownloaded data point, or
The download speed is adjusted according to the network conditions during transmission. Because the standard HTTP server can send this form of streaming media without the support of other special protocols, it is often called HTTP
Stream Transmission. Ordered stream transmission is suitable for high-quality multimedia clips, such as titles, credits, and advertisements.

 

Real-time stream transmission

 

Real-time stream transmission ensures that the media signal bandwidth can match the current network conditions, so that streaming media data is always transmitted in real time. Therefore, it is particularly suitable for on-site events. Real-time stream transmission supports random access, that is, user
You can view the content in front or back by performing fast forward or backward operations. Theoretically, Real-Time Streaming Media will not pause once played, but in fact it is still possible to pause periodically, especially in the Network
Even worse. Different from Sequential stream transmission, real-time stream transmission requires a specific Streaming Media Server and supports specific network protocols.

 

Ii. Streaming Media Protocol


Real
Real-time transport
Protocol (PRT) is a network protocol used to process multimedia data streams over the Internet. It can be used in one-to-one (unicast, unicast) or one-to-multiple
(Multicast) implements real-time transmission of streaming media data in the network environment. RTP usually uses UDP for multimedia data transmission, but you can use TCP or
The RTP protocol is composed of two closely related parts: the RTP data protocol and the RTP control protocol. Real Time Streaming
Protocol, RTSP) first by real
Networks and Netscape jointly propose that it is located on top of RTP and RTCP, and its purpose is to transmit multimedia data effectively through an IP network.

 

2.1 RTP data protocol

 

The RTP data protocol is used to package streaming media data and implement real-time transmission of media streams. Each RTP data packet consists of header and payload, the first 12 bytes in the header are fixed, while the load can be audio or video data. The Header Format of the RTP datagram is 1:

 

Figure 1 RTP Header Format

 

The important domains and their meanings are as follows:

 

• CSRC count (cc)
  Indicates the number of CSRC identifiers. The CSRC Mark follows the fixed RTP Header to indicate the number of RTP
  It is reported that the RTP protocol allows multiple data sources to exist in the same session, which can be combined into one data source through the RTP mixer. For example, a CSRC list can be generated to indicate a call
  This meeting combines the voice data of all speakers into an RTP data source through an RTP mixer.
• Load type (PT)
  Indicates the RTP load format, including the encoding algorithm, sampling frequency, and bearer channel used. For example, type 2 indicates that the RTP data packet carries voice data encoded using the ITU g.721 algorithm. The sampling frequency is 8000Hz and the single channel is used.
• Serial number
  It is used to provide a method for the receiver to detect data loss, but how to handle the lost data is the application's own business. The RTP protocol itself is not responsible for data retransmission.
• Timestamp
  The sampling time of the first byte in the load is recorded. the receiver can timestamp to determine whether the arrival of data is affected by latency jitter, but how to compensate for latency jitter is the application's own business.

From RTP
The format of the datagram is not hard to see. It contains the media type, format, serial number, timestamp, and whether there is additional data. This provides a foundation for Real-Time Streaming Media transmission. RTP protocol
The objective is to provide end-to-end transmission services for real-time data (such as interactive audio and video). Therefore, there is no connection concept in RTP, it can be based on underlying connection-oriented or non-connection-oriented transmission protocols
RTP does not depend on the special network address format, but only needs the underlying transmission protocol to support framing and segmentation. In addition, RTP
No reliability mechanism is provided, which must be ensured by the transmission protocol or application itself. In typical application scenarios, RTP
Generally, it is implemented as part of an application on top of the transport protocol, as shown in Figure 2:

 

Figure 2 Relationship Between RTP and various network protocols

 

2.2 RTCP Control Protocol

 

RTCP
The control protocol must be used together with the RTP data protocol. When an application starts an RTP session, both ports are occupied
And RTCP. RTP itself does not provide a reliable guarantee for data packets transmitted in sequence, nor does it provide traffic control and congestion control, which are completed by RTCP. Generally, RTCP uses
The same distribution mechanism of RTP sends control information cyclically to all members of a session. The application receives the data and obtains the relevant information of the session participant, as well as the network conditions and the probability of packet loss.
To control service quality or diagnose network conditions.

 

The functions of the RTCP protocol are implemented through different RTCP datagram, mainly including the following types:

 

• Sr
  The sender report refers to the application or terminal that sends the RTP datagram. The sender can also be the receiver.
• Rr
  The acceptor reports that an application or terminal receives but does not send RTP data packets.
• Sdes
  The source description mainly serves as a carrier for the identity information of session members, such as the user name, email address, and phone number. In addition, it also provides the ability to send session control information to session members.
• Bye
  The main function of the notification is to indicate that one or more sources are no longer valid, that is, other members in the notification session will quit the session.
• App
  Defined by applications, this solution solves the scalability of RTCP and provides great flexibility for Protocol Implementers.

RTCP datagram carries the necessary information of service quality monitoring, which can dynamically adjust service quality and effectively control network congestion. Because RTCP datagram adopts the multicast mode, all members in the session can use the control information returned by the RTCP datagram to understand the current situation of other participants.

 

In a typical application, the application that sends a media stream periodically generates the sender Report SR. The RTCP datagram contains
Synchronization information, as well as the sent datagram and byte count. Based on this information, the receiving end can estimate the actual data transmission rate. On the other hand, the acceptor sends the acceptor report to all known senders.
Rr. The RTCP datagram contains the maximum serial number of the received datagram, the number of lost datagram, latency jitter, timestamp, and other important information. Based on this information, the sender application can estimate the round-trip latency and
You can dynamically adjust the transmission rate based on the probability of data loss and latency Jitter to improve network congestion or smoothly adjust the service quality of applications based on network conditions.

 

2.3 RTSP real-time stream Protocol

 

As an application layer protocol, RTSP provides a scalable framework, which makes real-time streaming media data controlled and On-Demand Streaming possible. Total
RTSP is a streaming media protocol that is mainly used to control data transmission with real-time features. However, RTSP does not transmit data, but must rely on some services provided by lower-layer transmission protocols.
RTSP can provide streaming media operations such as playing, pausing, and fast forward. It defines specific control messages, operation methods, status codes, and other operations. It also describes the interaction with RTP.

 

RTSP
The HTTP/1.1 protocol is widely referenced during the preparation, and many descriptions are exactly the same as those of HTTP/1.1. RTSP uses the syntax and operations similar to HTTP/1.1.
To a large extent, it is designed to be compatible with the existing web infrastructure. For this reason, most HTTP/1.1 extensions can be directly introduced to RTSP.

 

A media stream set controlled by RTSP can be described in a representation (Presentation
Description). The so-called representation refers to the set of one or more media streams provided by the Streaming Media Server to the client. The description contains a message indicating each media stream.
Information, such as data encoding/decoding algorithms, network addresses, and media streams.

 

Although the RTSP server uses identifiers to differentiate each session, the RTSP connection is not bound to the transport layer.
Connection (such as TCP), that is, during the entire RTSP connection, the RTSP user can enable or disable multiple reliable transmission connections to the RTSP server to send RTSP
Request. In addition, RTSP connections can also be based on connectionless transmission protocols (such as UDP ).

 

The RTSP protocol currently supports the following operations:

 

• Retrieve Media
  Allows users to submit a description to the Media Server through HTTP or other methods. For example, if the description is multicast, the description includes the multicast address and port number used for the media stream. If the description is unicast, to ensure security, only the target address is provided in the description.
• Invite to join
  The Media Server can be invited to an ongoing meeting, play back the media in the presentation, or record all media or its subsets in the presentation, which is very suitable for distributed teaching.
• Add Media
  It is particularly useful for on-site lectures to notify users of new available media streams. Similar to HTTP/1.1, RTSP requests can also be handled by proxy, channel, or cache.

Iii. streaming media programming


RTP
It is currently the best solution to the problem of Real-Time Streaming Media transmission. If you need to program Real-Time Streaming Media on the Linux platform, you can consider using some open-source RTP libraries, such as librtp,
Jrtplib. Jrtplib is an object-oriented RTP Library, which fully complies with RFC
The 1889 design is a very good choice in many cases. The following uses jrtplib as an example to describe how to use the RTP protocol for real-time streaming media programming on the Linux platform.

 

3.1 Environment Construction

 

Jrtplib
Is an RTP Library implemented in C ++. Currently, it can run on Windows, Linux, FreeBSD,
Solaris, UNIX, and VxWorks. To install jrtplib for Linux, first go to the jrtplib website (http:
// Lumumba. Luc. ac. be/Jori/jrtplib/jrtplib.html) download the latest source code package. Here jrtplib-
2.7b.tar.bz2. If the downloaded source code package is saved in the/usr/local/src directory, run the following command to decompress it:

 

[root@linuxgam src]# bzip2 -dc jrtplib-2.7b.tar.bz2 | tar xvf -

Next we need to configure and compile jrtplib:

 

[root@linuxgam src]# cd jrtplib-2.7 [root@linuxgam jrtplib-2.7b]# ./configure [root@linuxgam jrtplib-2.7b]# make

Run the following command to install jrtplib:

 

[root@linuxgam jrtplib-2.7b]# make install

3.2 Initialization

 

Before using jrtplib for Real-Time Streaming Media data transmission, an rtpsession instance should be generated to represent the RTP
Session, and then call the CREATE () method to initialize it. The create () method of the rtpsession class has only one parameter to specify
The port number. Listing 1 provides a simple initialization framework, which completes the initialization of RTP sessions and does not have any practical functions.

 

Code List 1: Initial. cpp

#include "rtpsession.h" int main(void) { RTPSession sess; sess.Create(5000); return 0; }

If the RTP session fails to be created, the CREATE () method returns a negative number, although it can easily determine whether the function call is
But it is difficult to understand the cause of the failure. Jrtplib adopts a unified error handling mechanism. If all functions provided by jrtplib return negative numbers, it indicates that some form of error occurs.
And the specific error information can be called
The rtpgterrorstring () function is obtained. The rtpgterrorstring () function passes in the error code as a parameter, and then returns the error corresponding to the error code.
Information. Listing 2 provides a more complete initialization framework that can better handle errors generated during RTP Session Initialization:

 

Code List 2: Framework. cpp

#include <stdio.h> #include "rtpsession.h" int main(void) { RTPSession sess; int status; char* msg; sess.Create(6000); msg = RTPGetErrorString(status); printf("Error String: %s//n", msg); return 0; }

Setting an appropriate timestamp unit is another important task in the RTP Session Initialization Process. This is done by calling the rtpsession class.
Settimestampunit () method. This method also has only one parameter, indicating the timestamp unit in seconds. For example, when the RTP session is used to transmit 8000Hz
When sampling audio data, the time stamp increases by 8000 per second, so the time stamp unit should be set to 1/8000:

 

sess.SetTimestampUnit(1.0/8000.0);

3.3 Data Transmission

 

When RTP
After the session is successfully established, the streaming media data can be transmitted in real time. First, you must set the destination address for data transmission,
The RTP protocol allows the same session to have multiple destination addresses. You can call the adddestination (),
Deletedestination () and cleardestinations () methods. For example, the following statement indicates that the RTP session sends data to the local host.
Server Port 6000:

 

unsigned long addr = ntohl(inet_addr("127.0.0.1")); sess.AddDestination(addr, 6000);

After all the target addresses are specified, you can call the sendpacket () method of the rtpsession class to send streaming media data to all target addresses. Sendpacket () is an overload function provided by the rtpsession class. It has the following forms:

 

int SendPacket(void *data,int len) int SendPacket(void *data,int len,unsigned char pt,bool mark,unsigned long timestampinc) int SendPacket(void *data,int len,unsigned short hdrextID,void *hdrextdata,int numhdrextwords) int SendPacket(void *data,int len,unsigned char pt,bool mark,unsigned long timestampinc, unsigned short hdrextID,void *hdrextdata,int numhdrextwords)

The most typical use of sendpacket () is similar to the following statement. The first parameter indicates the data to be sent, and the second parameter indicates the length of the data to be sent, the following are the RTP load types, identifiers, and timestamp increments.

 

sess.SendPacket(buffer, 5, 0, false, 10);

For the same RTP Session, the load types, identifiers, and timestamp increments are usually the same, and jrtplib allows them to be set as the session's
Parameter. This is done by calling the setdefaultpayloadtype (), setdefaultmark (), and
Setdefatimetimestampincrement () method. Setting these default parameters for RTP sessions can simplify data transmission. For example, if
The default parameters are set for RTP sessions:

 

sess.SetDefaultPayloadType(0); sess.SetDefaultMark(false); sess.SetDefaultTimeStampIncrement(10);

Then, when sending data, you only need to specify the data to be sent and its length:

 

sess.SendPacket(buffer, 5);

3.4 Data Reception

 

For the receiver of streaming media data, you must first call the polldata () method of the rtpsession class to receive the RTP or
RTCP datagram. Since multiple participants (sources) are allowed in the same RTP session, you can call the gotofirstsource () and
The gotonextsource () method can traverse all sources. You can also call the gotofirstsourcewithdata () and
Gotonextsourcewithdata () method to traverse those sources with data. After a valid data source is detected in the RTP session, you can call
The getnextpacket () method of the rtpsession class extracts RTP datagram from it. When the received RTP datagram is processed, remember to release it in time. The following code
Demonstrate how to process received RTP data packets:

 

If (sess. gotofirstsourcewithdata ()){ Do { Rtppacket * pack; Pack = sess. getnextpacket (); // Process received data Delete pack; } While (sess. gotonextsourcewithdata ()); }

Jrtplib defines three receiving modes for RTP datagram, each of which specifies which RTP datagram will be accepted and which RTP datagram will be rejected. You can set the following receiving modes by calling the setreceivemode () method of the rtpsession class:

 

• Receivemode_all
  In the default receiving mode, all received RTP data packets are accepted;
• Receivemode_ignoresome
  Except for some specific senders, all incoming RTP datagram data will be accepted, and the list of rejected senders can be called by addtoignorelist (), deletefromignorelist (), and clearignorelist () method;
• Receivemode_acceptsome
  Except for some specific senders, all incoming RTP datagram data is rejected, and the list of accepted senders can be set by calling the addtoacceptlist (), deletefromacceptlist, and clearacceptlist () methods.

3.5 control information

 

Jrtplib
It is a highly encapsulated RTP Library. When using it, programmers often do not care about how RTCP datagram is sent and received, because these can be done by jrtplib. As long
If the polldata () or sendpacket () method is successfully called, jrtplib can automatically
The RTCP datagram is processed, and the RTCP datagram is sent as needed to ensure the correctness of the entire RTP session process.

 

On the other hand, by calling the setlocalname (),
Setlocalemail (),
Setlocallocation (), setlocalphone (), setlocaltool (), and setlocalnote () methods,
Jrtplib allows programmers to set the control information of RTP sessions. All these methods are called with two parameters. The first parameter is a char pointer pointing to
The second parameter is an int-type value, indicating the number of first characters in the data. For example, the following statement can be used to set the email address in the control information:

 

Sess. setlocalemail (" Xiaowp@linuxgam.comxiaowp @ linuxgam.com This email address is protected by spam bots and needs to be viewed using the JavaScript function. ", 19 );

During RTP sessions, not all control information needs to be sent.
Enablesendname (), enablesendemail (), enablesendlocation (), enablesendphone
(), Enablesendtool (), and enablesendnote () methods can be used to select the control information to be sent for the current RTP session.

 

3.6 practical application

 

Finally, we use a simple streaming media sending and receiving instance to introduce how to program Real-Time Streaming Media Using jrtplib. Listing 3 shows the complete code of the Data sender, which is used to send RTP packets continuously to the IP address and port specified by the user:

 

Code List 3: sender. cpp

# Include <stdio. h> # Include <string. h> # Include "rtpsession. H" // Error handling function Void checkerror (int err) { If (ERR <0 ){ Char * errstr = rtpgeterrorstring (ERR ); Printf ("error: % S // n", errstr ); Exit (-1 ); } } Int main (INT argc, char ** argv) { Rtpsession sess; Unsigned long destip; Int destport; Int portbase = 6000; Int status, index; Char buffer [128]; If (argc! = 3 ){ Printf ("Usage:./sender destip destport // n "); Return-1; } // Obtain the IP address and port number of the acceptor. Destip = inet_addr (argv [1]); If (destip = inaddr_none ){ Printf ("Bad IP address specified. // n "); Return-1; } Destip = ntohl (destip ); Destport = atoi (argv [2]); // Create an RTP session Status = sess. Create (portbase ); Checkerror (Status ); // Specify the RTP data receiving end Status = sess. adddestination (destip, destport ); Checkerror (Status ); // Set the default parameters of the RTP session Sess. setdefapaypayloadtype (0 ); Sess. setdefamark mark (false ); Sess. setdefatimetimestampincrement (10 ); // Send streaming media data Index = 1; Do { Sprintf (buffer, "% d: RTP packet", index ++ ); Sess. sendpacket (buffer, strlen (buffer )); Printf ("Send packet! // N "); } While (1 ); Return 0; }

Listing 4 shows the complete code of the data receiver, which is responsible for continuously reading RTP data packets from the specified port: Code list 4:


Receiver. cpp

# Include <stdio. h> # Include "rtpsession. H" # Include "rtppacket. H" // Error handling function Void checkerror (int err) { If (ERR <0 ){ Char * errstr = rtpgeterrorstring (ERR ); Printf ("error: % S // n", errstr ); Exit (-1 ); } } Int main (INT argc, char ** argv) { Rtpsession sess; Int localport; Int status; If (argc! = 2 ){ Printf ("Usage:./sender localport // n "); Return-1; } // Obtain the user-specified port number Localport = atoi (argv [1]); // Create an RTP session Status = sess. Create (localport ); Checkerror (Status ); Do { // Accept RTP data Status = sess. polldata (); // Retrieve the RTP data source If (sess. gotofirstsourcewithdata ()){ Do { Rtppacket * packet; // Obtain the RTP Datagram While (packet = sess. getnextpacket ())! = NULL ){ Printf ("got packet! // N "); // Delete the RTP Datagram Delete packet; } } While (sess. gotonextsourcewithdata ()); } } While (1 ); Return 0; }

Download the source code of this Article

Iv. Summary


With
The role of multimedia data on the Internet is becoming more and more important. More and more scenarios require real-time transmission of audio, video, and other multimedia data, such as IP phones, on-demand videos, and online
Meetings. RTP is a protocol used for Real-Time Streaming Media transmission over the Internet. It has been widely used in various occasions. jrtplib is an object-oriented RTP encapsulation.
Library, which can be used to conveniently complete real-time streaming media programming on the Linux platform.

 

References

 

• 1. Website http://lumumba.luc.ac.be/jori/jrtplib/jrtplib.html at jrtplib
  You can download the latest jrtplib source code package and find resources related to RTP.
• 2. Edited by Gu shuzhen, an example of broadband value-added service development, Beijing: Machinery Industry Press, 2002
• 3. Edited by Huang Yongfeng, multimedia communication technology for IP networks, Beijing: People's post and telecommunications Press, 2003