RTP/RTCP/RTSP material essence!

7.3 Streaming Media Technology
Streaming media refers to the continuous, time-based media transmitted using streaming (sreaming) transmission technology in the network, such as audio or video data streams, rather than a new media. Streaming Media Technology (or stream media technology) compresses continuous image and sound information and puts it on a network server, allowing viewers to download and listen to it, it does not need to wait until the entire multimedia file is downloaded. Streaming Media Technology is not a single technology. It is a technology that integrates multiple technologies such as stream media data collection, compression, storage, transmission, and network communication.
Streaming media has brought huge changes to the Internet. For users, there is almost no difference between watching streaming media files and viewing traditional audio and video files. The only difference is that in terms of audio and video quality, streaming media uses a high compression ratio to solve bandwidth problems and shorten download time, so users cannot feel high image and sound quality.

1. Stream Transmission

Streaming Transmission is widely defined. It is a general term for the technology that transfers media (such as videos and audios) over the network. Its specific meaning is to transmit video programs to PCs over the Internet. There are two methods to implement stream transmission: realtime streaming and progressive streaming ).

1. Sequential stream transmission

Ordered stream transmission is a sequential download. When downloading an object, you can watch the record media. at a given time point, you can only watch the downloaded part, instead of jumping to the first part that has not been downloaded, unlike real-time stream transmission, Sequential stream transmission is not adjusted according to the speed of user connection during transmission. Because the standard HTTP server can send files in this form and does not need other special protocols, it is often called HTTP stream transmission. Ordered stream transmission is suitable for high-quality short video segments, such as titles, credits, and advertisements. This method ensures the final quality of video playback because the content to be viewed before playback is not downloaded. This means that users must experience latency before watching, especially for slow connections.

For short video segments published through the modem, Sequential stream transmission is very useful, which allows you to create video clips at a higher data rate than the modem. Despite the latency, you can publish high-quality video clips.
Ordered streaming files are stored on standard HTTP or FTP servers, which are easy to manage and are basically irrelevant to firewalls. Sequential stream transmission does not fit long clips and videos with random access requirements, such as lectures, presentations, and demos. It also does not support on-site broadcast. Strictly speaking, it is an on-demand broadcast technology.

2. Real-time stream transmission

Ii. Technical Principles of streaming media
Multimedia data must be pre-processed to be suitable for stream transmission, because the current network bandwidth is far from enough for the huge data stream of multimedia. Preprocessing mainly includes two aspects: first, quality reduction; second, advanced and efficient compression algorithms.

Basic Principles

The process of stream transmission is generally as follows:

(1) After selecting a first-class media service, the web browser and the Web server use HTTP/tcp to exchange control information, in order to retrieve the real-time audio/video stream data to be transmitted from the original information.

(2) The web browser on the client starts the Client Program (some playing programs) and uses http to retrieve relevant parameters from the Web server to initialize the client program. These parameters may include the directory information, the encoding type of audio/video data, or the server address related to A/V retrieval.

(3) The client program and the Streaming Media Server run the real-time stream Control Protocol (RTSP) to exchange the control information required for audio/video data stream transmission. RTSP acts as a remote control for the client to remotely control the Streaming Media Server and control the playback, fast forward, fast backward, pause, and other commands of the media data stream.

(4) The streaming media server uses the RTP/UDP protocol to transmit audio/video data to the client. Once audio/video data arrives at the client, the audio/video client program can play the output.

It should be noted that in stream transmission, two different communication protocols, RTP/udp and RTSP/tcp, are used to establish a connection with the Streaming Media Server, this is to redirect the server output to a destination address different from the client where the client program is running. Dedicated servers and players are generally required for stream transmission.

Iii. network protocols supporting streaming media transmission
1. Real-time transmission protocol RTP and RTCP

RTP (Real-timetransportprotocol) is a transmission protocol for multimedia data streams on the Internet. RTP is defined to work during one-to-one or one-to-many transmission. It aims to provide time information and implement stream synchronization. RTP usually uses UDP to transmit data, but RTP can also work on other protocols such as TCP or ATM. When an application starts a RTP session, two ports are used: one for RTP and the other for RTCP. RTP itself does not provide a reliable transmission mechanism for transmitting data packets in sequence, nor does it provide traffic control or congestion control. It relies on RTCP to provide these services. Generally, the RTP algorithm is not implemented as an independent network layer, but as part of the application code.
Real-time transmission control protocol RTCP. RTCP (Real-timetransportcontrolprotocol) and RTP provide traffic control and congestion control services. During the RTP session, each participant periodically transmits the RTCP package. The RTCP package contains statistics such as the number of sent data packets and the number of lost data packets. Therefore, the server can use this information to dynamically change the transmission rate or even the payload type. RTP and RTCP can be used together to optimize transmission efficiency with effective feedback and minimal overhead. Therefore, RTP is particularly suitable for transmitting real-time online data.

RTCP performs the following four functions:
It mainly provides quality feedback on data publishing. It is part of the RTP transmission protocol and is related to the stream and blocking control of other transmission protocols. Feedback directly applies to adaptive encoding control, but the experience of IP multicast shows that it is important to diagnose sending errors when feedback is received from the sender. Send a receiving feedback report to all participants to allow the problem observer to estimate whether the problem is local or global. Publishing mechanisms such as IP multicast allow network service providers and groups to receive feedback and act as third-party monitors to diagnose network problems. The feedback function is reported by the RTCP sender and receiver.

RTCP carries the RTP source persistent transport layer identifier known as the canonical name (cname. In the event of a conflict or Program Restart, since the SSRC identifier can be changed, the recipient needs a cname to track the participants. The receiver also needs to connect the cname with several data streams specified in the relevant RTP connection.

The first two features require all participants to send RTCP packets. Therefore, the rate must be controlled to scale RTP To a large number. Each participant is allowed to send a control package to other participants to observe the number of participants independently. This number of terms calculate the packet sending rate.

The fourth optional function is to transmit the minimum connection control information, such as participant identification. It is most likely to be used for \ "loose control \" connections where participants enter or leave freely without member control or parameter coordination. RTCP serves as a convenient channel to all participants, however, it does not need to support all control communication requirements of applications.

2. Real-time stream protocol RTSP

Real-time stream protocol RTSP (realtimestreamingprotocol) is jointly proposed by RealNetworks and Netscape. This Protocol defines how one-to-multiple applications can effectively transmit multimedia data over an IP network. RTSP is located on RTP and RTCP in the architecture. It uses TCP or RTP for data transmission. Compared with RTSP, HTTP transmits HTML, while RTP transmits multimedia data. HTTP requests are sent by the client and the server responds. When RTSP is used, both the client and server can send requests, that is, RTSP can be bidirectional.

3. Resource Reservation Protocol (RSVP)

Because audio and video data streams are more sensitive to network latency than traditional data streams, high-quality audio and video information must be transmitted over the network. In addition to bandwidth requirements, more conditions are required. RSVP (resourcereserveprotocol) is a Resource Reservation Protocol under development on the Internet. It uses RSVP to reserve some network resources (that is, bandwidth), which can provide QoS for streaming media transmission to a certain extent. RSVP is integrated into some experimental systems, such as the network video conferencing tool Vic.

Iv. Composition of the streaming media system

A basic streaming media system must contain three modules: encoder, server, and player. Modules communicate with each other through specific protocols and exchange file data in specific formats. The encoder is used to convert the original audio/video into a suitable stream format file. The server sends the encoded media stream to the client, and the client player decodes and plays the received media data.

1. Encoder

The encoder function is to compress and encode the input original audio and video signals. Different streaming media businesses have different performance requirements for encoders. Currently commonly used video encoding solutions include MPEG-4, H.264 and Microsoft Company Windows Media video using AC-1; Audio Encoding solutions include MP3, MPEG-2, AAC, AMR and AMR-WB. The bitstream generated by the Multimedia Encoder only contains the information necessary for decoding the bitstream, and does not contain system information such as inter-media synchronization and random access, therefore, the encoded multimedia data must be organized into a streaming media file format for transmission or storage.

2. Streaming Media Server

The following three methods are generally used for streaming media playback:

1. unicast

A separate data channel must be established between the client and the Media Server. Each packet sent from a server can only be sent to one client. This transmission method is called unicast. Each user must send a separate query to the Media Server, and the Media Server must send the requested data packet copy to each user. This huge redundancy first causes a heavy load on the server, and the response takes a long time or even stops playing the video. Managers are also forced to purchase hardware and bandwidth to ensure certain service quality.

2. Multicast

The IP multicast technology is used to build a network with multicast capabilities, allowing the router to copy data packets to multiple channels at a time. Using multicast, a single server can send continuous data streams to hundreds of thousands of clients without delay. The Media Server only needs to send one information package, rather than multiple. All clients sending requests share the same information package. Information can be sent to clients of any address to reduce the total amount of information packets transmitted over the network. The network utilization efficiency is greatly improved, and the cost is greatly reduced.

3. On-Demand and broadcast

Vod is an active connection between the client and the server. In the VOD connection, you can select a content item to initialize the client connection. You can start, stop, back, fast forward, or pause a stream. The VOD Connection provides the maximum convection control, but this method quickly uses up network bandwidth because each client connects to the server.

Broadcast refers to the passive receiving of streams by users. In the broadcast process, the client receives streams, but the control flow is not allowed. For example, you cannot pause, fast forward, or backward the stream. A separate copy of data packets in broadcast mode will be sent to all users on the network. When sending unicast data packets, You need to copy multiple copies of the data packets and send them to the users who need them in multiple point-to-point ways. Instead, the data packets are sent in broadcast mode, A separate copy of the data packet will be sent to all users on the network, regardless of whether the user needs it or not, the above two transmission modes will waste network bandwidth. Multicast absorbs the strengths of the above two sending methods, overcomes the weaknesses of the above two sending methods, and separately copies the data packets to the customers who need them. Multicast does not copy multiple copies of data packets to the network, nor send data packets to customers who do not need them. This ensures that multimedia applications on the network occupy the minimum bandwidth of the network.

3. Client player

After the audio/video RTP data packets are transmitted to the client over the network, they enter a Buffer Queue for waiting. All the data packets in the buffer queue are sorted by the serial number of the packet header. If there is a late packet, re-insert the correct position according to the serial number, so as to avoid disorder.
Each time the client reads a frame of data from the queue header, the playback time of the frame is obtained from the packet header time mark, and the audio/video is synchronized. The synchronized data is sent to the decoder for decoding, And the decoded data is sent to a buffer for loop reading. Once the frame playback time reaches, the decoded data is retrieved from the buffer and sent to the playback module for display or playback.

V. Streaming Media File Format
The streaming media file format supports stream transmission and playback. The stream file format is specially encoded so that it can be played while downloading on the network, rather than waiting until the entire file is downloaded. To encode a compressed media file into a streaming file, you must consider additional information, such as timing, compression, and copyright information. Encoding Process

　　

Common stream file formats file extensions media types company client players
ASF Advanced Streaming Format Microsoft Windows meadia player
WMV windows mesia video Microsoft Windows meadia player
WMA windows mesia audio Microsoft Windows meadia player
Rm (rmvb) real video/audio Real Networks Real Player
RA Real Audio Real Networks Real Player
RP real pix Real Networks Real Player
RT real text Real Networks Real Player
SWF Shockwave Flash Macromedia Shockwave Plug-in
Qt QuickTime Apple QuickTime player

Vi. Streaming Media Applications

1. Online live broadcast
With the popularity of the Internet, documents on the Internet are no longer limited to text and graphics. Many video applications need to be broadcast online, such as live broadcast of the World Cup and Spring Festival Gala. For TV stations, the use of streaming media technology to achieve online live broadcasting can cover the audience to the maximum extent, and can be roughly publicized, advertised, or meet the needs of viewers like live TV.

2. VOD
With the development of multimedia technology, communication technology and hardware storage technology, people no longer meet the previous single and passive single-direction information acquisition methods. Video on Demand (VOD) interactive services using streaming media technology are becoming popular. Nowadays, many online cinemas on the Internet basically use RealSystem of RealNetwork or Windows meadia System of Microsoft.

3. Distance Education
Compared with traditional school education, the distance education system breaks through the restrictions of time and space, increases the learning opportunities, and is conducive to expanding the education scale, improving the teaching quality, and reducing the teaching cost. Learners can use a wide range of teaching resources to teach at their desired speed and in their own convenient time and place. At present, many universities have adopted streaming media technology to achieve distance education.

In addition, streaming media technology has been successfully applied in many aspects such as e-commerce, telemedicine, and video conferencing. Currently, streaming media technology is mainly used in both broadband and narrowband modes. Narrowband methods include multimedia live broadcasting, remote teaching, e-learing, stock evaluation analysis, and video conferencing. Broadband methods include network TV, KTV, enterprise training, and multimedia IDC.