Android IOS WebRTC Audio and Video development Summary (57)--a QoS scheme on network transmission

This article mainly introduces a QoS solution, the article from the blog Park Rtc.blacker, welcome to the public number blacker, more details on www.rtc.help

Background of the advent of QoS:

And when the network congestion, all the traffic can be discarded, in order to meet the different application of different service quality requirements, the network can be based on the requirements of users to allocate and dispatch resources, different data streams to provide different quality of service:

1, the real-time strong and important data report priority of the text;

2, for the real-time non-strong ordinary data packets, to provide a lower processing priority, network congestion even discarded.

To meet the above requirements, QoS arises, defined as follows:

QoS (Quality of service) refers to the ability of a network to use a variety of basic technologies to provide better service for a given network communication, a security mechanism for the network, and a technique for solving problems such as network latency and congestion. Therefore, when the network is overloaded or congested, QoS ensures that critical traffic is not delayed or discarded, while ensuring efficient operation of the network.

A QoS-enabled device that provides transport quality services; For a class of data flows, it can be given a level of transport priority to identify its relative importance, and use the various priority forwarding policies provided by the device, congestion avoidance, and other mechanisms to provide special transport services for these traffic. The network environment with QoS is configured, which increases the predictability of network performance, and can effectively allocate network bandwidth and make use of network resources more rationally.

In the RTC development we understand that the actual network situation is very complex, if necessary to ensure that the communication quality, and real-time audio and video communications including acquisition, encoding, network transmission, decoding, playback and other links, where the acquisition, codec and playback is not affected by network conditions, Only limited by the codec algorithm and playback strategy and other factors. However, the impact of packet loss, jitter and chaos on QoS is the most significant, so the QoS solution described below addresses the poor impact of packet loss, jitter, and chaos on the quality of service, as follows:

1, the transmission end principle:

For real-time audio and video communication, the UDP protocol is used to transmit multimedia data, and the following is a UDP-based RTP protocol for transmitting audio and video data. For different formats of encoded data, there will be different RTP packaging protocols, such as for the H. V video data, document rfc3984 to NAL u RTP Packaging specification, please refer to this document for details. For the packaging of video data, because the length of the video data may be greater than the MTU, the relevant packaging protocol will stipulate that the length of the frame is greater than the MTU is cut, chunked encapsulated in multiple RTP packets for transmission. In order to avoid the impact of packet loss, jitter and chaos on the quality of service, the scheme establishes a cyclic buffer of equal number of nodes on both the sending and receiving sides, which is used to cache the sending data and the receiving end data.

When sending the data, the SEQ of one RTP packet is SEND_SEQ, and the sending side sends the packet through the UDP socket, which

The data of the RTP packet is copied to the buffer of the send_seq corresponding node, so that the sender can resend the RTP packet when the RTP packet receiver is confiscated.

One thing to note here is that the number of cyclic buffer nodes at the sending and receiving ends is divisible by 65536, so that the RTP SEQ increases to a maximum of 65535

The last node, the SEQ of the next RTP packet, corresponds to the first node at 0, preventing a vulnerability when RTP seq turns around.

2, the receiving end principle:

Similar to the sending side, the receiver also opens a loop buffer that can be divisible by 65536 for the cache of the received RTP packets. The receiving end receives the RTP

Packet, you need to parse the RTP header, take out the received RTP packet seq, corresponding to the RECEIVED_SEQ.

When the first packet is received, both START_SEQ and End_seq are set to Received_seq and the received RTP packets are sent to the decoding unit. There are two things to do when you receive the RTP packet later:

First, the receiving module copies the received RTP packets to the buffer of the node pointed to by Received_seq, and sets the data flag for this node (to mark whether the node is populated with data) to true, and according to Start_seq, End_ Seq and received_seq Relationship to determine whether to update End_seq to RECEIVED_SEQ value, if received_seq corresponding package should be end_seq before the corresponding package arrives, the value of End_seq is not updated, otherwise it is updated.

Second, every once in a while to scan Start_seq to end_seq corresponding to each node, first, if the current time and start_seq corresponding data arrival time difference exceeds a certain threshold (such as 500ms), will Start_seq and End_ The data for each node between SEQ is discarded, the data flag for each node is set to False, and the update start_seq is END_SEQ. Second, if the data falg of the next node of the START_SEQ node is true, the data of the node is sent to the decoding unit, and the START_SEQ is updated to the SEQ of that node, and the data flag of the node is set to false; If flag is false, And the current time and start_seq corresponding data arrival time difference exceeds a certain threshold (such as 50ms), then the SEQ (lost_seq) of the node is sent to the sending side, the request sender will be the SEQ corresponding RTP data again.

In this way, when some packages for a long time (greater than 500ms) did not receive, it is thought that it can not come, directly discard them, some packets short (less than 50ms) did not come, send a retransmission request to the sending side, the request sent to send the packet again, trying to fill up these packages.

3. Result description

3.1, without the QoS module, two mobile phone video communication in the case of loss of the video frame is not complete, play the mosaic phenomenon, coupled with the QoS module, video playback smooth, the effect greatly improved. At the same time, in order to test the function of the program, the sender of the 10% and 20% respectively discarded video RTP packets, the receiver side decoding good performance, no mosaic phenomenon.

3.2, adding the QoS module will bring a certain delay and lag, because packet retransmission is time-required.

3.3, the above plan is WEBRTC inside the nack concrete realization way.

The above scheme is provided by Peng Zuyuan, a senior audio and video expert from the ring, with some adjustments, and Kelly for editing and finishing.

Peng has many years of audio and video codec development experience, in Android,ios and other platform audio and video collection, coding, transmission, decoding, playback and other aspects have a wealth of experience, familiar with the development of streaming media server.

Android IOS WebRTC Audio and Video development Summary (57)--a QoS scheme on network transmission