Implementation of UDP-based audio jitter Buffer

1. Audio package structure:

The key member variables are as follows, and their fields are self-explanatory and need not be further described:

Public var lastseq: uint;

Public var from: uint;

Public var Sid: uint;

Public var seq: uint;

Public var time_stamp: uint;

Public var fec_id: uint;

Public var type: uint;

Public var voice_type: uint;

Public var payload: bytearray;

Public var wavbytes: bytearray;

Public var isbroadcast: Boolean;

Public var playtime: uint;

Public var flvbytes: bytearray;

Public var resend: Boolean;

2. Storage Structure of the audio package:

Two-way linked list is used for storage. Each node corresponds to an audio package. The two-way linked list has the chain table size attribute and the header node head. each time an audio package comes in, it is inserted into the appropriate position in the two-way linked list (after considering packet loss, out-of-order, delay, determine the location of the audio package ), the linked list is sorted by the serial number of the audio package. The serial number of the head is the smallest.

3. Add the audio package to the two-way linked list:

Return Value description:

Res = 0; it indicates that it is a normal audio package received in ascending order.

Res = 1; indicates that the audio package is repeatedly received.

Res = 2; packet loss occurs.

Res = 3; it indicates that the received audio package is late.

Perform corresponding processing based on different returned values:

Res = 0 or res = 3, directly flush () to determine the number of audio packets to play

Res = 1. Duplicate packets are discarded directly.

Res = 2. When packet loss occurs, all packet loss sequences are retransmitted.

4. Flush function:

(1) The sequence number of the last playback audio package and the head node of the current linked list is continuous,

That is, lastseq + 2 = head. Data. seq:

From the current head node, find a continuous audio package, which can be a maximum of the length packages of the linked list,

Set the number of audio packages found to flushcnt.

(2) The sequence number of the last audio package and the current head node is not consecutive,

That is, lastseq + 2

If the total playback time of all audio packets in the linked list is greater than the maximum playback time,

That is, max_play_time, call flushlater (0, buftime). For the function, see the description of this function.

5. playaudio functions:

Play the flushcnt audio packets identified in Step 4

(1) If the serial number of the last audio package and the head node of the current linked list is not consecutive, packet loss is required.

Compensation. If the number of packet loss is 1, that is, lastseq + 4 = head. data. seq, then play back the audio package that was last played: specifically, if the serial number is: 2, 4, 6, 10, 12 ,...

The last playback ends with the serial number 6. This time, the audio package with the serial number 8 will be lost from the serial number 10. Therefore, the serial number 6 will be played again before the serial number 10; that is, the playback sequence is:

2, 4, 6, 6, 10, 12 ,...

If the number of packet loss is greater than or equal to 2, that is, lastseq + 4

2, 4, 6, 12, 14 ,... the last playback ends with the serial number 6, but this time it starts from the serial number 12. Two audio packages with the serial number 8 and 10 are lost. Therefore, you must first play the serial number 6 and the serial number 12, then play the serial number 12, and then play the serial number after it, that is, the playing sequence is: 2, 4, 6, 12, 12, 14 ,...

For example, the implementation of packet loss compensation is as follows:

If (_ lastaudioplayres ){

VaR seqlen: uint = voiceres. seq-_ lastaudioplayres. seq;

If (seqlen = 4 ){

_ Ns. appendbytes (_ lastaudioplayres. flvbytes );

} Else if (seqlen> 4 ){

_ Ns. appendbytes (_ lastaudioplayres. flvbytes );

_ Ns. appendbytes (voiceres. flvbytes );

Appendbytes (voiceres. flvbytes );

}

}

6. flushlater function:

When the return value of the flush function is 0, it indicates that the flush function does not find a small audio package to be played. At this time, if the total playback time of the audio package in the linked list is greater than max_play_time, the parameter passed to flushlater () is minplaytime = 0, _ audiobuftimemax, indicating that you only need to find an audio package to play the video, so that the audio package is not found in every flush.

Otherwise, if the buffer time of the audio package in the linked list in netstream is less than audio_flush_time, And the netstream playback buffer is not full, the minimum playback time is calculated. minplaytime = uint (audio_buf_time_max-buflen) * 1000), and then find several audio packages in the linked list so that the total playback time (including the playback time of the lost package) is greater than the minplaytime.

Specifically, if the audio package sequence is: 2, 4, 6, 10, 14, 16, 18...

Last played until the serial number is 6, and this time when flush is called, due to lastseq + 2

7. Smooth RTT calculation. The reference formula is as follows:

VaR smoothrtt: uint = (N * _ smoothrttsum/_ rttcnt + currtt)/m;

VaR deltartt: Int = currtt-smoothrtt;

VaR smoothdeltartt: Int = (N * _ smoothrttdeltasum/_ rttcnt + deltartt)/m;

VaR RTT: Int = smoothrtt + smoothdeltartt;

Note: The value of N and M is worth the experience value based on the actual situation. The above max_play_time and audio_buf_time_max must also obtain the experience value.

 

Reprinted please indicate the source: zhujian blog, http://blog.csdn.net/linyanwen99/article/details/8985656