Introduction to the WEBRTC audio processing process

This article provides an overview of the WEBRTC audio processing flow, as shown in the following diagram:

WebRTC an audio session is abstracted into a channel, such as A and b for audio calls, a needs to establish a channel and b for audio data transmission. The figure above has three channel, each channel contains codec and real-time Transport protocol (real-time Transport Protocol,RTP)/ The real-time Transport Control Protocol (real-time control PROTOCOL,RTCP) sends the function .

In the case of a channel, the application will contain three active threads, a recording thread, an audio receive thread, and a playback thread .

1) Recording thread : responsible for the microphone audio collection, see the Red path in the picture, after the acquisition of audio, cache to a certain length, audio processing, mainly including aEC,AGC and NS . Then sent to the channel, after the audio codec module encoding, encapsulated into RTP packets, sent through the socket;

2) receive thread : see Blue Path, responsible for receiving the remote sent over the audio packet, unpack the RTP packet, decode the audio data, fed into the Neteq module cache.

3) Play thread : responsible for headphone sound playback, see green Path. The playback thread goes to the outmixer to get the audio data to play, first obtains the audio frame Neteq stored in the channel that participates in the session, can do the AGC and NS processing, then mixes the audio signal of multiple channel, obtains the mixed audio, Passed to the Audioprocessing module for remote analysis. Finally play it out.

The following is a local loopback recording and playback code:

[CPP]   View plain  copy voiceengine* ve = voiceengine::create ();    Voebase* base = voebase::getinterface (VE);   base->init ();   Int chid = base->createchannel ();   base->setsenddestination (chId,3000, " 127.0.0.1 ", 4000);   Base->setlocalreceiver (chid,3000,3001," 127.0.0.1 ");   base-> Startplayout (chId);   base->startreceive (chId);   base->startsend (chId);              //....sleep...wait.....      Base->stopsend (chId);      base->stopreveive (chId);      base-> Stopplayout (chId);      base->terminate ();