WEBRTC Study (ii): The Opensles of Audio_device

The Audio_device is a WEBRTC audio device module. Encapsulates audio device-related code for each platform

Audio device encapsulates two sets of sound code in Android.

1. Use JNI to invoke Java's media.

2. Operate directly through the native C interface of the OpenSL es.

The native interface is naturally more efficient, but the downside is that OpenSL requires Android 2.3+.

OpenSL ES (Open sound Library for Embedded systems) is a hardware audio acceleration API optimized for embedded systems with no licensing fees, cross-platform

OpenSL very little information, Google once again, also found two articles a little use.

OpenSL ES for Android is native-audio in the NDK samples. Lock-free Audio IO with OpenSL ES on Android another OPENSL application.

An example of OpenSL es is provided in WEBRTC's example. OPENSL_LOOPBAKC (opensldemo-debug.apk) is used to demonstrate the use of OPENSL (echo sound).

It took a little time to analyze the whole process, because it couldn't be debugged, so it looked annoying. The thread processing place adds the log to understand.

There are several main categories:

1. Audiodevicebuffer

Cache class, method Registeraudiocallback. Use callback to inform data acquisition (record), or request data (playout).

2. Openslesinput

The implementation of the record.

3. Openslesoutput

The implementation of playout.

4. Singlerwfifo

Implements a lock-free queue.

The process of playback:

1. Create Openslesoutput and Attachaudiobuffer, initialize OPENSL related information (engine, outmix, etc.). Initializes the required playback cache.

2. In Startplayout, create the OPENSL audio player to register the player cache playback callback. and enqueue for all the playback caches, and then create the audio data processing thread Cbthreadimpl

Description: Audio data enqueue to player. Will be played back, and the player will callback the registered callback after each playback is complete.

3. The wake of the Cbthreadimpl is controlled by Event_. There are two states of Kunderrun and Knounderrun. Kunderrun indicates that the audio data is below the expected value. Knounderrun indicates that the audio data is normal.

This is the case when the callback (Playersimplebufferqueuecallbackhandler) callback is processed.

Wake Cbthreadimpl with Kunderrun when no data in the Fifo_ needs to be played.

When there is data in the Fifo_, the audio data is enqueue into the player. Wake Cbthreadimpl with Knounderrun

voidopenslesoutput::P layersimplebufferqueuecallbackhandler (SLANDROIDSIMPLEBUFFERQUEUEITF sles_player_sbq_itf) { /c1>if(Fifo_->size () <=0|| Number_underruns_ >0) {    ++Number_underruns_; Event_.    Signalevent (Kunderrun, number_underruns_); return; } int8_t* Audio = Fifo_->Pop (); if( Audio) opensl_return_on_failure (*SLES_PLAYER_SBQ_ITF)Enqueue (SLES_PLAYER_SBQ_ITF, audio, buffer_s  Ize_bytes_), Void_return); Event_. Signalevent (Knounderrun,0);}

4. When the Cbthreadimpl is awakened. If it is Kunderrun, the player restarts. and re-enqueue all the playback caches.

opensl_return_on_failure (      *sles_player_itf_), Setplaystate (sles_player_itf_,                                        Sl_ playstate_stopped),      true);  Enqueueallbuffers ();  Opensl_return_on_failure (      *sles_player_itf_), Setplaystate (sles_player_itf_,                                        Sl_ playstate_playing),      true);

If it is knounderrun, it starts processing.

 while (Fifo_->size () < num_fifo_buffers_needed_ && playing_) {    int8_t* Audio = Play_buf_[active_ Queue_]. Get ();    Fine_buffer_-getbufferdata (audio);    Fifo_,Push (audio);     1)% totalbuffersused ();  }

Fine_buffer_ 's getbufferdata automatically handles 10ms of data. If the data is insufficient, request data from the callback–> Needmoreplaydata of audio buffer. If too much data is stored in the cache.

Fifo_ the acquired data into the stack. When the size of the fifo_ equals Num_fifo_buffers_needed_ (the number of pre-allocated play caches), Cbthreadimpl stops processing and waits for the next wakeup.

5.

The thread created by Threadwrapper::create in WEBRTC. The processing code for start is this.

this);

Startthread's Code:

BOOL true ;   BOOL true ;    while (Alive) {    = run_function_ (obj_);    Criticalsectionscoped CS (crit_state_);     if (! run) {      false;    }     = alive_;  }

Run_function_ is the function that is passed in when create.

So OpenSL's cbthreadimpl processing is constantly being called. This is something I was very puzzled about (I didn't know that Cbthreadimpl would have been called until I looked at the Threadwrapper code).

The recording process will not be discussed. There's no difference.

OPENSL Demo is Fakeaudiodevicebuffer inherited the Audiodevicebuffer, in Getplayoutdata the record of the data delivered to playout. Rather than through external callback.

WEBRTC Study (ii): The Opensles of Audio_device