Android OpenSL ES Development: Android OPENSL Introduction and development process description

First, Android OpenSL ES Introduction

OpenSL ES (Open sound Library for Embedded systems) is a hardware audio acceleration API that is optimized for embedded systems with no licensing fees, cross-platform. It provides a standardized, high-performance, low-response-time audio feature implementation method for native application developers on embedded mobile multimedia devices, and enables direct cross-platform deployment of soft/hardware audio performance, reducing execution difficulty and facilitating the development of advanced audio markets. In short, OpenSL es is an embedded cross-platform, free audio processing library.

Android OpenSL ES Library is the corresponding CPU type in the NDK's platforms folder corresponding to the Android platform first, such as:

Second, Android OpenSL ES development process

The development process of OPENSL ES consists of the following 6 steps:

1. Create an Interface object

2. Setting mixer

3. Create player (recorder)

4. Set buffer queue and callback function

5. Set Playback status

6. Start callback function

Of these, 4 and 6 are used to play the audio of data formats such as PCM.

2.1 OpenSL ES develops the most important interface class SLOBJECTITF

Through the SLOBJECTITF interface class we can create the various types of class interfaces we need, such as:

• Create engine Interface object: SLOBJECTITF engineobject
• Create Mixer Interface object: SLOBJECTITF outputmixobject
• To create a player interface object: SLOBJECTITF playerobject

The above and so on are all created through SLOBJECTITF.

2.2 Using SLOBJECTITF to create an instance of a specific interface object

OpenSL ES also has specific interface classes, such as (Engine: SLENGINEITF, Player: SLPLAYITF, voice controller: SLVOLUMEITF, etc.).

2.3 Creating the engine and implementing

The first step in OpenSL ES is to declare the engine interface object Engineobject of the SLOBJECTITF interface type, and then create an engine interface object with the method Slcreateengine, and when the engine interface object is created, It is necessary to implement engineobject with the realize method of SLOBJECTITF, and finally initialize the GetInterface object instance with the Slengnineitf method of SLOBJECTITF. Such as:

SLOBJECTITF engineobject = NULL;//declaring engine interface objects with SLOBJECTITFSLENGINEITF engineengine = NULL;//declaring a specific instance of an engine object voidCreateengine () {slresult result;//return Resultsresult = Slcreateengine (&engineobject,0Null0, NULL, NULL);//The first step is to create the engineresult = (*engineobject)->realize (Engineobject, Sl_boolean_false);//implementing the (realize) Engineobject interface Objectresult = (*engineobject)->getinterface (Engineobject, Sl_iid_engine, &engineengine);//Initialize Engineengine with the GetInterface method of the Engineobject}

2.4 Creating additional interface objects with engine objects

Other interface objects (SLOBJECTITF OUTPUTMIXOBJECT,SLOBJECTITF playerobject) are created with engine interface objects (the parameters required for the specific interface object are stated here, which can be referenced in the NDK example), such as:

//Audio MixerSLOBJECTITF outputmixobject = NULL;//Creating a Mixer Interface object with SLOBJECTITFSLENVIRONMENTALREVERBITF outputmixenvironmentalreverb = NULL;////Create a concrete mixer object instanceresult= (*engineengine)->createoutputmix (Engineengine, &outputmixobject,1, mids, Mreq);//Creating mixer interface objects with engine interface objectsresult = (*outputmixobject)->realize (Outputmixobject, Sl_boolean_false);//implementing the (realize) Mixer interface Objectresult = (*outputmixobject)->getinterface (Outputmixobject, Sl_iid_environmentalreverb, & OUTPUTMIXENVIRONMENTALREVERB);//initializing a specific mixer instance using the Mixer interface object//playerSLOBJECTITF playerobject = NULL;//Creating a player interface object with SLOBJECTITFSLPLAYITF playerplay = NULL;//to create a specific instance of a player objectresult= (*engineengine)->createaudioplayer (Engineengine, &playerobject, &AUDIOSRC, &audioSnk,3, IDs, req);//creating player interface objects with engine interface objectsresult = (*playerobject)->realize (Playerobject, Sl_boolean_false);//implementing the (realize) Player interface Objectresult = (*playerobject)->getinterface (Playerobject, Sl_iid_play, &playerplay);//Initializes a specific instance of the Player object

The last is to use the specific object instance created to achieve the specific function.

Three, OpenSL ES use example

First import the OpenSL es and other required libraries:

-lopensles-landroid

3.1 Play Assets file

Create an engine--create a mixer--create player--set playback status

Jniexportvoidjnicalljava_com_renhui_openslaudio_mainactivity_playaudiobyopensl_1assets (jnienv*Env, jobject instance, Jobject assetmanager, jstring filename)    {release (); Const Char*utf8 = (*env)getstringutfchars (env, filename, NULL); //Use asset Manager to open asset by filenameaassetmanager* mgr =Aassetmanager_fromjava (env, Assetmanager); Aasset* Asset =aassetmanager_open (Mgr, UTF8, aasset_mode_unknown); (*ENV)releasestringutfchars (env, filename, UTF8); //Open Asset as file descriptoroff_t start, length; intFD = aasset_openfiledescriptor (Asset, &start, &length);     Aasset_close (asset);      Slresult result; //The first step is to create the engineCreateengine (); //Step Two, create the mixer    ConstSlinterfaceid mids[1] ={Sl_iid_environmentalreverb}; ConstSlboolean mreq[1] ={Sl_boolean_false}; Result= (*engineengine)->createoutputmix (Engineengine, &outputmixobject,1, Mids, mreq); (void) result; Result= (*outputmixobject)realize (Outputmixobject, sl_boolean_false); (void) result; Result= (*outputmixobject)->getinterface (Outputmixobject, Sl_iid_environmentalreverb, &Outputmixenvironmentalreverb); if(Sl_result_success = =result) {Result= (*outputmixenvironmentalreverb)->setenvironmentalreverbproperties (Outputmixenvironmentalreverb, &reverbsettings); (void) result; }    //step three, set player parameters and create player//1. Configure Audio sourceSLDATALOCATOR_ANDROIDFD LOC_FD ={SL_DATALOCATOR_ANDROIDFD, FD, start, length}; Sldataformat_mime Format_mime={sl_dataformat_mime, NULL, sl_containertype_unspecified}; Sldatasource audiosrc= {&LOC_FD, &Format_mime}; //2. Configure Audio sinkSldatalocator_outputmix Loc_outmix ={sl_datalocator_outputmix, outputmixobject}; Sldatasink audiosnk= {&loc_outmix, NULL}; //Create player    ConstSlinterfaceid ids[3] ={sl_iid_seek, Sl_iid_mutesolo, sl_iid_volume}; ConstSlboolean req[3] ={sl_boolean_true, sl_boolean_true, sl_boolean_true}; Result= (*engineengine)->createaudioplayer (Engineengine, &fdplayerobject, &AUDIOSRC, &audioSnk,3, IDs, req); (void) result; //Implementing the Playerresult = (*fdplayerobject)realize (Fdplayerobject, sl_boolean_false); (void) result; //get the Player interfaceresult = (*fdplayerobject)->getinterface (Fdplayerobject, Sl_iid_play, &fdplayerplay); (void) result; //Get the Sound Control interfaceresult = (*fdplayerobject)->getinterface (Fdplayerobject, Sl_iid_volume, &fdplayervolume); (void) result; //fourth step, set playback status    if(NULL! =Fdplayerplay) {Result= (*fdplayerplay)setplaystate (Fdplayerplay, sl_playstate_playing); (void) result; }     //Set Playback Volume (100 *-50: Mute)(*fdplayervolume)->setvolumelevel (Fdplayervolume, -* - -); }

3.2 Playing PCM files

(When integrated into FFmpeg, it is also the data that plays FFmpeg converted into PCM format), here in order to simulate audio files that are directly read in PCM format.

1. Create player and mixer

//The first step is to create the engineCreateengine (); //Step Two, create the mixer    ConstSlinterfaceid mids[1] ={Sl_iid_environmentalreverb}; ConstSlboolean mreq[1] ={Sl_boolean_false}; Result= (*engineengine)->createoutputmix (Engineengine, &outputmixobject,1, Mids, mreq); (void) result; Result= (*outputmixobject)realize (Outputmixobject, sl_boolean_false); (void) result; Result= (*outputmixobject)->getinterface (Outputmixobject, Sl_iid_environmentalreverb, &Outputmixenvironmentalreverb); if(Sl_result_success = =result) {Result= (*outputmixenvironmentalreverb)setenvironmentalreverbproperties (Outputmixenvironmentalreverb,&reverbsettings); (void) result; } sldatalocator_outputmix Outputmix={sl_datalocator_outputmix, outputmixobject}; Sldatasink audiosnk= {&outputmix, NULL};

2. Set the number of frequency digits in the PCM format and create the player

//step Three, configure the PCM format informationSldatalocator_androidsimplebufferqueue Android_queue={sl_datalocator_androidsimplebufferqueue,2}; SLDATAFORMAT_PCM PCM={SL_DATAFORMAT_PCM,//play data in PCM format            2,//2 channels (stereo)Sl_samplingrate_44_1,//Frequency of 44100hzSl_pcmsampleformat_fixed_16,//number of digits 16 bitsSl_pcmsampleformat_fixed_16,//the same as the number of digits.Sl_speaker_front_left | Sl_speaker_front_right,//Stereo (front left front right)Sl_byteorder_littleendian//End Flag    }; Sldatasource Sldatasource= {&android_queue, &PCM}; ConstSlinterfaceid ids[3] ={sl_iid_bufferqueue, sl_iid_effectsend, sl_iid_volume}; ConstSlboolean req[3] ={sl_boolean_true, sl_boolean_true, sl_boolean_true}; Result= (*engineengine)->createaudioplayer (Engineengine, &pcmplayerobject, &sldatasource, &audioSnk,3, IDs, req); //Initializing the player(*pcmplayerobject)realize (Pcmplayerobject, sl_boolean_false);//get interface after call get player interface(*pcmplayerobject)->getinterface (Pcmplayerobject, Sl_iid_play, &pcmplayerplay);

3. Set buffer queue and callback function

//     Register callback buffer get buffer queue Interface    (*pcmplayerobject)->getinterface (Pcmplayerobject, Sl_iid_bufferqueue, &pcmbufferqueue);     // Buffer Interface Callback    (*pcmbufferqueue)->registercallback (Pcmbufferqueue, Pcmbuffercallback, NULL);

callback function:

void* Pcmbuffercallback (SLANDROIDBUFFERQUEUEITF BF,void*context) {    //assert (NULL = = context);Getpcmdata (&buffer); //for streaming playback, replace this test by logic to find and fill the next buffer    if(NULL! =buffer)        {Slresult result; //enqueue Another bufferresult = (*pcmbufferqueue)->enqueue (pcmbufferqueue, buffer,44100*2*2); //the most likely and result is sl_result_buffer_insufficient,//which for this code example would indicate a programming error    }}

To read a file in PCM format:

voidGetpcmdata (void**PCM) {     while(!feof (Pcmfile)) {fread (Out_buffer,44100*2*2,1, Pcmfile); if(Out_buffer = =NULL) {Logi ("%s","Read End");  Break; } Else{Logi ("%s","Reading"); }        *PCM =Out_buffer;  Break; }}

4. Set playback status and start calling callback function manually

//     Get Playback status interface    (*pcmplayerplay)///     active call callback function to start work    Pcmbuffercallback (pcmbufferqueue , NULL);

Note: In the callback function, result = (*pcmbufferqueue)->enqueue (pcmbufferqueue, buffer, 44100 * 2 * 2), and the Last "44100*2*2" is the size of the buffer, Because I have specified here is not read from the PCM file read the "44100*2*2" byte, so it can play normally, if the use of FFmpeg to obtain PCM data source, then the actual size according to the specific size of each avframe, so as to play the sound of the normal!!! (44100 * 2 * 2 means: 44100 is the frequency hz,2 is stereo dual channel, 2 is the use of 16-bit sampling is 2 bytes, so the total number of bytes is: 44100 * 2 * 2)

Android OpenSL ES Development: Android OPENSL Introduction and development process description