Android Audio Development (6): Using OpenSL ES API (top)

The previous articles show how to capture audio using the Audiorecord provided by Android in the Java layer, play audio with audiotrack, and encode and decode using MEDIACODEC, which are Java layer APIs provided by Android , whether it is acquisition, playback, or codec, these API interfaces need to copy audio data from Java to the native layer, or from the native layer to Java, if you want to reduce the copy and develop more efficient Android audio applications, it is recommended to use the Android NDK OpenSL ES API Interface, which supports direct processing of audio data at the native layer.

OpenSL es is available from Android 2.3, but Android official about OpenSL ES documents very little, the online data is relatively poor, coupled with its interface is the object-oriented C interface, not particularly easy to understand, so learning is not so easy, I plan to Two articles to introduce this framework, hope to be helpful to beginners.

This article is not intended to introduce the working principles and functions of OpenSL es (which will be introduced in the next article), but to introduce a brief overview of OpenSL es from a macro perspective, so that beginners understand what the API is doing, what can be done, what cannot be done, Finally, some reference sample code and resource links are given.

1. What is OpenSL ES?

OpenSL ES full name is: Open sound Library for Embedded systems, is a set of unlicensed, cross-platform, optimized for embedded systems hardware audio Acceleration API. It provides a standardized, high-performance, low-response-time audio feature implementation method for native application developers on embedded mobile multimedia devices, and enables direct cross-platform deployment of hardware and software audio performance, reducing execution difficulty and facilitating the development of advanced audio markets. (from Baidu Encyclopedia)

In a nutshell: OpenSL ES is a set of audio standards for embedded platforms.

2. What is the relationship between Android and OpenSL ES?

Android 2.3 (API 9) is starting to support the OpenSL ES standard, with the appropriate API development interface through the NDK, is the official Android diagram (Source: here):

650) this.width=650; "src=" Http://s4.51cto.com/wyfs02/M01/7F/16/wKioL1cTSITSrsv9AADPiynT6O8079.png "title=" Image.png "alt=" Wkiol1ctsitsrsv9aadpiynt6o8079.png "/>

As can be seen from the figure, Android implementation of OpenSL ES is only a subset of OpenSL 1.0.1, and has been extended, so for the use of OpenSL ES API, we also need to pay special attention to what is Android support, which is not supported, the specific relevant documents The address is located in the NDK Docs directory:

Ndkroot/docs/additional_library_docs/opensles/index.htmlndkroot/docs/additional_library_docs/opensles/opensl_ Es_specification_1.0.1.pdf

3. What can I do? Can't do what?

The following summarizes the features of Android OpenSL ES and unsupported features.

Characteristics:

(1) C language interface, compatible with C + +, need to be developed under the NDK, better integration in native applications

(2) running on the native layer, need to manage the resources of the application and release, no Dalvik virtual machine garbage collection mechanism

(3) Support the acquisition of PCM data, supported configuration: 16bit bit width, 16000 Hz sampling rate, single channel. (Other configurations are not guaranteed to be compatible with all platforms)

(4) Support the playback of PCM data, supported configuration: 8bit/16bit bit width, single/dual channel, small end mode, sampling rate (8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100, 48000 Hz)

(5) Audio data sources that support playback: Audio in res folder, audio under Assets folder, audio in SDcard directory, online audio, audio binary data defined in code, etc.

Not supported:

(1) devices with version less than Android 2.3 (API 9) are not supported

(2) Not all features and functions defined by OPENSL ES are implemented

(3) MIDI not supported

(4) Direct playback of DRM or encrypted content is not supported

(5) Does not support audio data codec, if need to encode and decode, need to use Mediacodec API or third-party library

(6) In the audio delay aspect, compared to the upper level API, does not have the particularly obvious improvement

4. What are the benefits of developing audio applications with OpenSL ES?

(1) To avoid audio data frequently in the native layer and Java layer copy, improve efficiency

(2) More flexibility to control parameters compared to Java APIs

(3) because it is C code, it can be done in depth optimization, such as using NEON optimization

(4) Code details are more difficult to decompile

(5) Others, to be added

5. Sample Code

A foreigner (Victor Lazzarini) wrote a nice introduction to the Android OpenSL ES API and gave some sample code, which is linked as follows:

https://audioprograming.wordpress.com/2012/03/03/android-audio-streaming-with-opensl-es-and-the-ndk/

I extracted two of these files, modified them, sorted into my Audiodemo example, and personally feel better to demonstrate the effect, the address is as follows:

Https://github.com/Jhuster/AudioDemo

6. What are the good resources?

"OpenSL ES Basics"

"Native audio:opensl ES for Android"

"Android Audio Architecture"

"OpenSL ES for Android"

"Tutorials for OpenSL ES for Android"

"Android.NDK.Beginner ' S.guide.pdf"

"Getting Started with OpenSL on Android"

"Android audio streaming with OpenSL ES and the NDK"

7. Summary

about how Android OpenSL ES API is introduced here, the article is not clear of the place to welcome messages or letters [email protected] exchange, or follow my Sina Weibo @ Lu _ June or the public number @Jhuster to obtain the latest articles and information.

650) this.width=650; "src=" http://s4.51cto.com/wyfs02/M01/7F/16/wKioL1cTShuBlswIAACb8XAe6Uo968.jpg "title=" Weixin _jhuster.jpg "alt=" Wkiol1ctshublswiaacb8xae6uo968.jpg "/>

This article is from the "jhuster column" blog, be sure to keep this source http://ticktick.blog.51cto.com/823160/1764778

Android Audio Development (6): Using OpenSL ES API (top)