Basic knowledge about Android audio Development

Basic knowledge about Android audio Development

Let's start with a piece of gossip. Some time ago I had a sentiment: Android development is not a way to go. It is just a platform that provides many encapsulated APIs, this allows you to quickly develop applications for specific services.

 

The real value lies in the integration of Android with specific business directions, such as Android and audio and video technology, Android and smart hardware interaction, Android and front-end technology integration and exploration, and Android information security, android source code in-depth customization and so on.

 

I have always been optimistic about Audio and Video/Multimedia, and hope to gain in-depth experience and exploration here. Some time ago, I released an Android VoIP network phone application "Feige phone ", I wrote an article titled PigeonCall: Architecture Analysis of an Android VoIP network phone App. You are welcome to take a look at this article.

 

Many things have been learned during the development of this application, so I want to write some articles and share them. This article is the first part of this article. It mainly introduces some basic knowledge of audio development in the form of Q &, it is very important because you do not know this knowledge. You do not know how to configure parameters of many Android APIs or third-party libraries.

 

1. What are the main applications of audio development?

 

Audio players, recorders, voice phones, audio and video monitoring applications, live audio and video applications, audio editing/processing software, Bluetooth headsets/speakers, and more.

 

2. What are the specific content of audio development?

 

(1) Audio Acquisition/playback

(2) Audio algorithm processing (noise reduction, mute detection, Echo elimination, sound processing, power amplifier/enhancement, sound mixing/separation, etc)

(3) Audio Codec and format conversion

(4) Development of audio transmission protocols (SIP, A2DP, AVRCP, etc)

 

3. What are the difficulties in audio application?

 

Latency-sensitive, lagging-sensitive, Denoise, AEC, VAD, and audio mixing algorithms.

 

4. What are the basic concepts of audio development?

 

The following concepts are often used in audio development.

 

(1) samplerate)

 

Sampling is the process of digitization of analog signals. Not only does the audio need to be sampled, but all analog signals need to be converted to digital signals that can be expressed by 0101 through sampling, as shown below:

 

 

Blue represents the analog audio signal, and red points represent the quantified values obtained by sampling.

 

The higher the sampling frequency, the more intensive the red interval. The larger the data volume used to record the audio signal, and the higher the audio quality.

 

According to the nequest theory, as long as the sampling frequency is no less than twice the maximum frequency of the audio signal, the original sound can be restored without losing ground.

 

Generally, the human ears can hear a frequency range of about 20Hz ~ To ensure sound distortion, the sampling frequency should be above 40 kHz. Common audio sampling frequencies include 8 kHz, 11.025 kHz, 22.05 kHz, 16 kHz, 37.8 kHz, 44.1 kHz, 48 kHz, 96 kHz, and 192kHz.

 

(2) quantization precision (Bit Width)

 

In, each Red sampling point requires a value to represent the size. The data type of this value can be 4-bit, 8-bit, 16-bit, 32-bit, and so on. The more digits, the more detailed the representation, the better the sound quality. Of course, the larger the data volume.

 

The common bits are 8-bit or 16-bit.

 

(3) number of channels (channels)

 

As Audio Acquisition and playback can be superimposed, sound can be collected from multiple audio sources and output to different speakers at the same time, therefore, the number of Audio Channels generally indicates the number of audio sources during sound recording or the number of speakers during playback.

 

Mono and Stereo are common. As the name suggests, the former has 1 channels and the latter has 2 channels.

 

(4) Audio frame)

 

This concept is very important in application development. Many articles on the Internet have not specifically introduced this concept.

 

The audio is very different from the video. Each frame of the video is an image. From the above xuanbo, we can see that the audio data is stream and there is no clear concept of a frame, in practical applications, for the convenience of audio algorithm processing/transmission, it is generally set to Ms ~ 60 ms indicates that the unit of data is one frame of audio.

 

This time is called "sampling time", and there is no special standard in its length. It is determined based on the requirements of the decoder and specific applications, we can calculate the size of an audio frame:

 

Assume that the audio signal of a channel is a sampling rate of 8 kHz, the bit width is 16 bit, 20 ms a frame, two channels, then the size of the audio data of one frame is:

 

Int size = 8000 x 16bit x 0.02 s x 2 = 5120 bit = 640 byte

 

5. What are common audio encoding methods?

 

As mentioned above, analog audio signal conversion to digital signal requires sampling and quantification. The quantization process is called encoding. According to different quantization policies, many different encoding methods are produced, common encoding methods include PCM and ADPCM. These data represent lossless original digital audio signals. You can store them as WAV Files by adding some file header information, it is a standard for Audio Digital Storage jointly developed by Microsoft and IBM and can be easily parsed and played.

 

During audio development, we often involve reading and writing WAV files to verify the correctness of the collected, transmitted, and received audio data.

 

6. What are common audio compression formats?

 

First, we will briefly introduce the most basic principle of audio data compression: compression is possible because of redundant information.

 

(1) spectrum masking effect: the frequency range of audible signals that human ears can perceive is 20Hz ~ 20 KHz. audio signals out of this frequency range are redundant signals.

 

(2) Time-Domain masking: When both strong and weak sound signals appear, the weak signal cannot be heard. Therefore, the weak signal is also a redundant signal.

 

Common audio compression formats are listed below:

 

MP3, AAC, OGG, WMA, Opus, FLAC, APE, m4a, AMR, etc.

 

7. What open-source applications are involved in Adndroid VoIP?

 

Imsdroid, sipdroid, csipsimple, linphone, WebRTC, etc.

 

8. What are the open-source libraries for audio algorithm processing?

 

Speex, ffmpeg, webrtc audio module (NS, VAD, AECM, AGC), etc.

 

10. What audio development APIs does Android provide?

 

Audio collection: MediaRecoder, AudioRecord

Audio Playback: SoundPool, MediaPlayer, and AudioTrack (for the differences between them, refer to this Article)

Audio Codec: MediaCodec

Ndk api: OpenSL ES

 

11. What is the latency standard for audio development?

 

ITU-TG.114 specifies that the acceptable latency for high-quality speech is 300 ms. Generally, if the latency is between 300 and ~ 400 ms, the call interaction is poor, but acceptable. When the latency is greater than ms, interactive communication is very difficult.

 

12. Summary

 

There are actually a lot of audio development knowledge points, and an article cannot be detailed. Therefore, it is not comprehensive and detailed enough. Please search for professional materials for in-depth understanding.