Introduction to the Android Audio Framework

Introduction to the Android Audio Framework
Basic audio knowledge

What are the important attributes of sound?

Loudness)

The sound level is the volume of all kinds of sounds that humans can perceive. The sound intensity is directly related to the amplitude of the acoustic wave.

Pitch)

The tone is related to the frequency of the voice. When the voice frequency increases, the voice that the human ears perceive increases. Otherwise, the voice is lower.

Quality)

Different materials are used to make the same instrument. The effects of different sounds are determined by the structural characteristics of the object.

How can we digitize various media sources?

Audio sampling

The process of converting acoustic wave signals to computer-supported binary data through ADC is called Audio Sampling ). The core of Sampling is to convert continuous analog signals into discrete digital signals.

Sample)

This is the initial data for sampling, such as a continuous sound waveform.

Sampler)

Samplerger is the key to converting a sample to a final state signal. It can be a sub-system, an operation process, or an algorithm, depending on different signal processing scenarios. The ideal samplerger requires as much signal distortion as possible.

Quantization)

The value after sampling also needs to be quantified, that is, the processing process of approximation of a continuous value to a finite number of discrete values in a certain range. Because the original data is a simulated continuous signal, while the digital signal is discrete, its expression range is limited, so quantization is an essential step.

Encoding (Coding)

In the computer world, all values are represented in binary, so we need to encode the quantified values in binary format. This step is usually done at the same time as quantification.

Nequest sampling theory

"When the simulated signal is restored, the highest frequency is only half of the sampling frequency ".

In other words, if we want to completely reconstruct the original analog signal, the sampling frequency must be more than twice that of it. For example, the human voice range is 2 ~ 20 kHZ, the sampling frequency should be around 40 kHZ. If the value is too small, the sound will be distorted, and the value is too large to significantly improve the sound quality that human ears can perceive.

Recording Process
Audio collection devices (such as Microphone) Capture sound information. Analog signals are processed into binary data that can be accepted by computers through an analog-to-analog converter (ADC. Perform necessary rendering as needed, such as sound effect adjustment and filtering. Theoretically, the processed audio data can be stored on computer devices, such as hard disks and USB devices. However, because the volume of audio data is relatively large, it is not conducive to storage and transmission, and it is usually compressed. For example, our common mp3 music is actually produced by using the corresponding Compression Algorithm for the raw data. Depending on the sampling rate, bit depth, and other factors, the final audio file may be distorted to a certain extent. In addition, the audio/video codec can be completed by pure software, you can also use dedicated hardware chips. Playback Process
Extract related files from the storage device and perform Decoding Based on the encoding method used in the recording process. The audio system selects the final matching audio playback device for the playback instance. The decoded data is transmitted through the path designed by the audio system. The audio data signal is transformed into a analog signal through a digital-to-analog converter (DAC. The analog signal is played back to restore the original sound. Audio Framework
APP

A music player software written by the manufacturer according to specific requirements.

Framework

Android also provides two classes with similar functions, namely AudioTrack and AudioRecorder. The internal implementation of MediaPlayerService is achieved through these classes, but MediaPlayer/MediaRecorder provides more powerful control functions, it is easier to use than the former. In addition, the Android system also provides AudioManager, AudioService, and AudioSystem for our audio control system. These are all designed by the framework to facilitate upper-layer application development.

Libraries

The framework serves as a bridge for applications to access the Android library. The specific functions are implemented in the library. For example, the above AudioTrack, AudioRecorder, MediaPlayer and MediaRecorder can find the corresponding classes in the library.

1. frameworks/av/media/libmedia [libmedia. so]

2. frameworks/av/services/audioflinger [libaudioflinger. so]

3. frameworks/av/media/libmediaplayerservice [libmediaplayerservice. so]

4. HAL

In terms of design, the hardware abstraction layer is the object directly accessed by AudioFlinger. This illustrates two problems. On the one hand, AudioFlinger does not directly call the underlying driver; on the other hand, the upper-layer module of AudioFlinger only needs to interact with it to implement audio-related functions. Therefore, we can think that AudioFlinger is the real "isolation board" in the Android audio system. The upper-layer implementation can be compatible regardless of the following changes.

The audio hardware abstraction layer consists of AudioFlinger and AudioPolicyService. In fact, the latter is not a real device, but a virtual device is used to allow manufacturers to conveniently customize their own policies. The abstract layer task is to associate AudioFlinger/AudioPolicyService with hardware devices, but it must provide a flexible structure to cope with changes-especially for systems with frequent Android updates. For example, in the Android system, the Audio system was dependent on ALSA-lib, but later it was changed to tinyalsa. Such a change should not cause damage to the upper layer. Therefore, Audio HAL provides a unified interface to define the communication mode between it and AudioFlinger/AudioPolicyService. This is the purpose of audio_hw_device, audio_stream_in, audio_stream_out, and so on, most of these Struct data types are defined as function pointers and are "shells ". When AudioFlinger/AudioPolicyService is initialized, they will find the most matched implementations in the system (these implementations reside in audio. primary. *, audio. a2dp. * fill in the "shells" in the various libraries named ". According to different products, there are great differences in audio devices. In the Android audio architecture, these problems are caused by the HAL-layer audio. primary and so on, without the need to modify the upper-layer implementation on a large scale. In other words, the focus of the manufacturer in customization is how to provide efficient implementation of this sub-database.

AudioRcorder and AudioTrack are APIs provided by the Audio system. AudioRcorder is mainly used to collect Audio data, while AudioTrack is used to output Audio data. AudioFlinger manages the input and output Audio streams in the system and performs Audio data mixing. It reads and writes Audio hardware to implement Audio data input and output functions. AudioPolicyService is the policy control center of the Audio system, in charge of the selection and switching of sound devices in the system, volume control, etc.