Introduction to the Android System audio framework

Audio Basics

What are the important attributes of sound?

1. Loudness (loudness)

Loudness is the size of the various sounds that humans can perceive, that is, the volume. Loudness is directly related to the amplitude of sound waves.

2. Tone (Pitch)

The tone is related to the frequency of the sound, and the greater the frequency of the sound, the higher the tone the human ear perceives, or the lower it will become.

3. Tone (Quality)

The same instrument, the use of different materials to make, the performance of the sound effect is not the same, which is determined by the structure of the object itself.

How do you digitize a variety of media sources?

Audio sampling

The process of converting a sonic waveform signal to a computer-supported binary via the ADC is called Audio sampling. The core of sampling (sampling) is the conversion of continuous analog signals into discrete digital signals.

1. Samples (sample)

This is the initial data we sampled, such as a continuous sound waveform.

2. Sampling Device (Sampler)

A sampler is the key to converting a sample into an end state signal. It can be a subsystem or an operating procedure, or even an algorithm, depending on the signal processing scenario. The ideal sampler requires as little signal distortion as possible.

3. Quantification (quantization)

The sampled values also need to be quantified, which is the process of approximating successive values to a finite number of discrete values within a range. Because the original data is an analog continuous signal, and the digital signal is discrete, its expression range is limited, so quantization is an essential step.

4. Encoding (Coding)

In the world of computers, all values are represented in binary notation, so we also need to encode the quantization values into binary. This step is usually done at the same time as quantification.

Nyquist Sampling theory

"When the sampled analog signal is restored, its maximum frequency is only half the sampling frequency."

In other words, if we were to reconstruct the original analog signal completely, the sampling frequency would have to be more than twice times that of it. For example, the human voice range is 2~ 20kHZ, then the selection of sampling frequency should be around 40kHZ, the value is too small, the sound will produce distortion, and the value is too large can not significantly improve the sound quality of the human ear to perceive.

Recording process

1. Audio acquisition devices (such as microphone) capture sound information.
2. Analog signals are processed into binary data that can be accepted by the computer via an analog-to-digital converter (ADC).
3. Perform the necessary rendering processing as needed, such as sound adjustment, filtering, and so on.
4. The processed audio data can theoretically be stored in a computer device, such as a hard drive, USB device, and so on. However, due to the relatively large volume of audio data at this time, it is not conducive to the preservation and transmission, it is usually compressed processing. For example, our common MP3 music, is actually the original data using the corresponding compression algorithm obtained. Compression process According to the sampling rate, bit depth and other factors, the resulting audio files may have a certain degree of distortion, in addition, audio and video codec can be done by pure software, but also by the use of specialized hardware chip to complete.

Playback process

1. Remove the relevant files from the storage device and decode them according to the encoding used in the recording process.
2. The audio system selects the final matching audio playback device for this playback instance.
3. The decoded data is transmitted through the path of the audio system design.
4. The audio data signal is transformed into an analog signal by means of a digital-analog converter (DAC).
5. The analog signal passes through the playback device to restore the original sound.

Audio frame

1. APP

A music player software that is written by the manufacturer according to the specific needs and so on.

2. Framework

Android also provides another two similar functions of the class, that is, audiotrack and audiorecorder,mediaplayerservice internal implementation is done through them, but mediaplayer/ The Mediarecorder provides more powerful control capabilities and is easier to use than the former. In addition, the Android system provides Audiomanager, Audioservice and Audiosystem classes for our control of the audio system. These are the frameworks designed to facilitate upper-level application development.

3. Libraries

The framework simply provides the application with a bridge to access the Android Library, and the specific functionality is implemented in the library. For example, the above audiotrack,audiorecorder,MediaPlayer , and mediarecorder can find corresponding classes in the library.

1, Frameworks/av/media/libmedia "libmedia.so"

2, Frameworks/av/services/audioflinger "libaudioflinger.so"

3, Frameworks/av/media/libmediaplayerservice "libmediaplayerservice.so"

4. HAL

From a design point of view, the hardware abstraction layer is an object that Audioflinger directly accesses. This illustrates two issues, on the one hand Audioflinger not directly invoke the underlying driver, on the other hand, the Audioflinger upper module only need to interact with it to achieve audio-related functions. As a result, we can assume that Audioflinger is the real "isolator" in the Android audio system, and no matter how the following changes, the implementation of the upper layer can be kept compatible.

The hardware abstraction layer of audio is mainly divided into two parts, namely Audioflinger and Audiopolicyservice. In fact, the latter is not a real device, but the use of virtual devices to make it easy for vendors to customize their own strategy. The task of the abstraction layer is to actually associate the Audioflinger/audiopolicyservice with a hardware device, but it must provide a flexible structure to cope with the change--especially for Android, a system that has a very frequent update. For example, the audio system in the previous Android system relied on Alsa-lib, but later became TINYALSA, and such a shift should not be disruptive to the upper layer. The audio HAL thus provides a unified interface to define how it communicates with Audioflinger/audiopolicyservice, which is Audio_hw_device, audio_stream_in, and AUDIO_STREAM_ For the purpose of existence, these struct data types are mostly simply definitions of function pointers, and are some "shells". When Audioflinger/audiopolicyservice is initialized, they look for the most matching implementations in the system (these implementations reside in various libraries in the name of audio.primary.*,audio.a2dp.*) to populate these "shells". Depending on the product, the audio device is very different, in the Android audio architecture, these problems are solved by the HAL layer Audio.primary, and so on, without the need to modify the upper level implementation on a large scale. In other words, the focus of the vendor's customization is how to provide the efficient implementation of this part of the library.

Audiorcorder and Audiotrack is the audio system to provide API class, Audiorcorder is mainly used to complete the acquisition of audio data, and Audiotrack is responsible for the output of audio data. The Audioflinger manages the input and output audio stream in the system, and undertakes the mixing of audio data, and realizes the input and output function of audio data through the reading and writing hardware. Audiopolicyservice is the Strategic control center of the audio system, Take charge of the selection and switching of sound devices in the system, volume control, etc.

Introduction to the Android System audio framework