Audio frame of Android audio system

1.1 Audio Frame

Reprint please specify, from LXS, http://blog.csdn.net/uiop78uiop78/article/details/8796492

Android's audio system has been the focus of criticism for a long time. It is true that early Android systems have a certain gap in audio processing compared to iOS, which is also an important reason why many professional music player software developers have not launched the Android platform product. This does not mean that its audio framework is useless, instead, Linux-based Android platforms have a lot to learn from.

1.1.1 Audio frame under Linux

In the early days of computer development, the computer's sound processing equipment was composed of a very simple loudspeaker plus sounder (Tone Generator) with relatively limited function. Later people thought of plug-in in the form of expansion of audio equipment, "Sound Blaster" is one of the most famous. This early sound card is connected to the PC motherboard in plug-in mode and provides more sophisticated audio devices. It can be imagined that the independent hardware design also means that the cost increases, so with the development of technology, then there is a board-borne sound card, which is commonly known as "Integrated sound card." The onboard sound card is also divided into "soft sound card" and "Hard sound card". If the sound card itself does not have the main processing chip, but only the decoding chip, needs to perform the processing work through the CPU operation, then is "the soft sound card", the converse is "the Hard sound card". Computers that are typically oriented to the low end market will contain an integrated sound card device to reduce costs.

A typical sound card typically consists of three parts:

· Connectors

The connection between the sound card and the speaker device, such as speakers and headphones, is also called "Jacks"

· Audio circuits

The main implementation of the sound card, which is responsible for signal amplification, mixing, and analog digital conversion and other operations

· Interface

A unit that connects a sound card to a computer bus, such as a PCI bus

We can use the "cat/proc/asound/cards" command to view the sound card device installed on your computer, as shown in the following example:

There are many types of sound cards on the market, both complex high-performance and low-end simple, then for an operating system, how it manages these audio devices, and to provide a unified interface to the upper application?

Android is strictly a Linux system that relies on various driver support provided by the kernel, including audio drivers. Therefore, it is necessary to take some time to learn the two main audio driver architectures under the Linux platform:

· OSS (Open sound System)

Earlier versions of Linux used the OSS framework, which is an audio system widely used in Unix and Unix-like systems. OSS can either refer to the OSS interface itself, or it can be used to represent the implementation of an interface. The author of OSS is Hannu Savolainen, who works for 4Front Technologies company. Due to intellectual property issues, the support and improvement of OSS is not very good, which is one of the reasons why Linux kernel finally abandons Oss.

In addition, OSS has also been questioned in some ways, such as:

Insufficient support for new audio features;

Lack of support for the latest kernel features and more.

Of course, the OSS as the UNIX under the Unified Audio processing operation of the early implementation, itself is relatively successful. It conforms to the "everything is a document" design concept, and as an architectural framework, it is more than just the application and operating system audio driver interaction, so each system can be tailored to the actual needs of the development. In general, OSS uses the device nodes as shown in the following table:

Table 13?1 device nodes used by OSS

Device node	Description
/dev/dsp	Write data to this file à output to the speaker speaker Read data to this file-record from microphone
/dev/mixer	Mixer for settings related to audio devices, such as volume adjustment
/dev/midi00	First MIDI port, MIDI01,MIDI02, etc.
/dev/sequencer	Used to access synths (synthesizer), often used for the production of games and other effects

For more information, refer to the official OSS Note: http://www.opensound.com/

· ALSA (Advanced Linux sound Architecture)

ALSA is a framework proposed by the Linux community to replace OSS and is a fully open source system (following the GNU GPL and GNU LGPL). ALSA was formally introduced in the kernel 2.5 release, the OSS was gradually excluded from the kernel. Of course, the OSS itself is constantly being maintained, but it is no longer used for kernel.

ALSA provides more and more complex API interfaces than OSS, so the difficulty of development is relatively increased. To do this, ALSA provides a library of tools that developers use to help them better use the ALSA API. According to the official documentation, ALSA has the following features:

? Efficient support for most types of audio interface (either consumer or professional multi-channel sound cards)

? Highly modular sound Driver

? SMP and Thread Safety (THREAD-SAFE) design

? Alsa-lib is provided in user space to simplify application authoring

? Maintains compatibility with OSS APIs to ensure that older OSS programs run correctly in the system

Alsa mainly consists of several parts as shown in the following table:

Table 13?2 Alsa-project Package

Element	Description
Alsa-driver	Kernel driver Package
Alsa-lib	Library of user-space functions
Alsa-utils	Contains a lot of practical small programs, such as ALSACTL: For saving device settings Amixer: is a command-line program for sound volume and other sound control Alsamixer:amixer version of Ncurses Acconnect and Aseqview: Making a MIDI connection and checking the list of connected ports Aplay and Arecord: two command-line programs for playing and recording audio in a variety of formats, respectively
Alsa-tools	Includes a range of tool programs
Alsa-firmware	Audio Firmware Support Package
Alsa-plugins	Plug-in packages, such as Jack,pulse,maemo
Alsa-oss	Simulation packages for OSS-compatible
Pyalsa	Alsa Lib for compiling the Python version

Alsa The main file nodes are as follows:

1. Information Interface (/proc/asound)
2. Control Interface (/DEV/SND/CONTROLCX)
3. Mixer Interface (/DEV/SND/MIXERCXDX)
4. PCM Interface (/DEV/SND/PCMCXDX)
5. Raw MIDI Interface (/DEV/SND/MIDICXDX)
6. Sequencer Interface (/DEV/SND/SEQ)
7. Timer Interface (/dev/snd/timer)

For more information on ALSA, it is recommended that you refer to the documentation yourself, which is useful for understanding the entire audio system later.

1.1.2 Tinyalsa

A look at the word "Tiny", you should be able to guess this is a alsa version of the reduction. In fact, in other parts of the Android system can see similar practices-both want to use open source projects, but also too much work is too cumbersome, how to do? That will only lose weight, so a lot of tiny-xxx appeared.

In earlier versions, the audio architecture of the Android system was primarily based on ALSA, and its upper-level implementation could be seen as an "application" of ALSA. Later, it may be due to some of the shortcomings of Alsa, Android later version is no longer dependent on the implementation of the user space layer provided by ALSA, so we have not found the ALSA related Lib in its library folder, as shown in:

Figure 13?7 Differences between Android4.1 and earlier versions on the audio library

Instead it is a TINYALSA-related library file, as shown in:

At the same time we can see the EXTERNL directory is more than a "Tinyalsa" folder, which contains a few of the few source files, as follows:

Form 13?3 Tiny-alsa Project Document

Description

TD valign= "Top" style= "background: #BFBFBF" >

Source File
android.mk	Makefile
mixer.c	Mixer interface implementation
pcm.c	PCM interface implementation
tinycap.c	Caputer Tools
tinymix.c	Mixer Tools
tinyplay.c	Play tool
include/ Tinyalsa/asoundlib.h	Header file

Visible Tinyalsa and the original ALSA difference is still quite large, it only partially supports two of the interface, and like Raw MIDI, Sequencer, timer and so on interface did not involve- Of course, this is enough for general embedded devices.

Tinyalsa as a substitute for alsa-lib, since the advent of the public has come to the evaluation has been praised, can not generalize-for each manufacturer, the right one is the best. And the manufacturers can also expand their functions on this basis, the real use of ALSA to the extreme.

1.1.3 an audio frame on an Android system

A good system architecture requires as little as possible the coupling of the upper layer to the specific hardware, which is both the design purpose of the operating system and for the audio system as well. The embryonic framework of an audio system can be simply used to denote:

Figure 13?8 The embryonic framework of the audio system

In this diagram, the entire Android audio implementation is considered user, except for the audio driver of Linux itself. Therefore, we can think that the audio driver is the "isolation board" between the upper layer and the hardware. However, if the audio system is designed using only the framework shown, it is not a small burden to use the audio function for upper-layer applications, and it is clear that the Android development team will further refine the "User" section based on its own realities.

Refinement is based on the nature of several hierarchies of Android, including the application layer, the framework layer, the library layer, and the HAL layer, as shown in:

Figure 13?9 Further refinement of the audio frame in the Android system

We can combine the knowledge we have now and think about which modules will be included in each level (regardless of the Bluetooth audio section)?

· APP

This is the top level of the entire audio system and is not the focus of Android system implementation. For example, the manufacturer according to the specific needs of a music player, the game used to sound, or adjust the audio of a class of software and so on.

· Framework

I believe you can immediately think of MediaPlayer and Mediarecorder, as this is the most widely used two classes in the development of audio-related products. In fact, Android also provides another two similar functions of the class, that is, Audiotrack and audiorecorder,mediaplayerservice inside the implementation is done through them, but mediaplayer/ The Mediarecorder provides more powerful control capabilities and is easier to use than the former. We'll have a detailed introduction later.

In addition, the Android system provides Audiomanager, Audioservice and Audiosystem classes for our control of the audio system. These are the frameworks designed to facilitate upper-level application development.

· Libraries

We know that many of the classes in the framework layer are really just "intermediaries" for applications that use the Android library file. Because the upper-level applications are written in the Java language, they require the support of the most straightforward Java interfaces, which is one of the implications of the framework layer's existence. As "intermediaries", they do not actually implement specific functions, or only some of the functions, but the main focus on the library to complete. For example, the above Audiotrack, Audiorecorder, MediaPlayer, and Mediarecorder can find corresponding classes in the library.

This part of the code is centrally placed in the frameworks/av/media/libmedia of the project, mostly written in the C + + language.

In addition to the class library implementation above, the audio system requires a "core control", or, in the general implementation of Android, requires a system service (such as ServiceManager, Locationmanagerservice, Activitymanagerservice and so on), this is Audioflinger and Audiopolicyservice. Their code is placed in Frameworks/av/services/audioflinger, and the most important library generated is called Libaudioflinger.

Another important system service in the audio system is Mediaplayerservice, which is located in Frameworks/av/media/libmediaplayerservice.

Because there are so many libraries and related classes involved, it is recommended that you divide them into two clues when you understand them.

First, take the library as a clue. For example, Audiopolicyservice and Audioflinger are all in the Libaudioflinger library, while Audiotrack, Audiorecorder and other implementations are in the Libmedia library.

Second, take the process as a clue. The library does not represent a process, and the process relies on the library to run. Although some classes are implemented in the same library, they do not mean that they are called in the same process. For example, Audioflinger and Audiopolicyservice reside in a system process called mediaserver, while Audiotrack/audiorecorder and mediaplayer/ Mediarecorder is actually just part of the application process, which communicates with other system processes through the binder service.

In the process of analyzing the source code, it is important to grasp these two clues, so as not to feel confused.

· HAL

From a design point of view, the hardware abstraction layer is an object that Audioflinger directly accesses. This illustrates two problems, on the one hand Audioflinger not directly invoke the underlying driver, on the other hand, Audioflinger upper layer (including the same layer of Mediaplayerservice) module only need to interact with it to achieve audio-related functions. As a result, we can assume that Audioflinger is the real "isolator" in the Android audio system, and no matter how the following changes, the implementation of the upper layer can be kept compatible.

The hardware abstraction layer of audio is mainly divided into two parts, namely Audioflinger and Audiopolicyservice. In fact, the latter is not a real device, but the use of virtual devices to make it easy for vendors to customize their own strategy.

The task of the abstraction layer is to actually associate the Audioflinger/audiopolicyservice with a hardware device, but it must provide a flexible structure to cope with the change--especially for Android, a system that has a very frequent update. For example, the audio system in the previous Android system relied on Alsa-lib, but later became TINYALSA, and such a shift should not be disruptive to the upper layer. The audio HAL thus provides a unified interface to define how it communicates with Audioflinger/audiopolicyservice, which is Audio_hw_device, audio_stream_in, and AUDIO_STREAM_ For the purpose of existence, these struct data types are mostly simply definitions of function pointers, and are some "shells". When Audioflinger/audiopolicyservice is initialized, they look for the most matching implementations in the system (these implementations reside in various libraries in the name of audio.primary.*,audio.a2dp.*) to populate these "shells".

Depending on the product, the audio device is very different, in the Android audio architecture, these problems are solved by the HAL layer Audio.primary, and so on, without the need to modify the upper level implementation on a large scale. In other words, the focus of the vendor's customization is how to provide the efficient implementation of this part of the library.

Based on the above analysis, we give a complete Android audio system framework for reference (no Linux layer implementations are listed, such as Alsadriver, etc.), as follows:

Figure 13?10 Android audio system frame full view

In the next section, we will introduce several key modules in the above framework diagram, including the Audioflinger,audiotrack/audiorecorder,audiomanager/audiopolicyservice, and simply introduce the upper layer of some modules such as Mediaplayerservice and so on.

Audio frame for Android audio system