iOS audio playback (i): Overview turn

Today see very good introduction audio development articles, reproduced

Original address: http://msching.github.io/blog/2014/07/07/audio-in-ios/

Preface

Music-related app development has been for some time, in the process of the app's player several changes I also therefore for iOS audio playback implementation has been a certain research. Write this series of blog purpose on the one hand hope to be able to stimulate, on the other hand is also hope to help other iOS developers and enthusiasts less detours (I have encountered a lot of pits =. =).

This article is the first in the series "iOS audio playback", which will provide an overview of how audio playback is implemented under iOS.

Basic

Let's take a quick look at some basic audio knowledge.

At present, we need to rely on audio files for audio playback on the computer, audio file generation process is the sound information sampling, quantization and encode the digital signal generated by the process, the sound that the human ear can hear, the lowest frequency is from 20Hz to the highest frequency 20KHZ, So the maximum bandwidth for the audio file format is 20KHZ. According to the Nyquist theory, only if the sampling frequency is higher than twice times the maximum frequency of the sound signal, the voice of the digital signal can be restored to the original sound, so the audio file sampling rate is generally 40~50khz, such as the most common CD quality sampling rate of 44.1KHZ.

The process of sampling and quantifying sound is called Pulse Code modulation (modulation), or short PCM . PCM data is the most original audio data completely lossless, so the PCM data, although the sound quality and large volume, in order to solve this problem has been born a series of audio formats, these audio formats use different methods to compress audio data, including lossless compression (ALAC, APE, FLAC) and lossy compression (MP3, AAC, OGG, WMA) are two kinds.

Now the most commonly used audio format is MP3, MP3 is a lossy compression of the audio format, the purpose of this format is to significantly reduce the amount of audio data, it discards the PCM audio data in the human auditory insensitive parts, from the comparison chart below we can obviously see the MP3 data compared to the PCM data significantly shorter (image from the Imp3 forum).

For PCM data is MP3 data

The MP3 format of the code rate (bitrate) represents the MP3 data compression quality, now commonly used in the code rate of 128kbit/s, 160kbit/s, 320kbit/s, and so on, the higher the value of sound quality is higher. There are two fixed bit rates (Constant bitrate,cbr) and variable bitrate (Variable bitrate,vbr) commonly used in MP3 encoding methods.

The data in the MP3 format is usually made up of two parts, some of which are ID3 to store the song name, the artist, the album, the number of tracks, and the other part is the audio data. The Audio data section is stored in frames (frame) and each audio has its own frame header, which is a MP3 file frame chart (the image is also from the Internet). Each frame in the MP3 has its own frame header, which stores the necessary information such as sample rate decoding, so each frame can be independent of the file existence and playback, this feature plus the high compression ratio makes the MP3 file become the mainstream format of audio stream playback. After the frame head is stored the audio data, these audio data is several PCM data frame compression algorithm compression obtains, to CBR's MP3 data each frame contains the PCM data frame is fixed, and the VBR is variable.

iOS audio playback overview

Having learned the basics, we can list a classic audio playback process (take MP3 as an example):

2. Read MP3 file
4. Parse the sample rate, bitrate, length and other information, separate the audio frames in the MP3
6. Decode the isolated audio frame to get the PCM data
8. Sound processing of PCM data (equalizer, reverb, etc., not required)
10. Decodes PCM data into audio signals
12. Give the audio signal to the hardware to play
14. Repeat 1-6 steps until playback is complete

In iOS, Apple encapsulates the above process and provides different levels of interface (images from official documents).

Interface Hierarchy of CoreAudio

The following is a description of the middle-and high-level interfaces:

• Audio File Services: Read and write the data, you can complete the 2nd step in the playback process;
• Audio File Stream Services: Decodes the sound and can complete the 2nd step in the playback process;
• Audio Converter Services: The 3rd step in the playback process can be accomplished by the conversion of the sound data;
• Audio processing Graph Services: The sound processing module, which can complete the 4th step in the playback process;
• Audio Unit Services: Playback of Sound data: the 5th and 6th steps in the playback process can be completed;
• Extended audio file Services:audio The combination of file Services and audio Converter services;
• Avaudioplayer/avplayer (avfoundation): Advanced interface, can complete the entire audio playback process (including local files and network stream playback, except for the 4th step);
• Audio Queue Services: Advanced Interface, can be recorded and played, can complete the playback process of the 3rd, 5, 6 steps;
• OpenAL: For game audio playback, no discussion

You can see that Apple offers a wide range of interface types to meet a variety of categories of requirements:

• If you just want to achieve audio playback, no other requirements avfoundation will be good to meet your needs. Its interface is simple to use, do not care about the details;

• < Span style= "Font-family:comic Sans MS; font-size:18px; Background-color:rgb (255,255,255) "> If your app needs to stream audio and store it at the same time, then Audiofilestreamer plus audioqueue can help you, you can first download the audio data locally, While downloading the local audio file with Nsfilehandler and other interface to Audiofilestreamer or audiofile parsing and separating the audio frame, the separated audio frame can be sent to Audioqueue for decoding and playback. If it is a local file, read the file resolution directly. (Both are straightforward practices that can be implemented in the same way as avfoundation+ local server, Avaudioplayer sends the request to the local server, which is forwarded by the local server. After the data is fetched, it is stored in the local server and forwarded to Avaudioplayer. Another way to compare trick is to first download the audio to a file, after downloading to a certain amount of data to the file path to Avaudioplayer playback, of course, this practice after the audio seek back a problem. );

• If you are developing a professional music playback software that requires sound effects (equalizer, reverb), you need to use Audioconverter to convert audio data to PCM data in addition to data reading and parsing, and then by audiounit+ Augraph for sound processing and playback (but most bands apps are developing their own sound modules to take care of PCM data, which is a bit more self-developing in customization and extensibility.) PCM data can be played using AudioUnit after the sound is processed, and of course the Audioqueue also supports direct playback of the PCM data. ）。 Describes the process of using Audiofile + Audioconverter + audiounit for audio playback (images from official documents).

Next Trailer

The next article will tell you about the difficult (DA) problem (Keng) that you must face in iOS audio playback, audiosession.

References

Audio file format

Pulse coded modulation

Sample Rate

Nyquist frequency

MP3

ID3

Core Audio Essential

Common Tasks in OS X