iOS audio playback (a): overview

(This article transferred from the Code farm Life )

Objective

Music-related app development has been for some time, in the process of the app's player after several changes, I also for iOS audio playback implementation has a certain research. Write this series of blog purpose on the one hand hope to be able to stimulate, on the other hand is also hope to help other iOS developers and enthusiasts less detours (I have encountered a lot of pits). This article is the first in the series "iOS audio playback", which will provide an overview of how audio playback is implemented under iOS. BasicLet's take a quick look at some basic audio knowledge. At present, we need to rely on audio files for audio playback on the computer, audio file generation process is the sound information sampling, quantization and encode the digital signal generated by the process, the sound that the human ear can hear, the lowest frequency is from 20Hz to the highest frequency 20KHZ, So the maximum bandwidth for the audio file format is 20KHZ. UnderNyquistTheory, only the sampling frequency is higher than twice times the maximum frequency of the sound signal, in order to restore the voice of the digital signal as the original sound, the audio file sampling rate is generally 40~50khz, such as the most common CD quality sampling rate of 44.1KHZ. The process of sampling and quantifying sound is calledPulse coded modulation(Pulse Code modulation), referred to as PCM. PCM data is the most original audio data completely lossless, so the PCM data, although the sound quality and large volume, in order to solve this problem has been born a series of audio formats, these audio formats use different methods to compress audio data, including lossless compression (ALAC, APE, FLAC) and lossy compression (MP3, AAC, OGG, WMA) are two kinds. The most commonly used audio format is Mp3,mp3 is a lossy compression of the audio format, the purpose of this format is to significantly reduce the amount of audio data, it discards the PCM audio data in the human auditory insensitive parts, From the comparison chart below, we can obviously see that the MP3 data is significantly shorter than the PCM data (image citationImp3 Forum）。 For PCM data MP3 data MP3 format of the code rate (bitrate) represents the MP3 data compression quality, now commonly used in the code rate 128kbit/s, 160kbit/s, 320kbit/s, and so on, the higher the value of sound quality is higher. There are two kinds of MP3 encoding methods commonly usedfixed bit rate(Constant BITRATE,CBR) andvariable bit rate(Variable BITRATE,VBR). The data in the MP3 format is usually made up of two parts, one divided intoID3It is used to store information such as song name, artist, album, track number, and the other part is audio data. The Audio data section is stored in frames (frame) and each audio has its own frame header, which is a MP3 file frame chart (the image is also from the Internet). Each frame in the MP3 has its own frame head, which stores the code rate, sampling rate, and other decoding necessary information, so each frame can be independent of the existence and playback of the file, this feature plus a high compression ratio makes the MP3 file become the mainstream audio streaming format. After the frame head is stored the audio data, these audio data is several PCM data frame compression algorithm compression obtains, to CBR's MP3 data each frame contains the PCM data frame is fixed, and the VBR is variable. iOS audio playback overviewAfter understanding the basic concepts we can list a classic audio playback process (take MP3 as an example): 1. Read MP3 file 2. Parse the sample rate, bitrate, length and other information, separate the audio frame in the MP3 3. Decode the isolated audio frame to get PCM data 4. Audio processing of PCM data (equalizer, reverb, etc., not required) 5. Decodes the PCM data into an audio signal 6. Give the audio signal to the hardware playback 7. Repeat 1-6 steps until playback is complete on the iOS system Apple encapsulates the above process and provides a different level of interface (image citationOfficial Documents）。 CoreAudio interface level   Below is a description of the middle-to-high-level interface:  audio file Services: Read and write audio data, you can complete the 2nd step in the playback process; Audio File Stream Services: Decode the audio to complete the 2nd step in the playback process; Audio Converter Services: Data Conversion to complete the 3rd step in the playback process; audio processing Graph services: Sound processing module , you can complete the 4th step of the playback process; audio Unit Services: Play audio data: You can complete steps 5th and 6th in the playback process; Extended audio file Services:audio file Services and audio Converter Services, Avaudioplayer/avplayer (avfoundation): Advanced interface to complete the entire audio playback process (including local files and network stream playback, except for the 4th step); Audio Queue Services: Advanced interface, can be recorded and played, can complete the playback process of the 3rd, 5, 6 steps; OpenAL: for game audio playback, no discussion   You can see that Apple offers a wide variety of interface types to meet various categories of requirements:  If you just want to achieve audio playback, no other requirements avfoundation will be good to meet your needs. Its interface is simple, don't care about the details;  if your app needs to stream audio and store it at the same time, then Audiofilestreamer plus audioqueue can help you, you can first download the audio data locally, While downloading the local audio file with Nsfilehandler and other interface to Audiofilestreamer or audiofile parsing and separating the audio frame, the separated audio frame can be sent to Audioqueue for decoding and playback. If it is a local file, read the file resolution directly. (Both are straightforward practices that can be implemented in the same way as avfoundation+ local server, Avaudioplayer sends the request to the local server, which is forwarded by the local server. After the data is fetched, it is stored in the local server and forwarded to Avaudioplayer. Another way to compare trick is to first download the audio to a file, after downloading to a certain amount of data to the file path to AvaudioplayeR plays, of course, this practice after the audio seek is back to a problem. );  If you are developing a professional music playback software that requires sound effects (equalizer, reverb), you need to use Audioconverter to convert audio data to PCM data in addition to data reading and parsing, and then by audiounit+ Augraph for sound processing and playback (but most bands apps are developing their own sound modules to take care of PCM data, which is a bit more self-developing in customization and extensibility.) PCM data can be played using AudioUnit after the sound is processed, and of course the Audioqueue also supports direct playback of the PCM data. ）。 Describes the process of using Audiofile + Audioconverter + audiounit for audio playback (image citationOfficial Documents）。 Next TrailerThe next article will tell you about the difficult (DA) problem (Keng) that you must face in iOS audio playback, audiosession. ReferencesAudio file FormatPulse coded modulationSample RateNyquist FrequencyMP3ID3Core Audio EssentialCommon Tasks in OS X Related reading:iOS audio Playback (ii): Audiosession

iOS audio playback (a): Overview