iOS audio AAC video H264 coded push flow best practices

iOS audio AAC video H264 coded push flow best practices

Projects are personal research and experimentation, there may be many bad or wrong places please forgive.

1 Overview of features

* Realization of audio and video data collection

* Realize the encoding of audio and video data, video encoding into H264, audio encoding into AAC

* To achieve the release of audio and video data, the encoded audio and video transmission to the server

2 video and audio coding scheme

Video hard-coding requires the use of avassetwriter, but he only supports encoding data directly into H264 and writing files, not providing the interface to get video data processing, we need to read the data in the saved file

According to the top adoption of soft coding, the mainstream open source codec Xvid,x264,ffmpeg,xvid is based on the MPEG4 protocol codec, x264 is an encoder based on the H. S protocol, FFmpeg sets up a variety of audio, video codec protocol, by setting parameters can be completed based on mpeg4,h.264 and other protocols, based on efficiency, decided to use x264 soft

IOS supports AAC audio soft coding and hard coding, hard-coded without CPU consumption, high efficiency, and decision to use hard knitting. Using the audio Queue

3. Technical flowchart

4 Related Technologies

(1) X264 encoder

x264 is a GPL-licensed video encoding free software [1]. The main function of the x264 is to perform h.264/mpeg-4 AVC video coding, rather than as a decoder (decoder).

(2) Compile the x264 library under Xcode

* Download the x264 library

* Open the shell, perform the compilation, because our diet channel amount acquisition on the simulator can not be done, so do not compile i386 library, directly compile ARMV7 library

* Execute make, compile successfully get libx264.a

(3) Audioqueue Introduction

To play and record audio on an iOS device, Apple recommends that we use the Avaudioplayer and Avaudiorecorder classes in the Avfoundation framework. Although the usage is relatively simple, but does not support streaming, this means: Before playing audio, you must wait until the entire audio load is complete before you can start playing audio, recording, you must wait until the end of the recording, to obtain the recording data. This has caused great limitations to the application. In order to solve this problem, we need to use audio Queue services to play and record sound.

Audio Queue Services provides direct recording and playback of various formats on iOS and MAC OS x, enabling your application to use hardware recording and playback devices, such as microphones and loudspeakers, without the knowledge of hardware interfaces.

(4) Audioqueue recording audio process

A recording Audio queue, including a buffer queue (buffer) consisting of buffer, and a callback (callback)

The recording process is as follows:

L fill the audio into the first buffer

L The next buffer is automatically populated when the first buffer in the queue fills up. At this point, the callback is triggered.

l need to write audio data stream to disk in callback function

L Then, you need to put the buffer back into the buffer queue in the callback function, so that the buffer can be reused. Repeat step 2.

Audio Queue records AAC audio data as follows:

5 rtmp Send Audio Video

Librtmp Library compiled in Xcode

Use the Librtmp Library in Xcode to package the audio and video data, follow the RTMP protocol, send the data to the specified server, if you need to compile the Librtmp library and use the library in Xcode

* Download Rtmpdump Source code

* The shell compiles librtmp into Librtmp.a and joins the Xcode project

* Copy the rtmpdump include folder to the appropriate project under Xcode and add the path to the user Header searchpaths

* Introduction of the use of librtmp in the project

6. Audio and video synchronization

Using time stamp technology, time stamp is the acquisition time of a frame, audio and video take the same reference time, to each frame timestamp

Cond...

iOS audio AAC video H264 coded push flow best practices