C + + implements RTMP protocol to send H. Encode and AAC encoded audio and video, webcam live

C + + implements RTMP protocol to send H. Encode and AAC encoded audio and video

RTMP (Real time Messaging Protocol) is a streaming protocol specifically designed to transfer audio and video data, originally created by Macromedia and later owned by Adobe, a proprietary protocol used primarily to contact Flash Player and Rtmpserver, such as FMS, Red5, crtmpserver , etc. The RTMP protocol can be used for live streaming, on-demand applications, and FMLE (Flash Media live Encoder) to push audio and video data to Rtmpserver for real-time camera live streaming. However, after all, FMLE application scope is limited, want to embed it in their own programs, or to implement the RTMP protocol push. I realized a rtmpliveencoder, by capturing the camera video and microphone audio, and the H. E and AAC encoding, and then sent to the FMS and Crtmpserver, to achieve real-time live, can be through the flash The player is watching normally and the effect is good and the delay time is about 2 seconds . This article introduces the main ideas and key points of rtmpliveencoder, in order to help those who need this technology.

Technical Analysis

The following four key technologies are required to implement Rtmpliveencoder:

• Capture camera video and microphone audio
• H264 Encoding and AAC encoding
• Video and audio data encapsulated as a playable stream that can be recognized by the streaming media server
• RTMP protocol for message sending

It is a difficult point to encapsulate the audio and video data as a playable stream. With a closer look, you'll find that the audio and video streams in the RTMP packet are the same as the flv package for both video and AV data, so we just need to follow the FLV package H264 and AAC to generate a playable stream.

Let's look at the RTMP protocol again. Adobe has published a document, "rtmp specification," but Wikipedia points out that the document hides a lot of detail, independently of its inability to properly implement rtmp. However, it is of reference significance. In fact, before Adobe release, the RTMP protocol has been cracked almost, and now has a relatively complete implementation, such as: Rtmpdump, which provides the C language interface, which means it can be easily called in other languages.

Program Framework

Using DirectShow technology to achieve audio and video capture, encoding and video encoding, looping through the respective threads (Audioencoderthread and Videoencoderthread), rtmp pushes another thread (rtmpthread). After two coded threads encode the audio and video data in real time, the data is delivered to the rtmp thread, which is packaged as rtmp Packet by the rtmp thread and then sent out.

Data exchange between threads, implemented by a queue databufferqueue. Audioencoderthread and Videoencoderthread post the data pointer to the Databufferqueue and return immediately, which avoids the normal execution time of the encoded thread due to the sending of rtmp packets.

The main job of Rtmpthread is to send the decoding information header of the audio data stream and the decoding information header of the video data stream, and extract the data from the Databufferqueue continuously, encapsulating it as rtmp Packet and sending it out. The process is shown in the following code: (Process_buf_queue_, that is, databufferqueue in)

librtmp first, compile librtmp

Download the Rtmpdump code and you will find that it is an authentic Linux project, except for a simple makefile, nothing else. It seems that librtmp does not depend on the system, and we can compile it on Windows without much effort. However, Librtmp relies on OpenSSL and zlib, and we need to compile them first.

　　1. Compiling openssl1.0.0e

A) download and install ActivePerl

b) Download and install NASM (http://nasm.sourceforge.net/)

c) Unzip the OpenSSL compression pack

d) Run the cmd command line, cut to the OpenSSL directory and execute the following command separately

>perl Configure vc-win32--prefix=c:\some\dir >ms\do_nasm

e) Run Visual Studio command Prompt (2010), cut to the OpenSSL directory, and execute the following commands, respectively.

>nmake-f Ms\nt.mak >nmake-f Ms\nt.mak Install

f) After compiling, the compiled SDK can be found in the directory specified in the first command.

　　2. Compiling zlib

A) unzip the zlib compression pack

b) Run Visual Studio command Prompt (2010), cut to the OpenSSL directory, and execute the following command separately

>CD contrib\masmx86 >bld_ml32.bat

c) Go back to the zlib directory, go to the CONTRIB\VSTUDIO\VC10 directory, open the VS2010 solution file,

In Zlibstat Engineering properties, remove the precompiled macro Zlib_winapi

d) Select Debug or Release compile to

　　3. Compiling librtmp

A) First open Visual Studio 2010 and create a new Win32 console project, designated as a static link library

b) Import the LIBRTMP code into the project and put the OpenSSL, zlib header files and librtmp together to put together the compiled OpenSSL and zlib static libraries

c) in the project setup, add the previously compiled OpenSSL and zlib libraries and compile.

Second, the use of librtmp

Initialize the RTMP structure first

After you start, you will initiate a handshake connection message to RTMP server.

Once the connection is successful, you can start looping the message, where you need to specify the timestamp and data type (Audio, Video, Metadata). One thing to note here is that before calling send, the data in BUF must be an already encapsulated H264 or AAC data stream.

Shut down

and finally the release.

H264 and AAC data streams

As mentioned in this paper, rtmp push audio and video streaming package form similar to the FLV format, it can be seen, to the FMS push H264 and AAC live stream, need to first send "AVC sequence header" and "AAC sequence Header", these two data contain important coding information, without them, the decoder will not be decoded.

The AVC sequence header is the Avcdecoderconfigurationrecord structure in the standard document "iso-14496-15 AVC file Format" are described in detail.

The AAC sequence header stores the audiospecificconfig structure, which is described in "iso-14496-3 Audio". The description of the AUDIOSPECIFICCONFIG structure is very complex, here I do a simplification, in advance to set the audio format to be encoded, wherein, select "AAC-LC" for audio encoding, audio sampling rate of 44100, The audiospecificconfig is then simplified to the following table:

In this way, AVC sequence header and AAC sequence header content can be basically determined, more detailed information, you can go through the relevant documents.

Run effect

Rtmpliveencoder Start Running

Play with a Flash player that comes with a FMS

C + + implements RTMP protocol to send H. Encode and AAC encoded audio and video, webcam live