Decryption Multimedia Encapsulation Solution package Framework

In the previous article we built the environment and compiled the required FFmpeg Library, this article discusses how to use the API functions provided by FFmpeg for the Solution Encapsulation (DEMUX) process of multimedia files. Before we explain, we need to understand some basic knowledge of multimedia files, prawns please drift over.

• Container format: Whether it is an audio file or a video format file, is a multimedia container, that is, container, such as the common video container format has AVI, MP4,mkv, FLV,rm/rmvb, MOV , TS, VOB,dat, audio container format has MP3, WAV,AAC, APE, FLAC and so on, it accommodates video, audio, subtitles (subtitle ) such as one or more basic stream data, and some even a container with multiple video, audio, and subtitles.

• compression format: The basic stream of video and audio data is compressed in the form of audio and video compression. Common video compression formats such as Mpeg2 compression ratio can reach 25:1

• ES: Elementarystream, also known as basic flow, component flow, and so on, is a separate video, an audio, a subtitle caption, or a single additional piece of data. Obviously a common multimedia file has a video es,audio es, andsome also contain multiple video es and audio es as well as Subtitlees. For example, Blu-ray original TS generally contain multiple audio tracks es and subtitles es, but not all subtitles have subtitles es, may have been embedded into the video subtitles, such subtitles are actually part of the video.

• Demux: When playing, it is necessary to separate the basic streams such as AV and subtitle, this process is called Demux, or solution encapsulation, also known as reusability. Separate basic streams (ES) are sent to the video decoder, audio decoder and other decoding to get the image sound. Demux processes such as (subtitle may also need decoding):

• Remux: Of course Demux in turn the basic audio, video, subtitles and other combinations into a complete multimedia is Remux or encapsulation, also known as reuse. For example, many movie website audio and video suppression people need to do demux, separated into Es, after adding the necessary Chinese subtitles and audio tracks, re-encapsulation. All transcoding tools must also have a Remux and re -Demux process. The concept of reuse and reuse should be more clear to readers familiar with the DVB industry.

• PTS: That is, the time stamp, the point at which the image or sound should be displayed or audible after decoding. Audio and video is not a decoding out on the air, or chaos, the performance of a good decoder playing fast, poor playback slow, and video and audio is not on the number. All of this is synchronized by PTS . The DTS decoding timestamp is less important now than the previous large decoding memory buffer.

With these basic multimedia knowledge, we can continue to explain how to use FFmpeg to Demux the process. Let's start by introducing some of the main API functions:

Intavformat_open_input (Avformatcontext **ps, const char *filename,

Avinputformat *fmt, avdictionary **options);

This function is used to open the multimedia file and read the relevant file header information.

Voidavformat_close_input (Avformatcontext **ps);

This function is used to close the multimedia file opened above and release the related resources.

Intavformat_find_stream_info (Avformatcontext *ic, avdictionary**options);

This function reads all kinds of information through the registered file format parser, such as playback duration, audio and video compression format, audio track information, subtitle information, frame rate, sample rate, and so on.

int Av_read_frame (avformatcontext*s, Avpacket *pkt);

This function is the most important function for the DEMUX process, which reads a frame of video from a file, a frame or multi-frame audio, subtitles and other ES packets, in addition to the data itself, including PTS, duration, reference frames and other important information.

void Av_free_packet (Avpacket *pkt);

This function is used to release the ES packet, which is used in pairs with the function above.

With these functions and the basics above, let's implement a simple Demux framework instance. The function of this instance is to extract the audio and video ES data from the multimedia files to write different files separately. For the sake of simplicity, we do not handle return errors here and add the error handling mechanism to the actual project. This paper tries to explain the basic frame of ffmpeg solution package in the simplest and most primitive way.

1#include"libavformat/avformat.h"2 3 Static Const Char*media_file ="Test_media.mp4";4 intMainvoid)5 {6 intI, Vid_idx, Aud_idx;7FILE *fp_vides = NULL, *fp_audes =NULL;8Avformatcontext *pformatctx =NULL;9 Avpacket pkt;Ten  One Av_register_all (); AAvformat_open_input (&pformatctx, media_file, NULL, NULL); - Avformat_find_stream_info (Pformatctx, NULL); -  theFp_vides = fopen ("Vid_es.dat","WB"); -Fp_audes = fopen ("Aud_es.dat","WB"); - //1, handle stream info -  for(i=0; i<pformatctx->nb_streams; i++) + { - if(Pformatctx->streams[i]->codec->codec_type = =Avmedia_type_video) +Vid_idx =i; A Else if(Pformatctx->streams[i]->codec->codec_type = =Avmedia_type_audio) atAud_idx =i; - Else -;//such as Subtitile - } -  while(Av_read_frame (Pformatctx, &AMP;PKT) >=0) - { in //2, handle PKT data - if(Pkt.stream_index = =vid_idx) toFwrite (Pkt.data, Pkt.size,1, fp_vides); + Else if(Pkt.stream_index = =aud_idx) -Fwrite (Pkt.data, Pkt.size,1, fp_audes); the Else *;//such as Subtitile $Av_free_packet (&pkt);Panax Notoginseng } - fclose (fp_vides); the fclose (fp_audes); +Avformat_close_input (&pformatctx); A return 0; the}

In the place of note 1 , we need to deal with the basic flow index and audio and video corresponding relationship and important information records, this relationship will be used in the note 2 , and is also the subsequent multi-track, subtitle switch credential, this example only processed the simplest only one audio and video situation, and no other information recorded, such as frame rate, video width height, encoding type, time scale, first pts, and so on. In principle these have nothing to do with the DEMUX framework, and everyone has their own way of handling it, and it is not posted here.

For more detailed information or demo code, please note: Programmer Interaction Alliance , sweep the QR code below or search number Coder_online can be concerned, we can communicate online.

If you want to reprint please specify the source: Thank you for your cooperation!

Decryption Multimedia Encapsulation Solution package Framework