Decryption Multimedia Encapsulation Solution package Framework

in the previous article we set up the environment and compiled the required ffmpeg Library, this article we discuss how to use the API function provided by FFmpeg for the solution of multimedia file encapsulation (Demux Process Before we explain, we need to understand some basic knowledge of multimedia files, prawns please drift over.

• container format: Whether it's an audio file or a video format file, it's a multimedia container,Container, such as the common video container format hasAvi,mp4,mkv,flv,Rm/rmvb,mov,Ts,Xo,Dat, the audio container format hasMP3,Wav,Aac,APE,Flacand so on, it accommodates video, audio, subtitles (Subtitle) such as one or more basic stream data, and some even a container with multiple video, audio, and subtitles.

• compression format: The basic stream of video and audio data is compressed in the form of audio and video compression. Common video compression formats such asMpeg2,Mpeg4,H264,VC1,Rm/rmvb, common audio compression formats such asMPA,Aac,AC3,Dts. Note that some of the names here are the same as above, but with different meanings, above is the encapsulation format, here is the compression format. Why compress it? Because without compression, it takes a lot of space to store images or sounds, such asMpeg2compression ratio can be achieved25:1about, whileH264can even reach102:1The amazing degree!

• Es: namelyElementarystream, also known as basic flow, component flow and so on, is a separate video, an audio, aSubtitlecaptions, or individual additional data. Obviously a common multimedia file that has a videoEs, AudioEs, some also contain multiple videosEsand AudioEsas wellSubtitlees. Like the original Blu-ray.Tstypically contains multiple tracksEsand SubtitlesEs, but not all subtitles have subtitlesEs, perhaps subtitles have been embedded in the video, such subtitles are actually part of the video.

• Demux: When playing, it is necessary to separate the basic streams such as AV and subtitle, this process is called Demux, or solution encapsulation, also known as reusability. Separate basic streams (ES) are sent to the video decoder, audio decoder and other decoding to get the image sound. Demux processes such as (subtitle may also need decoding):

• remux: Of course Demux in turn, the basic audio, video, subtitles and other combinations into a complete multimedia is Remux or encapsulation, also known as multiplexing. For example, many movie site audio and video suppression people need to do demux es remux Demux dvb

• PTS: That is, the time stamp, the point at which the image or sound should be displayed or audible after decoding. Audio and video is not a decoding out on the air, or chaos, the performance of a good decoder playing fast, poor playback slow, and video and audio is not on the number. All of this is synchronized by PTS . the DTS decoding timestamp is less important now than the previous large decoding memory buffer.

with these basic multimedia knowledge, we can continue to explain how to use ffmpeg to Demux the process. Let's start by introducing some of the main API functions:

Intavformat_open_input (Avformatcontext **ps, const char *filename,

Avinputformat *fmt, avdictionary **options);

This function is used to open the multimedia file and read the relevant file header information.

Voidavformat_close_input (Avformatcontext **ps);

This function is used to close the multimedia file opened above and release the related resources.

Intavformat_find_stream_info (Avformatcontext *ic, avdictionary**options);

This function reads all kinds of information through the registered file format parser, such as playback duration, audio and video compression format, audio track information, subtitle information, frame rate, sample rate, and so on.

int Av_read_frame (avformatcontext*s, Avpacket *pkt);

This function is for The Demux process is the most important function, it reads a frame of video from a file, a frame or multi-frame audio, subtitles and other ES packets, in addition to the data itself, including PTS, duration, reference frames and other important information.

void Av_free_packet (Avpacket *pkt);

This function is used to release ES Packet, used in pairs with the above function.

with these functions and the basics above, let's implement a simple Demux the framework instance. The function of this instance is to extract the audio and video ES data from the multimedia files to write different files separately. For the sake of simplicity, we do not handle return errors here and add the error handling mechanism to the actual project. This paper tries to explain the basic frame of ffmpeg solution package in the simplest and most primitive way.

#include <stdio.h> #include "libavformat/avformat.h" static const char *media_file = "TEST_MEDIA.MP4"; int main ( void) {int I, VID_IDX, aud_idx; FILE *fp_vides = null, *fp_audes = NULL; Avformatcontext *pformatctx = NULL; Avpacket Pkt;av_register_all (); Avformat_open_input (&pformatctx, media_file, NULL, NULL); AVFORMAT_FIND_STREAM_ Info (pformatctx, NULL); fp_vides = fopen ("Vid_es.dat", "WB"), Fp_audes = fopen ("Aud_es.dat", "WB");//1, Handle stream info for (i=0; i<pformatctx->nb_streams; i++) {if (Pformatctx->streams[i]->codec->codec_type ==AVMEDIA_ Type_video) Vid_idx = I;else if (pformatctx->streams[i]->codec->codec_type ==avmedia_type_audio) aud_idx = i; Else;//such as Subtitile}while (Av_read_frame (Pformatctx, &AMP;PKT) >= 0) {//2, handle PKT dataif (Pkt.stream_index = = VID_IDX) fwrite (Pkt.data, Pkt.size, 1, fp_vides); else if (Pkt.stream_index = aud_idx) fwrite (Pkt.data, Pkt.size, 1, Fp_ Audes); else;//such as Subtitileav_free_packet (&AMP;PKT);} Fclose (fp_vides); FCLOSE (fp_audes); Avformat_close_input (&pformatctx); return 0;} 

in note 1 , the basic flow index and audio and video corresponding to the relationship and important information records, the relationship will be used in the place of note 2 , and is also the subsequent multi-track, subtitle switch credentials, this example only to deal with the simplest only one audio and video situation , and no other information is recorded, such as frame rate, video width height, encoding type, time scale, first PTS, and so on. In principle these have nothing to do with the DEMUX framework, and everyone has their own way of handling it, and it is not posted here.

the first time to get blog updates, for more detailed information and demo Code, please note: Programmer Interaction Alliance , sweep the QR code below or search number Coder_online can be concerned, we can communicate online.

If you want to reprint please specify the source: Thank you for your cooperation!

Decryption Multimedia Encapsulation Solution package Framework