Decryption Multimedia Encapsulation Solution package Framework

In the previous article we built the environment and compiled the required FFmpeg library, this article discusses how to use the API functions provided by FFMPEG for the Solution Encapsulation (DEMUX) process of multimedia files. Before we explain, we need to understand some basic knowledge of multimedia files, prawns please drift over.

• Container format: Whether it is audio files or video format files, is a multimedia container, that is, container, such as the common video container format has AVI, MP4, MKV, FLV, RM/RMVB, MOV, TS, VOB, DAT, audio container format has MP3, WAV, AAC, Ape,flac, and so on, it accommodates one or more basic streaming data, such as video, audio, subtitles (subtitle), or even a container with multiple video, audio, and subtitles.

• Compression format: The basic stream of video and audio data is compressed in the form of audio and video compression. Common video compression formats such as MPEG2, MPEG4, H264, VC1, RM/RMVB, common audio compression formats such as MPA, AAC, AC3, DTS. Note that some of the names here are the same as above, but with different meanings, above is the encapsulation format, here is the compression format. Why compress it? Because it does not compress, to store the image or sound requires a lot of space, such as MPEG2 compression ratio can reach about 25:1, and H264 can even reach 102:1 amazing degree!

• ES: Elementarystream, also known as basic flow, component flow, and so on, is a separate video, an audio, a subtitle caption, or a single additional piece of data. Obviously a common multimedia file has a video es, audio es, and some also contain multiple video es and audio es as well as subtitlees. For example, Blu-ray original TS generally contain multiple audio tracks es and subtitles es, but not all subtitles have subtitles es, may have been embedded into the video subtitles, such subtitles are actually part of the video.

• Demux: When playing, it is necessary to separate the basic streams such as AV and subtitle, this process is called DEMUX, or solution encapsulation, also known as reusability. Separate basic streams (ES) are sent to the video decoder, audio decoder and other decoding to get the image sound. Demux processes such as (subtitle may also need decoding):

• Remux: Of course Demux in turn the basic audio, video, subtitles and other combinations into a complete multimedia is REMUX or encapsulation, also known as reuse. For example, many movie website audio and video suppression people need to do demux, separated into ES, after adding the necessary Chinese subtitles and audio tracks, re-encapsulation. All transcoding tools must also have a remux and re-demux process. The concept of reuse and reuse should be more clear to readers familiar with the DVB industry.

• PTS: That is, the time stamp, the point at which the image or sound should be displayed or audible after decoding. Audio and video is not a decoding out on the air, or chaos, the performance of a good decoder playing fast, poor playback slow, and video and audio is not on the number. All of this is synchronized by PTS. The DTS decoding timestamp is less important now than the previous large decoding memory buffer.

With these basic multimedia knowledge, we can continue to explain how to use FFmpeg to demux the process. Let's start by introducing some of the main API functions:

Intavformat_open_input (Avformatcontext **ps, const char *filename,

Avinputformat *fmt, Avdictionary **options);

This function is used to open the multimedia file and read the relevant file header information.

Voidavformat_close_input (Avformatcontext **ps);

This function is used to close the multimedia file opened above and release the related resources.

Intavformat_find_stream_info (Avformatcontext *ic, avdictionary**options);

This function reads all kinds of information through the registered file format parser, such as playback duration, audio and video compression format, audio track information, subtitle information, frame rate, sample rate, and so on.

int Av_read_frame (avformatcontext*s, Avpacket *pkt);

This function is the most important function for the DEMUX process, which reads a frame of video from a file, a frame or multi-frame audio, subtitles and other ES packets, in addition to the data itself, including PTS, duration, reference frames and other important information.

void Av_free_packet (Avpacket *pkt);

This function is used to release the ES packet, which is used in pairs with the function above.

With these functions and the basics above, let's implement a simple Demux framework instance. The function of this instance is to extract the audio and video ES data from the multimedia files to write different files separately. For the sake of simplicity, we do not handle return errors here and add the error handling mechanism to the actual project. This paper tries to explain the basic frame of ffmpeg solution package in the simplest and most primitive way.

?

12345678910111213141516171819202122232425262728293031323334353637383940414243

#include <stdio.h>#include "libavformat/avformat.h"staticconstchar*media_file =  "test_media.mp4" ;int main( void ){inti, vid_idx, aud_idx;FILE*fp_vides = NULL, *fp_audes = NULL;AVFormatContext *pFormatCtx = NULL;AVPacket pkt;av_register_all();avformat_open_input(&pFormatCtx, media_file, NULL, NULL);avformat_find_stream_info(pFormatCtx, NULL);fp_vides =  fopen ( "vid_es.dat" ,  "wb" );fp_audes =  fopen ( "aud_es.dat" ,  "wb" );// 1, handle stream infofor(i=0; i<pFormatCtx->nb_streams; i++){if(pFormatCtx->streams[i]->codec->codec_type ==AVMEDIA_TYPE_VIDEO)vid_idx = i;elseif(pFormatCtx->streams[i]->codec->codec_type ==AVMEDIA_TYPE_AUDIO)aud_idx = i;else; //such as subtitile}while(av_read_frame(pFormatCtx, &pkt) >= 0){// 2, handle pkt dataif(pkt.stream_index == vid_idx)fwrite (pkt.data, pkt.size, 1, fp_vides);elseif(pkt.stream_index == aud_idx)fwrite (pkt.data, pkt.size, 1, fp_audes);else; // such as subtitileav_free_packet(&pkt);}fclose (fp_vides);fclose (fp_audes);avformat_close_input(&pFormatCtx);return0;}

In the place of note 1, the basic flow index and audio and video corresponding to the relationship and important information records, the relationship will be used in the note 2, and is the subsequent multi-track, subtitle switch credentials, this example only processed the simplest only one-way audio and video, and no other information recorded, such as frame rate, Video height, encoding type, time scale, first pts, and so on. In principle these have nothing to do with the DEMUX framework, and everyone has their own way of handling it, and it is not posted here.

The first time to get blog updates, get more detailed information and demo code, please note: Programmer Interaction Alliance, sweep the QR code below or search number Coder_online can pay attention to, we may exchange online.

Decryption Multimedia Encapsulation Solution package Framework