Explanation of FFMPEG framework

1. ffmepg Structure Description
1.1 Introduction
FFmpeg (Fast Forward Moving Pictures Experts Group) is a complete solution for audio and video separation, conversion, encoding and decoding, and streaming media. The most important solution is the libavcodec library, it is a complete open-source solution integrating recording, conversion, audio/video encoding and decoding functions. FFmpeg is developed based on Linux, but can be compiled and used in most operating systems. FFmpeg supports more than 40 types of code, including MPEG, DivX, MPEG4, AC3, DV, and FLV, and more than 90 types of decoding. tcpmp, such as Avi, MPEG, Ogg, matroska, and ASF,
FFmpeg is used for Open Source players such as VLC and mplayer.
The main directory of FFMPEG contains subdirectories such as libavcodec, libavformat, and libavutil. Where
Libavcodec is used to store various encode/decode modules. codec is short for coder/decoder, that is, the decoder. It is used for various types of sound/image codec.
Libavformat is used to store muxer/demuxer modules and parse audio and video formats. It is used to generate and parse various audio and video encapsulation formats, including obtaining the information required for decoding to generate the decoding Context Structure and reading audio and video frames;
Libavcodec and libavformat are used to process media files, such as format conversion;
Libavutil collection tool, which contains some common tool functions. It is used to store memory operations and other auxiliary modules. It is a common small function library that generates CRC verification codes, 128-bit integer mathematics, maximum common divisor, integer open side, integer logarithm, memory allocation, big-end small-end format conversion, and other functions
Libavdevice: supports output input devices;
Libpostproc: used for post-effect processing;
Libswscale: used for proportional scaling and color ing conversion in video scenarios;
FFmpeg: A tool provided by this project, which can be used for format conversion, decoding, or instant video card encoding;
Fsever: an HTTP multimedia instant broadcast streaming server;
Ffplay: A simple player that uses the FFMPEG library for parsing and decoding and is displayed through SDL;
After compilation, the FFMPEG package generates three executable files: FFMPEG, ffserver, and ffplay. FFmpeg is used to process media files, ffserver is an HTTP Streaming Media Server, and ffplay is a simple SDL-based player.

Note:
Muxer/demuxer and encoder/decoder:
The biggest difference is that muxer and demuxer have different structures avoutputformat and avinputformat;
Both encoder and decoder use the avcodec structure.
Muxer/demuxer are saved in the global variables avoutputformat * first_oformat and avinputformat * first_iformat respectively. Both encoder and decoder are saved in the global variable avcodec * first_avcodec.
Muxer/demuxer and encoder/decoder are the same:
All are initialized in the av_register_all () function starting with main ().
They are all stored in global variables as linked lists.
Using function pointers as open public interfaces

1.2 download and compilation
Official download URL http://ffmpeg.org/download.html
Compile./configure
# Make
# Make install
Install to/usr/local/bin,/usr/local/include (including various header files),/usr/local/lib (Generate. A file), after compilation
A. Execute./FFMPEG. The result is as follows:
FFmpeg version SVN-r17579, copyright (c) 2000-2009 Fabrice bellard, et al.
Configuration:
Libavutil 49.15. 0/49.15. 0
Libavcodec 52.19. 0/52.19. 0
Libavformat 52.30. 0/52.30. 0
Libavdevice 52. 1. 0/52. 1. 0
Built on Mar 25 2011 17:30:17, GCC: 4.3.4
At least one output file must be specified
B. Execute./ffplay. The result is as follows:
Ffplay version SVN-r17579, copyright (c) 2003-2009 Fabrice bellard, et al.
Configuration:
Libavutil 49.15. 0/49.15. 0
Libavcodec 52.19. 0/52.19. 0
Libavformat 52.30. 0/52.30. 0
Libavdevice 52. 1. 0/52. 1. 0
Built on Mar 25 2011 17:30:17, GCC: 4.3.4
An input file must be specified
C. Run./ffserver. The result is as follows:
Ffserver version SVN-r17579, copyright (c) 2000-2009 Fabrice bellard, et al.
Configuration:
Libavutil 49.15. 0/49.15. 0
Libavcodec 52.19. 0/52.19. 0
Libavformat 52.30. 0/52.30. 0
Libavdevice 52. 1. 0/52. 1. 0
Built on Mar 25 2011 17:30:17, GCC: 4.3.4
/Etc/ffserver. conf: no such file or directory
Incorrect config file-exiting.
Note: If the fserver. conf file is missing, add the ffserver. conf file to/etc.

2. FFMPEG encoding and decoding
The main process of 2.1 is as follows:
Input stream initialization input streams initializing
Output stream initialization output streams initializing
Encoders and decoders initializing are initialized by encoders and decoder.
If necessary, set the meta data information from the input file to set meta data information from input file if required.
Write output file header file write output files Header
Loop processing of each data unit loop of handling each frame (frame refers to a data unit in Stream)
Read frame from input file:
Decode frame data in a data unit
Encode the data in a data unit. encode new frame data
Write a new data unit to the output file write new frame to output file
Write output files Trailer
Close Each encoder and decoder for Each encoder and decoder
Note:
Av_encode is the most important function in FFMPEG. Most functions such as encoding, decoding, and output are completed in this function. Av_encode (avformatcontext ** output_files,
Int nb_output_files,
Avformatcontext ** input_files,
Int nb_input_files,
Avstreammap * stream_maps, int nb_stream_maps)

Avformatcontext is the main structure for implementing Input and Output Functions and saving relevant data during FFMPEG format conversion. Each input and output file has corresponding entities in the global variables of the pointer array defined below.
Static avformatcontext * output_files [max_files];
Static avformatcontext * input_files [max_files];
For input and output, because they share the same struct, You need to assign values to the iformat or oformat members defined in the structure.
Struct avinputformat * iformat;
Struct avoutputformat * oformat;
For an avformatcontext, the two members cannot have values at the same time, that is, an avformatcontext cannot contain both demuxer and muxer. After matching muxer and demuxer are found in the parse_options () function starting with main (), the avformatcontext structure of each input and output is initialized based on the input and output argv parameters, and saved in the output_files and input_files pointer arrays. In the av_encode () function, output_files and input_files are passed in as function parameters and are not used elsewhere.
 
Avcodeccontext stores avcodec pointers and CODEC-related data, such as video width, height, and audio sample rate. The codec_type and codec_id variables in avcodeccontext are the most important for Encoder/decoder matching.
Enum codectype codec_type;/* See codec_type_xxx */
Enum codecid codec_id;/* See codec_id_xxx */
Codec_type stores media types such as codec_type_video and codec_type_audio. codec_id stores codec_id_flv1 and codec_id_vp6f encoding methods.

The avstream structure stores data stream-related codecs, data segments, and other information. There are two important members:
Avcodeccontext * codec;/** <codec context */
Void * priv_data;
The codec pointer stores the encoder or decoder structure. The priv_data pointer saves
Data related to the encoding/decoding stream.

Avinputstream/avoutputstream according to different input and output streams, the aforementioned avstream structure is encapsulated in the avinputstream and avoutputstream structures and used in the av_encode () function. Avinputstream also saves time-related information. Avoutputstream also stores information related to audio/video synchronization.

2.2 video file decoding process
A initializes the libavcodec library and registers all container formats, codec, parsers, and bitstream filters. When a file is opened, automatically select the corresponding file format and Encoder:
Avcodec_register_all ();
Avdevice_register_all ();
Av_register_all ();
Avformat_alloc_context (); allocates the context for playing avformat and distributes the output media content.

B open the file: av_open_input_file ()
Int av_open_input_file (avformatcontext ** ic_ptr, const char * filename,
Avinputformat * FMT,
Int buf_size,
Avformatparameters * AP)
{
......
If (! FMt ){
/* Guess format if no file can be opened */
FMt = av_probe_input_format (PD, 0 );
}
......
Err = av_open_input_stream (ic_ptr, Pb, filename, FMT, AP );
......
}
There are two main tasks:
Detect the container file format (in the avformatcontext definition );
Retrieving stream information from the container file is the process of calling demuxer of a specific file to separate stream. The description is as follows:
Av_open_input_file
Av_probe_input_format2 () traverses all registered demuxer from first_iformat to call the corresponding probe function
Av_open_input_stream () calls the read_header function of the specified demuxer to obtain information about the related stream. IC-> iformat-> read_header

C. Extract stream information from the file: av_find_stream_info () fills up the streaming field of avformatcontext with valid information. For audio/video, each packet contains a complete or multiple Composite Frames. Read packet from the file and decode the corresponding frame from packet.
Av_find_stream_info (avformatcontext * ic) consists of the following two parts:
In part, av_open_input_file () is used for demuxer)
Then use av_read_frame (avformatcontext * s, avpacket * Pkt) and avcodec_decode_video () to decode (decode)

D. traverse all the streams and find the codec_type_video type:
Int I;
Avcodeccontext * pcodecctx;

// Find the first video stream
Videostream =-1;
For (I = 0; I <pformatctx-> nb_streams; I ++)
If (pformatctx-> streams [I]-> codec-> codec_type = codec_type_video ){
Videostream = I;
Break;
}
If (videostream =-1)
Return-1; // didn't find a video stream

// Get a pointer to the codec context for the video stream
Pcodecctx = pformatctx-> streams [videostream]-> codec;

E. Find the corresponding decoder: avcodec_find_decoder (). If it is successful, open the decoder avcodec_open () and use the given avcodec to initialize avcodeccontext. The description is as follows:
Avcodec * pcodec;

// Find the decoder for the video stream
Pcodec = avcodec_find_decoder (pcodecctx-> codec_id );
If (pcodec = NULL ){
Return-1; // codec not found
}
// Open Codec
If (avcodec_open (pcodecctx, pcodec) <0)
Return-1; // cocould not open Codec

F allocates memory for decoding frames: avcodec_alloc_frame (), which is used to store frame data.

G keeps extracting frame data from the decoding stream: av_read_frame ()
Int framefinished;
Avpacket packet;

I = 0;
While (av_read_frame (pformatctx, & Packet)> = 0 ){
// Is this a packet from the video stream?
If (packet. stream_index = videostream ){
// Decode Video Frame
Avcodec_decode_video (pcodecctx, pframe, & framefinished,
Packet. Data, packet. size );

// Did we get a video frame?
If (framefinished ){
// Convert the image from its native format to rgb32
Img_convert (avpicture *) pframergb, pix_fmt_rgb32,
(Avpicture *) pframe, pcodecctx-> pix_fmt,
Pcodecctx-> width, pcodecctx-> height );

// Save the frame to disk
......
}
}

// Free the packet that was allocated by av_read_frame
Av_free_packet (& Packet );
}

H: Determine the frame type. Call the decoding function of the specified codec for the video frame: avcodec_decode_video ()
After decoding, release the decoder avcodec_close ()
J. Close the input file av_close_input_file ()

3. code mark log
Based on the video decoding process described in section 2.2, log mark (output using printf () method) and track the video decoding process. Start with the ffplay player that comes with FFMPEG and track the calling functions involved in the main () function of ffplay. C.
/* Called from the main */
Int main (INT argc, char ** argv)
{
/* Register all codecs, Demux and protocols */
Avcodec_register_all ();
Avdevice_register_all ();
Av_register_all ();
......
Avformat_opts = avformat_alloc_context ();
Sws_opts = sws_getcontext (16, 16, 0, 16, 0, sws_flags, null, null );
Show_banner ();
Parse_options (argc, argv, options, opt_input_file );
......
Cur_stream = stream_open (input_filename, file_iformat );
Event_loop ();

/* Never returns */
Return 0;
}
The tracking result is as follows:
Root @ localhost/work/FFMPEG> ffplay/work/test/AVI/output. Avi
Beginning avcodec_register_all... _ by Jay remarked
Beginning avdevice_register_all... _ by Jay remarked
Beginning av_register_all... _ by Jay remarked
Registering muxdemux MP3.. _ by Jay remarked
Returning av_register_all's initialized

Avctx_opts [0]
Avctx_opts [1]
Avctx_opts [2]
Avctx_opts [3]
Avctx_opts [4]
Returning avformat_alloc_context value... _ by Jay remarked
Returning sws_getcontex value... _ by Jay remarked

Showing version banner... _ by Jay remarked
Ffplay version SVN-r17579 _ by Jay remarked, copyright (c) 2003-2009 Fabrice bellard, et al.
Configuration:
Libavutil 49.15. 0/49.15. 0
Libavcodec 52.19. 0/52.19. 0
Libavformat 52.30. 0/52.30. 0
Libavdevice 52. 1. 0/52. 1. 0
Built on Apr 1 2011 09:29:06, GCC: 4.3.4

Beginning parse_options... _ by Jay remarked
Returning optindex = [2]
Beginning av_init_packet... _ by Jay remarked
Beginning cur_stream... _ by Jay remarked
Returning av_open_input_file's Pd-> filename = [T]
[MP3 @ 0x9b26d20] MDB: 432, lastbuf: 0 skipping granule 0
Last message repeated 1 times
[MP3 @ 0x9b26d20] MDB: 432, lastbuf: 0 skipping granule 1
Last message repeated 1 times
[MP3 @ 0x9b26d20] MDB: 460, lastbuf: 216 skipping granule 0
Last message repeated 1 times
[MP3 @ 0x9b26d20] MDB: 460, lastbuf: 216 skipping granule 1
Returning av_close_input_file successful