Turn: Interpretation of FFMPEG frame structure

1, FFMEPG structure description
1.1 "Introduction
FFmpeg (Fast Forward movingpictures experts Group) is a complete solution for the separation, conversion, encoding and decoding of audio and video and streaming media, the most important of which is the Libavcodec library, which is a set recording, conversion, sound/ Video encoding and decoding functions as a complete open source solution. The development of FFmpeg is based on the Linux operating system, but it can be compiled and used in most operating systems. FFmpeg support MPEG, DivX, MPEG4, AC3, DV, FLV and more than 40 kinds of encoding, AVI, MPEG, OGG, Matroska, ASF and more than 90 kinds of decoding. TCPMP, VLC, MPlayer and other open source players all use the ffmpeg.
FFmpeg main directory has libavcodec, Libavformat and Libavutil subdirectories. which
L LIBAVCODEC is used for storing each Encode/decode module, codec is actually coder/decoder abbreviation, namely codec; for various types of sound/image codec
L Libavformat is used for storing muxer/demuxer modules, parsing audio and video formats, and for generating and parsing various formats of audio and video encapsulation, including obtaining the information needed for decoding to generate decoding context structure and reading audio and video frames.
Where library Libavcodec,libavformat is used to process media files, such as format conversion;
L Libavutil set of tools, including some common tool functions, for storing memory operations and other auxiliary modules, is a general small function library, the library realizes the generation of CRC check code, 128-bit integer math, greatest common divisor, integer root, integer take logarithm, memory allocation, Functions such as conversion of big-endian small-end format
L Libavdevice: Support for output input device;
L Libpostproc: For post-effect processing;
L Libswscale: For video scene scale scaling, color mapping conversion;
L FFmpeg: This project provides a tool that can be used for format conversion, decoding or TV card instant encoding, etc.;
L Fsever: An HTTP multimedia instant broadcast streaming server;
L Ffplay: is a simple player that uses the FFmpeg library to parse and decode through the SDL display;
The FFmpeg package is compiled and generates three executables, Ffmpeg,ffserver,ffplay. Where ffmpeg is used to process media files, Ffserver is an HTTP streaming media server, Ffplay is a simple SDL-based player.

Description
The difference between Muxer/demuxer and Encoder/decoder:
n The biggest difference is that muxer and demuxer are different structures avoutputformat and Avinputformat respectively;
N and encoder and decoder are used AVCODEC structures.
n muxer/demuxer are stored in global variables Avoutputformat *first_oformat and Avinputformat*first_iformat respectively. Encoder/decoder are stored in the global variable AVCODEC*FIRST_AVCODEC.
The similarities between Muxer/demuxer and Encoder/decoder:
n is initialized within the Av_register_all () function starting at main ().
n are stored in a global variable as a linked list.
n is used as an open public interface in the form of a function pointer

1.2 "Download and compile
Official Download URL http://ffmpeg.org/download.html
Compile./configure
#make
#makeinstall
Install to/usr/local/bin,/usr/local/include (contains each header file),/usr/local/lib (generate. A file), after compilation
A "Implementation./ffmpeg, the results are as follows:
FFmpeg version svn-r17579, Copyright (c) 2000-2009 Fabrice Bellard, et al.
Configuration
Libavutil 49.15.0/49.15. 0
Libavcodec 52.19. 0/52.19. 0
Libavformat 52.30. 0/52.30.0
Libavdevice 52. 1.0/52. 1.0
Built on Mar 17:30:17, gcc:4.3.4
At least one output file must is specified
B "Implementation./ffplay, the results are as follows:
Ffplay version svn-r17579, Copyright (c) 2003-2009 Fabrice Bellard, et al.
Configuration
Libavutil 49.15.0/49.15. 0
Libavcodec 52.19. 0/52.19. 0
Libavformat 52.30. 0/52.30.0
Libavdevice 52. 1.0/52. 1.0
Built on Mar 17:30:17, gcc:4.3.4
An input file must is specified
C "Implementation./ffserver, the results are as follows:
Ffserver version svn-r17579, Copyright (c) 2000-2009 Fabrice Bellard, et al.
Configuration
Libavutil 49.15.0/49.15. 0
Libavcodec 52.19. 0/52.19. 0
Libavformat 52.30. 0/52.30.0
Libavdevice 52. 1.0/52. 1.0
Built on Mar 17:30:17, gcc:4.3.4
/etc/ffserver.conf:no such file or directory
Incorrect config file-exiting.
Description: If the fserver.conf file is missing, you need to add the ffserver.conf file in the/etc/.

2, ffmpeg encoding, decoding
2.1 The main process is as follows:
n input stream initialization input streams initializing
N output stream initialization output streams initializing
N Encoder and decoder initialization encoders and decoders initializing
n if necessary, set meta-data information from the input file sets meta datainformation from input to file if required.
N Write output file header file write outputs files header
n cycles through each data unit loop of handling each frame (frame refers to a data unit in the stream)
U read the data unit from the input file read frame from input files:
U decode data Unit decode frame data
U coded data cell encode new frame data
U write new data unit to output file, write new frame to outputs files
U write the end file of the output file write outputs files trailer
U close each encoder and decoder close each encoder and decoder
Description
The Av_encode function is the most important function in FFmpeg, and most functions such as encode/decode and output are done in this function. Ffmpeg.c in Av_encode (Avformatcontext**output_files,
int Nb_output_files,
Avformatcontext **input_files,
int Nb_input_files,
Avstreammap *stream_maps, int nb_stream_maps)

Avformatcontext is the main structure for implementing input and output functions and saving related data during the FFmpeg format conversion process. Each input and output file has a corresponding entity in the global variable of the pointer array defined below.
Static Avformatcontext*output_files[max_files];
Static Avformatcontext*input_files[max_files];
For inputs and outputs, because the same struct is shared, you need to assign values to the Iformat or Oformat members defined in the structure, respectively.
struct Avinputformat*iformat;
struct Avoutputformat*oformat;
For a avformatcontext, these two members cannot have values at the same time, i.e. a avformatcontext cannot contain both demuxer and muxer. After matching muxer and demuxer are found in the parse_options () function at the beginning of the main () function, the avformatcontext structure of each input and output is initialized according to the incoming argv parameters and saved in the corresponding Output_ Files and input_files pointers in arrays. In the Av_encode () function, output_files and input_files are passed in as function arguments and are not used anywhere else.

Avcodeccontext saves Avcodec pointers and codec-related data, such as the width of the video, the Height,audio sample rate, and so on. The codec_type,codec_id two variables in Avcodeccontext are the most important for encoder/decoder matching.
Enum Codectypecodec_type; /* See CODEC_TYPE_XXX */
Enum codecidcodec_id; /* See codec_id_xxx*/
Codec_type Save is Codec_type_video,codec_type_audio and other media types, codec_id save is codec_id_flv1,codec_id_vp6f and other encoding methods.

The Avstream structure holds information such as codecs, data segments, and so on that are related to data flow. It is more important to have the following two members:
Avcodeccontext *codec;/**< Codec Context */
void *priv_data;
Where the codec pointer holds the encoder or decoder structure. The Priv_data pointer is saved with a
Data related to the volume codec stream.

Avinputstream/avoutputstream depending on the input and output streams, the Avstream structure described above is encapsulated in the avinputstream and avoutputstream structures and used in the Av_encode () function. Time-related information is also saved in the Avinputstream. The Avoutputstream also holds information related to audio and video synchronization.

2.2 "Video File decoding process
A "Initialize the LIBAVCODEC library and register all container formats (format), codec codec, parser (parsers), and bitstream filter (bitstream filters), and automatically select the appropriate file format and encoder when opening a file:
Avcodec_register_all ();
Avdevice_register_all ();
Av_register_all ();
Avformat_alloc_context (); Assigns the context of the playback Avformat and assigns the output media content.

B "Open File: Av_open_input_file ()
Intav_open_input_file (Avformatcontext **ic_ptr, const char *filename,
Avinputformat *fmt,
int Buf_size,
Avformatparameters *ap)
{
......
if (!FMT) {
/* Guess format if no file can be opened */
FMT = Av_probe_input_format (PD, 0);
}
......
Err =av_open_input_stream (ic_ptr, Pb, filename, FMT, AP);
......
}
There are two main things:
N detects the container file format (in the Avformatcontext definition);
N Gets the stream information from the container file, which is the process of invoking the demuxer of a particular file to separate the stream, as described below:
Av_open_input_file
U av_probe_input_format2 () traverses all registered demuxer from the First_iformat to invoke the corresponding probe function
U Av_open_input_stream () invokes the Read_header function that specifies demuxer to get information about the associated stream Ic->iformat->read_header

C extract stream information from a file: Av_find_stream_info () fills the Avformatcontext watershed (streams field) with useful information. For audio/video each packet contains a full or multiple composite frame. Reads the packet from the file and decodes the corresponding frame from the packet.
Av_find_stream_info (Avformatcontext *ic) is mainly two parts:
Part n is using Av_open_input_file () multiplexing (demuxer)
n then decoding (*PKT) using Av_read_frame (Avformatcontext *s, Avpacket Avcodec_decode_video) and Decode ()

D "traverse all the streams to find the type Codec_type_video, described as follows:
int i;
Avcodeccontext *pcodecctx;

Find the first video stream
Videostream=-1;
for (i=0; i<pformatctx->nb_streams; i++)
if (Pformatctx->streams[i]->codec->codec_type==codec_type_video) {
Videostream=i;
Break
}
if (videostream==-1)
return-1; Didn ' t find a video stream

Get A pointer to the codec context for Thevideo stream
pcodecctx=pformatctx->streams[videostream]->codec;

E "Find the corresponding decoder: Avcodec_find_decoder (); If successful, the decoder Avcodec_open () is initialized with the given avcodec to initialize the Avcodeccontext, described as follows:
Avcodec *pcodec;

Find the decoder for the video stream
Pcodec=avcodec_find_decoder (pcodecctx->codec_id);
if (pcodec==null) {
return-1; Codec not found
}
Open codec
if (Avcodec_open (Pcodecctx, Pcodec) <0)
return-1; Could not open Codec

F "Allocating memory for decoding frames: Avcodec_alloc_frame (), for the presence of frame data

G "continuously extracts the frame data from the decoding stream: Av_read_frame ()
int framefinished;
Avpacket packet;

i=0;
while (Av_read_frame (pformatctx,&packet) >=0) {
Is this a packet from the Videostream?
if (Packet.stream_index==videostream) {
Decode Video Frame
Avcodec_decode_video (Pcodecctx, Pframe, &framefinished,
Packet.data, packet.size);

Did we get a video frame?
if (framefinished) {
Convert the image from itsnative format to RGB32
Img_convert (Avpicture *) Pframergb, PIX_FMT_RGB32,
(avpicture*) pframe, PCODECCTX-&GT;PIX_FMT,
Pcodecctx->width, Pcodecctx->height);

Save the frame to disk
......
}
}

Free the packet it was allocated byav_read_frame
Av_free_packet (&packet);
}

H "Determine the type of frame, for the video frame call the specified codec decoding function: Avcodec_decode_video ()
I "After decoding, release decoder: Avcodec_close ()
J "Close input file: Av_close_input_file ()

3. Code Tag Log
According to the video decoding process described in 2.2, the log tag (output with the printf () method) and the video decoding process are tracked. Starting with the FFmpeg ffplay player, trace the calling function involved in main function main () of FFPLAY.C.
/* Called from the main */
int main (int argc, char **argv)
{
/* Register all codecs, Demuxand protocols */
Avcodec_register_all ();
Avdevice_register_all ();
Av_register_all ();
......
Avformat_opts =avformat_alloc_context ();
Sws_opts =sws_getcontext (16,16,0, 16,16,0, Sws_flags, null,null,null);
Show_banner ();
Parse_options (argc, argv,options, opt_input_file);
......
Cur_stream =stream_open (Input_filename, File_iformat);
Event_loop ();

/* Never returns */
return 0;
}
Trace results are as follows:
[Email protected]/work/ffmpeg>ffplay/work/test/avi/output.avi
Beginning Avcodec_register_all ... _by jayremarked
Beginning Avdevice_register_all ... _by jayremarked
Beginning Av_register_all ... _by Jay remarked
Registering Muxdemux MP3 ... _by Jay remarked
Returning Av_register_all ' s initialized

Avctx_opts[0]
AVCTX_OPTS[1]
AVCTX_OPTS[2]
AVCTX_OPTS[3]
AVCTX_OPTS[4]
Returning Avformat_alloc_context value..._by jayremarked
Returning Sws_getcontex Value..._by Jay remarked

Showing version banner..._by Jay remarked
Ffplay version svn-r17579 _by Jay remarked,copyright (c) 2003-2009 Fabrice Bellard, et al.
Configuration
Libavutil 49.15.0/49.15. 0
Libavcodec 52.19. 0/52.19. 0
Libavformat 52.30. 0/52.30.0
Libavdevice 52. 1.0/52. 1.0
Built on APR 1 09:29:06, gcc:4.3.4

Beginning parse_options ... _by Jay remarked
Returning optindex=[2]
Beginning Av_init_packet ... _by Jay remarked
Beginning Cur_stream ... _by Jay remarked
Returning Av_open_input_file ' spd->filename=[t]
[mp3 @ 0x9b26d20]mdb:432, lastbuf:0 skippinggranule 0
Last message repeated 1 times
[mp3 @ 0x9b26d20]mdb:432, lastbuf:0 skippinggranule 1
Last message repeated 1 times
[mp3 @ 0x9b26d20]mdb:460, lastbuf:216 skippinggranule 0
Last message repeated 1 times
[mp3 @ 0x9b26d20]mdb:460, lastbuf:216 skippinggranule 1
Returning Av_close_input_file Successful

Turn: Interpretation of FFMPEG frame structure