"Reprint" FFMpeg SDK Development Manual _ffmpeg

The more important functions and data structures in FFMpeg are as follows:

1. Data structure:

(1) Avformatcontext

(2) Avoutputformat

(3) Avinputformat

(4) Avcodeccontext

(5) Avcodec

(6) Avframe

(7) Avpacket

(8) Avpicture

(9) Avstream

2. Initialization function:

(1) Av_register_all ()

(2) Avcodec_open ()

(3) Avcodec_close ()

(4) Av_open_input_file ()

(5) Av_find_input_format ()

(6) Av_find_stream_info ()

(7) Av_close_input_file ()

3. Audio and Video codec function:

(1) Avcodec_find_decoder ()

(2) Avcodec_alloc_frame ()

(3) Avpicture_get_size ()

(4) Avpicture_fill ()

(5) Img_convert ()

(6) Avcodec_alloc_context ()

(7) Avcodec_decode_video ()

(8) Av_free_packet ()

(9) Av_free ()

4. File Operation:

(1) Avnew_steam ()

(2) Av_read_frame ()

(3) Av_write_frame ()

(4) Dump_format ()

5. Other functions:

(1) Avpicture_deinterlace ()

(2) Imgresamplecontext ()

The following is based on the above data structure and function in the FFmpeg test code output_example.c appeared in the forward and backward analysis. Let's talk about this before.

FFmpeg compile problem. In Linux, the compiler is relatively simple, here is not much to say. Compilations under Windows can refer to the following Web pages:

Http://bbs.chinavideo.org/viewthread.php?tid=1897&extra=page%3D1

It is worth mentioning that the following two questions may be asked during the compilation process using the compiled SDK for testing (using the OUTPUT_EXAMPLE.C in the FFmpeg directory)

Problem:

1. Output_example.c used the snprintf.h header file. However, this header file is different under Win and Linux. Specifically under the win

You can resolve it in the following ways:

http://www.ijs.si/software/snprintf/

2. If you compile using VC6 or the VC6 command line, inline may not recognize it. The error appears in the Common.h file and can be added to the Common.h

#ifdef _msc_var

#define Inline __inline

#endif

Confessed to get to the point.

A Data structures in FFMpeg:

I. Avformatcontext

Typically in the code using the FFmpeg SDK, Avformatcontext is a data structure that runs through all the Times, and many functions use it as a parameter. FFmpeg code

The comments on this data structure are: Format I/o context

This structure contains the format content of a video stream. There are avinputformat (or avoutputformat within the same time Avformatcontext can only be stored

In one), and Avstream, avpacket these important data structures as well as some other relevant information, such as Title,author,copyright.

There are also some possible information to be used in decoding, such as: Duration, file_size, bit_rate, etc. Refer to the Avformat.h header file.

Useage:

Statement:

Avformatcontext *oc; (1)

Initialization: Because the AVFORMATCONEXT structure contains a lot of information, the initialization process is done step-by-step, and some variables are not initialized if no value is available

Of. However, since a general declaration is made with a pointer, an allocated memory procedure is not less than:

OC = Av_alloc_format_context (); (2)

The avinputformat* (or avoutputformat*) in the structure must be initialized, which is basically the basis of what the compiler uses to codec:

Oc->oformat = FMT; or Oc->iformat = FMT; (3)

where avoutputformat* FMT or avinputformat* fmt. (The initialization of Avinputformat and Avoutputformat is described later.) Then in the parameter

There is a line in the OUTPUT_EXAMPLE.C code:

snprintf (oc-filename, sizeof (Oc->filename), "%s", filename); (4)

It is not clear what the role is, it is estimated that first in the output file to write some header information.

After completing the steps above (initialization is complete avinputformat* (or avoutputformat*) and Avformatcontext) The next step is to take advantage of OC Initial

The second important structure in the avformatcontext that begins in this section. Avstream (assuming there is already a statement Avstream *video_st.) In the reference code

A function is used to complete initialization, and of course it can be done in the main function, and the arguments passed into the function are OC and FMT->VIDEO_CODEC (this next section describes

(29)):

Vdeo_st = Add_video_stream (OC, FMT->VIDEO_CODEC); (5)

This function is analyzed later in the avstream structure.

Avformatcontext The last setup work is:

if (Av_set_paramters (Oc,null) < 0) {(6)

Handle error;

}

Dump_format (OC, 0, filename, 1); (7)

The function is to see if the parameters set during the previous initialization are compliant, or they will be an error.

All of this is the initialization process, including the Avformatcontext itself and the use of Avformatcontext to initialize other data structures. I'll talk about the whole thing.

Coding and decoding process. I want to first describe the codec function framework within the main function in ouput_example.c. This is clearer, and the coder ends it.

Structure clearly, in the process of writing ouput_example.c also basically in the main function only maintain avformatcontext and avstream two data structures

(Avoutputformat is in fact but included in the Avformatcontext).

Open Video codec and allocate the necessary encode buffers

if (Video_st)

Open_video (OC, Video_st); (8)

Write the stream header, if any

Av_write_header (OC); (9)

Encode and decode process

for (;;) {

Write_video_frame (OC, Video_st); (10)

Break Condition...here

}

Close Codec

if (Video_st)

Close_video (OC, Video_st); (11)

Write the trailer, if any

Av_write_trailer (OC); (12)

Free the Streams

For (i=0 i<oc->b_streams; i++) {

Av_freep (&OC->STREAMS[I]->CODEC); (13)

Av_freep (&oc->streams[i]); (14)

}

Close the ouput file

if (!) ( Fmt->flags & Avfmt_nofile)) {

Url_fclose (&OC->PB); (15)

}

Av_free (OC); (16)

By using the code above, it is clear that avformatcontex* oc and avstream* Video_st are running through the development of the FFmpeg SDK

Two data structures. Here's a brief introduction to the three functions labeled red, which are reference code output_example.c the developer's own definition of the function. Such

Can make the entire code structure clear, of course, you can also use the FFmpeg SDK in the main function to complete the corresponding functions. In the back, we will specifically address these three letters.

Number to do analysis.

1. Open_video (OC, Video_st);

This function is primarily the initialization process for a video encoder (or decoder). The initialized data structure is avcodec* codec and avcodeccontext* C including the

The SDK functions that are available are:

c = st->codec;

codec = Avcodec_find_encoder (c->codec_id); When coding, find the encoder (17)

codec = Avcodec_find_decoder (c->codec_id); When decoding, find decoder (18)

Avcodeccontex is a data structure in the structure avstream, so after Avstream initialization (5) the direct complex value to C.

Internal Open Video Codec

Avcodec_open (C,CODEC); (19)

Allocate video stream Buffer

Avframe *picture

uint8_t *video_outbuf

video_outbuf_size=200000;

Video_outbuf = Av_maloc (video_outbuf_size); (20)

Allocate video frame Buffer

Picture = Alloc_picture (C->pix_fmt, C->width, c->height); (21)

The above three steps are easy to understand, turn on video codec codec, allocate output stream cache size, allocate each frame image cache size. Where Avframe is also ffmpeg

One of the main data structures. This step (8) is the initialization process for the codec.

2. Write_video_frame (Avformatcontext *oc, Avstream *st)

This function does a real codec work, where the functions are more complex first listed to slowly analyze.

The data structure used is Avcodeccontext *c, Swscontext *img_convert_ctx. Where Swscontext is used to transform the image format. Like what

yuv422 to yuv420 and so on, of course, use the function, see the following list.

Fill_yuv_image (Tmp_picture, Frame_count, C->width, c->height); (22)

Sws_scale (Img_convert_ctx, Tmp_picture->, Tmp_picture->linesize,

0, C->height, Picture->data, picture->linesize); (23)

Img_convert_ctx = Sws_getcontxt (C->width, C->height, pix_fmt_yuv420p, (24)

C->width, C->heigth, c->pix_fmt, sws_flags, NULL, NULL, NULL);

Because the reference code does an encoding. Therefore, it always requires the encoder to enter the YUV file, and it is yuv420 format. There will be some of these places

Process. Next, the encoder code is invoked, the data is Avpacket (packaged), which is a more difficult place to understand in FFmpeg.

Out_size = Avcodec_encode_video (c, Video_outbuf, video_outbuf_size, picture); (25)

Avpacket PKT;

Av_init_packet (&PKT); (26)

... handle PKT process, we'll analyze later

ret = Av_write_frame (OC, &PKT); (27)

If you have encode, you must have decode. and ffmpeg specifically for decoding, but why in the reference code only used encoder it. A personal guess is because

Encode only use yuv420 to encode, such yuv420 generation is easier, if used to decode, but also in the code with a other format audio and video text

Thing There is a apiexample.c reference code in the source code Libavcodec folder, in which the codec is done. I'll analyze it when it's empty.

3. Close_video (Avformatcontext *oc, Avstream *st)

Avcodec_close (ST->CODEC);

Av_free (Picture->data[0]);

Av_free (picture);

Av_free (VIDEO_OUTBUF);

It's easier to understand, not to say more.

A large section of the above, though called Introduction Avformatcontext. But basically put the OUPUT_EXAMPLE.C video coding part of the framework to go over again, one is to explain the knot

The importance of Avformatcontext, on the other hand, also wants to have a general framework for using FFMPEG SDK developers.

In fact, some of the real coding functions, memory allocation functions in the SDK has been encapsulated, as long as the structure can be used to understand. And the developers are going to do something

The initialization process is basically the initialization of the data structure 1.

Ii. Avoutputformat

Although simple (initialization) is important, he is the "indication" of which codec the codec will use. The most important thing in its member data is about video

Codec: enum Codecid Video_codec;

Avoutputformat *fmt;

FMT = Guess_format (null, filename, null); (28)

According to filename to determine the file format, but also initialize the use of what encoder. Of course, if you're using Avinputformat *fmt, that's what fix uses.

Decoder. (Specifies the output sequence->fix encoder, specifying the input sequence->fix decoder.) ）

Iii. Avstream

Avstream as the second avformatcontext after the continuous structure is a reason. There's avcodeccontext in his membership data, which is basically

The parameters of the video codec used are set (including bit rate, resolution and other important information). Also as "stream", it contains the "stream"

Some of the data in this concept, such as frame rate (r_frame_rate), Base time measurement Unit (time_base), and (need to be encoded) the first frame position

(start_time), duration (duration), number of frames (nb_frames), and some IP information. Of course, some of the information behind it is not necessary to start

Initialized, but Avcodeccontex is a must to initialize and is the most important part of initializing Avstream. We talked about it in the front.

Avstream initialization function (5), now let's see how he does it:

Declaration

Avstream *video_st;

Video_st = Add_video_stream (OC, FMT->VIDEO_CODEC);

Static Avstream *add_video_stream (Avformatcontex *oc, int codec_id) {(29)

Avcodeccontext *c; Member of Avstream, which is initialized here

Avstream *st; Temporary data, would be returned

st = Av_new_stream (OC, 0); (30)

c = st->codec;

The following is basically the initialization process for C. Including bit rate, resolution, GOP size and so on.

......

The following two lines need to be noted, especially the use of MP4

if (!strcmp (Oc->oformat->name, "mp4") | |!strcmp (oc->oformat->name, "mov") | |!strcmp (oc->oformat-> Name

"3GP"))

C->flags |= Codec_flag_global_header;

Pass St to Video_st;

Return St;

}

In the code above, there are a few points to note. One is (30) and C = St->codec is sure to do, of course, this is the most basic programming problem, (30) is the St

This avsteam is bound to the avformatcontext* oc. The C = St->codec at the back is the binding of C to the St Avcodeccontext. The second is the initial of C

In the process, ouput_example.c is to do some basic configuration, of course, as a user you also want to codec add some other codec conditions.

can refer to avcodec.h about the AVCODECCONTEXT structure of the introduction, note more detailed.

The use of Avstream is already covered in the previous introduction of Avformatcontext, in three codec functions (8), (10), and (11) in the main function. Observation phase

Off the code, you can find that the main or avstream in the avcodeccontext extracted, and then extracted from the AVCODEC structure as in (8):

Open_video (OC, Video_st);

Avformatcontext *oc, Avstream *st

Avcodec *codec;

Avcodeccontext *c;

c = st->codec;

codec = Avcodec_find_encoder (c->codec_id); (31)

Open the Codec

Avcodec_open (c, codec); (32)

Similarly, we can see that in (Write_video_frame ()) The Avframe is also present as a carrier for the transfer of AVCODECCONTEXT structures. (11)

(Close_video ()) relatively simple, not boil down.

Iv. Avcodeccontext

The comments in the FFmpeg SDK for this structure are as follows: The main external API structure its importance. And in Avcodec its definition, each of its

Member variables, all given a very detailed description. It should be said that Avcodeccontext initialization is the most important part of codec use. Although in front of the

It's been mentioned in Avstream, but it's going to say it again. As a member structure of Avstream, Avcodeccontext must be at the beginning of Avstream

Initialization (30) is then initialized (the Avstream initialization is used to Avformatcontex). Although there are a lot of member variables, but here only to say

Output_example.c, please refer to the avcodec.h file for more information.

Static Avstream *add_video_stream (avformatcontext *oc, int codec_id)

Avcodeccontext *c;

st = Av_new_stream (OC, 0);

c = st->codec;

c->codec_id = codec_id;

C->codec_type = Codec_type_video;

C->bit_rate = 400000; kbits/s

C->width = 352;

C->height = 288; Cif

The frame rate is the denominator, the second is the molecule, then the time_base is the time of the frame. (Time base. ）

C->time_base.den = stream_frame_rate;

C->time_base.num = 1;

C->gop_size = 12;

Here define:

#define STREAM_PIX_FMT pix_fmt_yuv420p

Pixel format, pix_fmt_xxx

-encoding:set by user.

-decoding:set by LAVC.

C->PIX_FMT = stream_pix_fmt;

In addition to the above listed. There is also a method for specifying motion estimation: Me_method. Quantization parameter, maximum B frame number: Max_b_frames. The parameters of the code rate control,

Error concealment error_concealment, mode judgment mode: mb_decision (this parameter is quite interesting, you can see avcodec.h 1566),

Lagrange multipler Parameters: Lmin & LMAX and macro block level Lagrange Multipler parameters: Mb_lmin & Mb_lmax, constant

Quantization parameter rate control METHOD:CQP and so on.

It is worth mentioning that there are two member data structures in the Avcodeccontext: Avcodec, Avframe. The AVCODEC records the codec information to be used and contains

5 functions: Init, encoder, close, decode, flush to complete the codec (see Avcode.h 2072 lines). Avframe mainly contains a series of

The frame information after the code, including whether this frame is key frame, *data[4] defined Y, CB and CR information, and then detailed.

After initialization, it can be said that Avcodeccontext in (8) & (10). Initialize Avcodec *codec and avframe* in (8) Open_video () first

Picture

Avcodeccontext *c;

codec = Avcodec_find_encoder (c->codec_id);

......

Picture = Alloc_picture (pix_fmt_yuv420p, C->width, c->height);

The main parameters of a codec in Writer_video_frame (Avformatcontext *oc, Avstream *st) are then exploited:

Avcodeccontext *c;

c = st->codec;

......

Out_size = Avcodec_encode_video (c, Video_outbuf, video_outbuf_size, picture);

V. Avcodec

There are fewer member variables and member functions in structure AVCODEC, but they are very important. He contains the codecid, which is the codec,

Pixel format information. There are also 5 functions mentioned earlier (init, encode, close, decoder, flush). Incidentally, while in the reference code

The encoding function in OUTPUT_EXAMPLE.C is Avcodec_encode_video (), which I suspect is called the Avcodec encode function, and they

Passed parameters and return values are consistent, of course, have not been confirmed, interested to see the FFmpeg source code. In the reference code, the initialization of the AVCODEC

Subsequent use is dependent on Avcodeccontex, the former member of the latter. After the initialization of Avcodeccontext (Add_video_stream ()), Avcodec Also

can be well initialized:

Class

codec = Avcodec_find_encoder (c->codec_id); (33)

Open codec

Avcodec_open (c, codec) (34)

VI. Avframe

Avframe is a very interesting structure, which is defined in its own way:

typedef struct AVFRAME {

Ff_common_frame

}avframe;

Among them, Ff_common_frame appears as a macro. Because the data in the Avframe is to be accessed frequently in the decoding process. In order to accelerate, to take such

means of coding.

Avframe is used as a description of the "original image" (i.e. YUV or RGB ...). Is there anything else? The structure of his first two member data, uint8_t

*data[4],int Linesize[4], the first store is Y, Cb, Cr (YUV format), Linesize is what. The two data can also be extracted from another

Data:

typedef struct AVPICTURE {

uint8_t *data[4];

int linesize[4]; Number of bytes per line

}avpicture;

In addition, Avframe also contains some other member data, such as. Whether key_frame, encoded image book Coded_picture_number, as reference frame

Reference, macro block type *mb_type and so on (avcodec.h 446 lines).

The initialization of the avframe is not as simple as his structure looks. Because Avframe also has a task to host the image data (data[4]) so, he points

With memory should be carefully completed. Alloc_picute () was provided in the OUTPUT_EXAMPLE.C to complete the work. Two global variables are defined in the reference code:

Avframe *picture,*tmp_picture. (If you use the yuv420 format, then just use the previous data picture on the line, put the image information into the

In the picture. If it's a different format, first initialize the yuv420 format and put it into the tmp_picture to put it in the picture in the request format. ) in

Open_video () initializes the avframe after the codec is opened:

Picture = Alloc_picture (C->pix_fmt, C->width, c->height);

Tmp_picture = Alloc_picture (pix_fmt_yuv420p, C->width, c->height);

static Avframe *alloc_picture (int pix_fmt, int width, int height) {

Avframe *picture;

uint8_t *picture_buf; How about why use uint8_t? A byte!

Picture = Avcodec_alloc_frame (); (35)

if (!picture)

return NULL;

Size = Avpicture_get_size (pix_fmt, width, height); (36)

Picture_buf = av_malloc (size); (37)

if (!PICTURE_BUF) {

Av_free (picture); (38)

return NULL;

}

Avpicture_fill (avpicture *) picture, picture_buf, pix_fmt, width, height); (39)

return to picture;

}

As can be seen from the above code, the completion of the initialization of a avframe (in fact, memory allocation), basically has such a fixed pattern. As for (35)

(39) Completed those work separately, and why there are two steps, have not yet figured out, need to see the original code. My guess is (35) to Avframe did a basic

memory allocation, which retains the memory allocation to (39) of the first two data that can be extracted from the avpicture.

In this regard, we observe that there is a (avpicture *) picture,avpicture structure in (39) that is also useful. Basically, his size is going to be

The size of the packets transmitted over the network, we can see in the back of the Avpacket and Avpicture have a close relationship.

VII. Avpicture

Avpicture does not have its own declaration and initialization process in the reference code. Both of the two occurrences are extracted as mandatory type conversions by Avframe:

In Open_video ()

Avpicture_fill (avpicture *) picture, picture_buf, pix_fmt, width, height); (40)

In Write_video_frame

Avpacket PKT;

if (Oc->oformat->flags & avfmt_rawpicture) {

......

pkt.size = sizeof (avpicture); (41)

}

In (40), the memory is actually allocated for Avframe's data[4], linesize[4]. Because of how these two data sizes are allocated, there really needs to be pix_fmt,

width, height to determine. If the output file format is a raw picture (such as YUV and RGB), Avpacket as the basic data to write the encoded data to the file

Unit, his unit size and data are all made up of avpacket.

To sum up, avpicture exists for the following reasons, avpicture the concept of the picture from the frame, only by the pictures

Body information, brightness, chroma, and row size. And frame is like whether it's a key frame or something. This kind of "grading" is the whole concept clearer.

VIII. Avpacket

The existence of avpacket exists as a basic unit of writing files. We might think that simply writing the encoded bit stream to a file is not a possibility, why

And trouble to set up a avpacket structure. In my opinion, such coding settings are very necessary, especially in the video real-time transmission, synchronization, boundary problems can be

To be solved by avpacket. The Avpacket member data has two timestamps, data (usually encoded data), size, and so on (see

Avformat.h 48 lines). The use of Avpacket will have to mention the codec function, because the avpacket of some of the information can only be decoded after the knowledge. In

In the reference code (OUPUT_EXAMPLE.C from 362 to 394 lines), make a branch of judgment. If the output file format is a RAW image (that is, YUV or RGB) then

There is no coding function, write directly to the file (because the program itself generates a YUV file), here the code, although this seems to have little value, but if it is decoding

function to solve the YUV file (or RGB) so the basic write file operation is this:

if (Oc->oformat->flags & avfmt_rawpicture) {

Avpacket PKT; There is no pointer here.

Av_init_packet (&PKT);

Pkt.flags |= pkt_flag_key//Raw picture, each frame is a KEY frame?

Pkt.stream_index = st->index;

Pkt.data = (uint8_t *) picture;

pkt.size = sizeof (avpicture);

ret = Av_write_frame (OC, &PKT);

}

Output is not raw picture, after encoding:

else{

Video_outbuf & Video_outbuf_size is initialized in Open_video ().

Out_size = Avcodec_encode_video (c, Video_outbuf, video_outbuf_size, picture); (42)

if (Out_size > 0) {

Avpacket PKT;

Av_init_packet (&PKT); (43)

Pkt.pts= av_rescale_q (c->coded_frame->pts, C->time_base, st->time_base); (44)

if (c->coded_frame->key_frame)

Pkt.flags |= Pkt_flag_key;

pkt.stream_index= st->index;

Pkt.data= Video_outbuf;

Pkt.size= out_size;

/* Write the compressed frame in the media file * *

ret = Av_write_frame (OC, &PKT); (45)

} else {

ret = 0;

}

if (ret!= 0) {

fprintf (stderr, "Error while writing video frame\n");

Exit (1);

}

The initialization of Video_outbuf and Video_outbuf_size in Open_video () is as follows:

Video_outbuf = NULL;

The output is not a raw picture, but it does use the encoding codec

if (!) ( Oc->oformat->flags & Avfmt_rawpicture)) {

Video_outbuf_size = 200000;

Video_outbuf = Av_malloc (video_outbuf_size);

}

(43) is the initialization function of the avpacket structure. (44) More difficult to understand, and why there are some time stamps I do not understand. The other Avpacket

The assignment of member data is relatively easy to understand, and attention is given to the initialization of Video_outbuf and Video_outbuf_size, due to initialization in the reference code and

Use is not in the same function, so it is easier to ignore. (45) is a write file function, avformatcontext* OC contains information such as file name, return value RET due to the

is how much data information is written altogether, and if you return 0, you write a failure. (42) and (45) as the more important SDK functions, which will be described later.

Ix. Conclusion

The above analysis is the important data structure in ffmpeg. The following generation relationship follows the idea: (->)

Avformatcontext->avstream->avcodeccontext->avcodec

|

Avoutputformat or Avinputformat

Avframe->avpicture....>avpacket

Two Functions in FFMpeg:

In the previous section of the analysis we have seen that the FFmpeg SDK provides a number of initialization functions and coding functions. What we're going to do is start with the right primary data structure

To start, and correctly use the appropriate codec functions and read/write (I/O) operation functions. As a holistic code sdk,ffmpeg has some of his own standards

The use of the process. such as function Av_register_all (); is a "registration function" that starts with the call, and he initializes the Libavcodec, "register"

All of the codec and video file formats (format). Below, I introduce the correlation function along the thread of the Reference Code (OUPUT_EXAMPLE.C).

/******************************************************************

Main ()

******************************************************************/

1. Av_register_all ();

Usage:initialize ibavcoded, and register all codecs and formats

Every project that uses the FFmpeg SDK must call the function. Register for codec and format before you can use it. The statement in the ALLFORMATS.C, are all

Macros are interested to see.

2. Avoutputformat Guess_format (const char *short_name, const char *filename, const char *mime_type)

Usage: By file suffix name, guessing file format, in fact, is to determine the use of what encoder (or decoder).

Avoutputformat *fmt;

FMT = Guess_format (null, filename, null);

3. Avformatcontext *av_alloc_format_context (void)

Usage:allocate the output media context. is actually initializing the Avformatcontext member data avclass:

Avformatcontext *ic;

Ic->av_class = &av_format_context_class;

where

Format_to_name, options are pointer to function <