The important structure of ffmpeg

1. Relationships between the most critical structures in the FFmpeg

There are many structures in ffmpeg. The most critical structures can be divided into the following categories:

A) Solution protocol (HTTP,RTSP,RTMP,MMS)

Aviocontext,urlprotocol,urlcontext primarily stores the type and status of the protocol used by AV. The Urlprotocol stores the encapsulation format used by the input video audio. Each of these protocols corresponds to a URLPROTOCOL structure. (Note: Files in FFmpeg are also treated as a protocol "file")

b) Solution Encapsulation (FLV,AVI,RMVB,MP4)

The Avformatcontext mainly stores the information contained in the video-Audio encapsulation format, and the Avinputformat stores the encapsulation format used by the input video audio. Each video and audio package format corresponds to a AVINPUTFORMAT structure.

c) decoding (H264,mpeg2,aac,mp3)

Each Avstream stores data related to a video/audio stream, and each avstream corresponds to a avcodeccontext that stores the data that the video/audio stream uses for decoding; a avcodec in each avcodeccontext that contains the video /audio corresponding to the decoder. Each decoder corresponds to a AVCODEC structure.

d) Storage data

Video, each structure is usually saved one frame; audio may have several frames

Pre-decoded data: Avpacket

Decoded data: Avframe

The correspondence between them is as follows:

1) Avpacket Structural body

Avpacket is a structure that stores information about compressed encoded data.

In the avpacket structure, there are several important variables:

uint8_t *data: Compresses the encoded data.

For example, for H. Data for 1 Avpacket usually corresponds to a nal.

Note: Here is just the correspondence, not the exact same. There is a slight difference between them: using the FFmpeg class library to isolate the H. D stream in a multimedia file

Therefore, when using ffmpeg for AV processing, it is often possible to write the resulting avpacket data directly into a file, resulting in audio-visual stream files.

The size of the int size:data

int64_t pts: Displaying timestamps

int64_t DTS: Decoding timestamps

int Stream_index: Identifies the video/audio stream to which the avpacket belongs.

Description: FFmpeg uses Avpacket to temporarily store the media data (a sound/video frame, a caption packet, etc.) and additional information (decoding timestamp, display timestamp, length, etc.) after the de-duplication and decoding. Where: DTS represents the decoding timestamp, and PTS represents the time stamp displayed, and their units are the time baselines of the owning media stream. Stream_index gives the index of the owning media stream, data is the buffer pointer, size is length, duration is the duration of the data, and is the time base of the owning media stream; Pos represents the byte offset of the data in the media stream; Destruct is the function pointer used to release the data buffer; Flags is a flag field, where a minimum of 1 indicates that the data is a keyframe.

The avpacket structure itself is simply a container that uses the data member to refer to the actual buffer. This buffer is usually created by av_new_packet, but it may also be created by FFmpeg's API (such as Av_read_frame). When a data buffer for a avpacket structure is no longer being used, it needs to be freed by calling Av_free_packet. Av_free_packet calls the destruct function of the struct itself, which has two conditions: 1) av_destruct_packet_nofree or 0;2) Av_destruct_packet, of which, in case 1) Only the value of data and size is cleared by 0, which is 2) to actually release the buffer.

The FFmpeg internally uses the Avpacket structure to build the buffer load data, providing the destruct function, and if FFMPEG intends to maintain the buffer itself, destruct is set to Av_destruct_packet_nofree and the user calls Av_ Free_packet is not able to release the buffer when it is cleared, and if FFmpeg intends to give the buffer to the caller completely, set destruct to Av_destruct_packet, which means it can be freed. To be safe, if the user wishes to freely use a avpacket structure created internally by the FFmpeg, it is best to call Av_dup_packet to clone the buffer and convert it to a buffer that can be freed avpacket, so as not to cause an exception to the improper usage of the buffer. Av_dup_packet creates a new buffer for the avpacket of the destruct pointer for Av_destruct_packet_nofree, then copies the data from the original buffer to the new buffer, and the value of the set data is the address of the new buffer. Also set the destruct pointer to Av_destruct_packet.

2) Avformatcontext

When using ffmpeg for development, Avformatcontext is a constant data structure that many functions use as parameters. It is the structure of ffmpeg encapsulation (Flv,mp4,rmvb,avi) function. Here are a few of the main variables (consider the case of decoding here):

struct Avinputformat *iformat: Encapsulation format for input data

Aviocontext *PB: Caching of input data

unsigned int nb_streams: The number of video audio streams

Avstream **streams: Video audio streaming

Char filename[1024]: file name

int64_t Duration: Duration (in microseconds us, converted to seconds, divided by 1000000)

int bit_rate: Bit rate (unit bps, converted to kbps needs to be divided by 1000)

Avdictionary *metadata: Meta data 3) aviocontext struct, urlcontext struct, urlprotocol struct aviocontext struct There are several variables that are more important:

unsigned char *buffer: Cache start Location

int buffer_size: Cache size (default 32768)

unsigned char *buf_ptr: Where the current pointer is read

unsigned char *buf_end: where the cache ends

void *opaque:urlcontext Structural Body

In the case of decoding, buffer is used to store ffmpeg read-in data. For example, to open a video file, first read the data from the hard disk into buffer, and then send the decoder for decoding.

Where opaque points to the urlcontext. Note that this struct is not in the header file provided by FFmpeg, but in the source code of the FFmpeg. The definition from the FFmpeg source code is as follows:

[CPP]  View Plain  copy typedef struct urlcontext {       const  avclass *av_class; ///< information for av_log ().  Set by url_ Open () .       struct URLProtocol *prot;        int flags;       int is_streamed;  /**< true  if streamed  (no seek possible), default = false */       int max_packet_size;  /**< if non zero, the  stream is packetized with this max packet size */       void *priv_data;       char *filename; /** < specified URL */       int is_connected;       AVIOInterruptCB interrupt_callback;  } urlcontext;  
There is also a structural body urlprotocol in the urlcontext structure. Note: Each protocol (rtp,rtmp,file, etc.) corresponds to a urlprotocol. This struct is also not in the header file provided by FFmpeg. To ffmpeg its definition from the source code:

[CPP] view plain copy typedef struct URLPROTOCOL {const char *name;       Int (*url_open) (Urlcontext *h, const char *url, int flags);       Int (*url_read) (Urlcontext *h, unsigned char *buf, int size); Int (*url_write) (Urlcontext *h, const