Object-oriented design of C language--discussion on x264/ffmpeg architecture

1. Why to use C language

Until today, although the C language is not the most used language, but C is not old, in many core system code, still running is beautifully designed C, the vast majority of embedded development core library software is developed by C, most of the standard algorithm is based on the standard C design. C language with its concise, flexible and superior performance, still in the heart of the core software designers have an unshakable position.

2. Why object-oriented

The process is often considered to be a strict top-down, progressive subdivision of the design approach, the step-by large-scale design decomposition into a small concrete implementation. Object-oriented is based on the object model to describe the problem domain, more close to people's understanding of the objective world process. The following object-oriented benefits are listed in the General Software Engineering Lesson Plans: (1) modularity (2) Abstraction (3) information hiding (4) weak coupling (5) strong cohesion (6) reusable. These benefits come from the use of three basic methods of object-oriented: encapsulation, inheritance, and polymorphism. In the actual software engineering project, it is precisely because of these characteristics of object-oriented, this method of analysis is widely welcomed and continues to develop, the recent development of the framework of the Java EE project is the best example, whether the object injection or Pojo to the object-oriented approach to add more new vitality.

From "1" you can see a simple example of process-oriented and object-oriented analysis of the same project, so that the object-oriented approach is to divide the problem by functions rather than steps. In the series "2", the author analyzes how to simulate encapsulation, inheritance and polymorphism with C language structure and function pointers, and analyzes the performance loss with simple experiments, the result is that the loss of C language simulation class can be neglected. The standard C object-oriented code example can be seen from LINUX,GTK and other sources, this article only analyzes the basic architecture of ffmpeg and X264. Object-oriented is an efficient method of analysis and design, and C language does not directly support the object-oriented syntax, c to imitate C + + is not necessary, when considering the use of C language to build large projects, the use of object-oriented design, and appropriate construction C syntax support such design ideas are needed.

3.FFMPEG Architecture Analysis

FFmpeg is currently the most widely used codec software library, supporting a variety of popular codecs, it is implemented in C language, not only is integrated into a variety of PC software, but also often ported to a variety of embedded devices. Using an object-oriented approach to imagine such a codec library, the first thing to think about is to construct classes of various codecs, and then for their abstract base class to determine the rules of running the data flow, according to the algorithm to transform the input and output objects.

In the actual code, these codecs are divided into encoder/decoder,muxer/demuxer and device three objects, respectively, corresponding to the codec, input and output format and device. At the beginning of the main function, these three types of objects are initialized. In Avcodec_register_all, many codecs are registered, including the video's codec and the X264 encoder, etc.

Register_decoder (H264, H264);
Register_encoder (LIBX264, libx264);

Find the relevant macro code below

#define Register_encoder (x,x) {\
extern avcodecx# #_encoder; \
if (config_# #X # #_ENCODER) avcodec_register (&x# #_encoder); }
#define Register_decoder (x,x) {\
extern Avcodec x# #_decoder;
if (config_# #X # #_DECODER) avcodec_register (&x# #_decoder); }

This is actually in the code according to config_# #X # #_ENCODER这样的编译选项来注册libx264_encoder和h264_decoder, the process of registration occurs in Avcodec_register (Avcodec *codec) function, is actually to add Libx264_encoder, H264_decoder specific codec to the global list First_avcodec, the input parameter avcodec is a struct, can be understood as the base class of the codec, which not only contains the name, ID attributes, and contains the following function pointers for each specific codec extension class implementation.

Int (*init) (Avcodeccontext *);
Int (*encode) (Avcodeccontext *, uint8_t *buf, int buf_size,void *data);
Int (*close) (Avcodeccontext *);
Int (*decode) (Avcodeccontext *, void *outdata, Int*outdata_size,
Const uint8_t *BUF, int buf_size);
void (*flush) (Avcodeccontext *);

Continue to trace libx264, also known as X264 's static encoding library, which was introduced as a h. A encoder at FFmpeg compile time. The following code is in the LIBX264.C

Avcodec Libx264_encoder = {
. Name = "Libx264",
. Type = Codec_type_video,
. id = codec_id_h264,
. priv_data_size = sizeof (X264context),
. init = X264_init,
. Encode = X264_frame,
. Close = X264_close,
. capabilities = Codec_cap_delay,
. Pix_fmts = (enum pixelformat[]) {Pix_fmt_yuv420p,pix_fmt_none},
. Long_name = Null_if_config_small ("libx264 h.264/avc/mpeg-4 avc/mpeg-4 Part 10"),
};

The property and method assignments from AVCODEC are specified here. which
. init = X264_init,
. Encode = X264_frame,
. Close = X264_close,
The function pointer is pointed to a specific function, and these three functions will be implemented using the API provided in the libx264 Static library, which is the main interface function of X264. PIX_FMTS defines the supported input formats, where 4:2:0
pix_fmt_yuv420p,///< planar YUV 4:2:0, 12BPP, (1 Cr & cbsample per 2x2 Y samples)

The x264context shown above encapsulates the context management data required by X264,
typedef struct X264CONTEXT {
x264_param_t params;
x264_t *enc;
x264_picture_t pic;
Avframe Out_pic;
} X264context;
It belongs to the void*priv_data variable of the struct Avcodeccontext, which defines the context properties of each codec's private, Avcodeccontext is similar to the context base class, and provides other contextual properties that represent the screen resolution rate, quantization scope, and so on RTP_ Callback such as function pointers for codec use.

Back to the main function, you can see the completion of various types of codecs, input and output format and device registration, the context initialization and codec parameters are read, and then call the Av_encode () function for specific codec work. View the process of the function according to its comments:

1. Input and output stream initialization.
2. Determine the required codec based on the input and output stream and initialize it.
3. Write the parts of the output file

Focus on STEP2 and 3 to see how to use the codec base class of the previous analysis to achieve polymorphism. Probably look at the relationship of this code, found in the FFmpeg, you can use class diagram to represent the approximate codec combination.

You can refer to "3" to understand the meaning of these structures (see appendix). Here you will invoke a series of functions from UTILS.C, where the Avcodec_open () function, which is called by the Open codec, will run the following code:
Avctx->codec = codec;
avctx->codec_id = codec->id;
Avctx->frame_number = 0;
if (avctx->codec->init) {
RET =avctx->codec->init (AVCTX);
The codec that is specifically adapted is initialized, and here Avctx->codec->init (AVCTX) is the specific initialization function that invokes the function pointer definition in AVCODEC, such as X264_init.

The Avcodec_encode_video () and Avcodec_encode_audio () are called by the Output_packet () to encode the audio and video, and will also use the function pointer avctx->codec-> Encode () calls the encoding function of the adapter, such as X264_frame, for specific work.

From the above analysis, we can see how FFmpeg uses object-oriented to abstract codec behavior by combining and inheriting relationships to materialize each codec entity. Suppose you want to add a new decoder H265 to FFmpeg, here's what to do:
1. Add Config_h265_decoder in config compile configuration
2. Register H265 decoder with macros
3. Define AVCODEC 265_decoder variables, initialize properties and function pointers
4. Use the decoder API to materialize 265_decoder's init and other function pointers

By completing the above steps, you can put the new decoder into ffmpeg, and the external matching and running rules are implemented by the multi-state of the base class.

4. X264 Architecture Analysis

X264 is a French university student from 2004, launched by the open-source H/S encoder, the PC to compile-level code optimization, discard the chip set and multi-reference frame and other performance-efficient features to improve the coding efficiency, it was ffmpeg as the introduction of. 264 code library, also ported to many DSP embedded platforms. An example of the X264 in FFmpeg has been analyzed in the previous section, which will continue to deepen the understanding of the relevant content in conjunction with the X264 framework.

Before you look at the code, think about how to object-oriented analysis for a specific encoder. The abstraction of different algorithms in the entropy coding part, and the abstraction of various estimation algorithms in frame or inter-frame coding can be constructed as classes.

In X264, we see external APIs and context variables declared in X264.h, and in API functions, functions about accessibility are defined in COMMON.C
void X264_picture_alloc (x264_picture_t *pic, int i_csp, int i_width, inti_height);
void X264_picture_clean (x264_picture_t *pic);
int X264_nal_encode (void *, int *, int b_annexeb, x264_nal_t *nal);
and the coding function is defined in ENCODER.C
x264_t *x264_encoder_open (x264_param_t *);
int X264_encoder_reconfig (x264_t *, x264_param_t *);
int X264_encoder_headers (x264_t *, x264_nal_t * *, int*);
int X264_encoder_encode (x264_t *, x264_nal_t * *, int*, x264_picture_t *, x264_picture_t *);
void X264_encoder_close (x264_t *);
In the x264.c file, the main function of the program can be seen as an example of how the API is used, and it also implements the actual functionality by invoking the API and context variables in X264.h.

X264 The most important structural body of the record context data x264_t defined in Common.h, which contains all the H-Encoding related variables from the thread control variable to the specific SPS, PPS, quantization matrix, Cabac context, and so on. It contains the following structure
    x264_predict_t     predict_16x16[4+3];
    x264_predict_t     predict_8x8c[4+3];
    x264_predict8x8_t   predict_8x8[9+3];
    x264_predict_t     predict_4x4[9+3];
    x264_predict_8x8_filter_t predict_8x8_filter;

X264_pixel_function_t pixf;
x264_mc_functions_t MC;
X264_dct_function_t DCTF;
X264_zigzag_function_t ZIGZAGF;
X264_quant_function_t quantf;
X264_deblock_function_t Loopf;
The trace view can be seen either as a function pointer or as a struct consisting of a function pointer, which is intended to be interface to an object-oriented interface declaration. These function pointers will be initialized in the X264_encoder_open () function, where initialization first provides different functions to implement code snippets based on the different CPUs, many of which are implemented with the possible assembler to improve the efficiency of code operation. Next, the functions of similar functions are centrally managed, such as nine predictive functions of 4 and INTRA4 similar to INTRA16 are managed with array of function pointers.

X264_encoder_encode () is the main function responsible for coding, and the X264_slice_write () contained therein is responsible for the specific coding of the layers, including intra-frame and inter-frame macro block coding. Here, the behavior of Cabac and CAVLC is differentiated according to H->param.b_cabac, respectively, running X264_macroblock_write_cabac () and X264_macroblock_write_ CAVLC () to write the stream, in this part, the function function according to the file definition collation, basically according to the coding flowchart to run, looks more like the process-oriented writing, in has initialized the concrete function pointer, the program has been according to the coding process logic realization. If we look at the overall architecture, x264 uses this kind of interface to realize the weak coupling and reusability, and uses the x264_t as the consistent context to realize the information encapsulation and polymorphism.

This article is about to analyze the code architecture of ffmpeg/x264, focus on the implementation of C language to achieve object-oriented coding, although not forcibly to C + +, but also the implementation of features, to ensure practicality. It is worth to plan C language software project to draw lessons from.

"References"

1. "Use examples to illustrate object-oriented and process-oriented differences"
2. liyuming1978, "liyuming1978 's Column"
3. "FFmpeg Framework Code reading"

Appendix: Excerpt from "3"

3. Match of current Muxer/demuxer
In the FFmpeg file conversion process, the first thing to do is to match the suffix name of the incoming file and the outgoing file [Fixme].
Suitable for demuxer and muxer. The Demuxer and muxer on the match are saved as shown below, defined in the FFMPEG.C
In global variables File_iformat and File_oformat:
Static Avinputformat *file_iformat;
Static Avoutputformat *file_oformat;

3.1 Demuxer Match
The Staticavinputformat *av_probe_input_format2 in LIBAVFORMAT\UTILS.C (
Avprobedata *pd, int is_opened, int *score_max) function use is based on the incoming probe data
, calling the Read_probe interface of each demuxer, in turn, to see if the Demuxer matches the contents of the incoming file.
Judge. The order of invocation is as follows:
void parse_options (int argc, char **argv, const optiondef *options,
void (* parse_arg_function) (const char *));
static void Opt_input_file (const char *filename)
int Av_open_input_file (...)
Avinputformat*av_probe_input_format (Avprobedata *pd,
int is_opened)
Static AVINPUTFORMAT*AV_PROBE_INPUT_FORMAT2 (...)
The Opt_input_file function is stored in the const OPTIONDEF options[] Array and is used to
void parse_options (int argc, char **argv, const optiondef *options) in parsing argv
The "-i" parameter, which is called when the file name is entered.

3.2 Muxer Match
Unlike the demuxer, the Muxer match is called the Guess_format function, according to the argv of the main () function.
The output file suffix name to be performed.
void parse_options (int argc, char **argv, const optiondef *options,
void (* parse_arg_function) (const char *));
void Parse_arg_file (const char *filename)
static void Opt_output_file (const char *filename)
Avoutputformat *guess_format (Constchar *short_name,
const Char *filename,
const char *mime_type)

3.3 Matching of current Encoder/decoder
In addition to parsing incoming parameters and initializing the parse_options () function of demuxer and muxer in the main () function,
Other functions are done in the Av_encode () function.
The following two functions are available in LIBAVCODEC\UTILS.C:
Avcodec *avcodec_find_encoder (enum codecid ID)
Avcodec *avcodec_find_decoder (enum codecid ID)
Their function is to find the matching encoder and decoder according to the incoming codecid.

At the beginning of the Av_encode () function, the individual avinputstream and Avoutputstream are initialized first, then respectively
With the above two functions, and the matching encoder and decoder are respectively guaranteed to exist:
Avinputstream->avstream *st->avcodeccontext *codec->struct Avcodec*codec and
Avoutputstream->avstream *st->avcodeccontext *codec->struct avcodec*codec variable.

4. Other major data structures
4.1 Avformatcontext
Avformatcontext is the main structure for implementing input and output functions and saving related data during the FFmpeg format conversion process.
Each input and output file has a corresponding entity in the global variable of the pointer array defined below.
Static Avformatcontext *output_files[max_files];
Static Avformatcontext *input_files[max_files];
For inputs and outputs, because the same struct is common, it is necessary to separate the iformat defined in the structure as follows
or the Oformat member assignment value.
struct Avinputformat *iformat;
struct Avoutputformat *oformat;
For a avformatcontext, these two members cannot have values at the same time, i.e. a avformatcontext cannot simultaneously
Contains Demuxer and muxer. The matching muxer and parse_options are found in the () function at the beginning of the main () function.
After demuxer, the avformatcontext structure of each input and output is initialized according to the incoming argv parameter, and
The corresponding array of output_files and input_files pointers exists. In the Av_encode () function, output_files
And Input_files are passed in as function parameters and are not used anywhere else.

4.2 Avcodeccontext
Save Avcodec pointers and codec related data, such as the width of video, Height,audio's sample rate, and so on.
CODEC_TYPE,CODEC_ID two variables in avcodeccontext for Encoder/decoder, the most
Important.
Enum Codectype Codec_type; /* See CODEC_TYPE_XXX */
Enum Codecid codec_id; /* SEECODEC_ID_XXX */

As shown above, Codec_type saves the media types such as Codec_type_video,codec_type_audio,
codec_id Save is the codec_id_flv1,codec_id_vp6f and other encoding methods.

To support the FLV format as an example, in the Av_open_input_file (...) function described above, match to the correct
After Avinputformat Demuxer, Avinputformat is called through the Av_open_input_stream () function.
Read_header interface to execute the Flv_read_header () function in FLVDEC.C. In the Flv_read_header () function
, create the corresponding video or audio Avstream according to the data in the file header, and set the Avstream
The correct codec_type value for the Avcodeccontext. The codec_id value is the Flv_read_packet () letter during the decoding process.
Number is set according to the data in each packet header.

4.3 avstream
The Avstream structure holds information such as codecs, data segments, and so on that are related to data flow. It is more important to have the following two members:
Avcodeccontext *codec; /**< Codec Context */
void *priv_data;
Where the codec pointer holds the encoder or decoder structure described in the previous section. The Priv_data pointer is saved with a
Volume codec stream related data, as shown in the following code, in the ASF decoding process, Priv_data is
The data for the asfstream structure.
Avstream *st;
Asfstream *asf_st;
... ...
St->priv_data = Asf_st;

4.4 Avinputstream/avoutputstream
The aforementioned avstream structures are encapsulated in Avinputstream and Avoutputstream, depending on the input and output streams
Structure, used in the Av_encode () function. Time-related information is also saved in the Avinputstream.
The Avoutputstream also holds information related to audio and video synchronization.

4.5 Avpacket
The avpacket structure is defined as follows, which is used to hold the read packet data.
typedef struct AVPACKET {
int64_t pts; <presentation time stamp in time_base units
int64_t DTS; <decompression time stamp in time_base units
uint8_t *data;
int size;
int stream_index;
int flags;
int duration; <presentation Duration in time_base units (0 if not available)
void (*destruct) (struct avpacket *);
void *priv;
int64_t POS; < byteposition in stream, 1 if unknown
} Avpacket;

In the Av_encode () function, call the Avinputformat
(*read_packet) (Struct Avformatcontext *, avpacket *PKT) interface, reads a frame number of input files
It is stored in the Avpacket member of the current input avformatcontext.

Original: http://jmvc.blog.sohu.com/120705757.html

Object-oriented design of C language--discussion on x264/ffmpeg architecture