Internal video buffer management and transfer mechanism of FFMPEG

This article mainly introduces the principle and process of the FFMPEG decoder's internal management of video buffer. The videobuffer of FFmpeg is an internal management, and its process is roughly as follows: register the processing function-> frame-level release-> frame-level application-> clear.

1 register get_buffer () and release_buffer ()

Ffapi_initcodec ()

Avcodec_alloc_context ()

Avcodec_alloc_context2 ()

Avcodec_get_context_default2 (avcodeccontext * s ,...){

......

S-> get_buffer = avcodec_default_get_buffer;

S-> release_buffer = avcodec_default_release_buffer;

......

}

2-Frame Memory application and release call

Figure 1 function call for Frame-level memory application and release

2.1 The ffapi function calls the codec (vc1 for wmv3) function corresponding to libavcodec for decoding, and calls the internal buffer processing function during the process. Buffer Management is encapsulated in the mpegvideo interface (including codec such as H.261, H.263, H.264, mpeg12, rv10, rv34, svq1, and vc1)

Ffapi_decode ()

Avcodec_decode_video2 ()

Avctx-> codec-> decode () // register codec during initialization. The decoding function of wmv3 is

Vc1_decode_frame (){

Decode_vc1_header;

Mpv_frame_start (); // 2.2.2

Vc1_decode_blocks ();

Mpv_frame_end (); // 2.2.3

}

2.2 mpv_frame_start () // call get_buffer () to apply for the video buffer of the current frame.

Mpv_frame_start ()

// Call release_buffer () to release the video buffer of non-reference frames.

For (I = 0; I <max_picture_count; I ++)

If (S-> picture [I]. Data [0] &! S-> picture [I]. Reference)

Free_frame_buffer (S, & S-> picture [I]); // call S-> avctx-> get_buffer () to call back avcodec_default_release_buffer ()

Ff_alloc_picture ()

Alloc_frame_buffer ()

S-> avctx-> get_buffer () // callback avcodec_default_get_buffer ()

2.3mpv _ frame_end () // complete video edge adding and other operations

How to apply for and release three frames of memory

3.1 Internal buffer Data Structure

-Typedef struct internalbuffer {

-Int last_pic_num;

-Uint8_t * base [4];

-Uint8_t * Data [4];

-Int linesize [4];

-Int width, height;

-Enum pixelformat pix_fmt;

-}Internalbuffer;

-Typedef struct avcodeccontext {

-......

-Int internal_buffer_count; // record the current number of internal buffers. Maintenance is required for get_buffer and release_buffer.

-Void * internal_buffer; // initialize to the array internalbuffer [internal_buffer_size]

-......

-}Avcodeccontext;

Codec implements efficient memory management by maintaining internal_buffer_count and internal_buffer.

3.2 reference frame management related data structures

-Typedef struct picture {

-Uint8_t * Data [4];

-Int linesize [4];

-Uint8_t * base [4];

-Int reference;

-......

-}Picture;

-Typedef struct mpegenccontext {

-......

-Picture * picture; // initialize to the array picture [internal_buffer_size]

-Picture * last_picture_ptr; // point to the previous Frame

-Picture * next_picture_ptr; // point to the next frame in bidirectional prediction.

-Picture * current_picture_ptr; // point to the current frame

-......

-}Mpegenccontext;

3.3 How applications and releases work

Figure 2 principle of memory application and release

(1) reset all internal_buffer during initialization

(2) When the buffer is released, the released buffer is exchanged with the last valid buffer instead of av_free.

Avcodec_default_release_buffer (avcodeccontext * s, avframe * pic ){

S-> internal_buffer_count --;

Last = & (internalbuffer *) S-> internal_buffer) [S-> internal_buffer_count];

// Exchange the last buffer with the buffer to be released, so that the last buffer becomes invalid buffer and can be applied for the next time get_buffer is used.

Ffswap (internalbuffer, * Buf, * Last );

For (I = 0; I <4; I ++ ){

PIC-> data [I] = NULL;

}

}

(3) when applying for a buffer, check whether the base address of internal_buffer [internal_buffer_count] is not empty. If not empty, use internal_buffer [internal_buffer_count] directly. If empty, use the av_malloc () function to apply.

This solution avoids frequent calls to malloc () and free (), thus improving the efficiency.

Avcodec_default_get_buffer (avcodeccontext * s, avframe * pic ){

......

Buf = & (internalbuffer *) S-> internal_buffer) [S-> internal_buffer_count];

Get_size_info (size []);

Buf-> base [0, 1, 2] = av_malloc (size [0, 1, 2]);

Buf-> data [0, 1, 2] = Buf-> base [0, 1, 2] + padding_offset [0, 1, 2];

......

}

(4) After decoding each frame, the output frame is determined based on the current frame type and reference information.

If (S-> pict_type = ff_ B _type | S-> low_delay ){

* PICT = * (avframe *) S-> current_picture_ptr;

} Else if (S-> last_picture_ptr! = NULL ){

* PICT = * (avframe *) S-> last_picture_ptr;

}

3.4 example -- decode the non-H.264 code stream of ipbpb.

(1) The status after Initialization is shown in. IBC is CTX-> internal_buffer_count, curptr is s-> current_picture_ptr, lastptr is s-> last_picture_ptr, and nextptr is s-> next_picture_ptr.

GpavpictureThe pointer is the pointer to the output image.

Figure 3 initialization status

(2) decode the first I frame. During the process, release_buffer () is not called, and get_buffer () is used to obtain picture [0]. At this time, no image is output.

Figure 4 Status After decoding the first I frame

(3) decode the first p frame without calling release_buffer (). get_buffer () gets picture [1] and outputs picture [0].

Figure 5 State After decoding the first p Frame

(4) decode the first B frame without calling release_buffer (). get_buffer () gets picture [2] and outputs picture [2].

Figure 6 State After decoding the first B frame

(5) decode the second p frame, call release_buffer (& picture [2]), call get_buffer (), obtain picture [2], and output picture [1].

Figure 7 decoding the status of the second p Frame