FFmpeg internal video buffer management and delivery mechanism

This paper mainly introduces the principle and process of internal management video buffer of FFmpeg decoder, the videobuffer of ffmpeg is internal management, and its flow is basically: Register processing function, frame level release, frame level application, and empty.

1 registered Get_buffer () and Release_buffer ()

Ffapi_initcodec ()

Avcodec_alloc_context ()

AVCODEC_ALLOC_CONTEXT2 ()

AVCODEC_GET_CONTEXT_DEFAULT2 (Avcodeccontext *s,...) {

......

S->get_buffer = Avcodec_default_get_buffer;

S->release_buffer = Avcodec_default_release_buffer;

......

}

2 frame-level memory request and release calls

Figure 1 Frame-level memory request and release of function calls

The 2.1 Ffapi function calls the libavcodec corresponding codec (WMV3 corresponding codec) function to decode, calling the internal buffer handler function during the process. Where buffer management is uniformly encapsulated into the MPEGVideo interface (including codec with H.261, H.263, H, Mpeg12, rv10,rv34, svq1 and VC1)

Ffapi_decode ()

Avcodec_decode_video2 ()

Avctx->codec->decode ()//The decoding function of registering codec,wmv3 during initialization is

Vc1_decode_frame () {

Decode_vc1_header;

Mpv_frame_start (); 2.2.2

Vc1_decode_blocks ();

Mpv_frame_end (); 2.2.3

}

2.2 Mpv_frame_start ()//By calling Get_buffer () to request the video buffer of the current frame.

Mpv_frame_start ()

First call Release_buffer () to release the video buffer from the non-reference frame

for (i=0; i<max_picture_count; i++)

if (S->picture[i].data[0] &&!s->picture[i].reference)

Free_frame_buffer (S, &s->picture[i]); Call S->avctx->get_buffer (), Callback Avcodec_default_release_buffer ()

Ff_alloc_picture ()

Alloc_frame_buffer ()

S->avctx->get_buffer ()//callback Avcodec_default_get_buffer ()

2.3mpv_frame_end ()//finish video plus edge operation

3 frame-level memory request and release processing methods

3.1 Internal buffer data structure

–typedef struct internalbuffer{

–int Last_pic_num;

–uint8_t *base[4];

–uint8_t *data[4];

–int Linesize[4];

–int width, height;

–enum PixelFormat pix_fmt;

–}Internalbuffer;

–typedef struct Avcodeccontext {

– ......

–int Internal_buffer_count; Records the current number of internal buffer, both Get_buffer and release_buffer need to be maintained.

–void *internal_buffer;//initialized to array internalbuffer [Internal_buffer_size]

– ......

–} Avcodeccontext;

Codec enables efficient memory management through maintenance of Internal_buffer_count and Internal_buffer.

3.2 Reference frame management related data structures

–typedef struct picture{

–uint8_t *data[4];

–int Linesize[4];

–uint8_t *base[4];

–int reference;

– ......

-} picture;

–typedef struct mpegenccontext{

– ......

–picture* picture; Initialize to array picture[internal_buffer_size]

–picture* last_picture_ptr; Refers to a forward frame

–picture* next_picture_ptr; Bidirectional prediction, refers to a frame backward

–picture* current_picture_ptr;//pointing to the current frame

– ......

–} Mpegenccontext;  

3.3 Application and Release principle

Figure 2 Memory request and release principle

(1) All Internal_buffer are zeroed when initializing

(2) When Buffer is released, the released buffer is swapped with the last valid buffer instead of freeing the memory with Av_free ().

Avcodec_default_release_buffer (Avcodeccontext *s, Avframe *pic) {

s->internal_buffer_count--;

Last = & ((internalbuffer*) s->internal_buffer) [s->internal_buffer_count];

Swap the last buffer with the buffer to be released so that last buffer becomes invalid buffer and can be applied at the next get_buffer.

Ffswap (Internalbuffer, *buf, *last);

for (i=0; i<4; i++) {

pic->data[i]=null;

}

}

(3) When applying for buffer, check whether the base address of Internal_buffer[internal_buffer_count] is not empty, if not empty, use internal_buffer[internal_buffer_count]; , use the Av_malloc () function to apply.

The benefit of this approach is to avoid frequent calls to malloc () and free (), which improves efficiency.

Avcodec_default_get_buffer (Avcodeccontext *s, Avframe *pic) {

......

Buf= & ((internalbuffer*) s->internal_buffer) [s->internal_buffer_count];

Get_size_info (size[]);

Buf->base[0, 1, 2] = Av_malloc (size[0, 1, 2]);

Buf->data[0, 1, 2] = buf->base[0, 1, 2] + padding_offset[0, 1, 2];

......

}

(4) determines that the output frame is decoded after each frame, depending on the current frame type and reference information to determine the output frame.

if (S->pict_type = = Ff_b_type | | s->low_delay) {

*pict= * (avframe*) s->current_picture_ptr;

} else if (s->last_picture_ptr! = NULL) {

*pict= * (avframe*) s->last_picture_ptr;

}

3.4 Example-Suppose decoding a non-H. IPBPB Stream.

(1) After the initialized state as shown, IBC is ctx->internal_buffer_count,curptr to s->current_picture_ptr,lastptr for s->last_picture_ Ptr,nextptr for S->next_picture_ptr.

The gpavpicture pointer is a pointer to the output image.

Figure 3 Initialization status

(2) Decoding the first I-frame, the process does not call Release_buffer (), Get_buffer () get picture[0], at this time do not output any image.

Figure 4 Decoding the state after the first I-frame

(3) Decoding the first P-frame, the process does not call Release_buffer (), Get_buffer () get picture[1], output PICTURE[0].

Figure 5 Decoding the state after the first P-frame

(4) Decoding the first B-frame, the process does not call Release_buffer (), Get_buffer () get picture[2], output PICTURE[2].

Figure 6 Decoding the state after the first B-frame

(5) Decode the second P-frame, call Release_buffer (&picture[2]), then call Get_buffer (), get picture[2], output PICTURE[1].

Figure 7 Decoding the state of a second P-frame

FFmpeg internal video buffer management and delivery mechanism