FFmpeg internal video buffer management and delivery mechanism

Source: Internet
Author: User

This paper mainly introduces the principle and process of internal management video buffer of FFmpeg decoder, the videobuffer of ffmpeg is internal management, and its flow is basically: Register processing function, frame level release, frame level application, and empty.

1 registered Get_buffer () and Release_buffer ()

Ffapi_initcodec ()

Avcodec_alloc_context ()

AVCODEC_ALLOC_CONTEXT2 ()

AVCODEC_GET_CONTEXT_DEFAULT2 (Avcodeccontext *s,...) {

......

S->get_buffer = Avcodec_default_get_buffer;

S->release_buffer = Avcodec_default_release_buffer;

......

}

2 frame-level memory request and release calls

Figure 1 Frame-level memory request and release of function calls

The 2.1 Ffapi function calls the libavcodec corresponding codec (WMV3 corresponding codec) function to decode, calling the internal buffer handler function during the process. Where buffer management is uniformly encapsulated into the MPEGVideo interface (including codec with H.261, H.263, H, Mpeg12, rv10,rv34, svq1 and VC1)

Ffapi_decode ()

Avcodec_decode_video2 ()

Avctx->codec->decode ()//The decoding function of registering codec,wmv3 during initialization is

Vc1_decode_frame () {

Decode_vc1_header;

Mpv_frame_start (); 2.2.2

Vc1_decode_blocks ();

Mpv_frame_end (); 2.2.3

}

2.2 Mpv_frame_start ()//By calling Get_buffer () to request the video buffer of the current frame.

Mpv_frame_start ()

First call Release_buffer () to release the video buffer from the non-reference frame

for (i=0; i<max_picture_count; i++)

if (S->picture[i].data[0] &&!s->picture[i].reference)

Free_frame_buffer (S, &s->picture[i]); Call S->avctx->get_buffer (), Callback Avcodec_default_release_buffer ()

Ff_alloc_picture ()

Alloc_frame_buffer ()

S->avctx->get_buffer ()//callback Avcodec_default_get_buffer ()

2.3mpv_frame_end ()//finish video plus edge operation

3 frame-level memory request and release processing methods

3.1 Internal buffer data structure

–typedef struct internalbuffer{

–int Last_pic_num;

–uint8_t *base[4];

–uint8_t *data[4];

–int Linesize[4];

–int width, height;

–enum PixelFormat pix_fmt;

–}Internalbuffer;

–typedef struct Avcodeccontext {

– ......

–int Internal_buffer_count; Records the current number of internal buffer, both Get_buffer and release_buffer need to be maintained.

–void *internal_buffer;//initialized to array internalbuffer [Internal_buffer_size]

– ......

–} Avcodeccontext;

Codec enables efficient memory management through maintenance of Internal_buffer_count and Internal_buffer.

3.2 Reference frame management related data structures

–typedef struct picture{

–uint8_t *data[4];

–int Linesize[4];

–uint8_t *base[4];

–int reference;

– ......

-} picture;

–typedef struct mpegenccontext{

– ......

–picture* picture; Initialize to array picture[internal_buffer_size]

–picture* last_picture_ptr; Refers to a forward frame

–picture* next_picture_ptr; Bidirectional prediction, refers to a frame backward

–picture* current_picture_ptr;//pointing to the current frame

– ......

–} Mpegenccontext;  

3.3 Application and Release principle

Figure 2 Memory request and release principle

(1) All Internal_buffer are zeroed when initializing

(2) When Buffer is released, the released buffer is swapped with the last valid buffer instead of freeing the memory with Av_free ().

Avcodec_default_release_buffer (Avcodeccontext *s, Avframe *pic) {

s->internal_buffer_count--;

Last = & ((internalbuffer*) s->internal_buffer) [s->internal_buffer_count];

Swap the last buffer with the buffer to be released so that last buffer becomes invalid buffer and can be applied at the next get_buffer.

Ffswap (Internalbuffer, *buf, *last);

for (i=0; i<4; i++) {

pic->data[i]=null;

}

}

(3) When applying for buffer, check whether the base address of Internal_buffer[internal_buffer_count] is not empty, if not empty, use internal_buffer[internal_buffer_count]; , use the Av_malloc () function to apply.

The benefit of this approach is to avoid frequent calls to malloc () and free (), which improves efficiency.

Avcodec_default_get_buffer (Avcodeccontext *s, Avframe *pic) {

......

Buf= & ((internalbuffer*) s->internal_buffer) [s->internal_buffer_count];

Get_size_info (size[]);

Buf->base[0, 1, 2] = Av_malloc (size[0, 1, 2]);

Buf->data[0, 1, 2] = buf->base[0, 1, 2] + padding_offset[0, 1, 2];

......

}

(4) determines that the output frame is decoded after each frame, depending on the current frame type and reference information to determine the output frame.

if (S->pict_type = = Ff_b_type | | s->low_delay) {

*pict= * (avframe*) s->current_picture_ptr;

} else if (s->last_picture_ptr! = NULL) {

*pict= * (avframe*) s->last_picture_ptr;

}

3.4 Example-Suppose decoding a non-H. IPBPB Stream.

(1) After the initialized state as shown, IBC is ctx->internal_buffer_count,curptr to s->current_picture_ptr,lastptr for s->last_picture_ Ptr,nextptr for S->next_picture_ptr.

The gpavpicture pointer is a pointer to the output image.

Figure 3 Initialization status

(2) Decoding the first I-frame, the process does not call Release_buffer (), Get_buffer () get picture[0], at this time do not output any image.

Figure 4 Decoding the state after the first I-frame

(3) Decoding the first P-frame, the process does not call Release_buffer (), Get_buffer () get picture[1], output PICTURE[0].

Figure 5 Decoding the state after the first P-frame

(4) Decoding the first B-frame, the process does not call Release_buffer (), Get_buffer () get picture[2], output PICTURE[2].

Figure 6 Decoding the state after the first B-frame

(5) Decode the second P-frame, call Release_buffer (&picture[2]), then call Get_buffer (), get picture[2], output PICTURE[1].

Figure 7 Decoding the state of a second P-frame

FFmpeg internal video buffer management and delivery mechanism

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.