FFmpeg codec detailed process (reprint)

1. Register all container formats and Codec:av_register_all ()

2. Open file: Av_open_input_file ()

3. Extracting stream information from a file: Av_find_stream_info ()

4. Exhaustive all flow, look for the type of Codec_type_video

5. Find the corresponding decoder: Avcodec_find_decoder ()

6. Open codec: Avcodec_open ()

7. Allocating memory for decoding frames: Avcodec_alloc_frame ()

8. Continuously extracting frame data from the stream: Av_read_frame ()

9. Determine the type of frame, for video frame call: Avcodec_decode_video ()

10. After decoding, release decoder: Avcodec_close ()

11. Close the input file: Av_close_input_file ()

The first thing to do is to open a video file and get a stream from it. The first thing we need to do is use Av_register_all () to initialize the Libavformat/libavcodec:

This step registers all available file formats and encoders in the library so that when a file is opened, they will be able to automatically select the appropriate file format and encoder. Av_register_all () is called only once, so it is placed in the initialization code. You can also simply register your personal file format and encoding.

Next, open the file:

Avformatcontext *pformatctx;
const char *filename= "Myvideo.mpg";
Av_open_input_file (&pformatctx, filename, null, 0, NULL);//Open video file
The last three parameters describe the file format, buffer size and format parameters, and we tell Libavformat to automatically probe the file format and use the default buffer size by simply indicating null or 0. The format parameter here refers to the video output parameters, such as the coordinates of the wide height.

Next, we need to remove the stream information contained in the file:
Av_find_stream_info (PFORMATCTX);//Remove stream information

Avformatcontext Structural Body

Dump_format (pformatctx, 0, filename, false);//We can use this function to get all the output parameters.

for (i=0; i<pformatctx->nb_streams; i++)//differentiate video streams and audio streams
if (pformatctx->streams->codec.codec_type==codec_type_video)//Find the video stream, you can also switch to audio
{
Videostream=i;
Break
}

Next we need to find the decoder.

Avcodec *pcodec;
Pcodec=avcodec_find_decoder (pcodecctx->codec_id);

Avcodec_open (Pcodecctx, PCODEC);//Open decoder
Allocate space for video frames to store decoded pictures:

Avframe *pframe;
Pframe=avcodec_alloc_frame ();

Start decoding///////////////////////////////////////////

The first step is of course reading the data:

What we're going to do is read the entire video stream by reading the package, then decode it into frames, finally convert the format and save it.

while (Av_read_frame (Pformatctx, &packet) >=0) {//Read data

if (Packet.stream_index==videostream) {//Determine if the video stream

Avcodec_decode_video (Pcodecctx,pframe, &framefinished,

Packet.data, packet.size); Decoding

if (framefinished) {

Img_convert (Avpicture *) Pframergb, pix_fmt_rgb24, (avpicture*) pframe, pcodecctx->pix_fmt, PCodecCtx->width, Pcodecctx->height);//Conversion}

Saveframe (Pframergb, Pcodecctx->width,pcodecctx->height, i); Save data

Av_free_packet (&packet); Release

Av_read_frame () reads a package and saves it to the avpacket structure. The data can be released later via Av_free_packet (). The function Avcodec_decode_video () converts the package to a frame. However, when decoding a packet, we may not get the information we need about the frame. So, when we get to the next frame, Avcodec_decode_video () sets the frame end flag for us framefinished. Finally, we use the Img_convert () function to convert frames from the original format (PCODECCTX->PIX_FMT) to RGB format. Keep in mind that you can convert a pointer of a avframe struct to a pointer to the avpicture struct body. Finally, we pass the frame and height width information to our saveframe function.

By the time the decoding is complete, the display process uses SDL to finish taking into account that we will later use firmware for display operations, which SDL ignores.

Audio and video synchronization

DTS (decode timestamp) and PTS (display timestamp)

When we call Av_read_frame () to get a package, PTS and DTS information is also saved in the package. But the pts we really want is the PTS of the original frame we just decoded, so we know when to show it. However, the frame we get from the Avcodec_decode_video () function is just a avframe, which does not contain a useful PTS value (Note: Avframe does not contain timestamp information, but when we wait for the frame it is not what we want). We save a frame of the first packet of PTS: This will act as the whole of this frame of PTS. We can use the function Avcodec_decode_video () to calculate which package is the first packet of a frame. How to achieve it? At any time when a package starts a frame, avcodec_decode_video () invokes a function to request a buffer for a frame. Of course, FFmpeg allows us to redefine the function that allocates memory. Calculates the time stamp of the previous frame and the current frame to predict the time of the next timestamp. At the same time, we need to sync video to audio. We will set an audio time audioclock; An internal value records the location of the audio we are playing. Just like the numbers read out of any MP3 player. Now that we've synced the video to audio, the video thread uses this value to figure out if it's too fast or too slow.

FFmpeg codec detailed process (reprint)