FFmpeg basic knowledge

Container: indicates the file format. After a video file is processed, an abstraction is provided for the video file. The abstraction is the container that stores the video file. In FFMPEG, the container used to abstract the file format is avformatcontext;

Stream: a data stream is a multimedia data stream that we usually see. It contains several basic data streams, including video streams, audio streams, and Subtitle streams. According to my understanding, these three basic data streams are staggered on the timeline. Only in this way can the multimedia data streams be received and played while the data streams are abstracted as avstream in FFMPEG.

Demuxer (demuxer): FFMPEG regards the multimedia files to be processed as multimedia data streams. It first puts the multimedia data streams into the container (avformatcontext), and then sends the data streams to demuxer (demuxer ), demuxer is abstracted as avinputformat in FFMPEG. I prefer demuxer to be called a shunt, because demuxer recognizes and processes various basic data streams that are staggered separately, separate data streams are sent to the video, audio, and Subtitle codecs.

Packet (packet), of course, separate data streams must be placed in the cache before being sent to the codecs for processing, and some ancillary information, such as timestamp, must be added for later processing, therefore, the cache space is a data packet. Because data streams are staggered on the timeline, all videos, audios, and subtitles are divided into segments of data, after the data in the segment is parsed from the data stream, it is stored in the respective packet. Here we want to explain that a video packet can store a video frame, for pure audio frames, if the sampling rate (Sample-rate) is fixed, one audio packet can store several audio frames. If the sampling rate is variable, A data packet can only store one audio frame.

There is no data packet concept in h264. It is estimated that h264 only processes videos, so there is no need to stream, so there is no need to stream the cached videos.

Frame: The data frame is finally reached. The real data processed by the decoder is the data frame. The frame is used to facilitate the process of abstract concepts of image information. According to my own understanding, it is the original image data information + a lot of additional information = frames; additional information is the information that facilitates subsequent encoding, such as the frame decoding time and frame display time, this information is added for the convenience of encoding/decoding and playback. Of course, the additional information of frames is much more than the two information. If you are interested, you can view the avframe. This is the abstraction of frames in FFMPEG.

We can see that the multimedia files are large, because the additional information occupies a large storage space, and there are also protocols and other information, of course, there is no such additional information, this raw or compressed data stream is really hard to handle. For example, how to distinguish the previous frame from the next frame depends on the additional information.

Image (picture): avpictue is abstracted from FFMPEG, which is the real image pixel information.

FFmpeg basic knowledge