FFmpeg time stamp Processing

how video is displayed and stored

For a movie, the frame is displayed like this: I b b P. Now we need to know the information in P frames before we show B-frames. Therefore, the frame may be stored in such a way: Ipbb. That's why we have a decoding time stamp and a reason for displaying timestamps. Decoding the timestamp tells us when to decode and shows when the timestamp tells us when it needs to be displayed. So, in this case, our flow can be like this:

1 4 2 3  1234stream:i P b b

Normally pts and DTS will be different only if there are B frames in the stream.

DTS and pts

Both audio and video streams have some information about how fast and when to play them. The audio stream is sampled and the video stream has a frame rate per second. However, if we are simply synchronizing the video by a few frames and multiplying the frame rate, it is very likely that the synchronization will be lost. As a supplement, the package in the stream has a mechanism called DTS (decoding timestamp) and PTS (displaying timestamps). For these two parameters, you need to know how the movie is stored. Formats such as MPEG, using a method called B-frame (b for bidirectional bidrectional). The other two frames are called I-frames and P-frames (I represents keyframes and P represents the predicted frame). The I-frame contains a specific complete image. P frames are dependent on the previous I and P frames and are encoded using a comparison or differential method. B-frames are somewhat similar to P-frames, but they are dependent on the information of the front and back frames. This also explains why we may not get a frame image after calling Avcodec_decode_video.

time units in the FFmpegAv_time_base

The Internal timing unit (time Base) in the FFmpeg, all time in FFMEPG is a unit, such as duration in Avstream, which means that the length of the stream is duration av_time_base. Av_time_base is defined as:

#define         Av_time_base   1000000

av_time_base_q

FFmpeg the fractional representation of the internal time base, which is actually the reciprocal of the av_time_base. From its definition it is clear to see this:

#define         Av_time_base_q   (avrational) {1, av_time_base}

Avratioal is defined as follows:

struct avrational{int//numeratorint//denominator} avrational;

FFmpeg provides a function to convert the AVRATIOAL structure into a double:

Static Double av_q2d (Avrational a) { /* * * Convert rational to double.* @param a rational to convert* */    return a.num/(double) A.den;}

You can now calculate the time position of a frame over the entire video based on the PTS:

Timestamp (sec) = pts * AV_Q2D (St->time_base)

How to calculate video length:

Time (seconds) = St->duration * AV_Q2D (st->time_base)

The St here is a Avstream object pointer.

Time Base Conversion Formula

• Timestamp (ffmpeg internal timestamp) = Av_time_base * Time (seconds)
• Time (seconds) = Av_time_base_q * Timestamp (ffmpeg internal timestamp)

So when you need to jump the video to n seconds, you can use the following method:

int64_t timestamp = N * av_time_base; 2av_seek_frame (Fmtctx, Index_of_video, timestamp, avseek_flag_backward);

FFmpeg also provides us with a conversion function between different time bases:

int64_t Av_rescale_q (int64_t A, avrational BQ, Avrational CQ)

The function is to calculate a * BQ/CQ to adjust the timestamp from one time base to another. In the case of time-based conversions, we should prefer this function because it avoids overflow situations.

FFmpeg time stamp Processing