Key Frame Extraction-Based on Motion Information

Key Frame Extraction: extract the video frames with a descriptive significance in the video as the key frame to describe the content of the video in a concise manner. It is mainly used in video compression storage, video structured indexing, and other fields.

The selection of key frames is related to people's visual feelings and has some subjectivity. Currently, most key frames are extracted from videos in a certain field, such as sports videos, news, and movies.

In this way, key frame extraction can be improved based on the characteristics of the video itself.AlgorithmAccuracy.

I am mainly concerned with the extraction of key frames of monitoring videos. The methods described below are mainly for monitoring videos.

Basic Ideas:

1) Background Modeling and target detection technologies are used to detect foreground targets, that is, motion information.

2) For the foreground target in each frame image, extract the corresponding features and work to measure the features of motion information.

3) Calculate the motion information features of each frame of the image.

4). Select a key frame based on the motion information.

The key lies in what features are used to describe the motion information of each frame of the image, and the strategy used to select the key frame based on the motion information.

References a novel video key-frame-extraction algorithm based on perceived motion energy model IEEE Trans. Circuits syst. Video audio l 2003

Extraction of motion information features in this paper: In the video compression domain, the feature is extracted using the encoding information (not very familiar)

Key Frame selection policy:

1). Smooth and split Motion Information Features

2). In each segment, select the frame with the largest motion information as the key frame.

3) if the number of key frames is limited, calculate the motion information in each segment, sort it, and select the large motion information as the candidate segment of the key frame.

A triangle model is proposed. A triangle represents a segment, and the frame corresponding to the triangle vertex is selected as the key frame.

Disadvantages:

1). The final triangle model of the motion information is related to the smoothing and split window size. The filter_win = 20 and split_win = 200 set in this article.

For some videos, these two values have a greater impact on the final detection results.

2) because only the frame corresponding to the triangle vertex is selected as the key frame, it can only represent that all foreground targets appear in the video, but cannot show the characteristics of the moving direction and trajectory of the foreground target.

Taking vehicles and humans as examples, the general trajectory of vehicle and human movement cannot be determined.

PS: this is also related to specific applications. If the application does not need to pay attention to the motion track information of the foreground target, it can be ignored.

Extension:

Because the proposed motion information extraction method is obtained from the compression encoding of the compressed domain, I am not very familiar with it.

Therefore, I mainly change the method of obtaining motion information and use motion target detection to obtain motion information.

Secondly, the feature of motion information is simply the number of pixels in the foreground target.

The key frame selection policy is similar to the previous one.