It's time for AI to help you chase the play, take "hunting grounds" as an example

Preface

After a long break, we can finally have a double break this week:)

Originally intended to take advantage of the double Hugh nothing to do, the recent I hit the TV drama "Hunting Ground" gave, see 10 episodes time, more and more see, the amount, how to say, headhunting related not professional, more like "romance", but downloaded so many video complete, rm off a little pity, just recently on the video analysis this piece of interest, So want to try hackers, in addition, their own has been dependent on the face of the research and application of people, first from the face began. ：）

Above all is nonsense:) main techniques and steps video pumping frames

Directly with the CV2, I just use a second to draw a frame. Human face detection and feature extraction

Human face detection includes human face frame, face recognition features, and a variety of facial attributes: Gender, age, facial expression, Yan value and so on.

struct facefeature{
    1:region,
    2:double age,
    3:double Gender,
    4:list<double> racial,
    5:list<double> feature,
    6:double smile,
    7:double attractive,
    8:list<double> exps,< c12/>}

Among them, feature is mainly used for clustering, and facial feature clustering;

Face clustering is mainly used to identify a video complete in the end how many people, the method of clustering is more, it is not fine, on the GitHub search, chinese_whispers_clustering Clustering algorithm for this kind of problem is still reliable, and Dlib has the realization of this, Directly to use (why do you want to repeat the wheel:)) results of the character cluster

The recognition clustering results are as follows:

The above ID, which represents a character, will be used later.

A simple look, the main figures are all gathered together, a lot of people do not know (compared to only see 10 episodes), there are several problems, may be the clustering algorithm threshold selection is not appropriate, such as 0 and 6 are Zheng Autumn and winter, was separated, there are 346 and 763, plus 21st, this kind of aggregation is the human face of false detection The application of the character beauty value sort

It is said that men and women are generally in the play of the Yan value, but in this play, I do not agree (not only I disagree, I trained Yan value model also disagree:))

The results are as follows:

As expected, in statistical sense, the female actor's Yan value is higher than the male actor's, in addition, the model believes that the play's Yan value play is No. 363 weeks to play the clock Huai LAN, the second is No. 346 Jia, and 1th leading actress row fifth:)

Actor aspect: The most handsome is the number 1454th, should be behind the plot to appear, have not seen. The second handsome is number No. 345 (I also think he is the most handsome, in line with my straight male cancer of the aesthetic:), as for the No. 0 male lead is also row fifth:) character emotional curve

By analyzing the mood of a character in the play, it is generally possible to know whether the character is a tragic character or a comedy character.
We put emotions into 3 categories: Neutral 0, positive expression 1, negative expression 1;

Exp2scole = {
    |   " Angry ":-1,
    |   " Disgust ":-1,
    |   " Fear ":-1,
    |   " Happy ": 1,
    |   " Sad ":-1,
    |   " Surprise ": 1,
    |   " Neutral ": 0
    |   }

Take no. No. 363 Huai Lan as an example:

Can see Zhonghuai blue in the early negative mood-based, guess the plot appeared in Guangxi prison that paragraph. Meet expectations. visualization of characters ' appearance time

All along, one thing I want to do is for those groupies to put Mingming in the TV show all together, it can be avoided to see a star and need to see the whole drama of the trouble. By visualizing each character's exit point in each concentration, you can basically implement this function, just use ffmpeg to crop it.

Take 16 episodes as an example:
The results are as follows:

Can be seen in this focus, there are 1th, 3rd, two, Zheng-winter girlfriend at the same time, it is expected to be more beautiful:) Video highlight cropping

Now a lot of small video such as watermelon video has a lot of highlight, if you can automatically highlight found, that should also be a very interesting thing, such as in the play, will be 1th and 3rd at the same time cropped out, estimated will be very interesting:)

Can't post video:) PostScript

The implementation code is on GitHub, and the address is in this videoface.
, if it helps you, give a little star:)

Well, Big Weekend, what am I doing?