An attempt of personalized film recommendation based on RNN

An attempt of personalized film recommendation based on RNN

With more and more application of in-depth learning in industry, Youku potatoes try to use the depth learning method in the Video recommendation field, improve the accuracy of video recommendation, and provide high-quality video recommendation service for users. This time to share in personalized film recommendations on the attempt to use the RNN sequence model for user movie recommendations. Video Recommendation Issues

The current common personalized recommendations include: Recommendation algorithm based on association rules, content-based recommendation algorithm and collaborative filtering based recommendation algorithm. In the personalized recommendation of video website, the personalized recommendation algorithm based on collaborative filtering is the main recommendation, it can filter the information which is difficult to automate the machine based on content analysis, and can filter based on some complex and difficult concepts (information quality, grade). But the collaborative filtering algorithm cannot process the user's viewing sequence information, and can not recommend the video according to the user's viewing sequence, but only considers the user's individual viewing behavior information. Then how to tap the user's viewing sequence information, the user more accurate personalized video recommendation.
In the current sequence model, comparing fire and using the RNN model, it obtains good results in the fields of speech recognition, machine translation and text processing. Therefore, we use the RNN model to build and predict the user viewing sequence. RNN Model

This article RNN introduced the Circular Neural network (RNN, recurrent neural Networks) introduction, the following part of the diagram and explanation from the blog. RNN

RNN model is mainly used to deal with sequence problems, in the traditional neural network model, from the input layer to the hidden layer to the output layer, the layer and layer are all connected, the node between each layer is not connected. But this common neural network is incapable of many problems. For example, you have to predict what the next word in a sentence is, and you generally need to use the preceding word, because the word is not independent in a sentence. RNN is called a cyclic neural network, that is, the current output of a sequence is also related to the previous output. The specific manifestation is that the network will remember the preceding information and apply it to the current output calculation, that is, the nodes between the hidden layers are no longer connected but connected, and the input of the hidden layer includes not only the output of the input layer but also the output of the hidden layer at the last moment.

The Rnns contains an input units, the input set is labeled {x0,x1,..., xt,xt+1,...}, and the output unit (output units) is marked as {y0,y1,..., yt,yt+1.,..}. The RNN also contains hidden units (Hidden units), which we mark as {s0,s1,..., st,st+1,...}, and these hidden units do the most important work. You will find in the diagram: There is a one-way flow of information from the input unit to the hidden unit, while another one-way flow of information from the hidden unit to the output unit. In some cases, Rnns will break the latter limit by directing information back to the hidden unit from the output unit, which is known as "projections", and the input of the hidden layer includes the state of the previous hidden layer, that is, the nodes within the hidden layer can be connected and interconnected. lstm

Because in the RNN, the update parameter adopts the bpTT method, when the length of the sequence is long, the problem of the gradient disappears. In order to solve this problem, a lstm model is produced. In the LSTM model, the hidden layer becomes a complex block, in which the memory structure is referred to as the cells, which can be seen as a black box to hold the saved state before the current input XT ht−1, these cells more certain conditions determine which cell is inhibited by which cell excited. They combine the previous state, current memory, and current input. It has been proved that the network structure is very effective in the problem of long sequence dependencies. The LSTMS network structure is shown in the following figure. For LSTMS's study, see UNDERSTANDING-LSTMS
Film recommended Applications

The above is a brief introduction to RNN, if you want to learn more, you can refer to the specific literature and code. The next major introduction is how we use RNN to personalize recommendations in the Youku Potato film recommendation.
In the movie recommendation, we're going to work out the movie that predicts the future of the user based on the video viewed by the user's history, as shown in the following illustration:
Training Samples

In order to solve the above problem, we can now get what the user's data has. Obviously, the user's viewing history is available to us. Therefore, the training data for the model can be structured into his (U) ={v1,v2,v3,.. VN}. Where, V for users to watch the video, V1,v2. For the user to view the sequence of time. Of course, for the accuracy of the data, to watch the history of cleaning, such as the user to watch the video playback rate and so on.
In fact, for users and video data, the available data also have user portraits and video portraits. In order to simplify the model and consider the training time, only the user's viewing history is used in the experiment. model Target

With the above data, you need to train the model.
First, define the objective function and use the cross entropy function to define the objective function:
e=1n∑nylogp+ (1−y) log (1−p) E = \dfrac{1}{n} \sum_n{ylogp+ (1-y) log (1-P)}
Where Y is the label data and p is the prediction probability of the next video for the user.
The specific training flow diagram, the video prediction problem into the sequence prediction problem, such as the user viewing history as V1,v2,v3, when the model input as V1,V2, the goal is to maximize the probability of video v3.

User Forecasts

How to use the model after using it. In fact, it can be seen in the goal setting of the model above. As a sequence model, in the user video prediction, input the user history of the viewing sequence, select the user may watch the video (according to the probability of the forecast to sort) can. As shown in the figure:
Experimental Results

We selected the millions user to sample and build the training set. and uses TensorFlow to carry on the coding realization. As the RNN model has a longer training cycle, we use Aliyun HPC as a training machine to perform accelerated training on M40.
The sample number was tens, the video number was Chi Pe, 1 lstm layers, 50 cell, and the iteration number was 60. On this configuration, the training time is 40minue.
In order to verify its effect, we on the Youku potato on the ab small flow test, found that RNN click rate of the benefits of the effect is obvious. Summary

Although RNN has some effect in the current experiment, because of the complexity of the RNN training, it is still difficult to calculate the large-scale data, and it will take a long way to go on the large-scale use.