Deep Learning paper interpretation--session-based recommendations with recurrent neural Networks

Blog Address: http://www.cnblogs.com/daniel-D/p/5602254.html

Sina Weibo: http://weibo.com/u/2786597434

Welcome to Exchange ~

Main idea

The work of this paper is to talk about the RNN application to the recommendation system, the idea is that a session click on a series of item behavior as a sequence to train a RNN model. In the prediction phase, the click Sequence known to the session is used as input, and the Softmax is used to predict the next most likely clicked item in the session. Although the idea of the thesis is simple and straightforward, there are very few papers (except for RBM) which are directly recommended by the deep learning structure, so it is worth studying carefully.

Architecture

In this paper, the GRU unit in RNN is used as the basic structure, and a deep neural network is formed to predict the probability of the next clicked item of the session. For example, item has 2w, then through 1-of-n coding Way (one hot encoding), the first layer of the neural network has 2w nodes, and then through the second layer of embedding layer compression into a low-dimensional continuous vector (assuming 200-dimensional), Of course, these vectors are initialized randomly at first. Embedding this layer is actually a 2w * 200 Matrix dictionary, and each line of 200-dimensional vectors is a presentation of item. This 200-dimensional vector can be used as input to the stacked GRU, resulting in a low-dimensional continuous output (200-D), and then by multiplying with a 2w matrix, the next click of item in the 2w node prediction probability, overall, this is a sequence to Seque NCE's prediction problem.

In order to better parallel computing, the paper uses mini-batch processing, that is, the different session splicing together, the same sequence encounter the next session, you should pay attention to the GRU in some of the vector reinitialization. Because the dimension of item is very high, the probability of predicting a 2w item each time results in a large amount of computation, so in principle only the current positive sample (that is, the next clicked Item) is selected plus a randomly sampled negative sample. In this paper, the trickery method is used to reduce the computational amount required by sampling, that is, the next click item in the same Mini-batch is selected as a negative sample to train the whole neural network with these positive and negative samples.

Loss Function

The author tries the common Point-wise rank loss, which is that the negative sample is 0, the positive sample is 1 of the loss function, found that the trained model is not obvious, because in the recommendation, there is no absolute positive and negative samples. In the same scenario, the user can only click on one of the most preferred item, and there may be preferences for other item, which is not a black-and-white classification problem. The author then tries the Pair-wise rank loss, and has nearly 0 constraints on the score of the negative sample, which prevents all the predicted scores from getting higher in the case of "comparing" the scores to each other in the training process. In fact, I think the constraint method is approximate to the L2 constraint of the weight.

Results

Paper in the recSys2015 contest data set on the click on the item of the forecast rating, the comparison method has pop, S-pop, ITEM-KNN and BPR-MF, evaluation standards have [email protected] and [email protected], seemingly all Have achieved good results.

Deep Learning paper interpretation--session-based recommendations with recurrent neural Networks