Preliminary introduction of neural network and recommendation system

Author: one person 1. Deep neural networks are suitable for any field

Depth neural network (deep neural Networks,
DNN has made breakthrough advances in image classification, speech recognition, and natural language processing over the past few years. The application in practice has proved that it can be used as a very effective technical means in the field of big data correlation. Deep neural network through a number of simple linear transformation of the hierarchical non-linear transformation of the complex relationship in the data can be well fitted, that is, the deep-seated data characteristics of mining. therefore, as a technical means, deep neural networks are applicable in any field. 2. Referral System Introduction

The function of the recommender system is to help the user to find the personalized item which satisfies the preference and recommend it to the user proactively. In essence can be used as a personalized search engine, the input data for user behavior information, preference information, etc., the result is the most matching the query criteria list of items. A mathematical representation:

Item List =f (User preferences) −−−−−−−−−−− formula (1) Item list = F\left (user preference \right)-----------formula (1)

Our recommendation engine plays the role of the function here, it mainly needs to complete two parts of the work:

A > estimate the relevance of the item to the query criteria.

B > Choose TOPN Most relevant items.

Therefore, the key of recommendation system is to solve the above function.

The number of items in the actual application is very large, so it is impractical to evaluate all items using the evaluation function on the premise of satisfying the business needs. Therefore, in order to achieve the balance of performance and effect, most of the recommended system will be divided into two parts of the calculation process:

Referral recall

Recommended sort

Referral recall refers to the retrieval of a candidate set of user interest in all collections of items, with approximately hundreds of candidate lists selected. The purpose of sorting is to use the display, click (or convert) data, and then add more users, item characteristics, the recommended candidates for more granular correction, scoring. Another benefit of this pattern is the ability to take advantage of multiple candidate sets.

Therefore, the recommendation system needs to complete two-step calculation: Candidate set generation and sequencing, the two-phase estimation functions are expressed as G and H, that is:

F=g (H (x)) −−−−−−−−−−−−−−−−−−−−−−−−− formula (2) f= g (H (x))-------------------------formula (2)

3. Using neural network approximation to solve function reference 1

There are several ways to solve a function:

Deterministic solution: A direct solution to the data by modeling the law.

Deterministic approximate solution: solving by means of variational inference, EM.

Stochastic approximation Solution: The method of sampling to solve the function, Monte Carlo method.

Unstructured Solutions

No matter what this function is, there is always a neural network capable of obtaining the corresponding value F (x) f (x) for any possible input x x network (or some exact approximation)

Even if the function has many inputs or multiple outputs, the result is set, F=f (x1,..., xm) f=f (x_1,..., x_m). For example, here is a network with input of m=3 m=3 and output of n=2 n=2:

In conclusion, the neural network as an approximate solution method can be used to approximate the formula (2) two function g and H. 4. Recommended recalls

Google uses DNN to make YouTube video recommendations for its model diagram as shown in the figure below. By the user watching the video, search for keywords to do embedding, and then in the concatenation of the user's side
Information and other information, as the input of DNN, using a multi-layered dnn to learn the user's hidden vector, and then add a layer of Softmax to learn the hidden vector of the item, and then you can do top-n recommendations for users.

Autoencoder (AE) is an unsupervised learning model (like matrix decomposition) that uses a reverse propagation algorithm to make the output of the model equal to the input. Using AE to predict the user's missing value of the item, the input of the model is a row (user-based) or a column (item-based) in the scoring matrix, and its objective function optimizes the model by calculating the loss of input and output. The scoring value of missing in the scoring matrix is predicted by the output of the model and then recommended for the user, and the model is shown in the figure below. Subsequently, the denoising Autoencoder (DAE) is based on AE, adding noise to the input training data. So Dae must learn to remove these noises and get real input data that is not contaminated by noise. As a result, this forces the encoder to learn more robust representations of the input data, and the Dae's generalization ability is usually stronger than the general AE. Stacked
Denoising
Autoencoder (Sdae) is a multilayer ae composed of a neural network, its previous layer of self-encoder output as a subsequent layer of self-encoder input. and a Bayesian.
Sdae and many other methods are homologous to this.
5. Recommended sortingWide & Deep model, Google uses a combination of DNN and traditional generalized linear models to implement Google Play
Recommended in the application.
DNN has a good generalization and the generalized linear model has good memory, and by combining the two, the irrelevant article rules are suppressed on the basis of achieving good generalization. At the input layer, the class feature is connected through embedding and continuous values to form an input embedding vector and the input hidden vector is formed via a three-layer network, and the app-related features are cross-multiplied at the input layer, along with the hidden vector input into a logical output unit, and the final output is scored for a particular app.
6. Other applications of neural networks Word vector representation, using the shallow neural network method to study. Using the information contained in the sequence data, the representation of an item is mapped to a low-dimensional dense representation by a high-dimensional sparse representation. Typical model methods are:Word2vec
[unsupervised] and glove[unsupervised] (Global Vectors for Word representation).
——————————————————————————————————–
The following figure shows the Word2vec based on the Cbow hierarchy network structure, the input layer is a word vector of several words, through the mapping layer to accumulate, the yellow node in the output layer is a non-leaf node representative of a category, and the leaf node represents a word vector, the entire output layer is a Huffman tree. It is assumed that for a particular context, the predicted probability of a particular intermediate word is maximal, and training is given to the low-dimensional dense representation of the word.
——————————————————————————————————–
For example: The statement "Modify this file directly", after the word "direct", "Modify", "This file". So for the word "modify" training, then the input context is "direct", "This file", we expect "modify" the probability of the most. After training with a large number of samples, a vector representation of the corresponding words can be trained at the leaf node. After that, the similarity between the word vectors can be calculated to represent the similarity between the words, and so on to facilitate further analysis.
——————————————————————————————————–
7. Difficulties in neural networks

Because the neural network can fit complex nonlinear relation with multilayer structure, it has huge parameters, and it becomes more and more difficult to train with the network in depth. Therefore, it has the following difficulties for practical applications:

Requires a lot of training data

There is no reasonable selection method for the parameter adjustment

There is no standard network structure for specific applications 8. Current data application depth model challenges

User behavior is sparse, so there is a lot of noise in the data

Less available fields for Media Library data

User image clutter, user attribute information collection not clear summary

Previously, the computational resources were valuable and the computing power was weak, so in order to realize the intelligent function, it was necessary for the researchers to integrate the functional rules into the algorithm indirectly by artificial means to reduce the computational amount. However, due to the complexity of the user's application scenario, there is often a situation that many developers cannot predict. And because a lot of approximate solutions require a lot of computation to get the exact results, forcing most scenarios to be impossible to implement and adopt, so in the past few years, the application level of intelligent development stalled. With the rapid development of computing power, it has become a feasible strategy to use large amount of computation to realize intelligent function. The Deep Neural Network algorithm is adapted to the development trend with its powerful fitting ability, and has made great achievements in the fields of image, speech and natural language.

Personalized recommendation as a member of many intelligent scenes, has attracted a large number of research and development staff, different from the image, voice and other rich features and the results of the algorithm and the real sample will not have mutual influence, due to the complexity of the recommended characteristics data, and the recommended results affect the collected data, There is no general-purpose structure and method in the present recommendation. There are also many people who apply neural network methods to the overall recommended sub-areas have achieved good results. It can be predicted that as more people get involved, personalized recommendations will be invaded by neural network methods.

In the industry, in the case of limited resources investment, keeping abreast of the development of the technological frontier, advanced methods are validated in the system. Or for the industry to achieve a stable effect of the method to verify and system integration, the product will be greatly benefited.

Report:

Word2vec effect "program VEC similarity": Mission Spy 5: The Mysterious Kingdom

# Crisis 13 hours, # Mission spy 4,# death Rook, # extreme agent 2,# Die Hard Dragon 5,#
Star Trek, # Conglinch, # Assassin League, # Spy Shadow 2, # Very human trafficking "86 version of the journey to the Monkey

# Journey to the Monkey cartoon, # The Monkey's Lock Demon Tower, # Monkey King (3D), #
Monkey King of the Journey to the Heaven, # Monkey King three Dozen Bones Jing, # Hee Travels, #
The return of the holy man of the journey to the West, # Western Mirror, # electric Blackadder Upbraid, # Monkey King Seven dozen nine Fox "Shoot Eagle Hero Biography

"# The 3rd episode of the hero of the shoot carving, # The hero of the Li Yapeng version, # The statue of God, #
The statue of God [Cantonese version],# Tianlong Eight, # Fangshiyu and Huhuiqian, # Yi Tian Tu long kee finale, #
The new god Vulture, # Huang Xiaoming version, # The end of the Moon sword "