YouTube recommended algorithm _ recommended algorithm

Recommended system architecture, candidate set generation, sorting

1. Recommended algorithm Framework
Millions raw data-"User history information and contextual information-" candidate set generation-"hunderds data-" sort

(1) First of all, focus on the next few words in the picture millions, hundreds, dozens: the level of data volume, all the video corpus is probably millions level, after candidate After the generation is probably the hundreds level, after ranking is probably dozens level

(2) candidate generation input includes millions video corpus, user history and context, designed to quickly and efficiently filter part of a set of videos

(3) Ranking input includes the hundreds video corpus, user history and context, the other candidate sources, and the video features, designed to get high precision top N
Users view historical data, search data, do a embedding, plus age, gender features as DNN input, followed by a few layers of the full join layer (activation function is Relu), the training phase using cross-entropy as the optimization loss function, The on-line phase obtains top N as output and ranking stage input through an approximate nearest neighbor lookup via the user vector and the video vector
The classification characteristics (including single value and multivalued) are embedding, the continuous feature is normalizing, the last layer of the training phase is the weighted logistic regression, and the prediction stage is directly based on the learned W to obtain the output results.


2. Lstm Time series analysis and prediction

Time series elements: long-term trends, seasonal change, cyclic change, irregular change, long-term trend (T) a general trend of change in the long term that is affected by a fundamental factor (S) The cyclical change of regularity in the period of a year as the Seasons change (C The irregular change of the wave form that the phenomenon takes over several years as a period (I) is a kind of irregular change, which includes strict random changes and irregular abrupt changes. Two kinds of direct prediction
Rolling forecast
Sliding window + Rolling forecast

3. Depth Belief Network
The depth belief network is a probabilistic generation model, which is relative to the traditional discriminant model's neural network, and the generative model is to establish a joint distribution between the observed data and the label, P (observation| Label) and P (label| Observation) are evaluated, and the discriminant model evaluates only the latter, which is P (label| observation).

4. Boltzmann machine

As shown in the figure is a Boltzmann machine, its blue node is hidden layer, the white node is the input layer.
Compared with the recursive neural network, the Boltzmann machine differs in the following points:
1, the nature of recurrent neural network is to learn a function, so there is the concept of input and output layer, and the use of Boltzmann machine is to learn a set of data "intrinsic representation", so it does not have the concept of the output layer.
2, the recurrent neural network nodes are linked to a forward ring, and the Boltzmann machine nodes are connected to a complete map.

(4) Limiting the Boltzmann machine

Limiting the Boltzmann machine compared to the Boltzmann machine, the main is to add the "limit". The so-called limitation is to turn the complete graph into a two-point graph. As shown in the figure, the limit Boltzmann machine consists of three explicit layer nodes and four hidden layer nodes.

The limit Boltzmann machine can be used for dimensionality reduction (less hidden layer), learning characteristics (hidden layer output is characteristic), depth belief network (multiple RBM stacked), etc. (with a more detailed description later).