Deep learning in layman's terms: Limited Boltzmann machine RBM (i) Basic concepts

Welcome reprint, Reprint Please specify: This article from Bin column Blog.csdn.net/xbinworld.
Technical Exchange QQ Group: 433250724, Welcome to the algorithm, technology, application interested students to join.

Recently, while reviewing the classical machine learning algorithms, we also looked at some typical algorithms of deep learning. Deep learning is the "New Wave" of machine learning, and its success is mainly due to the excellent effect of the deep "neural network Model". This small series intends to document some of the algorithms commonly used in deep learning in a more comprehensible way. First write the "Restricted Boltzmann machine" RBM, will be divided into a number of small paragraphs written, this is the first paragraph, about the basic concept of RBM.

Online has a lot of introduction to the RBM, but a lot of written relatively simple, skip a lot of details, this article as far as possible to pursue the details of the same time, to do simple. The recommended references can be seen in the final reference.

Required Background knowledge

To learn some basic statistical learning bases for RBM, including Bayes theorem, random sampling method (Gibbs sampling), etc. These can be read from some of my previous blog post can see the relevant introduction, in this article is not specifically expanded. In general, RBM is relatively independent of an algorithm, do not need to rely on too much prior knowledge.

Basic concepts of RBM

The restricted Boltzmann machine (Restricted Boltzmann MACHINE,RBM) is a treasure of Professor G.hinton. Professor Hinton is the pedigree of deep learning, and it is his work on deep belief network DBN in the 2006, as well as the training method of the level-by-layer training, which opens the preface of deep Learning. Among them, the RBM algorithm model is used in the pre-training of DBN in the layer. RBM is a non-graph model and a neural network model.

The RBM has two layers: visible layer (v layer), and hidden layer (h layer), and a more common graph on the network is [1]:

It can be seen that the two layers of neurons are all connected, but each layer is not connected to each other, that is to say, the graph structure of the RBM is a binary graph (bipartite graph). It is this feature that is called restricted Boltzmann and, Boltzmann machines are allowed to connect neurons between the same layer. An RBM is actually a simplified BM model.

Another feature is that the neurons in an RBM are two-valued, that is, only active and inactive states, 0 or 1, and the weights of edges between visible and hidden layers can be W to indicate that W is a |V |x|H | The size of the real matrix. Later, when the solution of the RBM can be seen, the difficulty of the algorithm is mainly w Span style= "Display:inline-block; width:0px; Height:2.563em; " > The derivation (and of course, the bias parameter) for gradient descent updates; but since V and H are all two valued, there is no continuous derivative function to calculate, the actual use of the sampling method to calculate, where the surface can be used such as Gibbs sampling method, of course, Hinton The method of contrast divergence CD, which is faster than Gibbs method, has become the standard solution to solve the RBM. The RBM Solution section will be described in the next small article.

OK, the first article is here.

Resources
[1] http://www.chawenti.com/articles/17243.html
[2] Zhang Chunxia, restricted Boltzmann machine introduction
[3] Http://www.cnblogs.com/tornadomeet/archive/2013/03/27/2984725.html
[4] Http://deeplearning.net/tutorial/rbm.html
[5] Asja Fischer, and Christian Igel,an Introduction to RBM
[6] G.hinton, A Practical Guide to Training Restricted Boltzmann machines
[7] http://blog.csdn.net/itplus/article/details/19168937

Deep learning in layman's terms: Limited Boltzmann machine RBM (i) Basic concepts