Restricted Boltzmann machine (RBM)

The RBM uses the energy model.

Simply summarize the energy model. Suppose an isolated system (the total energy $e$ must, the number of particles $n$ certain), the temperature is constant 1, each particle has $m$ a possible state, each state corresponds to an energy $e_i$. So, randomly selecting a particle in this system, the probability that the particle is in state $k$, or the proportion of particles with state $k$ is:

$ $p (state=k) =\frac{e^{-e_k}}{\sum e^{-e_i}}$$

Expands, in a regular ensemble, the probability of the system $i$ in the state $s_i$ is:

$ $P (state=i) =\frac{e^{-e_i}}{\sum e^{-e_i}}$$

Restricted Boltzmann machine is such a system, system structure for a two-point network. One node type is a visual node, the other is an implied node, and all nodes are two value nodes (0,1). Such as.

The energy of the system is defined as

$ $E =-(\sum a_iv_i+\sum w_{i,j}v_i h_j+\sum b_ih_j) ~~where~~w_{i,j}=w_{j,i} ~~~~~~~~~ (1) $$

That is, the energy of the system is the sum of the energy of each node and the coupling energy (edge energy). The probability that the system is in state ${v,h}$ is:

$ $P (v,h) =\frac{e^{-e (v,h)}}{\sum_{v,h} e^{-e (v,h)}}=\frac{e^{-e (v,h)}}{z}$$

$Z $ for the distribution function.

The probability of the visible layer node in the state $x$ is:

$ $P (v=x) =\frac{\sum _he^{-e (x,h)}}{z} \equiv \frac{e^{-e_x}}{\sum e^{-e_x}}~~~~~~~~~~ (2) $$

which

$ $E _x=-log (\sum _he^{-e (x,h)}) $$

Because the same type of nodes are not connected, so that the same nodes are independent of each other, so that each hidden layer node and the edge of its connected to a small system is a separate system. When the visible layer state is given as $x$, the small system has only two states (an implied node of 0 or 1), or two energy levels.

$$-E (h_i;x) =b_ih_i+\sum w_ix_ih_i= (b_i+\sum w_ix_i) h_i=b_ih_i~~where~~b_i=b_i+\sum w_ix_i$$

So

$ $E (h_i=0;x) =0,e (h_i=1;x) =-b_i$$

$ $P (h_i=1;x) =\frac{e^{b_i}}{e^0+e^{b_i}}=sigmoid (b_i) $$

Reference documents

Http://en.wikipedia.org/wiki/Restricted_Boltzmann_machine

Http://deeplearning.net/tutorial/rbm.html

Restricted Boltzmann machine (RBM)