A Matlab instance of deep Trust network DBN

Some personal opinions about deep learning:

Deep learning is usually a training depth (multilayer) neural Network for pattern recognition (e.g. speech, image recognition), and a deep network is a neural network with a deep (multilayer) network structure.

The deep network is capable of solving complex problems because of many neurons, many parameters, and strong ability to fit and perform.

However, deep network has many local optimal solutions, the training of deep network is easy to stay on local optimal, and the choice of initial parameters has great influence on the final convergence of the network in that position.

Using limit Boltzmann machine RBM to do layer-by-level unsupervised training on depth network, the parameters of each single-layer training are used as the initial parameters of each layer of deep network, and this parameter is a better position of depth network parameter space (easy).

After the RBM has trained the depth network parameters by layer, the deep network is trained by the traditional BP algorithm, so the parameters of the deep network will converge in a good position.

RBM, through unsupervised training that iterates large amounts of data, can refine the more essential features of the training data, which is considered a good initial parameter.

This example is written in Matlab, in order to use digital recognition to train a handwritten digit recognition of a deep neural network.

Also need DBN support code, can download from here: Http://download.csdn.net/detail/hzq20081121107/7857735,http://pan.baidu.com/s/1c0fBQsK

The network structure adopts 784,400,200,100,50,20,10 network structure.

function agetdeepnet () clcclear all% get Training data load (' Adata.mat ', ' train_digitdata ', ' train_targets '); X = Train_digitdata; Y = train_targets;% input data Initialization xmin = min (X); Xmax = max (X); X = Bsxfun (@rdivide, Bsxfun (@minus, X,xmin), (xmax-xmin)),%RBM training Gets the network parameters of the first hidden layer, the RBM input is the picture data RBM1 = RBM ([784,400]); rbm1 = Checkrbmtrain (@rbmtrain1, rbm1,x,50,0.1); net_rbm1 = Rbm2nnet (rbm1, ' up '); h1 = NNETFW (net_rbm1,x);%RBM training Gets the network parameters of the second hidden layer, The input is the output of the first hidden layer rbm2 = RBM ([400,200]), rbm2 = Checkrbmtrain (@rbmtrain1, rbm2,h1,50,0.1), net_rbm2 = Rbm2nnet (rbm2, ' up '); H2 = NNETFW (NET_RBM2,H1);%RBM training Gets the network parameters of the third hidden layer, the output of the second hidden layer RBM3 = RBM ([200,100]); rbm3 = Checkrbmtrain (@rbmtrain1, RBM3,H2, 50,0.1); net_rbm3 = Rbm2nnet (rbm3, ' up '); h3 = NNETFW (NET_RBM3,H2);%RBM training Gets the network parameter of the third hidden layer, and the input is the output of the second hidden layer RBM4 = RBM ([100,50]); RBM4 = Checkrbmtrain (@rbmtrain1, rbm4,h3,50,0.1); net_rbm4 = Rbm2nnet (RBM4, ' up '); h4 = NNETFW (NET_RBM4,H3);% The RBM training obtains the network parameter of the hidden layer, the input is the output of the fourth hidden layer RBM5 = RBM ([50,20]); RBM5 = Checkrbmtrain (@rbmtrain1, rbm5,h4,50,0.1); NET_RBM5 = Rbm2nnet ( RBM5, ' up '), H5 = NNETFW (NET_RBM5,H4),% build depth network, and initialize the parameterNumber is the parameter that the RBM trains. Net1 = Nnet ([784,400,200,100,50,20,10], ' Softmax '); Net1.w{1} = net_rbm1.w{1};net1.w{2} = net_rbm2.w{1};net1.w{3} = net_ Rbm3.w{1};net1.w{4} = net_rbm4.w{1};net1.w{5} = net_rbm5.w{1};% BP training for deep networks Net2 = Nnettrain (net1,x,y,1000);

A Matlab instance of deep Trust network DBN