"Reprinted" Neural Networks for Digit recognition with Pybrain

Neural Networks for Digit recognition with PybrainPosted on January. by powel talwar

Hi Everyone
As a part of my B.Tech project, we were required to make a neural network, among other things, which can train on given dat A and perform the task of Digit recognition. We chose Python to do with project in given the wide array of libraries.

We aim to identify digits from images. The dataset is a part of MNIST database and are provided in the online course machine learning on Coursera. The images is of the size 20*20. This was a classification problem with the output classes. The pixel values would be used as features. We Use the Pybrain implementation of neural networks to make our network.

Let us first import all the modules required.

Import osimport sysfrom numpy import *from scipy import ioimport matplotlib.pyplot as Pltfrom pybrain.structure import *FR Om pybrain.datasets import superviseddatasetfrom pybrain.utilities import Percenterrorfrom Pybrain.supervised.trainers Import Backproptrainer

Then we load the data from the. mat file. Number of classes is determined by the unique values taken by Y. Since ' 0 ' was denoted by class value, so we have it to ' 0 ' first.

data = Io.loadmat (' Ex4data1.mat ') X = data[' x '][m, n] = shape (x) Y = data[' y ']y = Reshape (Y, (Len (y),-1)) Numlabels = Len (un Ique (Y)) Y[y = = 10] = 0

Also, we add a bias term to the feature matrix.

x = Hstack ((Ones ((M, 1), x)) n = n+1

>>> X Array ([[1., 0., 0., ..., 0., 0., 0.], [1., 0., 0., ..., 0., 0., 0.], [1., 0., 0., ...., 0., 0., 0. [1., 0., 0.,..., 0., 0., 0.], [1., 0., 0.,.., 0., 0., 0.], [1., 0., 0.,. ..., 0., 0., 0.]] >>> Y Array ([[0], [0], [0], ..., [9], [9], [9]], dtype=uint8)

>>> shape (X) (401) >>> shape (Y) (5000, 1)

Our implementation Conatins 3 layers viz. input layer, hidden layer and output layer although adding more hidden layers is Relatively easy with Pybrain. The size of hidden layer can be set as per requirement. I have the seen people set it according to ' one neuron per output class ' or ' one neuron per input feature '. Both would do, or even any number would do but greater size of hidden layer leads to greater efficiency and more training TI Me.

Ninput = nnHidden0 = Int (N/5) noutput = Numlabels

Pybrain allows us to specify each layer separately to add more control. Once the layers is set, we define a feedforward network and add the layers as modules to the network. Again, we have the liberty to chose how the layers is interconnected. After adding the interconnections, we sort the network that sets everything on place to form a neural network.

Inlayer = Linearlayer (ninput) Hiddenlayer = Sigmoidlayer (nHidden0) Outlayer = Softmaxlayer (noutput) net = Feedforwardnetwork () net.addinputmodule (Inlayer) net.addmodule (hiddenlayer) net.addoutputmodule (outLayer) theta1 = Fullconnection (Inlayer, hiddenlayer) theta2 = Fullconnection (Hiddenlayer, Outlayer) net.addconnection (THETA1) Net.addconnection (THETA2) net.sortmodules ()

At this point, we network is a ready-to-work though it's not trained. So we check that it is on any random input. The image corresponding to the example are shown and a value is predicted with the initialised parameters. This would give a mismatch.

c = Random.randint (0, x.shape[0]) print ("Testing without training \nchoosing a random number" + str (c)) X1 = X[c,:]predict Ion = Net.activate (X1) net.activate (X1) p = Argmax (prediction, Axis=0) PlotData (X, Y, c) print ("predicted output is \ t" + str ( P))

>>> testing without training choosing a random number 1497true number is [2]predicted output is 5

The PlotData function is written as

def plotdata (x, Y, c):    m, n = shape (X)    image = Array (x[c,1:n])    plt.imshow ((Image.reshape (20, 20)). T, cmap= ' greys ')    plt.show ()    print ("True number is" + str (y[c]))

Now, to train the network in real dataset, we need to create an object of the DataSet class. Although, this is a classification problem and we should being going with an object of ' Classificationdataset ', I found some Problem and instead set it up as a supervised problem with more than one ouptut and is 2 classes (' 1 ' for Yes and ' 0 ' for NO). The output Y is changed into a Softmax layer using the function ' Converttooneofmany '

def converttooneofmany (y):    rows, cols = shape (y)    numlabels = Len (unique (y))    Y2 = Zeros ((rows, numlabels)) C3/>for i in range (0, rows):        y2[i, y[i]] = 1    return Y2

AllData = Superviseddataset (n, numlabels) Y2 = Converttooneofmany (Y) alldata.setfield (' input ', X) Alldata.setfield (' Target ', Y2)

>>>y2 Array ([[1., 0., 0., ..., 0., 0., 0.], [1., 0., 0., ..., 0., 0., 0.], [1., 0., 0., ...., 0., 0., 0. [0., 0., 0.,..., 0., 0., 1.], [0., 0., 0.,.., 0., 0., 1.], [0., 0., 0.,. ..., 0., 0., 1.]]

Each prudent machine learning implementation takes care of avoiding overfitting. So we too divide the dataset into training and test dataset. The following command shuffles data and divides it into training and test datasets according to the given proportion.

Datatrain, datatest = alldata.splitwithproportion (0.70)

For training, setup a ' back propagation trainer ' with parameters as the Feedforward network (network to modify while Tra ining), training dataset (dataset on which training'll take place). Other parameters is optional and can is referred in the docs. We Store the true values of examples to find out the accuracy. The network can be trained at once or step by step to see how accuracy rises.

Train = Backproptrainer (NET, Dataset=datatrain, learningrate=0.1, momentum=0.1) Truetrain = datatrain[' target '].argmax (axis=1) trueTest = datatest[' target '].argmax (axis=1) epochs = 20for i in range (epochs):    train.trainepochs (1)    Outtrain = Net.activateondataset (datatrain)    Outtrain = Outtrain.argmax (axis=1)    restrain = 100-percenterror ( Outtrain, truetrain)    outtest = Net.activateondataset (datatest)    outtest = Outtest.argmax (axis=1)    resTest = 100-percenterror (Outtest, trueTest)    print ("Epoch:%4d"% Train.totalepochs, "\ttrain acc:%5.2f%%"% restrain, "\ TTest acc:%5.2f%% "% restest)

We Train the network step by step and check the accuracy on Trainig and test datasets after each step. Epochs denote the number of cycles of training. The number of cycles can changed to see when convergence takes place. Finally, we check on the same test image as done earlier.

 prediction = net.activate (X1) print (prediction) p = Argmax (prediction, axis=0) print ("predicted output is \ t" + str (p) )

>>> epoch:1 train acc:82.37% test acc:78.53% epoch:2 train acc:92.66% test acc:87.00% Epoch:3 train Acc:9 5.11% test acc:88.87% epoch:4 train acc:95.00% test acc:89.67% epoch:5 train acc:95.89% test acc:88.93% Epoch:6 TR Ain acc:99.43% test acc:90.93% epoch:7 train acc:99.63% test acc:90.87% epoch:8 train acc:99.91% Test acc:90.47% E Poch:9 train acc:99.91% test acc:91.13% epoch:10 train acc:99.97% test acc:90.93% epoch:11 train acc:100.00% test acc:90.87% epoch:12 train acc:100.00% test acc:91.27% epoch:13 train acc:100.00% test acc:91.33% epoch:14 train AC c:100.00% test acc:91.07% epoch:15 train acc:100.00% test acc:91.13% epoch:16 train acc:100.00% Test acc:91.13% EP Och:17 train acc:100.00% test acc:91.27% epoch:18 train acc:100.00% test acc:91.33% epoch:19 train acc:100.00% tes T acc:91.33% epoch:20 train acc:100.00% test acc:91.20% [1.59597297e-05 2.25560490e-07 9.97647093e-01 6.55645310e-06 7.44035909e-08 9.97930857e-07 3.73833112e-07 8.80468650e-05 9.46062824e-04 1.29460949e-03] predicted output is 2 

I hope this post is useful and would help you in designing your own neural network. You is free to comment for any suggestions or mistakes. The whole code is Availabale athttps://github.com/poweltalwar/deeplearning/blob/master/neuralnets.py

Ps:will leave for a trips to Rajasthan with class on 20th Jan.

"Reprinted" Neural Networks for Digit recognition with Pybrain