Neural Network for Handwritten Digit Recognition

I. Artificial Neural Networks

Most of the reason why humans can think, learn, and judge is due to the complicated Neural Networks in the human brain. Although the mechanism of the human brain has not yet been completely deciphered, the connection between neurons in the human brain and the transfer of information are all known. So people want to simulate the function of the human brain to solve other problems, which leads to the development of artificial neural networks.

Artificial Neural Network (ANN) is a mathematical model or computing model that imitates the structure and functions of the biological neural network. Neural Networks are calculated by a large number of Artificial Neuron connections. In most cases, artificial neural networks can change the internal structure based on external information, which is an adaptive system. Modern neural networks are a non-linear statistical data modeling tool. They are often used to model complex relations between input and output, or to explore data models.

A Neural Network is an operational model that consists of a large number of nodes (or "neurons", or "units") and mutual connections. Each node represents a specific output function, called the activation function ). The connection between each two nodes represents a weighted value for the connection signal, called weight (weight), which is equivalent to the memory of the artificial neural network. The output of the network depends on the connection method of the network, the weight value and the excitation function. The network itself is usually an approximation of an algorithm or function in nature, or it may be an expression of a logic strategy.

1.1 neurons

Artificial Neural Networks are composed of thousands of neurons, so we need to learn about neurons first.

Neuron:

• A1 ~ An is the component of the input vector.
• W1 ~ Wn is the weight of each neuron's syn.
• B is bias
• F is a transfer function, usually a non-linear function. Generally, there are traingd (), tansig (), and hardlim (). The default value is hardlim ()
• T is the output of neurons.

Mathematical representation

• Weight Vector
• Is the input vector, for transpose
• Offset
• To pass Functions

It can be seen that the function of a neuron is to obtain the Inner Product of the input vector and the weight vector, and then obtain a scalar result through a nonlinear transfer function. The role of a single neuron: divides an n-dimensional vector space into two parts (called the judgment boundary) with a superplane. Given an input vector, the neuron can determine which side of the vector is located on the superplane.

The superplane equation:

• Weight Vector
• Offset
• Vector on the ultra-Plane

1.2 BP Neural Network Structure

A neural network is composed of many neurons, which can be divided into input, output, and hidden layers.

Features of the BP neural network: Signal forward transmission and reverse error propagation. If the output has an error, adjust the weight and threshold based on the error to bring the output of the network closer to expectation.

2. Description of Handwritten Character Recognition Dataset:

The dataset contains the handwriting of the 10 digits 0-9. Is placed in 10 folders, the folder name corresponds to the number of the handwritten digital image, each number 500, each image pixel is 28*28.

Samples:

Identification process:

First, we need to process the data. This is mainly the process of reading images and Feature Extraction in batches. There are many feature extraction methods. Here we only select the simplest method to achieve this, then a neural network model is trained and tested with test data. For the sake of this, the creation, training and testing of neural networks are implemented using MATLAB functions.

2.0 main functions:

CLC; clear all; close all; % read image root = '. /data'; IMG = read_train (Root); % extract feature img_feature = feature_lattice (IMG); % construct tag class = 10; numberpclass = 500; ann_label = zeros (class, numberpclass * class); ann_data = img_feature; for I = 1: class for J = numberpclass * (I-1) + 1: numberpclass * I ann_label (I, j) = 1; endend % selected training set and test set K = rand (1, numberpclass * class); [M, N] = sort (k); ntraindata = 4500; ntestdata = 500; train_data = ann_data (:, n (1: ntraindata); test_data = ann_data (:, n (ntraindata + 1: numberpclass * class); train_label = ann_label (:, N (1: ntraindata); test_label = ann_label (:, n (ntraindata + 1: numberpclass * class); % BP neural network creation, training and testing net = network_train (train_data, train_label); predict_label = network_test (test_data, net); % Accuracy Calculation [U, V] = find (test_label = 1 ); label = U'; error = label-predict_label; Accuracy = size (find (error = 0), 2)/size (Label, 2)

2.1 function for reading images in batches

File storage features: there are 10 sub-folders under data, and 500 images under each sub-folder. The function can be used to read any batch of images. The input is the folder path, and the output is a n (corresponding to the number of images) dimension cell. Each cell stores the image data.

Function [imglist] = read_train (Root) % ======= read folder =====% out_files = Dir (Root ); % expand tempind = 0; imglist = cell (0); n = length (out_files ); % ======== Read File =========% for I = 1: N; If strcmp (out_files (I ). name ,'. ') | strcmp (out_files (I ). name ,'.. ') else rootpath = strcat (root,'/', out_files (I ). name); in_filelist = Dir (rootpath); ni = length (in_filelist); for j = 1: Ni if strcmp (in_filelist (j ). name ,'. ') | strcmp (in_filelist (j ). name ,'.. ') | strcmp (in_filelist (j ). name, 'desktop _ 1. INI ') | strcmp (in_filelist (j ). name, 'desktop _ 2. INI ') else tempind = tempind + 1; imglist {tempind} = imread (strcat (rootpath,'/', in_filelist (j ). name); end endendend

 

2.2 Feature Extraction

Extract features from all images, binarization-resize-extract features

Function feature = feature_lattice (IMG) % input: a black-and-white binary image. Output: 35-dimensional grid feature % =======extract features, convert them into 5*7 feature vectors, and divide and add each 10*10 points in the image, add to a point ====%%%%====== that is, count the percentage of image pixels in each small area as feature data ===% for I = 1: length (IMG); bw2 = im2bw (IMG {I}, graythresh (IMG {I}); bw_7050 = imresize (bw2, [70, 50]); for CNT = for cnt2 = atemp = sum (bw_7050 (CNT * 10-9) :( CNT * 10), (cnt2 * 10-9) :( cnt2 * 10); % 10*10 box lett (cnt-1) * 5 + cnt2) = sum (atemp); endendlett = (100-lett) /100); lett = lett'; feature (:, I) = lett; End

2.3 construct tags

To construct a label suitable for neural networks, there are 10 classes in this example. If a label is used, the value of this position is 1 and the remaining value is 0.

2.4 BP neural network creation, training and testing

It mainly involves several parameter settings, including the number of neurons in the hidden layer of the layer. Trainfcn: Training Algorithm

Function net = network_train (train_data, train_label) % input: Training image features and label. Output: trained neural network % BP network training % initialized network structure layer = 25; net = newff (train_data, train_label, layer); net. trainparam. epochs = 1; net. trainparam. LR = 0.1; net. trainparam. goal = 0.001; net. trainfcn = 'trainrp '; % network training net = train (net, train_data, train_label); End

 

Function out = network_test (test_data, net) % BP network prediction an = SIM (net, test_data); for I = 1: length (test_data) Out (I) = find (an (:, I) = max (an (:, I); endend

 

2.5 download the dataset and complete code

Http://pan.baidu.com/s/1pJz97pp

References:

[1] http://zh.wikipedia.org/zh-cn/%E4%BA%BA%E5%B7%A5%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C

Neural Network for Handwritten Digit Recognition