Simple neural network algorithm-handwritten digit recognition

In this paper, a simple handwriting recognition system is realized by BP neural network.

First, the basic knowledge

1 environment

python2.7

Need to numpy and other libraries

Can be installed with sudo apt-get install python-

2 Neural Network principle

Http://www.hankcs.com/ml/back-propagation-neural-network.html

It is particularly clear that the function of the NumPy library involved in matrix operations in the course of this experiment

Basic knowledge of 3.js

Http://www.w3school.com.cn/tags/html_ref_canvas.asp

Basic knowledge of canvas

http://blog.csdn.net/iamduoluo/article/details/7215639

Introduction of XMLHTTP

Use of 4.BaseHTTPServer libraries and NumPy

Ii. Important processes

1. How to convert handwritten images into data?

Dividing the canvas into a grid, calculating the squares that the mouse moves through, such as dividing the canvas into 20*20, you can build an array of data[400], the data[i]=1 of the mouse, you can get data input

Drawgrid:function (CTX) {
for (var x = this.) Pixel_width, y = this. Pixel_width; x < this. Canvas_width; X + = this. Pixel_width, Y + = this. Pixel_width) {
Ctx.strokestyle = this.　　BLUE; Set color
Ctx.beginpath ();
Ctx.moveto (x, 0); Move the focus to (x,0)
Ctx.lineto (x, this.   Canvas_width); Draw Line (x,0) to (X,canvas_width)
Ctx.stroke (); Submit Draw Line

Ctx.beginpath ();
Ctx.moveto (0, y);
Ctx.lineto (this. Canvas_width, y);
Ctx.stroke ();
}
},

Onmousemove:function (E, ctx, canvas) {
if (!canvas.isdrawing) {
Return
}
This.fillsquare (CTX, E.clientx-canvas.offsetleft, e.clienty-canvas.offsettop); E.clientx is the x distance between the edge of the browser and the mouse point, offsetleft is the distance between the canvas edge and the left edge of the browser
},

Onmousedown:function (E, ctx, canvas) {
Canvas.isdrawing = true;
This.fillsquare (CTX, E.clientx-canvas.offsetleft, e.clienty-canvas.offsettop);
},

Onmouseup:function (e) {
Canvas.isdrawing = false;
},

Fillsquare:function (CTX, x, y) {
var xpixel = Math.floor (x/this.　　　Pixel_width); Even if the coordinates
var ypixel = Math.floor (y/this. Pixel_width);
storing handwriting input data
this.data[((XPIXEL-1) * this. Translated_width + ypixel)-1] = 1;

Ctx.fillstyle = ' #ffffff ';
Ctx.fillrect (Xpixel * this. Pixel_width, Ypixel * this. Pixel_width, this. Pixel_width, this.　Pixel_width); Square Fill Color
},

2. Browser Interaction with Client

Primarily through JSON data interaction

Senddata:function (JSON) {
var xmlHttp = new XMLHttpRequest ();
Xmlhttp.open (' POST ', this. HOST + ":" + this. PORT, false);
Xmlhttp.onload = function () {this.receiveresponse (xmlHttp);}. Bind (this);
Xmlhttp.onerror = function () {this.onerror (xmlHttp)}.bind (this);
var msg = json.stringify (JSON); Serialized as JSON
Xmlhttp.setrequestheader (' Content-length ', msg.length); Refer to the link to the top
Xmlhttp.setrequestheader ("Connection", "close");
Xmlhttp.send (msg);
}

Server

Class Jsonhandler (Basehttpserver.basehttprequesthandler):
"" To process the received POST request ""
def do_post (self):
Response_code = 200
Response = ""
Var_len = Int (self.headers.get (' content-length '))
Content = Self.rfile.read (Var_len);
Payload = json.loads (content);

# if it is a training request, train then save the trained neural network
If Payload.get (' Train '):
#print payload[' Trainarray ']
Nn.train (payload[' Trainarray '],2)
Nn.save ()
# If it is a prediction request, return the predicted value
Elif payload.get (' Predict '):
Try
Print nn.predict (data_matrix[0])
Response = {"type": "Test", "Result": Str (nn.predict (payload[' image '))}
Except
Response_code = 500
Else
Response_code = 400

Self.send_response (Response_code)
Self.send_header ("Content-type", "Application/json")
Self.send_header ("Access-control-allow-origin", "*")
Self.end_headers ()
If response:
Self.wfile.write (Json.dumps (response))
Return
3. Realization of neural network

Refer to lines or less

the step of the back propagation algorithm

• Random initialization parameters, the input using forward propagation calculation output.

• For each output node, the delta is computed as follows:

• For each hidden node , the Delta is computed as follows:

• Calculates the gradient and updates the weight and offset parameters:. Here is the study rate, affecting the training speed.

# Forward propagation to get the result vector
Y1 = Np.dot (Np.mat (SELF.THETA1), Np.mat (data.y0). T
sum1 = y1 + np.mat (self.input_layer_bias)
Y1 = self.sigmoid (sum1)

y2 = Np.dot (Np.array (SELF.THETA2), y1)
y2 = Np.add (y2, Self.hidden_layer_bias)
y2 = self.sigmoid (y2)

# Back propagation to get the error vector
Actual_vals = [0] * 10
Actual_vals[data.label] = 1
Output_errors = Np.mat (actual_vals). T-np.mat (y2)
Hidden_errors = np.multiply (Np.dot (Np.mat (SELF.THETA2). T, Output_errors), Self.sigmoid_prime (sum1))//multiply function usage, can be simply seen as the corresponding multiplication

# Update weights matrix vs. biased vectors
Self.theta1 + = self. Learning_rate * Np.dot (Np.mat (hidden_errors), Np.mat (data.y0))
Self.theta2 + = self. Learning_rate * Np.dot (Np.mat (output_errors), Np.mat (y1). T
Self.hidden_layer_bias + = self. Learning_rate * output_errors
Self.input_layer_bias + = self. Learning_rate * hidden_errors

Write so much first.

The implementation of the neural network adopts a simplified method, the experimental effect is general, but also an interesting experiment, the next time to update the neural network is another complete realization.

Simple neural network algorithm-handwritten digit recognition