Neural Network algorithm

1. Background:

1.1 Inspired by neural networks in the human brain, there have been many different versions in history. 1.2 The most famous algorithms are the backpropagation of the 1980. 2. Multilayer forward neural networks (multilayer feed-forward neural network)The 2.1 backpropagation is used on a multilayer forward neural network with more than 2.2 layers of forward neural networks consisting of the following components: input layer, hidden layer (hidden layers), input layer (output lay ERs

2.3 Each layer consists of a unit (units) consisting of 2.4 input layer is passed by the instance eigenvector of the training set into 2.5 through the connection node weight (weight) into the next layer, the first layer of output is the next layer of input 2.6 the number of hidden layers can be arbitrary , the input layer has a layer, the output layer has a layer of 2.7 per unit (unit) can also be called a nerve node, according to the biological source of the definition of more than 2.8 become 2 layer of neural network (input layer does not count) 2.8 layer weighted sum, and then according to the nonlinear Equation transformation output 2.9 as Multilayer forward neural networks, theoretically, if there are enough hidden layers (hidden layers) and a large enough set of training, any equation can be simulated 3. Design the neural network structure     3.1 Before you can train data using a neural network, you must determine the number of layers of the neural network, and the number of units per layer      3.2 eigenvectors are typically normalized when they are passed into the input layer ( Normalize) to 0 and 1 (to speed up the learning process)      3.3 Discrete variables can be encoded into each input unit corresponding to a value that can be assigned to a feature value           For example: Eigenvalue a May take three values (a0, A1, A2), and 3 input units can be used to represent a.                     If a=a0, then the unit value for A0 is 1, others take 0;      &NB Sp             If A=A1, then the value of the A1de cell is taken 1, the other takes 0, and so on       3.4 neural network can be used to do the classification ( Classification) problem, can also resolve regression (regression) issues           3.4.1 for classification problems, if 2 classes, can be represented by an output unit (0 and 1 represent 2 classes respectively)                          ,         &NB Sp       If extra 2 classes, each category is represented by an output unit                     So the number of units in the input layer is Often equals the number of categories            3.4.2 There are no clear rules to design how many hidden layers are best,                   3.4.2.1 experiments and improvements based on experimental tests and errors, as well as accuracy                    4. Cross-validation method (Cross-validation)

K-fold Cross Validation 5. BackPropagation algorithm5.1 Instances of the training set are processed iteratively by comparing the input layer predictive value (predicted value) with the true value (target value) from the neural network to the 5.3 opposite direction (from the output layer and the hidden layer and the input layer) to minimize the error (error) to update the weight of each connection (weight) 5.4 Algorithm details Input: D: DataSet, L Learning Rate (learning), a multilayer feedforward neural network input: A trained neural network (a TRA                              ined Neural Network) 5.4.1 initialization weight (weights) and bias (bias): randomly initialized between 1 and 1, or 0.5 to 0.5, each unit has One bias 5.4.2 for each training instance x, perform the following steps: 5.4.2.1: Forward from the input layer

　　　　　　　　

                         ,         &NB Sp                                  &NBS P             5.4.2.2 error-based reverse transfer                 & nbsp                   for output layer:              &NBSP ;                     for hidden layers:         &NB Sp                          ,         &NB Sp                                  &NBS P           weight update &nbsp:              &NBSp                         &NB update  :  sp;                                                                   &NBS p;5.4.3 termination conditions                          5.4.3.1 weight updates are below a certain threshold & nbsp                        5.4.3.2 forecast error rates below a certain threshold     &NBSP ;                    5.4.3.3 to achieve a preset number of cycles                            6. BackPropagation Algorithm ExampleFor output layers: for hidden layers:
Weight update: Bias update:  

Neural Network algorithm