[Data Mining Course notes] artificial neural network (ANN)

The Artificial Neural network (Artificial neural Networks), as its name implies, is modeled after the neuron structure of the NPC. is an artificial neural network model with hidden layers. X = (x1,x2,.., xm) is the input of the Ann, which is the value of a record on M properties. Each attribute corresponds to an input node.

For the input layer, the input layer output OI is the input layer input XI.

For one node of the hidden layer J, the input of node J is Σoiwij (I is the value of all input layer nodes connected to node J). It can be found that the connection between nodes and nodes has a weight, this weight will affect the final classification results. And our training process for Ann is actually the process of adjusting weights.

And the output of the hidden layer introduces an excitation function, at which point the excitation function is set to

In the excitation function of the argument x, in the Ann is the input of this node, that is, Σoiwij

For the output layer, the input value is the output value of the hidden layer. The output value of the output layer is similar to the method of calculating the hidden layer. Usually, the same excitation function is taken.

Forward algorithm: Used to classify a piece of data. Through the above calculation method, we can obtain an output, compare this output with the threshold value, can produce the classification result.

Back algorithm: Used to train the model, adjust the weights.

e represents the error of the classification and true classification of Ann predictions. Our goal is to change e as small as possible.

Given a well-labeled data set, you can define an error function like this:

calculate the gradient of e

The "pace" of descent

Batch Neural Network Training

Thought: For every day record, calculate its o, then calculate e, find the gradient, adjust W.

Online Training

Thought: Revise weights according to each day's heads-Up Records.

Advantages: Faster convergence rates and reduced likelihood of falling into local optimal solutions.

Algorithm process

[Data Mining Course notes] artificial neural network (ANN)