Machine Learning---neural Network

Machine Learning:neural NetworkA: Preface

Definition of the neural network on 1,wikipedia:

InchMachine Learning, Artificial neural networks (anns) is a family of statistical learning algorithms inspired byBiological Neural Networks(TheCentral Nervous Systemsof animals, in particular theBrain) and is used to estimate orapproximatefunctionsThat can depend on a large number ofinput Sand is generally unknown. Artificial Neural networks is generally presented as systems of interconnected "neurons"which can compute values from inputs, and is capable ofMachine LearningAs well asPattern RecognitionThanks to their adaptive nature.

2, why should I introduce neural Network?

We have previously learned about regression and classification models, but their actual application is constrained by the size of the data (i.e., the dimension disaster (Curse of dimensionality)). The neural network is very advantageous for processing a large number of input features, such as extracting pixels from a picture in computer vision as an input feature, which will give you a large number of input feature sets, and if you still return and classify the model, the time required to learn the parameters will be unbearable;

II: Neural network-representation

1,neural Network Model

In a neural network, we call the first layer the input layer, and the last layer is called the output layer, and several layers in the middle are called the hidden layer.

Let's look at a very simple neural network:

This simple neural network is equivalent to a logical classifier. The result is that the neutron conversion function on the output layer is the sigmoid function. Of course, it is entirely possible to choose the right activation function according to our actual problem.

let's describe the neural network in detail:

, a (I,J) is expressed as the first I activation of Layer J, which is obtained by the activation function, which is the G (.) here. Generally, the activation function is the same for each layer (except for the output layer, which needs to be determined in the form of a final need, such as discrete, continuous, multi-classification, etc.), and of course you can choose to make them different. , which increases the difficulty of achieving it. The theta weight parameter matrix controls the mapping of Layer J to layer j+1.

2,forward propagation algorithm

The method of forward propagation is to compute h (x) from input Layeràhidden layeràoutput layer, and we do not model and tune the raw data directly, but instead use the result of the middle layer. However, the result of the middle tier is learned from the original data; in other words, there is great flexibility, and the conversion between each layer can be any linear combination or polynomial combination.

Here's a look at how to implement a logical expression using a neural network:

Through the analysis of multiple and, or, not, a small neural network is combined to realize the function of Xnor.

3, multi-classification Problem of neural network

For multi-classification, we use the Softmax activation function to represent the final output.

Three, neural network-learning

In the following, we will describe the learning process of neural network with the classification problem, the cost Function of the neural network is obtained by analogy logistic regression, and then the parameters are calculated by the gradient descent algorithm.

1,error Back propagation algorithm:

We know that the gradient descent algorithm consists of two steps:

(1), the partial derivative of the parameter theta is obtained for cost function;

(2), the parameter theta is updated and adjusted according to the partial derivative;

Error Back propagation algorithm provides an efficient method for partial derivative.

For example, in the neural network shown, we generalize the derivation:

Through the above deduction, we have obtained an efficient method to solve the partial derivative: the value of a (i) is obtained by forward propagation, and then the value of the delta is calculated by the back propagation, then it can be taken into the equation (6);

Now let's integrate the entire process of neural Network learning, and give a specific example:

The above back propagation algorithm directly refers to the results deduced above;

Summarize back propagation algorithm:

2, the following describes the neural network in MATLAB implementation tips:

Simply put, the weight parameter matrix is unroll into a vector, then using the existing gradient descent algorithm in the library to find the optimal parameters, and finally reshape into a matrix form; The reason for this is that the parameters of the ready-made gradient descent algorithm, the Inittheta requirement, must be in the form of a vector.

3,gradient Checking

This is a mathematical method to seek partial derivative.

It can be used to verify that the gradient descent algorithm is implemented correctly, when the data of the two is very similar, it indicates that the result of the operation is correct, and if the results differ greatly, the gradient descent algorithm does not work correctly.

4, how to initialize the value of the weight matrix theta

Four, summarize the whole process

Summary of haste, will inevitably have flaws and mistakes, but also hope that everyone is not hesitate to correct, enlighten;

--------------------------------------------------------------------------------------------------------------- -------

The courseware from Stanford University Andrew Ng Teacher's machine learning ppt

Machine Learning---neural Network