"Wunda deeplearning.ai Note two" on the neural network

The construction of Neural Networks (neural network) is inspired by the operation of biological neural network function. Artificial neural networks are usually optimized by a learning method based on mathematical statistics, so artificial neural networks are also a practical application of mathematical statistics methods.

Like other machine learning methods, neural networks have been used to solve a wide variety of problems, such as machine vision and speech recognition, which are difficult to solve by traditional rule-based programming.

1 Neural Networks

The neural network referred to in the field of machine learning refers to a mathematical or computational model that mimics the structure and function of a biological neural network for estimating or approximating functions.

For example, given some data about the size and price of a house on the market, it is necessary to build a model of house price prediction based on these data. That is, enter the area of a house and hope to output a forecast of the price by this model. Obviously, this is a linear regression problem, because in general the price and the size of the house are positively correlated. In this case, the relationship of the known data can be represented in a planar coordinate system:

The data is linearly fitted, and the house price is never negative, getting the relu function (rectified Linear unit, correcting the linear element)in the graph.

In this simple example, the size of the house as input, the price as the output, and the Relu function acts as a neuron to produce the output.

However, in addition to the size of the house, but also affected by the number of bedrooms, the location of the house and the wealth level of the region and other factors, it is necessary to build a more complex neural network model.

This forms the basic structure of a neural network model, and the neural network automatically generates hidden layers (Hidden Units) to process inputs and generate output. In this problem, if we have enough training data, we can generate a better neural network model and get more accurate results.

In simple terms, deep learning is a more complex neural network.

In this model, a cost function is established, and then the gradient descent method is used to find the optimal solution of the parameter. The final accuracy rate of the cat recognizer written by this algorithm is not high enough to further improve the recognition accuracy, it is necessary to establish a multilayer neural network to train the sample.

2 Symbolic conventions

As shown in the neural network, the front is the input layer, the middle is hidden layer, and finally the output layer. The middle tier is known as the hidden layer because in the course of training, you will see what the input samples are and what the output is, and the actual values generated by the neural nodes in the middle layer cannot be observed.

So the middle tier is called the hidden layer, just because you don't see it in the training set.

In the preceding logistic regression, an x is used to represent the input, which is replaced by the symbol a^[0], the number in the superscript "[]" denotes the first layer in the neural network, and the symbol a represents the activation (Activation), which refers to the values of the different levels of the neural network passed to the subsequent levels.

3 Representation of neural networks

In a logistic regression, all two parameters are initialized to zero directly. In a neural network, the parameter w is usually randomly initialized, and parameter B is initialized to 0.

All parameters except w,b, such as the learning rate alpha, the number of layers L of the neural network, the number of nodes in the layer L N^[K] and which activation functions in the hidden layer are called hyper-parameters (Hyper Parameters), because their values determine the last value of the parameter w,b.

OK, the contents of the last section to this, the next section on the activation function and other relevant knowledge points ~

Source:

http://mp.weixin.qq.com/s?__biz=MzI4MDYzNzg4Mw==&mid=2247487190&idx=1&sn= dfda6a615147a4510a04810a168f9dae&chksm= ebb43602dcc3bf14712ee33d6cc793921e1f0cbadf51e7a640274fcfc50c300e27d27e4feb62&mpshare=1&scene=23& Srcid=03200uuiaelyxszn6zfcos7u#rd