Deep Learning Basics (i): Logistic classification

This series mainly records I am learning each deep learning algorithm time note, because has already learned the general concept, therefore this round study focuses on the formula derivation and the concrete realization , but to the conceptual description is not many, therefore is suitable for this has the certain foundation the classmate .

Before formally beginning to write deep learning knowledge, there will be two traditional neural network content, because deep learning has used the knowledge of the previous neural network. It is necessary to understand how traditional neural networks work, which will help to lay a solid foundation for subsequent learning.

1. Logistic classification

Almost all textbooks start with the logistic classification, because the logistic classification is too classic and a basic part of the neural network, and each neuron (cell) can be considered as a logistic classification.

The so-called logistic classification, as its name implies, is a two taxonomy that divides data into 0 and 12 classes.

The flow of logistic classification is simple, there are 4 steps of linear summation, sigmoid function activation, calculation error and correcting parameter. The first two parts are used for judging, and the latter two steps are used for correction. This article is divided into 3 parts, the first 2 parts of the general logistic classification of the flow, the third part of the extension.

1.1 Linear summation and sigmoid functions

1th, 2 steps are used to determine the classification according to the input, so put together to say. Suppose there is an n-dimensional input column vector x, there is also an n-dimensional parameter column vector h, and a bias B, so that z can be summed linearly.
Z=htx+b

At this point, because Z's range is [−∞,+∞], it is not possible to determine whether X is 0 or 1 according to Z. So we need a function to map the value of z to [0,1], which is the activation function . There are many types of activation functions, and the activation function here is the sigmoid function.
Σ (x) =11+e−xσ′ (x) =σ (x) (1−σ (x))
Its shape is

Figure 1 sigmoid function
As you can see the larger the X, the closer the Σ (x) is to 1, the closer it is to 0. So at the time of judgment, we first get to the Z-sigmoid function
A=σ (z) =σ (htx+b)
When a is greater than 0.5, we decide that x should belong to Class 1, and if it is less than 0.5, it belongs to Class 0. In this way, the work of judging is completed.

1.2 error calculation and parameter correction

The parameter vector h and offset B are used in the process of the above judgment. It can be said that the values of H and B are directly related to the accuracy of the logistic judgments. So how are these two sets of parameters obtained? This involves the correction of the parameters. at the very beginning, the value in H is random, and the value of B is 0. We use constant training to enable H and B to achieve a better value as much as possible.

So how to train it. Assuming that we expect the input x to be a Y, and the actual result is a, then we define a loss function C (a,y), which modifies the values of H and B to minimize C, which is an optimization problem. In convex optimization problems, you can use the
∂c∂h=0,∂c∂b=0
To calculate the optimal solution of H and B directly. However, in some cases, such as large data size, or non-convex optimization problems, it is not possible to do so, but the iterative method to obtain the local optimal solution.
H:=h−η∂c∂hb:=b−η∂c∂b
where η indicates the learning rate. Here, we set the loss function as a square loss function, i.e. c=12 (a−y) 2 so you can get
∂c∂h====c′∂a∂h