Logistic regression model and its cost function derivation

1. Logistic regression

Logistic regression, the output variable range of the model is always between 0 and 1. The assumptions of the logistic regression model are:

g for logical functions (logistic function) is a common logical function for S-shape functions (Sigmoid function), the formula is:

, the image of the function is:

Together, we get the hypothetical model of the logistic regression model:

, the hθ (x) function is that for a given input variable, the probability of the output variable =1 (estimated probablity) is calculated based on the selected parameters:

2. Cost function for linear regression models, the cost function that we define is the sum of squares of all model errors we can also follow this definition to the logistic regression model, but we will get a non-convex function (Non-convex functions), as shown below:

This means that our cost function has many local minimums, which will affect the gradient descent algorithm to find the global minimum value. The cost function for redefining logistic regression is:

, where

The relationship between hθ (x) and cost (hθ (x), y) is shown in the following figure:

The Cost (hθ (x), y) function of this construction is characterized by the fact that when the actual Y=1 and hθ also have a 1 o'clock error of 0, when Y=1, but hθ is not 1 o'clock, the error increases as the hθ becomes smaller; when the actual y=0 and hθ are also 0 o'clock at 0, when y=0 but hθ is not 0 o'clock error with hθ To become larger and bigger. The cost of the build (hθ (x), y) is simplified as follows:

The bring-in cost function gets:

After obtaining such a cost function, we can use the gradient descent algorithm to find the parameters which can make the cost function least. The algorithm is:

After derivation, get:

Reference: Stanford University Open Class: Machine learning course http://open.163.com/special/opencourse/machinelearning.html