Lineage Logical Regression classification algorithm

Lineage Logical Regression classification algorithm

1. Overview

Lineage Logistic regression is a simple and effective classification algorithm .

What is regression: For example, we have two types of data, each with 10 points, when we draw these points out, there will be a line to distinguish between the two sets of data, we fit this curve (because it is likely to be nonlinear), is the regression. We use a lot of data to find this line, and to fit the expression of this line , and then new data, we use this line as a distinction to achieve classification.

is a data set of two sets of data, in the middle of a line that distinguishes between two sets of data.

Obviously, only this linear data distribution is suitable for linear logistic regression.

2. Algorithm idea

Lineage regression classification algorithm is the application of linear regression in the classification scene

In this scenario, the result is to get a classification label for the sample data, rather than get the regression line

2.1. Algorithm diagram

1) algorithm target ()?

Plain English: Calculates the vertical distance from the Y value of each point to the fitting line, if

Distance >0, divided into class A

distance <0, divided into class B

2) How to get fit floss?

Plain English: You can only assume that the function of a line or polygon can be expressed as

Y ( fit ) =w1*x1 + w2*x2 + w3*x3 + ...

where W is a pending parameter

and x is the characteristic value of each dimension of the data

Thus, the above problems have become Sample y (x)-Y ( fit ) >0? A:B

3) How to solve a set of optimal w parameters?

basic idea: substituting "prior data" to solve the problem by inverse pushing

But it is extremely difficult to solve the parameters for inequalities

A common solution is to make a conversion to the solution of inequalities:

1. Compresses the difference between sample y (x)-Y ( fit ) to a 0~1 Cell,
2. Then substituting a large number of sample eigenvalues to obtain a series of output results;
3. The output results are compared with the prior classes of the samples, and the parameter values of the fitting line are adjusted according to the comparison, which is the optimal parameter approximation of the fitting line.

To transform the problem into Approximation Solution A typical mathematical problem

2.2. sigmoid function

In the above algorithm, the sigmoid function is usually used as the conversion function.

function expression:

Note: The x here is a vector

Function curve:

The use of the sigmoid function, is to let the sample point after the operation of the results to limit the 0~1 between the compression of the large vibration of the data, so as to facilitate the classification of the sample Point label (classification to sigmoid whether the calculation result of the function is greater than 0.5 )

3. Algorithm implementation Analysis

3.1. Realization of Ideas

Mathematical representations of Algorithmic thinking:

Set the dataset's eigenvalues to X1,x2,x3 ...

Find their coefficient of regression WI

set z=w1*x1+w2*x2 ..... , and then substituting the z value into the sigmoid function and judging the result, you can get the classification label

The question is how to get a set of suitable parameters WI?

The analytic approach is difficult to solve, and the iterative method can be used to find the optimal solution conveniently.

To put it simply, we are constantly substituting the sample eigenvalues, calculating the results followed by their actual labels, correcting the parameters according to the difference, and then substituting the new sample values for calculation, and repeating them until there is no need to fix or have reached the preset iteration count.

Note: This process is achieved by the gradient rise method.

3.2. Gradient Rise algorithm

Popular explanation: Through small steps --"Adjust the direction-" continue to step forward-the final approximation of the optimal solution

gradient rise refers to the direction in which the function grows. In the implementation process, iterative operations continue until the value of W almost no longer changes.

：

Lineage Logical Regression classification algorithm