Logic regression-Gradient descent method Python implementation __python

the basic framework of machine learning:

Model, target (cost function), optimization algorithm

STEP1: For a problem, we need to first establish a model, such as regression or classification model;

Step2: The cost function of the model is established by the minimum classification error, maximum likelihood or maximum posterior probability;

Step3: Solving the optimization problem

A. If there is an analytic solution to the optimization function, it is possible to find a point with a reciprocal value of 0, that is, the maximum or the youngest child, by means of a general method of calculation-a derivative of the cost function;

B. If the derivative of the optimization function is more complex, the iterative algorithm can be used to solve the problem.

for regression issues:

(1) Model

The selectable models have linear regression and LR regression.

(2) Target-cost function

Linear regression Model:

(Least squares)

LR regression model:

(Maximum likelihood)

The assumption of linear regression is that y obeys the normal distribution, and the LR regression assumes that Y obeys two distribution (y Non-zero or 1).

(3) algorithm-optimization algorithm

Gradient Descent method, Newton method and so on;

Python code:

lrgradascent.py

From numpy import * def loaddata (filepath): datamat= [] labels = [] fr = open (filepath) for line in Fr.re Adlines (): str = Line.strip (). Split (' \ t ') datamat.append ([1.0,float (str[0]), float (str[1])] Labels  . append (int (str[2])) return Mat (Datamat), Mat (labels). Transpose () def sigmoid (InX): Return 1.0/(1+exp (-inx)) def Gradascent (Datamatrix,labelsmatrix): N,m = shape (datamatrix) weights = Ones ((m,1)) step = 0.001 iter = 50  0 for K in range (ITER): value = sigmoid (datamatrix*weights) Chazhi = Labelsmatrix-value Grad = Datamatrix.transpose () *chazhi weights = weights + Step * Grad return weights def plotbestfit (weights,f Ilepath): Import Matplotlib.pyplot as Plt # illustrate the samples Datamatrix,labelsmatrix = LoadData (filepath ) N,m = shape (datamatrix) xcord1 = []; Ycord1 = [] #store the coordination of sample having label 1 XCORD2 = []; Ycord2 = [] forI in range (n): If int (labelsmatrix[i]) = = 1:xcord1.append (datamatrix[i,1]) ycord1.append ( datamatrix[i,2]) else:xcord2.append (datamatrix[i,1]) ycord2.append (datamatrix[i,2)) F IG = plt.figure () ax = Fig.add_subplot () ax.scatter (xcord1,ycord1,s = 30,c = ' Red ', marker = ' s ') ax.scatter (xcord2,ycord2,s = 30,c = ' green ') #illustrate the classifying line min_x = min (datamatrix[:,1]) [0,0] Max _x = max (datamatrix[:,2]) [0,0] y_min_x = (-weights[0]-weights[1] * min_x)/weights[2] y_max_x = (-weights[0)- WEIGHTS[1] * max_x)/weights[2] #here, sigmoid (wx = 0) so wo + w1*x1 + w2*x2 = 0 Plt.plot ([min_x,max_x],[y_min_x[0,0 ],y_max_x[0,0]], ' G ') plt.show ()

Test program:

Import lrgradascent
import pdb

filename = ' testSet.txt '
datamat, Labelsmat = Lrgradascent.loaddata ( FileName)
weights = lrgradascent.gradascent (Datamat,labelsmat)
Lrgradascent.plotbestfit (weights,filename )

Reference: http://blog.csdn.net/zouxy09/article/details/20319673