Preliminary Study No. 1-before, after, and after logical regression learning No. 1
Preliminary Study No. 1-before, after, and 2 of Logistic regression learning
Some people think that mathematics is already very complicated. Is it more difficult to bring it into the coding process of implementation?
Actually, there are some difficulties, but when you do some research, you will find the rules and routines. In this case, it is better to patiently put this in the past, this is the real progress.
The code of this blog is from "Machine Learning pthon Practice". Let's use the logic regression in the previous article to see how the algorithm is implemented when it comes to implementation.
First, we need to download the numpy package first when using python. The address is as follows:
Http://www.lfd.uci.edu /~ Gohlke/pythonlibs/(find numpy and watch python directly)
With this python tool, it can be compared with matlab to a certain extent.
The first step serves as the basis of the entire model. The sigmoid function is always used. The Python method is also very simple to define:
<span style="font-family:Microsoft YaHei;font-size:14px;">def sigmoid(inX): return 1.0/(1+exp(-inX))</span>
The second step is to use an algorithm to make the model the best. First, we will give the simplest gradient descent Encoding Process:
<span style="font-family:Microsoft YaHei;font-size:14px;">def gradAscent(dataMatIn, classLabels): dataMatrix = mat(dataMatIn) #convert to NumPy matrix labelMat = mat(classLabels).transpose() #convert to NumPy matrix m,n = shape(dataMatrix) alpha = 0.001 maxCycles = 500 weights = ones((n,1)) for k in range(maxCycles): #heavy on matrix operations h = sigmoid(dataMatrix*weights) #matrix mult error = (labelMat - h) #vector subtraction weights = weights + alpha * dataMatrix.transpose()* error #matrix mult return weights</span>
Here we will understand this piece of code:
The first three rows of the method are data processing processes. The data sets and result sets are matrix respectively, and the number of data items in the dimension of the dataset is found.
Starting from line 1, we define the parameters. First, alpha, as a learning rate variable, represents the learning volume of each iteration gradient.
Maxcycles is the maximum number of iterations, that is, when it loops for 500 times, regardless of whether the calculation results meet the requirements, the calculation will be terminated. Of course, to improve the computing efficiency, we can also set a tolerance e. When the result of k + 1 is no greater than the value of e, settlement can also be terminated.
Weights is the actual weight we need, all initially 1
In the following loop, this sigmoid will be applied repeatedly. The process is as follows:
1. Find a result by yourself as the result of this round
2. Compare the result with the real result set labelMat to form an error vector, which is equivalent to finding the gradient. (It is hard to understand the difference operation in the computer based on the gradient definition)
3. Multiply this gradient by the learning rate and combine it with k weights as k + 1 results to participate in the next operation.
Bingo! In fact, this is the essence of Logistic regression. Just a few lines of code solves all the problems. Finally, we organized and verified several steps of data for the integrity of this model. The Code is as follows:
<span style="font-family:Microsoft YaHei;font-size:14px;">def classifyVector(inX, weights): prob = sigmoid(sum(inX*weights)) if prob > 0.5: return 1.0 else: return 0.0</span>
This method specifies that if the calculation result is greater than 0.5, it is regarded as 1, and if the calculation result is less than 0.5. The value is negative.
<span style="font-family:Microsoft YaHei;font-size:14px;">def loadDataSet(): dataMat = []; labelMat = [] fr = open('testSet.txt') for line in fr.readlines(): lineArr = line.strip().split() dataMat.append([1.0, float(lineArr[0]), float(lineArr[1])]) labelMat.append(int(lineArr[2])) return dataMat,labelMat</span>This method is customized for text data to sort out the data defined by the python list data structure. If any function does not understand, please Baidu this python function.
<span style="font-family:Microsoft YaHei;font-size:14px;">def colicTest(): frTrain = open('horseColicTraining.txt'); frTest = open('horseColicTest.txt') trainingSet = []; trainingLabels = [] for line in frTrain.readlines(): currLine = line.strip().split('\t') lineArr =[] for i in range(21): lineArr.append(float(currLine[i])) trainingSet.append(lineArr) trainingLabels.append(float(currLine[21])) trainWeights = gradAscent(array(trainingSet), trainingLabels) print trainWeights errorCount = 0; numTestVec = 0.0 for line in frTest.readlines(): numTestVec += 1.0 currLine = line.strip().split('\t') lineArr =[] for i in range(21): lineArr.append(float(currLine[i])) if int(classifyVector(array(lineArr), trainWeights))!= int(currLine[21]): errorCount += 1 errorRate = (float(errorCount)/numTestVec) print "the error rate of this test is: %f" % errorRate return errorRate</span>
This method calls the previous method, which is equivalent to the main function of a program, but the returned result is an error rate errorRate. We can use this to see whether this model is good or bad ~
Finally, the dataset and result set are provided. You can copy the code to python to practice it ~
Test Set: click here
Training set: click here