羅吉斯迴歸-梯度下降法 python實現__python

來源:互聯網
上載者:User

機器學習的基本架構:

模型(model)、目標(cost function)、最佳化演算法

Step1:對於一個問題,需要首先建立一個模型,如迴歸或分類模型;

step2:通過最小分類誤差、最大似然或最大後驗機率建立模型的代價函數;

step3:最佳化問題求解

a.如果最佳化函數存在解析解,則可以通過一般的求值方法-對代價函數求導,找到倒數為0的點,即是最大值或者最小孩子;

b.如果上述方法求最佳化函數導數比較複雜,可利用迭代演算法也求解。


對於迴歸問題:

(1)模型

可選擇的模型有線性迴歸和LR迴歸。

(2)目標-cost function

線性迴歸模型:

 (最小二乘)

LR迴歸模型:

(極大似然)


線性迴歸的假設前提是y服從常態分佈,LR迴歸的假設前提是y服從二項分布(y非零即1)。

(3)演算法-最佳化演算法

梯度下降法、牛頓法等;


Python代碼:

LRgradAscent.py

from numpy import *def loadData(filepath):    dataMat= []    labels = []    fr = open(filepath)    for line in fr.readlines():        str = line.strip().split('\t')        dataMat.append([1.0,float(str[0]),float(str[1])])        labels.append(int(str[2]))    return mat(dataMat), mat(labels).transpose()def sigmoid(inX):    return 1.0/(1+exp(-inX))def gradAscent(dataMatrix,labelsMatrix):    n,m = shape(dataMatrix)    weights = ones((m,1))    step = 0.001    iter = 500    for k in range(iter):        value = sigmoid(dataMatrix*weights)        chazhi = labelsMatrix - value        grad = dataMatrix.transpose()*chazhi        weights = weights + step * grad    return weights    def plotBestFit(weights,filepath):    import matplotlib.pyplot as plt    # illustrate the samples    dataMatrix,labelsMatrix = loadData(filepath)    n,m = shape(dataMatrix)    xcord1 = []; ycord1 = []  #store the coordination of sample having label 1    xcord2 = []; ycord2 = []    for i in range(n):        if int(labelsMatrix[i]) == 1:            xcord1.append(dataMatrix[i,1])            ycord1.append(dataMatrix[i,2])        else:            xcord2.append(dataMatrix[i,1])            ycord2.append(dataMatrix[i,2])    fig = plt.figure()    ax = fig.add_subplot(111)    ax.scatter(xcord1,ycord1,s = 30,c = 'red',marker = 's')    ax.scatter(xcord2,ycord2,s = 30,c = 'green')        #illustrate the classifying line    min_x = min(dataMatrix[:,1])[0,0]    max_x = max(dataMatrix[:,2])[0,0]    y_min_x = (-weights[0] - weights[1] * min_x) / weights[2]    y_max_x = (-weights[0] - weights[1] * max_x) / weights[2]  #here, sigmoid(wx = 0) so wo + w1*x1 + w2*x2 = 0    plt.plot([min_x,max_x],[y_min_x[0,0],y_max_x[0,0]],'-g')     plt.show()

測試程式:

import LRgradAscentimport pdbfilename = 'testSet.txt'dataMat, labelsMat = LRgradAscent.loadData(filename)weights = LRgradAscent.gradAscent(dataMat,labelsMat)LRgradAscent.plotBestFit(weights,filename)



參考:http://blog.csdn.net/zouxy09/article/details/20319673


聯繫我們

該頁面正文內容均來源於網絡整理,並不代表阿里雲官方的觀點,該頁面所提到的產品和服務也與阿里云無關,如果該頁面內容對您造成了困擾,歡迎寫郵件給我們,收到郵件我們將在5個工作日內處理。

如果您發現本社區中有涉嫌抄襲的內容,歡迎發送郵件至: info-contact@alibabacloud.com 進行舉報並提供相關證據,工作人員會在 5 個工作天內聯絡您,一經查實,本站將立刻刪除涉嫌侵權內容。

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.