Machine learning-Regression

Regression

What is the return, we all go to high school, in high school mathematics, there is a particularly important point of knowledge is the least squares, what meaning, give you a batch of data x, as well as the corresponding result y, find a straight line to fit these points well, such as consider the following example;

We want to predict how much a 3000 square foot house will cost, which involves a return.

Single-Variable linear regression

Let's take a look at the simple example, the relationship between house price and area is like this:

6.1101,17.5925.5277,9.13028.5186,13.6627.0032,11.8545.8598,6.82338.3829,11.8867.4764,4.3483

We want to fit an equation like this:

import numpy as npdata = np.loadtext(‘linear_regression_data1.txt‘, delimiter=‘,‘)X = np.c_[np.ones(data.shape[0]),data[:,0]]y = np.c_[data[:,1]]

np.c_this is a function to add a column, because we need to offset the amount, so the bias is set to 1. Let's look at what the result is:

plt.scatter(X[:,1],y,s=30,c=‘r‘,marker=‘x‘,linewidths=1)plt.xlim(4,24)plt.xlabel(‘Population of City in 10,000s‘)plt.ylabel(‘Profit in $10,000s‘)plt.show()

Our task is to find a straight line to fit the data nicely.

Error function

The cost function, which is a measure of the error between the fitted model and the real model, generally we use the following formula to indicate the degree of error:

So, our task is simpler, just to minimize the function, and here is the definition of the loss function:

def computerCost(X,y,theta=[[0],[0]]):    m = y.size    J = 0    h = X.dot(theta)    J = 1.0/(2*m)*(np.sum(np.square(h-y)))    return J

Let's calculate what the error is when theta=[0,0].

print computerCost(X,y) #32.0727338775

Stochastic gradient descent algorithm

Here we biased the loss function above to get

def gradientDescent(X, y, theta=[[0],[0]], alpha=0.01,  num_iters=1500):    m = y.size    J_history = np.zeros(num_iters)    for iter in np.arange(num_iters):        h = X.dot(theta)        theta = theta - alpha*(1.0/m)*(X.T.dot(h-y))        J_history[iter] = computeCost(X, y, theta)    return(theta, J_history)

If the derivation does not, go to fill the math bar ...

Following the above step, we get the values of the theta and loss functions for each gradient descent. Let's draw a value like this:

# 画出每一次迭代和损失函数变化theta , Cost_J = gradientDescent(X, y)print(‘theta: ‘,theta.ravel())plt.plot(Cost_J)plt.ylabel(‘Cost J‘)plt.xlabel(‘Iterations‘);

The results obtained are theta to[-3.63029144, 1.16636235]

So our equation is h=-3.63029144+1.16636235x , we draw a line like this,

plt.scatter(X[:,1],y,s=30,c=‘r‘,marker=‘x‘,linewidths=1)plt.xlim(4,24)plt.xlabel(‘Population of City in 10,000s‘)plt.ylabel(‘Profit in $10,000s‘)xx = np.arange(5,23)yy = theta[0] + theta[1]*xxplt.plot(xx,yy)plt.show()

Feel good, this is our own writing model, in fact, in Scikit-learn has encapsulated a lot of machine learning algorithms, can be used in industry.

regr = LinearRegression()regr.fit(X[:,1].reshape(-1,1),y.ravel())plt.plot(xx,regr.intercept_+regr.coef_*xx,label=‘scikit-learn‘)plt.legend(loc=0)plt.show()

By this, the simplest general linear regression has been learned, there is a multivariate regression, the same principle.

Learning machine learning, every algorithm to understand, and can achieve, that is the real ability!

Machine learning-Regression