Perceptual Machine PLA

The Perceptron is a linear classification model of class Two classification, its input is the characteristic vector of the instance, the output is the class of the instance, take +1,-1. Perceptual machine learning is designed to find out the discrete hyper plane that divides the training data linearly. The details are as follows:

• Perceptual Machine Model
• Perceptual Machine Learning Strategy
• Perceptual Machine Learning Algorithm

1 Perceptual Machine Model

Input space (feature space) is RN output space is {+1,-1}

The function f (x) =sign (W.X+B), which is the input space to the output space, is called the perceptual machine. W is the weighted value and B is biased. The hypothetical space of the Perceptron model is all linear classification models defined in the feature space. To obtain a perceptual machine model, it is necessary to require w,b.

2 Perceptual machine Learning strategies

Assuming that the training data set is linearly separable, the purpose of the perceptual learning is to obtain a separate super-plane which can separate the positive and negative instances completely correctly, in order to find out such a super-plane, that is to determine the parameter w,b of the Perceptron model, we need to determine a learning strategy, that is, to define the loss function and minimize it.

Loss function A natural selection is the total number of false classification points, but such loss function is not a continuous derivative function of parameter w,b, and is not easy to optimize; the other choice of loss function is the total distance from the wrong classification point to the super plane S.

First write out the distance from any point in the input space x0 to the super plane s:

Second, for the mis-classification point (Xi,yi),

The distance from the wrong classification point XI to the hyper-plane is

The total distance from all the wrong classification points to the hyper plane is

So, do not consider | | w| |, the loss function of the perceptron is obtained, and this function is the experiential risk function of the perceptual machine.

Obviously, the loss function is non-negative, if there is no mis-classification point, the loss function is zero, and the less the mis-classification point is, the closer the error classification point is to the super-plane, the smaller the value of the loss letter.

The strategy of perceptual machine learning is to select the model parameter w,b that minimizes the loss function in the hypothetical space.

3 Perceptual machine Learning algorithm

The Perceptual machine learning problem is transformed into the optimization problem of solving the loss function, and the optimization method is the stochastic gradient descent method.

Arbitrary selection of a super-plane, using gradient descent method to minimize the objective function, the minimization process is not to make the gradient of the wrong classification points in M, but a random selection of a false classification point to reduce its gradient.

The gradient of the loss function is as follows:

Then randomly select a mis-classification point (xi,yi) to update the W,B

The type of η (0<η<=1) is the step, also known as the learning rate.

The whole process is as follows:

With an example, the code of Practice is as follows:

ImportNumPy as NPImportrandomrows= [[3,3,1],[4,3,1],[1,1,-1]]#rows are training samples, n is the number of iterations, R is the learning ratedefPLA (rows,n=100,r=1):    #Initialize W,bw = [0.0]* (len (rows[0])-1) b= 0.0 forIinchxrange (n): Row= Rows[random.randint (0,len (rows)-1)]#randomly select a rowx = Np.array (row[0:-1]) Justice= row[-1]* (sum (w*x) +b)ifJustice <=0:w+ = r*row[-1]*x B+ = R*row[-1]    returnw,bPrintPLA (rows)

Perceptual Machine PLA