Machine Learning: Python implements the least mean square algorithm (lms) and pythonlms.

Machine Learning: Python implements the least mean square algorithm (lms) and pythonlms.

The main difference between the lms algorithm and the Rosenblatt sensor is that the weight correction method is different. Lms uses the batch correction algorithm, which is used by the Rosenblatt sensor.

Is a single sample correction algorithm. Both algorithms are single-layer sensors and are only applicable to linearly segmented situations.

The code and description are as follows:

''' Algorithm: the expectation value of the squared difference between the sample prediction output value and the actual output value. It is recorded as the sample value set to observed, if predicted is the predicted sample value, the formula is calculated: (it is converted to an easy-to-write method. It is not written in a standard way because it is difficult to write mathematical symbols here) MES = [(observed [0]-pridicted [0]) * (observed [0]-pridicted [0]) + .... (observed [n]-pridicted [n]) * (observed [n]-pridicted [n])]/n ''' variable conventions: upper case indicates a matrix or array, and lower case indicates a number X: indicates an array or matrix x: indicates a value ''''' of the corresponding array or matrix about learning efficiency (also called Step Size: controls the adjustment of weights in the nth iteration ). (Parameter a below): The learning efficiency is too high: the convergence speed is improved, the stability is reduced, that is, the output result is fast, but the results accuracy is poor. The learning efficiency is too small: the stability is improved, and the convergence speed is reduced, that is, the results are slow, accurate, and resource-consuming. There are dedicated algorithms for determining the learning efficiency. We will not study them here. In most cases, select the value '''import numpy as npa = 0.1 # learning rate 0 <a <1X = np. array ([[], [], [], [], []) # input matrix D = np. array ([,]) # expected output matrix W = np. array ([0.005]) # weight vector expect_e = # expected error maxtrycount = 20 # maximum number of attempts # Hard limiting function (standard), which is relatively simple: input v is greater than 0, and 1 is returned. if the value is less than or equal to 0, return-1) ''' and the final weight is W ([0.1, 0.1]), then: 0.1x + 0.1y = 0 ==> y =-x, that is: classification line equation: y =-x ''' def sgn (v): if v> 0: return 1 else: return 0 # The ratio of-1 in the previous Sensor Single Sample training is adjusted to 0 for testing purposes. -1 unable to train the result # Read the actual output ''' here is the multiplication of two vectors, corresponding mathematical formula: a (m, n) * B (p, q) = m * p + n * q in the following function, when xn = 1 in a loop (W = ([0.1, 0.1]): np. dot (W. t, x) = (0.1) * (0.1, 0.1) = 1*0.1 + 1*0.2 => 0 => sgn returns 1 ''' def get_v (W, x): return sgn (np. dot (W. t, x) # dot represents the multiplication of two matrices # Read error value def get_e (W, x, d): return d-get_v (W, x) # weight calculation function (batch correction) ''' corresponding mathematical formula: w (n + 1) = w (n) + a * x (n) * e corresponds to the following variables: w (n + 1) <= return value of neww w (n) <= oldw (old weight vector) a <= a (learning rate, value Range: 0 <a <1) x (n) <= x (input value) e <= error value or error signal ''' def neww (oldW, d, x, a): e = get_e (oldW, x, d) return (oldW + a * x * e, e) # modify the weight ''' the principle of this loop: weight Correction Principle (batch correction) ==> each time a neural network reads a sample, the algorithm is corrected to reach the expected error value or the maximum number of attempts, correction process ended ''' cnt = 0 while True: err = 0 I = 0 for xn in X: W, e = neww (W, D [I], xn,) I + = 1 err + = pow (e, 2) # core steps of the lms algorithm, namely: MES err/= float (I) cnt + = 1 print (u "% d Adjusted weight:" % cnt) print (W) print (u "error: % f" % err) if err <expect_e or cnt> = maxtrycount: breakprint ("Last weight:", W. t) # print the output result ("Start the verification result... ") for xn in X: print (" D % s and W % s => % d "% (xn, W. t, get_v (W, xn) # test accuracy: ''' the preceding description shows that the classification line equation is y =-x, which can be seen from the coordinate axis: (2, 3) belongs to + 1 category, (-2,-1) belongs to 0 category ''' print ("start testing... ") test = np. array ([2, 3]) print ("D % s and W % s => % d" % (test, W. t, get_v (W, test) test = np. array ([-2,-1]) print ("D % s and W % s => % d" % (test, W. t, get_v (W, test )))

Output result:

Weight after 1st adjustments: [0.1 0.1] error: 0.250000 weight after 2nd adjustments: [0.1 0.1] error: 0.000000 final weight: [0.1 0.1] Start verification results... D [1 1] and W [0.1 0.1] => 1D [1 0] and W [0.1 0.1] => 1D [0 1] and W [0.1 0.1] => 1D [0 0] and W [0.1 0.1] => 0 start testing... D [2 3] and W [0.1 0.1] => 1D [-2-1] and W [0.1 0.1] => 0

The results show that after two trainings, the optimal results are obtained.

Note: after multiple adjustments to the sample or weight, sometimes results are returned in 20 cycles, and sometimes the optimal solution is not found. Therefore, during the experiment

In the expected results, except for the insufficient number of cycles, the biggest possible cause is the problem of setting samples or weights.