Sensor and maximum interval classifier for Learning Theory

So far, the learning methods we have used are batch learning. That is, first, the training set is given to learn the parameters in the fitting hypothesis function, then use an independent test set when evaluating the effect. This blog post will introduce an online learning method, that is, algorithms must constantly make predictions during the learning process, rather than making predictions in batches, prediction is made only after the learning process ends.

In the online learning mode, the learning algorithm learns a sample sequence in sequence :. specifically, the algorithm first uses the assumption function (the parameter is first initialized to some suitable values) to give the predicted value. After the prediction is complete, the parameters are updated based on the actual values and predicted values. Then, based on the updated parameters, the calculated predicted values are used to update the parameters based on the actual values and predicted values; repeat until the last sample. It can be seen that online learning only pays attention to the error caused by the samples to be learned. Therefore, in the model application process, it is necessary to continuously predict the learning parameters, then, the parameters are constantly adjusted based on the current predicted values and actual values.

We provide a boundary for the online learning residual of the sensor algorithm. Label set used.

Sensor parameters.NYesXBecause one of the parameters represents a constant term in the function, this parameter does notXAn element corresponds to it. make predictions based on the following assumptions:

 

Where

A training sample is provided. The sensor learning rule is that if (that is, the predicted value is equal to the actual value), the parameter is not changed. If not, update the parameter as follows:

The following theory provides the boundaries of the number of online learning error predictions of the sensor algorithm, so that each online learning will produce a sample error. note that there is no dependency between the number of online learning error predictions and the number of samples and the input dimension.

 

Theorem (Block, 1962, and noviko, 1962) for a given sample sequence, we assume that for all samples, we further assume that there is a unit vector so that for all samples (that is, at that time ,, therefore, the minimum interval is used to classify the data at least. Therefore, we can conclude that the number of error predictions of the sensor Algorithm on this sample sequence does not exceed.

Proof: we can see from the above discussion that the algorithm updates parameters only when the function has made an incorrect prediction. use the weight of the table's k-th error. Therefore, because the weight is initialized to 0, if the sample prediction error is k-th error, it means:

According to the sensor learning rules, then:

Recurrence:

Known, so there are:

Same reasoning:

The same method as above can be obtained through gradual recursion:

Together, we can get:

The second inequality is derived based on the unit vector (and, yesZAnd). The result of the above export shows.

 

Sensor and maximum interval classifier for Learning Theory