The difference between bulk learning and online learning

First, bulk learning

In the bulk method of supervised learning, the adjustment of the prominent weights of multilayer perceptron occurs after all n examples of the training sample set , which constitute a round of training. In other words, the cost function of bulk learning is defined by the average error energy. The synaptic value adjustment of multilayer Perceptron is based on round-turn . Accordingly, a learning curve is achieved by depicting the average error energy on the graph of the number of rounds, and for each round of training, the sample of the training sample is randomly selected. The learning curve is calculated by averaging a large enough number of such implementations, where each implementation is randomly selected under different initial conditions.

Advantages:

(1) Eliminate the effect of sample order

(2) Accurate estimation of gradient vectors, therefore, under simple conditions, the method is guaranteed to reduce the convergence of the maximum speed to the local minimum point.

(3) The parallelism of learning.

Disadvantages:

(1) Storage requirements

Second, online learning

In the online method of supervised learning, the adjustment of synaptic weights of multilayer perceptron is based on sample-sample , and the cost function used to minimize is the instantaneous error energy of all. As with bulk learning, the learning curve for online learning is calculated by the overall average on a sufficiently large number of randomly selected initial conditions. For a given network structure, learning curve obtained under online learning is very different from the learning curve obtained under bulk learning.

Given training samples presented to the network in a random way, the use of online learning makes the search in the multi-safeguard value space in fact random; it is for this reason that online learning methods are sometimes called random methods.

Advantages:

(1) Easy to implement

(2) It provides effective solutions for large-scale and difficult pattern classification problems.

(3) Randomness makes it difficult to get into local extreme points

(4) Much less storage capacity

For this reason, a lot of content is about online learning.

References: "Neural networks and machine learning"

　　

The difference between bulk learning and online learning