The training algorithm of perceptual device

1. Source and basic idea of perceptron algorithmThe term "perceptron" comes from the the mid 1950s to the 60 's, a classification learning machine model, which belongs to the problem of the bionics field of animal learning.       Some researchers at the time thought that Perceptron was a powerful model for learning machines, and later found that estimates were too high, but some of the concepts associated with developing perceptron were still in use. Using the Perceptron algorithm (Perception approach), the coefficients of discriminant function can be obtained by "learning" the training mode sample set.       Because the algorithm does not need to make any assumptions about the statistical nature of the patterns in each category, it is called a deterministic method. Perceptron algorithm is essentially a reward and punishment process: – The correct classification of the pattern is "rewards", in fact, is "no penalty", that is, the right vector unchanged. – The pattern of the error classification is "penalty" so that W (k) plus a component proportional to the XK. – Once a round is trained with a full pattern sample, the next iteration is required if one of the patterns is a discriminant error, which is to be trained again with the full pattern sample. – This is repeated until all pattern samples are trained to get the correct classification results.

2. Perceptron Training algorithm

2.1 Two-class case algorithm:

It is known that two training mode sets belong to $\omega_1$ class and $\omega_2$ class respectively, and the initial value of weight vector is $w (1) $, which can be arbitrarily evaluated. If $x_k\in \omega_1,w^t (k) x_k>0$, if $x_k\in \omega_2,w^t (k) x_k\leq0$, when using the entire training mode set for iterative training, the first $k$ training steps are:

• If $x_k\in \omega_1$ and $w^t (k) X_k\leq 0$, then the classifier incorrectly classifies the $k$ of the $x_k$ mode, at which point the weight vector should be corrected so that the $w (k+1) = W (k) + cx_k$, where $c$ is an adjustment increment.
• If $x_k\in \omega_2$ and $w^t (k) X_k\geq 0$, the same classifier is classified incorrectly, then the weight vector should be corrected as follows: $w (k+1) = W (k)-cx_k$
  
• If the above conditions are not met, it indicates that the pattern sample is correctly classified in $k$, so the weight vector is unchanged, i.e.: $w (k+1) = W (k) $

If the pattern sample of the $x\in \omega_2$ is multiplied by $ (-1) $, then there are:

$w ^t (k) X_k\leq 0$, $w (k+1) = W (k) + cx_k$

At this point, the perceptron algorithm can be written uniformly:

\[w (k + 1) = \left\{{\begin{array}{*{20}{c}}
{W (k)}&{if\;{ W^t} (k) {X_k} > 0}\\
{W (k) + c{x_k}}&{if\;{ W^t} (k) {X_k} \le 0}
\end{array}} \right.\]

2.2 Multi-class case algorithm:

For M-class mode there is M discriminant function $\{d_i,i=1,2,\cdots,m\}$, if $x\in \omega_i$, then $d_i>d_j,\forall j\neq i$.

There is a M mode category $\omega_1,\omega_2,\cdots, \omega_m,$, if the $k$ iteration of the training process, a pattern sample belonging to the $\omega_i$ class $x$ fed into the classifier, the $m$ discriminant function should be calculated first:

\[d_j (k) =w_j (k) x,j=1,2,\cdots,m\]

If $d_i (k) >d_j (k), J=1,2,\cdots,m,\forall J\neq i$ condition is established, then the weight vector is unchanged, that is

\[w_j (k+1) =w_j (k), j=1,2,\cdots,m\]

If the L weight vector makes $d_i (k) \leq D_1 (k) $, then the corresponding weight vector should be adjusted, i.e.

\[\left\{{\begin{array}{*{20}{c}}
{{w_i} (k + 1) = {w_i} (k) + cx}\\
{{w_l} (k + 1) = {w_l} (k)-cx}\\
{{W_j} (k + 1) = {W_j} (k), \;j = \ldots, m,\;j \ne i,\;j \ne L}
\end{array}} \right.\]

where c is a normal number. The initial value of the weight vector is $w_i (1), I = 1,2,\cdots,m$ arbitrary selection.

3. Algorithmic discussion

– The classification algorithms here determine the coefficients of discriminant functions through pattern samples, but the judging performance of a classifier is ultimately tested by unknown samples that are not used for training. – In order for a classifier to be well designed, representative training data must be used, which can reasonably reflect the overall pattern data. – How many training samples are needed to obtain a linear classifier with good discriminant performance? The more intuitively, the better, but the number of samples that can actually be collected will be subject to objective conditions; too many training samples in the training phase will make the computer need a long computing time; In general, the appropriate number of samples is estimated: If K is the dimension of the pattern, so c=2 (k+1), The usual number of training samples selected is 10~20 times c.

The training algorithm of perceptual device