Learning Vector Quantization Neural network

On the basis of the competitive network structure, the learning vectorization (learning vector quantization,lvq) network is put forward to combine the characteristics of competitive learning and supervised learning algorithms, and the allocation categories of input samples are regulated by teacher signals. Thus overcoming the weakness of the lack of classified information in self-organizing networks using unsupervised learning algorithms.

1. Vector quantization

The idea of vector quantization is that the high-dimensional input space is divided into several different regions, and the center vector of each region is determined as the centre of the cluster, and the input vectors in the same region can be represented by the center vector, thus forming a set of points with each center vector as the center of the cluster. In the field of image processing, the coding of various regional centers (vectors) is used instead of the points in the region to store or transmit, and a variety of lossy compression techniques based on vector quantization are proposed.

The center vector distribution represented on the two-dimensional input plane is called the Voronoi graph, as shown in the first two blog posts introduced by the winner of the King's Learning rules and SOFM competitive learning algorithm is a vector quantization algorithm, can use a small number of clustering center to represent the original data, from the data compression. But the vectors of SOFM's cluster centers have some similar characteristics, but the center of general Vector quantization does not have this similarity.

Self-organizing mappings can play a clustering role, but cannot be categorized or identified directly, so it is only the first step in a two-step approach to adaptive problem-solving pattern classification. And let me take the second step: learning vector quantization, using the supervision mechanism, in the training to add the teacher signal as the classification information to fine tune the weight, and the output neuron pre-designated its category.

2. LVQ Network structure and working principle

The structure looks like this:

The competition layer has m neurons, the input layer has n neurons, and the two layers are fully connected to each other. Each neuron in the output layer is connected to only one group of neurons in the competition layer, the connection weights are fixed to 1, and the weights between the input layer and the competition layer are gradually adjusted to the cluster center during training. When a sample is fed into the LVQ network, the competing neurons generate the winning neurons by learning the rules of the winner, allowing their output to be 1, and the other neurons outputting 0. The output neuron that is connected to the winning neuron group outputs 1, while the other output neurons are 0, giving the pattern class of the current input sample. The classes learned from the competition layer become subclasses, and the classes learned from the output layer become the target classes.

3. LVQ Network Learning Algorithm

The learning rules of the LVQ network combine the rules of competitive learning with the rules of tutor learning, so the sample set should be {(XI,DI)}. Where di is L-dimensional, corresponding to the output layer of L neurons, it has only one component of 1, the other components are 0. Each neuron in the competition layer is usually assigned to an output neuron, and the corresponding weight is 1, thus obtaining the weight of the output layer. For example, a LVQ network competition layer 6 neurons, the output layer of 3 neurons, representing 3 classes. If the 1,3 of a competing layer is specified as the first output neuron, 2,5 is specified as the second output neuron, and 3,6 is specified as the third output neuron. The weight matrix of the competition layer to the output layer is:

Pre-defined competition layer to output layer weights before training, which specifies the output neuron category, and no longer changes in training. Network learning is done by changing the weight of the input layer to the competition layer. Depending on the input sample category and the category of the winning neuron, you can determine whether the current classification is correct. If the classification is correct, the weight vector of the winning neuron is adjusted to the input vector, and the classification error is adjusted in the opposite direction. As shown in the following:

The steps of the LVQ Network learning algorithm are as follows:

(1) Initialize. The weights of each neuron in the competition layer are randomly assigned small random numbers to determine the initial learning rate and the number of training times.

(2) Enter the sample vector.

(3) Finding the winning neurons.

(4) According to the classification is correct according to different rules to adjust the weight of the winning neurons: when the network classification results and the teacher signal consistent, the input sample direction to adjust the weight value:

When the network classification results are inconsistent with the teacher's signal, the weights are adjusted in the opposite direction to the input sample:

Other non-winning neuron weights remain unchanged.

(5) Update learning rate

(6) When the number of training times does not reach the set number of times, go to step (2) Enter the next sample, repeat the steps until the set number of training.

In the above training process, the η (t) is guaranteed to be a monotonically decreasing function.

Summary

LVQ Network is a kind of supervised extension of SOFM network, the two effective combination can better play the advantages of competitive learning and supervised learning.

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Learning Vector Quantization Neural network