Self-organizing feature map neural network (SOFM)

1981 T. of Helsink University, Finland Professor Kohonen proposed a self-organizing feature mapping network (self-organizing Feature map, SOFM), also known as the Kohonen network. Kohonen that when a neural network accepts external input patterns, it will be divided into different corresponding regions, each region has different response characteristics to the input mode, and the process is automatically completed. Self-organizing feature mapping is based on this view, and its characteristics are similar to the self-organizing characteristics of the human brain.

first, the biological basis of SOFM network

Biological studies have shown that in the sensory pathways of the brain, neurons are arranged in an orderly manner, with the input pattern approaching and the corresponding excitatory neurons similar. The corresponding characteristics of neurons in the cerebral cortex are not innate, but are formed by learning from the acquired form.

The specific excitation process of a graph or a certain frequency is the biological basis of the competition mechanism in the self-organizing feature Map network. The sequential arrangement of neurons and the continuous image of external information are also reflected in the self-organizing feature mapping network, where the neuron excitement in the network is random at the beginning of training when different samples are entered. But the self-organizing training will form the orderly arrangement of neurons in the competition layer, the neurons with similar functions are very close, and the neurons with different functions are farther away. This characteristic is very similar to the tissue principle of human brain neurons.

the topological structure of SOFM network and the adjustment domain of weight value1. Topological structure

The SOFM net has two layers, and the input layer neuron collects the external information into the output layer's neurons through the weight vector. The number of neurons in the input layer is equal to the number of sample dimensions. The output layer is the competitive layer in the competitive network. There are many forms of neuron arrangement, such as dimensional array, two-dimensional planar array and three-dimensional grid array, which are common in one and two dimensions. One-dimensional is the simplest, and the structure features as shown, each competing layer has a lateral connection between the neurons. Output according to two-dimensional planar organization is the most typical organization of SOFM network, more with the image of the cerebral cortex, each neuron of the output layer is connected to the other neurons around it, arranged into a checkerboard plane, the structure is as shown on the right:

2. Weight Adjustment field

The learning algorithm used in SOFM network is called Kohonen algorithm, which is improved on the basis of the winner's algorithm, and the main difference is that the method of adjusting weight vector is different from the side suppression. In the winner-King Learning rule, only the winning neuron can adjust the weight vector, and no other neuron has the right to adjust the weight vector, so its inhibition of all surrounding neurons is "blocked". The winning neurons of the SOFM net not only win the neurons themselves, but also adjust the weight vectors to varying degrees under the influence of the neuron in the learning algorithm. This adjustment can be represented by the three functions shown, where the function curve in (b) is composed of two normal curves in (a).

The 3 functions in the b-d can form a hat-like space surface after being rotated along the central axis, in order the Mexican Hat function, the Hat function and the chef hat function. Mexican Hat function is proposed by Kohonen, it shows that the winning neurons have the largest weight adjustment, the neighboring neurons have a slightly smaller adjustment, the greater the distance from the winning neurons, the lower the weight adjustment, until a certain distance r, the weight adjustment amount is 0. When the distance is further away, the weight adjustment amount is slightly negative, and further back to 0. The Mexican Hat function is very similar to the biological system, but the computational complexity affects the convergence of the network training. Therefore, a simplified function similar to the Mexican function is often used in the SOFM network, such as the bowler hat function and the further simplification of the chef's hat function.

Set a neighborhood radius centered on the winning neuron, which is called the winning neighborhood. In the SOFM Net Learning Network algorithm, all neurons in the winning neighborhood adjust weights according to their distance from the winning neurons in different degrees. The winning neighborhood starts very large, but its size shrinks as the number of sessions increases and eventually shrinks to a radius of zero.

operation principle and learning algorithm of self-organizing feature Map Network1. Operating principle

The operation of SOFM network is divided into two stages: training and working. At the training stage, a sample of the network's random input training concentration, a certain input mode, the output layer will have a neuron to produce the maximum response to win, and at the beginning of the training phase, the output layer where the neuron will be the type of input mode to produce maximum response is uncertain. When the class of the input pattern changes, the winning neurons in the two-dimensional plane also change. The neurons around the winning neuron have a greater response to the lateral excitation, so that the weights of the winning neurons and all the neurons in the winning neighborhood are adjusted to the direction of the input vectors, and the adjustment is gradually attenuated by the proximity of the neurons within the neighborhood to the winning neurons. Network through self-organization, with a large number of training samples to adjust the weight of the network, and finally make the output layer of neurons into a specific mode-sensitive neurons, the corresponding inner star weight vector becomes the central vector of each input mode class. And when the characteristics of the two pattern classes approach, the neurons that represent both classes are also close in position. Thus, an ordered feature map is formed in the output layer which can reflect the distribution of sample pattern classes.

After the training of SOFM nets, the specific relationship between each neuron of the output layer and each input mode class is completely determined, so it can be used as a pattern classifier. When a pattern is entered, the network output layer represents the specific neuron of the pattern class that produces the maximum response, which automatically classifies the input. It should be noted that when the mode of input to the network is not part of any pattern class that is seen in network training, the SOFM net can only classify it into the closest mode class.

2. Learning Algorithms

Kohonen algorithm

(1) Initialize

The weight vector of the output layer is assigned a small random number, and normalized processing. The initial winning neighborhood is established, and the learning rate is assigned the initial value.

(2) Accept input

An input pattern is randomly selected from the training set and normalized.

(3) Finding the winning neurons

Calculates the dot product of the input mode and the inner star weight vector, from which the winning neurons with the largest dot product are selected;

(4) Defining the winning neighborhood

At the center of the winning neuron, the weight adjustment domain of T time is determined, the general initial neighborhood n is larger, and N is gradually shrinking with the training time during training. As shown below:

(5) Adjustment weight value

Adjusts the weights for all neurons in the winning neighborhood. Such as:

,η (t,N) is a function of the topological distance N between the first J neurons and the winning neuron j* in the training time t and neighborhood, and the function generally has the following rules:

T↑→η↓,N↑→η↓
Many functions can satisfy the above rules, for example, the following functions can be constructed:
η (t) can adopt the monotone descent function of T, as shown, this function of monotonically decreasing with time is called annealing function.
< Span style= "font-family:symbol01" >
< Span style= "font-family:symbol01" > (6) End check SOFM Network Training does not exist similar to the BP network output error concept, because non-supervised learning, training when the end of the learning rate η (t) is attenuated to 0 or a predetermined positive decimal condition, does not meet the end condition back to step (2).
the complete process is as follows:

3. Functional Analysis

(1) function One: order-preserving mapping

One of the features of the SOFM network is the order-preserving mapping, which can map the sample pattern classes of the input space in an orderly manner to the output layer.

The following examples illustrate:

The different animals are mapped to the two-D output plane according to their attribute characteristics, so that similar animals ' position on the output plane of the SOFM network is similar. The training set 16 kinds of animals, each animal with 29-dimensional vector, of which the first 16 components constitute a symbol vector, the different animals 16 take 1 code, the latter 13 components constitute a property vector, describing the animal's 13 properties, with 1 or 0 to indicate the property of an animal or not. As shown in the following table:

SOFM net output plane has 10x10 neurons, with 16 animal mode turns input for training, the final output plane appears as shown in the case, you can see the properties of similar animals in the output plane position adjacent to achieve the characteristics of the order distribution.

(2) function Two: data compression

The second function characteristic of SOFM network is data compression.

The data compression is to map the samples of high latitude space to the low latitude space under the condition that the topological structure remains unchanged. In this respect, the SOFM network has obvious advantages. Regardless of how many dimensions the input sample space is, its pattern sample can be responded to in an area of the output layer of the SOFM network. After the SOFM network is trained, the input samples in the high dimensional space are close to the location of the response neurons of the output layer. Therefore, the data compression can be done by mapping to a one-dimensional or two-dimensional output layer of the SOFM network for any sample of n-dimensional input space. As shown in the example above, the input sample space is 29 dimensions, which is compressed into two-dimensional planar data after the SOFM network.

(3) function Three: feature extraction

The third function of SOFM network is feature extraction.

From the point of view of feature extraction, the mapping of high latitude spatial samples to low dimensional space is the equivalent of low dimensional feature space in the output layer of SOFM network. In the high-dimensional mode space, many patterns have complex structures, and it is difficult to find their intrinsic laws from data observation. When the SOFM network is mapped to the low-dimensional output space, its regularity is often at a glance, so this mapping is a feature extraction. The vector of high dimensional space can be expressed more clearly in the low dimensional feature space after the feature extraction, so the meaning of the map is not only the data compression, but also a law discovery.

The following is an example of sorting by character:

Use 32 characters as an input sample of the SOFM net, including 26 English letters and 6 digits (1-6). Each character corresponds to a five-dimensional vector, and the corresponding relationship between each character and the 5 components of the corresponding vector x is shown in the following table:

It can be seen from the table that there are four components in each vector representing A-B-C-D-E, so it represents a class, and so on, the corresponding word can be descriptor in the tree structure diagram shown:

The SOFM network output array is a two-dimensional planar array consisting of 70 neurons, each of which is connected to a five-dimensional input pattern with a five-dimensional internal star weight vector. The training set represents each character's input vector x randomly into the network for training, after 10000-step training, the weight vectors tend to stabilize, at this time the network output can be calibrated, that is, according to the output array neuron and the training set of the known pattern vector corresponding relationship labeling. For example, when input vector B, the upper-left corner neuron of the output plane produces the strongest response in the entire array, and the neuron is labeled B. Of the 70 neurons in the output layer, 32 neurons were labeled, while the other 38 were unused neurons.

The result of the output after learning by self-organization. After the SOFM network has completed the training, for each input character, the output plane has a specific neuron is most sensitive to it, the input-output mapping relationship in the output feature plane is very clear. The position relation between each character of the output plane of SOFM network is quite similar to the tree structure, and the consistency of the structure characteristics is very obvious. The dot on the output plane "·" Represents a free-state neuron that does not excite any of the input samples.

*********************************

2015-8-14

Less art

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Self-organizing feature map neural network (SOFM)