Paper 102: Extreme Learning Machines (Extreme learning machine)

Original address: http://blog.csdn.net/google19890102/article/details/18222103

The Extreme learning Machine ELM is a neural network algorithm proposed by Huangguang. The biggest feature of Elm is that the traditional neural network, especially the tow-layer feedforward neural Network (SLFNS), Elm is faster than the traditional learning algorithm.

ELM is a new fast learning algorithm, for the TOW layer neural network, ELM can randomly initialize the input weights and offsets and get the corresponding output
Out weights. For a single-hidden-layer neural network, suppose there is an arbitrary sample, of which,. For a single hidden layer neural network with a hidden layer node, it can be expressed as

Where, for the activation function, the input weight, for the output weight, is the bias of the first hidden layer unit. The inner product of the expression and.

The goal of TOW layer neural network learning is to minimize the error of the output, which can be expressed as

That exists, and that makes

can be expressed as

The output of the hidden layer node is the output weight, which is the desired output.

In order to be able to train a single hidden layer neural network, we want to get and make

Where this is equivalent to minimizing the loss function

Some traditional algorithms based on gradient descent, such as the BP learning algorithm and its variants, can be used to solve such problems, but the basic gradient-based learning algorithm needs to adjust all parameters in the iterative process. In the ELM algorithm, the output matrix of the hidden layer is determined only if the input weight and the implicit layer's bias are randomly determined. The training single hidden layer neural network can be transformed into a linear system. And the output weights can be determined

Among them, is the moore-penrose generalized inverse of the matrix. And the norm of the solution can be proved to be the smallest and only.

Extreme Learning Machine code Download: Click to open the link

Paper 102: Extreme Learning Machines (Extreme learning machine)