R using convolutional neural networks to make predictions

Deep learning over the past few years, the feature extraction capability of convolutional neural Networks has made this algorithm fire again, in fact, many years ago, but because of the computational complexity of deep learning problems, has not been widely used.

As a general rule, the convolution layer is calculated in the following form:

where x represents the J feature in the current convolution layer, the first characteristic of the first layer; K represents the convolution nucleus between the J feature of the current layer and the I feature of the previous layer; M represents the set of features that require the previous layer of the convolution, and b represents the offset of the J convolution core in the current convolution layer. f is the activation function.

Weights and thresholds in convolutional layers are obtained by means of a random gradient descent method:

In the formula, A is the learning rate.

The gradient of the loss function to the convolution layer parameter can be obtained by chain derivation, as follows:

Represents the gradient of the previous layer.

There are several forms of activation functions in convolutional neural networks:

A is a fixed parameter in the formula.

In the formula, each batch training sample is randomly sampled from the distribution of the mean value, which is taken in the test.

From the above convolution neural network, we can see that gradient iteration is needed in the learning process, and the time complexity is very high in practical applications such as industrial detection, so the academic field is optimized and a single-layer neural network limit learning machine is optimized to solve this problem, which has been widely used in the past.

To solve the above problems, the ultimate learning machine has emerged.

A special result solved by using least squares method, which is equivalent to a form of matrix inversion

For the moore-penrose generalized inverse.

1) because the limit learning machine is to calculate a generalized inverse, the training speed is much faster than the gradient-based learning algorithm.

2) There are many problems in gradient-based learning algorithms, such as difficult to determine the learning rate, local network minimization, and so on, the limit learning machine effectively improved this kind of problem, in the classification process has achieved better results;

3) Unlike other neural network algorithms, the Limit learning machine can select the non-micro function during the course of the activation function. ；

4) Limit Learning machine algorithm the training process is not complicated. The limit learning machine only needs three steps to complete the learning process.

Use the R code to explain the limit learning machine

# # #训练过程如下:

The training process can be 4 steps.

Elmtrain.default <-
function (X,y,nhid,actfun,...) {
Require (MASS)

if (Nhid < 1) Stop ("Error:number of hidden neurons must be >= 1")
####### #1. Select data, x and Y
T <-t (y)
P <-t (x)
####### #2. Randomly generated weights to change the X value

Inpweight <-Randommatrix (Nrow (P), nhid,-1,1)
temph <-inpweight%*% P
Biashid <-runif (nhid,min=-1,max=1)
Biasmatrix <-Matrix (Rep (Biashid, Ncol (P)), Nrow=nhid, Ncol=ncol (p), Byrow = F)

temph = temph + Biasmatrix
####### #3. High-dimensional mapping of the changed X-values, most commonly the SIG function
if (Actfun = = "sig") H = 1/(1 + exp ( -1*temph))
else {
if (Actfun = = "Sin") H = sin (temph)
else {
if (Actfun = = "Radbas") H = exp ( -1* (temph^2))
else {
if (Actfun = = "Hardlim") H = Hardlim (temph)
else {
if (Actfun = = "Hardlims") H = Hardlims (temph)
else {
if (Actfun = = "Satlins") H = Satlins (temph)
else {
if (Actfun = = "Tansig") H = 2/(1+exp ( -2*temph))-1
else {
if (Actfun = = "Tribas") H = Tribas (temph)
else {
if (Actfun = = "Poslin") H = Poslin (temph)
else {
if (Actfun = = "Purelin") H = temph
else Stop (Paste ("ERROR:", Actfun, "is not a valid activation function.", sep= ""))
}
}
}
}
}
}
}
}
}

####### #4. Model coefficients are fitted, i.e. a in Y=ax
Outweight <-Ginv (t (H), tol = sqrt (. machine$double.eps))%*% T (t)
Y <-t (t (H)%*% outweight)
Model = List (inpweight=inpweight,biashid=biashid,outweight=outweight,actfun=actfun,nhid=nhid,predictions=t (Y))
Model$fitted.values <-T (Y)
Model$residuals <-Y-model$fitted.values
Model$call <-Match.call ()
Class (model) <-"Elmnn"
Model
}

Test process, the process of 4 steps can be.

Function (object, NewData = NULL, ...)
{
if (Is.null (NewData))
Predictions <-fitted (object)
else {
if (!is.null (Object$formula)) {
X <-Model.matrix (Object$formula, NewData)
}
else {
X <-NewData
}
 
####### #1. Get the parameters in the training model
Inpweight <-Object$inpweight
Biashid <-Object$biashid
Outweight <-Object$outweight
Actfun <-Object$actfun
Nhid <-Object$nhid
TV. P <-t (x)

####### #2. Change the x value by parameter

Tmphtest = Inpweight%*% TV. P
Biasmatrixte <-Matrix (Rep (Biashid, Ncol (TV). P)), Nrow = Nhid,
Ncol = Ncol (TV. P), Byrow = F)
Tmphtest = tmphtest + biasmatrixte

####### #3. High dimensional mapping, usually select SIG function
if (Actfun = = "sig")
Htest = 1/(1 + exp ( -1 * tmphtest))
else {
if (Actfun = = "Sin")
Htest = sin (tmphtest)
else {
if (Actfun = = "Radbas")
Htest = exp ( -1 * (tmphtest^2))
else {
if (Actfun = = "Hardlim")
Htest = Hardlim (tmphtest)
else {
if (Actfun = = "Hardlims")
Htest = Hardlims (tmphtest)
else {
if (Actfun = = "Satlins")
Htest = Satlins (tmphtest)
else {
if (Actfun = = "Tansig")
Htest = 2/(1 + exp ( -2 * tmphtest))-
1