BP neural Network--the realization of C language

Reprint: http://www.cnblogs.com/jzhlin/archive/2012/07/30/bp_c.html

In the last article, we introduce the basic model of BP neural network, some terms in the model and the mathematical analysis of the model, and have a preliminary understanding of its principle. Then how to use the program language to specifically implement it, will be the next issue we need to discuss. This paper chooses the C language to realize a simple single hidden layer of BP neural network (by default, we understand the basic concept of BP neural network, there are some terms in this article that refer to the previous basic model, so for other C-class languages (C #, Java, and so on) you can simply modify the code in this article to migrate.

Some definitions of data

First, we introduce the definition of some important data in the program described below.

#define DATA  820
#define in 2
#define OUT 1
#define Neuron
#define TRAINC 5500

Data is used to indicate the number of samples that have been known, that is, the number of training samples. In indicates how many input variables are available for each sample; Out indicates how many output variables are available for each sample. Neuron represents the number of neurons, trainc to indicate the number of training. Let's look at the definition of the data in the neural network description, and look at the data types in this picture are double.

Figure 1

D_in[data][in] Stores Data samples, in input for each sample. D_out[data][out] Stores Data samples, out output of each sample. We use the Adjacency table method to represent the network in Figure 1, W[neuron][in] represents the weight of an input on a neuron, v[out][neuron] to indicate the weight of a neuron on an output, and the corresponding array of two modified quantities dw[neuron][in] and Dv[out][neuron]. Array O[neuron] records the output of a neuron through the activation function, Outputdata[out] to store the outputs of the BP neural network.

the process of executing a program

Here, do not consider the specific function of the implementation details, from the general introduction of the program execution process. In Pseudocode, the details are described in the following steps, as follows:

The main function main{

reads the sample data ReadData (),
       initializes the BP neural network  initbpnework () {

            including data normalization, neuron initialization w[neuron][in], v[out][ Neuron],
        BP neural Network Training Trainnetwork () {
            do{
                for (i less than sample size Data) {
                    
calculates the output of BP neural network in accordance with the input of the first sample, C Omputo (i);
                    accrual error accuracy;
                    feedback regulation of neurons in BP neural network to complete the Learning backupdate (i) of the first sample (i);
                }
            }while (the number of training times or the accuracy of error);

        storage of trained neuron information  writeneuron ();
        use some data to test the results of trained BP neural networks; return
        0;
}

The above is the process of processing, for reading data, saving data and so on processing this article will omit this content, highlighting the backbone part.

initialization of BP neural network

Initialization is mainly involved in two aspects of the function, on the one hand, to read the training sample data normalization process, normalized processing is to refer to the conversion of data into 0~1. In the theory of BP neural network, there is no requirement for this, but normalization is indispensable in the practical process. Because the theoretical model does not take into account the rate of convergence of BP neural networks, the output of neurons is generally very sensitive to the data between 0~1, and normalization can significantly improve the training efficiency. It can be normalized with the following formula, where a constant A is added to prevent 0 from occurring (0 cannot be the denominator).

y= (x-minvalue+a)/(Maxvalue-minvalue+a)

On the other hand, the weights of the neurons are initialized, the data is one to the (0~1), then the weights are initialized to the data between ( -1~1), and the correction value is assigned to 0. The implementation reference code is as follows:

void Initbpnework () {int i,j;
        /* Find data Minimum, maximum/for (i=0 i<in; i++) {minin[i]=maxin[i]=d_in[0][i]; For (j=0 j<data; j + +) {Maxin[i]=maxin[i]>d_in[j][i]?
            Maxin[i]:d _in[j][i]; Minin[i]=minin[i]<d_in[j][i]?
        Minin[i]:d _in[j][i];
        for (i=0; i<out; i++) {minout[i]=maxout[i]=d_out[0][i]; For (j=0 j<data; j + +) {Maxout[i]=maxout[i]>d_out[j][i]?
            Maxout[i]:d _out[j][i]; Minout[i]=minout[i]<d_out[j][i]?
        Minout[i]:d _out[j][i]; }/* Normalized processing/for (i = 0; i < i++) for (j = 0; J < Data; J +) D_in[j][i
    ]= (d_in[j][i]-minin[i]+1)/(maxin[i]-minin[i]+1); for (i = 0; i < out; i++) for (j = 0; J < Data; J +) d_out[j][i]= (d_out[j][i]-minout[i]+1)/(Maxou
    T[I]-MINOUT[I]+1);    
  /* Initialize Neuron * * for (i = 0; i < Neuron ++i) for (j = 0; J < in; ++j) {          W[i][j]= (rand () *2.0/rand_max-1)/2;
        dw[i][j]=0; for (i = 0; i < Neuron. ++i) for (j = 0; J < out; ++j) {v[j][i]= (rand () *2.0/rand_max
              -1)/2;
         dv[j][i]=0; }
}

  BP neural Network training

This part should be said to be the engine of the whole BP neural network, which drives the execution of the sample training process. According to the basic model of BP neural Network, the feedback learning mechanism consists of two parts, one is the result of BP neural network, the other is the comparison between the predicted results and the exact results of the sample, and then the error amount of the neuron is corrected. Therefore, we use two functions to represent such two processes, the training process is also the average error e monitoring, if the set accuracy can be completed training. Since it is not always possible to reach the desired precision requirements, we add a number of training parameters, if the number of times to also exit the training. The implementation reference code is as follows:

void  trainnetwork () {
    int i,c=0;
    do{
        e=0;
        for (i = 0; i < Data; ++i) {
            Computo (i);
            E+=fabs ((outputdata[0]-d_out[i][0])/d_out[i][0]);
            Backupdate (i);
        }
        printf ("%d  %lf\n", c,e/data);
        C + +;
    } while (C<trainc && e/data>0.01);
}

One of the functions, Computo (i) (O is output abbreviation) calculates the output of the BP neural network to predict the first sample, which is also the initial process. Backupdate (i) is an update of the weights of the neural network based on the predicted output of the first sample, and E is used to monitor the error.

  Here, our overall review, the skeleton of the BP Neural Network program has been introduced, the core of the training process of two functions Computo (i) and backupdate (i) to the next analysis, good night.