Contrast learning using Keras to build common neural networks such as CNN RNN

Keras is a Theano and TensorFlow-compatible neural network Premium package that uses him to component a neural network more quickly, and several statements are done. and a wide range of compatibility allows Keras to run unhindered on Windows and MacOS or Linux.

Today to compare learning to use Keras to build the following common neural network:

1. Regression
2. RNN regression
3. Classification
4. CNN Categories
5. RNN classification
6. Self-coding classification

Their steps are almost the same:

1. [Import module and create data]
2. [Build model]
3. [Define Optimizer]
4. [Activate model]
5. [Training model]
6. [Test model]
7. [Visualize Results]

In order to compare learning, the data used is almost the same,
So this article only focuses on the 2. [Build model] above, the other steps are similar, you can refer to the teaching site mentioned in the watch or directly read the source code.

1. Regression

The goal is to fit a set of data.

1. Build model with Sequential
2. Add the nerve layer with model.add and add the dense fully connected nerve layer.

There are two parameters, one is the dimension of input data and output data, and X and Y are one dimension in the example of this code.

If you need to add the next layer of nerves, you don't have to define the latitude of the input, because it defaults to the output of the previous layer as input to the current layer. In this case, just one layer is enough.

# build a neural network from the 1st layerto the last layer model = sequential () model.add (Dense (output_dim=1, Input_dim=1))

2. RNN regression

We're going to use the SIN function to predict the COS data, and we'll use the LSTM network.

RNN vs LSTM

1. Build the model and still use sequential.
2. Then add the LSTM nerve layer.

• Batch_input_shape is the size of the batch training data in the back, the number of points in time, and the number of data at each point in time.
• Output_dim means that there are 20 unit LSTM inside.
• Return_sequences means that at each point in time, you want to export output, the default is False, and now we define it as true. If equal to False, only one value is output at the last point in time.
• Stateful, the default is also false, meaning whether there is a connection between the batch and the batch. The intuitive understanding is that we are reading 20 steps, and the 21st step starts with 20 steps ahead. That is, the last step in the first batch is associated with the first step in the second batch.

3. A different point is timedistributed.

In the previous regression problem, we added the dense layer directly, because only the last output layer turned it into a fully connected layer.
The problem today is that there is an output at every point in time, which requires dense to perform an all-connected calculation of each output.

Model =Sequential ()#Build a LSTM RNNModel.add (LSTM (Batch_input_shape= (Batch_size, time_steps, Input_size),#or:input_dim=input_size, Input_length=time_steps,output_dim=cell_size, Return_sequences=true,#True:output at all steps. False:output as last step.Stateful=true,#true:the final State of batch1 was feed into the initial state of BATCH2))#Add output LayerModel.add (timedistributed (Dense (output_size))) Adam=Adam (LR) model.compile (Optimizer=Adam, loss.='MSE',)

3. Classification

The data is Keras with MNIST, which is divided into training sets and test sets. X is a picture, Y is the corresponding label for each picture, which is the number.

Simply introduce the relevant modules:

• Models. Sequential, used to build a layer of layers of nerve layer;
• Layers. Dense means that the nerve layer is a fully connected layer.
• Layers. Activation activates the function.
• Optimizers. The Rmsprop optimizer uses Rmsprop to accelerate neural network training methods.

Import NumPy as Npnp.random.seed (1337)  # For reproducibility  from Import mnist  from Import np_utils  from Import Sequential  from Import Dense, Activation  from Import Rmsprop

The use of Model.add in the regression network is to add a layer of nerve layer, and today's method is directly in the model with a plurality of nerve layers. Like a pipe, a paragraph, the data is from the above paragraph fell to the following paragraph, and then fell to the following paragraph.

• The first paragraph is to join the dense nerve layer. 32 is the dimension of the output, and 784 is the dimension of the input.
  The first layer of outgoing data has 32 feature, which is passed to the activation unit.
• The activation function uses the Relu function.
  After the activation function, it becomes non-linear data.
• This data is then passed on to the next neural layer, which dense we define it to have 10 output feature. Again, there is no need to define the input dimension, because it receives the output from the previous layer.
• Next, enter the following Softmax function to classify.

# another to build your neural netmodel = sequential ([    dense (+ input_dim=784),    Activation ('relu'),    dense(ten),    Activation ('  softmax'),])

4. CNN category CNN

The data is still using Mnist.

1. Set up the first layer of the network and set up a convolution2d with the number of filter parameters.

• Filter is the filter, with 32 filters to scan the same picture, each filter will summarize a feature. Each filter generates an entire picture, with 32 filters generating 32 images representing different characteristics,
• Nb_row Nb_col represents how many rows the filter has.
• Border_mode represents how this filter is filtered, and here we use same.
  Because it is the first layer, you need to define the dimension of the input data, 1, 28, 28 is the dimension of the picture picture.
  After the filter is complete, 32 layers of data are generated, but the length and width of the picture are constant and still 28x28.
• Then add a Relu activation function.

#another to build your CNNModel =Sequential ()#Conv Layer 1 output shape (+)Model.add (convolution2d (Nb_filter=32, Nb_row=5, Nb_col=5, Border_mode='same',#Padding Methoddim_ordering='th',#if use TensorFlow, to set the input dimension order to Theano ("th") style, but can change it.Input_shape= (1,#Channels28, 28,)#Height & Width)) Model.add (Activation ('Relu'))

2. Pooling is a downward sampling process.
It can narrow down the resulting length and width, and the height does not need to be compressed.

• Pool_size is a picture of how wide to consider when sampling down.
• Strides step, is to take a sample after a few steps to take a sampling, and then skip a few steps to sample.

# Pooling Layer 1 (max Pooling) output shape ( +, +) Model.add (maxpooling2d (    pool_size= (2, 2),    strides= (2, 2),    border_mode= ' same ',    #  Padding method))

3. Next, create a second neural layer

• There are 64 filter,5, 5 long widths, followed by an activation function.
• followed by a maxpooling2d sample.

# Conv Layer 2 output shape (model.add, 5, 5, border_mode='same (convolution2d) ')) Model.add (Activation ('relu'))#  Pooling Layer 2 (max pooling) output shape (7, 7)Model.add (Maxpooling2d (pool_size= (2, 2), border_mode=' same '))

4. Next, enter the full-join layer

• The three-dimensional layers that are rolled out with Flatten are flattened into two dimensions.
• Next, add a dense full-join layer, so that you can connect this one point to a single layer.
• Then add an activation function.

# Fully connected Layer 1 Input shape (7 * 7) = (3136), Output shape (1024x768) Model.add (Flatten ()) Model.add (Dense (1024x768)) Model.add (Activation ('relu') ))

5. In the second fully connected layer, output 10 unit, using Softmax as the classification.

# Fully connected Layer 2 to shape (Ten) for classesmodel.add (dense) Model.add (Activation(' 
    softmax'))

5. RNN Classification RNN classification

RNN is a serialized neural network, and when we process the image data, we need to consider it in a serialized way.
The picture is made up of a row of pixels, and we are going to read the data in a row and column. Finally, a summary is made to determine what kind of discrimination it is.

The meaning of the parameters used:

• Time_steps is how many points of time you want to read, and if you read a line 28 times.
• Input_size How many pixels each row reads.
• How many batch_size each batch of training.
• Batch_index is used to generate data.
• Output_size the length of the result of the classification, 0 to 9, so the length is 10.
• How many unit will be put in the hidden layer in the Cell_size network.
  LR is the learning rate.

1. Using sequential to build a model is to add a layer of nerve layer.

# build RNN modelmodel = sequential ()

2. Add Simplernn.
Batch_input_shape is the size of the batch training data in the back, the number of points in time, and the number of pixels at each point in time.

# RNN Cell  Model.add (simplernn (    #  for Batch_input_shape, if using TensorFlow as the backend, we had to put None for the batch_size.    # Otherwise, Model.evaluate () would get error.    Batch_input_shape= (None, Time_steps, input_size),       #  or:input_dim=input_size, input_length= Time_steps,    output_dim=cell_size,    unroll=True,))

3. Add dense output layer.
Output outputs are 10 in length, followed by Softmax activation functions for classification.

# Output Layer Model.add (Dense (output_size)) Model.add (Activation ('softmax'))

4. There is a small skill in training, that is, how to deal with bulk.
Output the accuracy and loss of the test set every 500 steps.

Need to use the Batch_index, a batch of land interception data, the next batch of time, the batch_index need to accumulate, the back of the time and the step size is no change is 28.
Y's batch and x processing is the same, except that Y has only two dimensions, so it has only two parameters.

There is a judgment statement, if the index is greater than the total number of training data, index becomes 0, and then a batch is started from scratch.

#Training forStepinchRange (4001):    #Data shape = (batch_num, steps, inputs/outputs)X_batch = x_train[batch_index:batch_index+batch_size,:,:] Y_batch= y_train[batch_index:batch_index+batch_size,:] cost=Model.train_on_batch (X_batch, Y_batch) Batch_index+=batch_size Batch_index= 0ifBatch_index >= X_train.shape[0]ElseBatch_indexifStep% 500 = =0:cost, Accuracy= Model.evaluate (X_test, Y_test, batch_size=y_test.shape[0], verbose=False)Print('Test Cost:', Cost,'Test accuracy:', accuracy)

6. Self-coded classification self-coding

Self-coding, in a nutshell, is the process of compressing and decompressing input data.
There are many Feature, compressed into a few to represent the original data, decompression and restore to the original dimension, and then compared with the original data.

The thing to do is to compress the data datasets.mnist of 28x28=784 dimension into 2-dimensional data, then visualize the effect of classification in a two-dimensional space.

Model structure:

encoding_dim, the dimension to compress into.

# In order-to- plot in a-2Dfigure Encoding_dim = 2# That's our input placeholderinput_img = Inpu T (shape= (784,))

Build encoded layers and decoded layers, and then build autoencoder them together. Use layers during training autoencoder .

1. encoded use a 4-layer Dense fully connected layer
In the activation function relu , the dimension entered is defined in the previous step input_img .
Next, define the next layer, its output dimension is 64, the input is the output of the previous layer.
In the final layer, we define its output dimension as desired encoding_dim＝2 .

2. Unzip the link, its process and the compression process is just the opposite.
The activation function of the corresponding layer is the same, but the activation function used in the last layer of decompression is tanh . Because the input value is from 0.5 to 0.5 this range, at the last layer with this activation function, its output is-1 to 1, can be as a good correspondence.

#Encoder Layersencoded = dense (+, activation='Relu') (INPUT_IMG) encoded= Dense (activation='Relu') (encoded) encoded= Dense (Ten, activation='Relu') (encoded) Encoder_output=Dense (Encoding_dim) (encoded)#Decoder Layersdecoded = dense (ten, activation='Relu') (encoder_output) decoded= Dense (activation='Relu') (decoded) decoded= Dense (+, activation='Relu') (decoded) decoded= Dense (784, activation='Tanh') (decoded)#construct the Autoencoder modelAutoencoder = Model (input=input_img, output=decoded)

Next, use Model this module to build the model.
The input is the picture, and the output is the final result of the decompression.

# construct the encoder model for plottingencoder = Model (input=input_img, output=encoder_output)

When we want to see from 784 compression to 2 dimensions, this result is what kind of time, also can only build a compressed plate, at this time its input is a picture, output is the final result of the compression link.

Visualization results for the final classification:

Compare learning using Keras to build common neural networks such as CNN RNN