RNN model of deep learning--keras training

RNN model of deep learning--keras training

RNN principle: (Recurrent neural Networks) cyclic neural network. It interacts with each neuron in the hidden layer and is able to handle the problems associated with the input and back. In RNN, the output from the previous moment is passed along with the input of the next moment, which is equivalent to a stream of data over time. Unlike Feedforward neural networks, RNN can receive serialized data as input, or it can return serialized values as output to model changes in time series. Because the time order of sample appears is very important for natural language processing, speech recognition, handwriting recognition and so on, the RNN model is widely recognized in this field.

Generate text with RNN training language model

Detailed Training code:

Thinking Arrangement: We use RNN to classify the datasets.mnist data, the resolution of the image 28x28 is understood as one (information row x time node) sequence data, the Rnn_cell classification training is established, and the error and precision are calculated. To import the relevant module (modules):

Import NumPy as NP
np.random.seed (1337)  # for reproducibility from

keras.datasets import mnist
from Keras.utils import np_utils from
keras.models import sequential from
keras.layers import Simplernn, Activation, Dense from
keras.optimizers import Adam

Initialization of parameters:

Time_steps = Same as the height of the     image
Input_size = same as the     width of the image
batch_s IZE =
Batch_index = 0
output_size =
cell_size = +
LR = 0.001

Output_size: The length of the result of the data collation of the output, expressed as a 10-D sequence of 0-1, for example (0,0,0,1,0,0,0,0,0,0) represents 4. To load a data set:

# Download the mnist to the path ' ~/.keras/datasets/' if it's the first time to be called
# X shape (60,000 28x28), Y Shape (
x_train, Y_train), (x_test, y_test) = Mnist.load_data ()

Data preprocessing (normalization):

# data pre-processing
X_train = X_train.reshape ( -1, +)/255.      # normalize
x_test = X_test.reshape ( -1, +)/255.        # normalize
Y_train = np_utils.to_categorical (Y_train, num_classes=10)
y_test = np_utils.to_categorical (y_ Test, num_classes=10)

The training data is normalized because the original data is a 8bit grayscale image and needs to be divided by 255. Build the RNN Model (serialization):

# build RNN model
model = sequential ()

To add a RNN layer:

The input is the training data and the output data size is defined by cell_size.

# RNN Cell
model.add (SIMPLERNN (
    # for Batch_input_shape, if using TensorFlow as the backend, we had to put None F or the batch_size.
    # Otherwise, Model.evaluate () would get error.
    Batch_input_shape= (None, Time_steps, input_size),       # or:input_dim=input_size, Input_length=time_steps,
    Output_dim=cell_size,
    unroll=true,
))

To add an output layer:

# Output Layer
model.add (dense (output_size))
Model.add (Activation (' Softmax '))

Set optimizer, loss function, and metrics method

# optimizer
adam = Adam (LR)
model.compile (Optimizer=adam,
              loss= ' categorical_crossentropy ',
              metrics=[' accuracy ')

Start training:

# Training
for step in range (4001):
    # Data shape = (batch_num, steps, inputs/outputs)
    X_batch = X_train[batch_ Index:batch_index+batch_size,:,:]
    y_batch = y_train[batch_index:batch_index+batch_size,:] Cost
    = Model.train_on_batch (X_batch, Y_batch)
    Batch_index + = batch_size
    batch_index = 0 if Batch_index >= X_ Train.shape[0] Else Batch_index

Error and accuracy of correct classification of output:

If step% = = 0: Cost
        , accuracy = model.evaluate (X_test, Y_test, batch_size=y_test.shape[0], verbose=false)
        PR Int (' Test cost: ', cost, ' Test accuracy: ', accuracy)

Training operation:

Refer to the previous Autoencoder model training method, run the 7-rnn_classifier_example.py, run the result is: