RNN lstm Cyclic neural Network (classification example)

Learning materials: Related code for TF 2017 built new visual instructional Code machine learning-Introduction series what is RNN machine learning-Introduction series What is Lstm RNN this code sets RNN parameters based on this code on the Web

This time we will use RNN to classify the training (classification). will continue to use the Mnist data set to the handwritten digits. Let RNN read the last line of pixels from the first row of each picture and then make a classification judgment. Next we import the Mnist data and determine the various parameters of the RNN (hyper-parameters):

Import TensorFlow as TF from
tensorflow.examples.tutorials.mnist import input_data
tf.set_random_seed (1)   # set Random seed

# import data
mnist = input_data.read_data_sets (' Mnist_data ', one_hot=true)

# Hyperparameters
lr = 0.001                  # learning rate
training_iters = 100000     # train step upper limit
batch_size = 128            
n_inputs =               # mnist data input (img shape:28*28)
n_steps = # time                steps
n_hidden_units = 1        # Neurons in hidden layer
n_classes = Ten              # mnist classes (0-9 digits)

Then we define the placeholder and weights of x, Y, and the initial state of biases.

# x y placeholder
x = Tf.placeholder (Tf.float32, [None, N_steps, n_inputs])
y = Tf.placeholder (Tf.float32, [None, N_classes])

# The definition of weights biases initial value
weights = {
    # shape (128) ' in
    ': TF. Variable (Tf.random_normal ([N_inputs, N_hidden_units]),
    # shape (128, ten)
    ' out ': TF. Variable ([N_hidden_units, n_classes])
}
biases = {
    # shape (128,)
    ' in ': TF. Variable (Tf.constant (0.1, Shape=[n_hidden_units,]),
    # shape (ten,)
    ' out ': TF. Variable (Tf.constant (0.1, shape=[n_classes,])
}

defining the body structure of a RNN

Then start defining the RNN body structure, which has a total of 3 components (Input_layer, cell, Output_layer). First we define Input_layer:

def RNN (x, Weights, biases):
    # The original X is 3-D data, we need to turn it into 2-dimensional data to use the weights matrix multiplication
    # X ==> (128 batches * steps, Inputs)
    x = Tf.reshape (x, [-1, N_inputs])

    # x_in = w*x + b
    x_in = Tf.matmul (x, weights[' in ']) + biases[' in ']
    # x_in ==> (128 batches, steps, 128 hidden) change back to 3 d
    x_in = Tf.reshape (x_in, [-1, N_steps, N_hidden_units])

Then there are two ways of computing in the cell: using TF.NN.RNN (cell, inputs) (not recommended). But if you use this method, you can refer to the reason; Use TF.NN.DYNAMIC_RNN (cell, inputs) (recommended). This practice will be used in this way.

Because of the TensorFlow version upgrade, State_is_tuple=true will become default in later versions. For lstm, state can be divided into (c_state, h_state).

    # Use Basic lstm Cell.
    Lstm_cell = Tf.contrib.rnn.BasicLSTMCell (N_hidden_units, forget_bias=1.0, state_is_tuple=true)
    init_state = lstm _cell.zero_state (Batch_size, Dtype=tf.float32) # Initialize all 0 state

If you use TF.NN.DYNAMIC_RNN (cell, inputs), we want to determine the inputs format. The time_major parameters in Tf.nn.dynamic_rnn have different values for different inputs formats. If inputs is (batches, steps, inputs) ==> time_major=false; If inputs is (steps, batches, inputs) ==> time_major=true;

    Outputs, final_state = Tf.nn.dynamic_rnn (Lstm_cell, x_in, Initial_state=init_state, Time_major=false)

The last is the value of Output_layer and return. Because of the particularity of this example, there are two ways to obtain results.

mode one:   direct call  h_state  (Final_state[1]

in final_state