Keras Introduction (i) Build deep Neural Network (DNN) to solve multi-classification problem

Keras Introduction

?? Keras is an open-source, high-level neural network API written by pure Python that can be based on TensorFlow, Theano, Mxnet, and CNTK. Keras is born to support rapid experimentation and can quickly turn your idea into a result. The Python version for Keras is: Python 2.7-3.6.
?? Keras, a Greek-like "horn" (horn), was first released in March 2015 and can run on Windows, Linux, Mac and other systems. So, since there is tensorflow (or Theano, MXNet, CNTK), why do you need Keras? This is because, although we can use TensorFlow and so on to create a deep neural network system, but TensorFlow, such as the use of relatively low-level abstraction, writing TensorFlow code directly has a certain challenge, and keras on the basis of TensorFlow, The more easily used abstraction layer is added, which is simpler and more efficient to use.
?? What kind of occasion is suitable for using Keras? If you have the following requirements, please select Keras:

• Simple and fast prototyping (Keras with highly modular, minimalist, and expandable features)
• Support for CNN and RNN, or the combination of both
• Seamless CPU and GPU switching

?? If you want to use Keras on your computer, you need the following tools:

• Python
• TensorFlow
• Keras

Here we choose TensorFlow as the back-end tool for Keras. Use the following Python code to output the version numbers of Python, TensorFlow, and Keras:

import sysimport keras as Kimport tensorflow as tfpy_ver = sys.versionk_ver = K.__version__tf_ver = tf.__version__print("Using Python version " + str(py_ver))print("Using Keras version " + str(k_ver))print("Using TensorFlow version " + str(tf_ver))

On the author's computer, the output results are as follows:

Using TensorFlow backend.Using Python version 3.5.1 (v3.5.1:37a07cee5969, Dec  6 2015, 01:54:25) [MSC v.1900 64 bit (AMD64)]Using Keras version 2.1.5Using TensorFlow version 1.6.0

?? Below, I will use the iris dataset (Iris DataSet, a classic machine learning dataset, suitable as test data for multi-classification problems), and use Keras to build a deep neural network (DNN) to solve the multi-classification problem of the iris dataset as the first example of the Keras primer.

Iris Data Set Introduction

?? Iris DataSet (Iris data Set) is a classic machine learning data set that is suitable as a test data for multi-classification problems: http://archive.ics.uci.edu/ml/machine-learning-databases/iris/.
?? The iris DataSet is the data set used to classify the iris, with a total of 150 samples, each containing the length of the calyx (sepal length in cm), the width of the calyx (sepal width in cm), the length of the petals (petal length in cm), Petal width (petal width in cm) four features, the iris is divided into three categories, respectively, Iris Setosa,iris Versicolour,iris virginica, each class has 50 samples.
?? The iris DataSet is as follows (only part of the data is displayed, the order is scrambled):

Reading data sets

?? The author's iris DataSet is stored in CSV format, the author uses pandas to read the iris DataSet and 0-1 encode the target variable (one-hot Encoding), and finally divides the dataset into training set and test set, the ratio is 7:3. The complete Python code is as follows:

Import pandas as Pdfrom sklearn.model_selection import train_test_splitfrom sklearn.preprocessing import Labelbinarizer # Read the CSV dataset and split it into training set and test set # The function's incoming parameter is csv_file_path:csv file path def load_data (csv_file_path): IRIS = Pd.read_csv (csv_file_path     Target_var = ' class ' # Target Variable # DataSet feature features = List (Iris.columns) Features.remove (Target_var) # category of the target variable class = Iris[target_var].unique () # The category Dictionary of the target variable class_dict = dict (Zip (class, range (len))) # Adds a column of target, Encode the target variable iris[' target ' = iris[target_var].apply (Lambda x:class_dict[x]) # 0-1 encoding of the target variable (one-hot Encoding) lb =  Labelbinarizer () Lb.fit (list (Class_dict.values ())) Transformed_labels = Lb.transform (iris[' target ') y_bin_labels = [] # 0-1-encoded variable for I in range (Transformed_labels.shape[1]): y_bin_labels.append (' y ' + str (i)) IR is[' y ' + str (i)] = transformed_labels[:, i] # divides the dataset into training set and test set train_x, test_x, train_y, test_y = Train_test_split (IRIS [Features], iris[y_bin_labels], train_size=0.7, test_size=0.3, random_state=0) return Train_ X, test_x, train_y, test_y, class_dict

Build DNN

?? Next, I will show how to use Keras to build a simple deep neural network (DNN) to solve this multi-classification problem. The structure of the DNN we want to build is as follows:

The DNN we build consists of input layer, hidden layer, output layer and Softmax layer, in which the input layer consists of 4 neurons, corresponding to 4 features in the iris dataset, as input vectors, the hidden layer has two layers, each layer has 5 and 6 neurons, then the output layer, consisting of 3 neurons, The number of categories that correspond to the target variable of the iris dataset, and finally, a softmax layer, which is created to solve the multi-classification problem.
?? Corresponding to the above DNN structure, with Keras to build, its Python code is as follows:

   import keras as K    # 2. 定义模型    init = K.initializers.glorot_uniform(seed=1)    simple_adam = K.optimizers.Adam()    model = K.models.Sequential()    model.add(K.layers.Dense(units=5, input_dim=4, kernel_initializer=init, activation=‘relu‘))    model.add(K.layers.Dense(units=6, kernel_initializer=init, activation=‘relu‘))    model.add(K.layers.Dense(units=3, kernel_initializer=init, activation=‘softmax‘))    model.compile(loss=‘categorical_crossentropy‘, optimizer=simple_adam, metrics=[‘accuracy‘])

In this model, we select the Neuron activation function as the Relu function, the loss function is the cross entropy (crosses entropy), the iterative optimizer (optimizer) chooses Adam, and the connection weights (weights) and the emphasis (biases) of the initial layers are randomly generated. So we're going to say this DNN model definition is complete. So simple? Yes, that ' s it!

Training and forecasting

?? OK, after defining the model, we need to train, evaluate, and predict the model. For model training, we have 1 batches for each training session, 100 iterations, and the code is as follows (follow the above code):

    # 3. 训练模型    b_size = 1    max_epochs = 100    print("Starting training ")    h = model.fit(train_x, train_y, batch_size=b_size, epochs=max_epochs, shuffle=True, verbose=1)    print("Training finished \n")

?? In order to evaluate the model, the performance of the perceptual model needs to output the value of the loss function of the DNN model and the accuracy rate on the test set, the Python code is as follows (the above code):

    # 4. 评估模型    eval = model.evaluate(test_x, test_y, verbose=0)    print("Evaluation on test data: loss = %0.6f accuracy = %0.2f%% \n"           % (eval[0], eval[1] * 100) )

Training 100 times, the results of the output are as follows (the middle part of the training show is ignored):

Starting training Epoch 1/100 1/105 [...]-eta:17s-loss:0.3679-acc:1.0000 42/105 [= = = = = = = = = = = = = = = = =======&gt, ...........] -eta:0s-loss:1.8081-acc:0.3095 89/105 [========================>.....]-eta:0s-loss:1.5068-acc:0.42701 05/105 [==============================]-0s 3ms/step-loss:1.4164-acc:0.4667epoch 2/100 1/105 [......... ...] ..........] -eta:0s-loss:0.4766-acc:1.0000 45/105 [===========>........]-eta:0s-loss:1.0813-acc:0.4889 9  3/105 [=========================> ...]-eta:0s-loss:1.0335-acc:0.4839105/105 [==============================]- 0s 1ms/step-loss:1.0144-acc:0.4857......epoch 99/100 1/105 [...]. ...]-eta:0s-loss:0.00, ........??????? 13-acc:1.0000 43/105 [===========&gt, ........]-eta:0s-loss:0.0447-acc:0.9767 84/105 [===============] ========&gt, ...] -eta:0s-loss:0.0824-acc:0.9524105/105 [==============================]-0s 1ms/step-loss:0.0711-acc:0.9619epoch 100/100 1/105 [...]-ETA:0S-LOSS:2.3032-ACC, .................. : 0.0000e+00 51/105 [=============>.....]-eta:0s-loss:0.1122-acc:0.9608 99/105 [===============] ============&gt, ...] -eta:0s-loss:0.0755-acc:0.9798105/105 [==============================]-0s 1ms/step-loss:0.0756-acc:0.9810t  Raining finished Evaluation on test Data:loss = 0.094882 accuracy = 97.78%

It can be seen that after 100 training, the accuracy rate on the test set has reached 97.78%, the effect is quite good.
?? Finally, to predict the new data set, we assume that the 4 features of an Iris are 6.1,3.1,5.1,1.1, and we want to know which class the DNN model predicts, and the Python code is as follows:

   import numpy as np    # 5. 使用模型进行预测    np.set_printoptions(precision=4)    unknown = np.array([[6.1, 3.1, 5.1, 1.1]], dtype=np.float32)    predicted = model.predict(unknown)    print("Using model to predict species for features: ")    print(unknown)    print("\nPredicted softmax vector is: ")    print(predicted)    species_dict = {v:k for k,v in Class_dict.items()}    print("\nPredicted species is: ")    print(species_dict[np.argmax(predicted)])

The results of the output are as follows:

Using model to predict species for features: [[ 6.1  3.1  5.1  1.1]]Predicted softmax vector is: [[  2.0687e-07   9.7901e-01   2.0993e-02]]Predicted species is: versicolor

If we carefully compare the iris data set, we will find that the prediction results are quite satisfactory, this iris pattern of the predicted results, in human eyes, should also be versicolor.

Summarize

?? So far, I have finished this demo example, as the first step in Keras, this example is still possible. Looking back at the model, we first use pandas to read the iris data set, divide it into training set and test set, then build a simple DNN model with Keras, train and evaluate the model, and finally look at the prediction ability of the model on the new data set. From this, the reader is not difficult to appreciate the superiority of keras, because, compared to tensorflow, building the same DNN model and model training, evaluation, prediction, its Python code will undoubtedly be longer than keras.
?? Finally, the complete Python code for the DNN model is attached:

# iris_keras_dnn.py# Python 3.5.1, TensorFlow 1.6.0, Keras 2.1.5# ==================================================== ====# Importing module Import osimport NumPy as Npimport Keras as Kimport TensorFlow as Tfimport pandas as Pdfrom sklearn.model_selecti On import train_test_splitfrom sklearn.preprocessing import labelbinarizeros.environ[' tf_cpp_min_log_level ']= ' 2 ' #    Read the CSV dataset and split it into training set and test set # The function's incoming parameter is csv_file_path:csv file path def load_data (csv_file_path): IRIS = Pd.read_csv (Csv_file_path)    Target_var = ' class ' # Target variable # DataSet features features = List (Iris.columns) Features.remove (Target_var) # The category of the target variable class = Iris[target_var].unique () # The category Dictionary of the target variable class_dict = dict (Zip (class, range (len))) # Adds a list of target The variable is encoded iris[' target ' = iris[target_var].apply (Lambda x:class_dict[x]) # 0-1 encoding of the target variable (one-hot Encoding) lb = Lab Elbinarizer () Lb.fit (list (Class_dict.values ())) Transformed_labels = Lb.transform (iris[' target ') Y_bin_labels = [] # 0-1-coded variable for i in RangE (transformed_labels.shape[1]): y_bin_labels.append (' y ' + str (i)) iris[' y ' + str (i)] = transformed_labels[:,                                                         I] # Divide the dataset into training sets and test sets train_x, test_x, train_y, test_y = Train_test_split (Iris[features], iris[y_bin_labels], train_size=0.7, test_size=0.3, random_state=0) return train_x, test_x, Train_y, Test_y, Class_dictdef Main (): # 0. Start Print ("\niris DataSet using Keras/tensorflow") np.random.seed (4) Tf.set_random_seed (13) # 1. Read the CSV DataSet print ("Loading Iris data into memory") Csv_file_path = ' e://iris.csv ' train_x, test_x, train_y, Test_y, Class_dict = Load_data (Csv_file_path) # 2.     Define MODEL init = K.initializers.glorot_uniform (seed=1) Simple_adam = K.optimizers.adam () model = K.models.sequential () Model.add (K.layers.dense (units=5, input_dim=4, Kernel_initializer=init, activation= ' Relu ')) Model.add (K.layers.Den Se (units=6, kernel_initializer=init, activation= ' Relu ')) MODEL.ADD (K.layers.dense (units=3, Kernel_initializer=init, activation= ' Softmax ')) model.compile (loss= ' Categorical_ Crossentropy ', Optimizer=simple_adam, metrics=[' accuracy ') # 3. Training model B_size = 1 Max_epochs = + print ("Starting training") H = Model.fit (train_x, train_y, batch_size=b_size , Epochs=max_epochs, Shuffle=true, verbose=1) print ("Training finished \ n") # 4. Evaluation Model eval = Model.evaluate (test_x, test_y, verbose=0) print ("Evaluation on test data:loss =%0.6f accuracy =%0.2f% % \ n "% (Eval[0], eval[1] * 100)) # 5. Using Models for Prediction Np.set_printoptions (precision=4) unknown = Np.array ([[6.1, 3.1, 5.1, 1.1]], Dtype=np.float32) predicted  = Model.predict (unknown) print ("Using model to predict species for features:") print (unknown) print ("\npredicted Softmax vector is: ") print (predicted) Species_dict = {v:k-k,v in Class_dict.items ()} print (" \npredicted SPE Cies is: ") print (Species_dict[np.argmax (predicted))) main ()

Reference documents

1. Keras Chinese Document: https://keras-cn.readthedocs.io/en/latest/
2. Keras succinctly:http://ebooks.syncfusion.com/downloads/keras-succinctly/keras-succinctly.pdf? Awsaccesskeyid=akiaj5w3g2z6f2zhareq&expires=1539315050&signature=r6qj%2bp7kueu442wmobsld2%2flkqw%3d
3. Iris Data Set: http://archive.ics.uci.edu/ml/machine-learning-databases/iris/

Keras Introduction (i) Build deep Neural Network (DNN) to solve multi-classification problem