Deeplearning Tutorial (6) Introduction to the easy-to-use deep learning framework Keras

Before I have been using Theano, the previous five deeplearning related articles are also learning Theano some notes, at that time already feel Theano use up a little trouble, sometimes want to achieve a new structure, it will take a lot of time to programming, so think about the code modularity, Easy to reuse, but because it's too busy to do it. Recently discovered a framework called Keras, which coincides with my ideas, is particularly simple to use and is suitable for rapid development. (There are many other deep learning frameworks that are relatively easy to use.) ）

1. Keras Introduction

Keras is a deep learning framework based on Theano, and its design references torch, written in Python language, is a highly modular neural network library that supports both GPU and CPU. Using the documentation in this: http://keras.io/, this frame seems to be just burning up, the use of the problem can go to GitHub to mention Issue:https://github.com/fchollet/keras

The following is a brief introduction to how to use Keras, to mnist database as an example, to write a CNN network structure, you will find it particularly simple.

2. Introduction of modules in Keras

• Optimizers

  As the name implies, Optimizers contains some optimization methods, such as the most basic random gradient drop SGD, plus Adagrad, Adadelta, Rmsprop, Adam, some new methods will be added in the future.

  keras.optimizers.SGD(lr=0.01, momentum=0.9, decay=0.9, nesterov=False)

  The above code is the use of SGD, LR represents the learning rate, momentum represents the momentum term, decay is the decay factor of the learning rate (once per epoch decay), the value of Nesterov is False or true, indicating that the Nesterov is not used Momentum Please refer to the documentation for additional information.

• Objectives

  This is the target function module, Keras provides Mean_squared_error,mean_absolute_error, Squared_hinge,hinge,binary_crossentropy,categorical_ Crossentropy these kinds of objective functions.

  Here Binary_crossentropy and Categorical_crossentropy are logloss.

• Activations

  This is the Activation function module, Keras provides linear, sigmoid, Hard_sigmoid, Tanh, Softplus, Relu, Softplus, In addition Softmax is also placed in the activations module (I think it is more reasonable to put in the layers module). In addition, the newer activation functions, such as Leakyrelu and Prelu, are provided Keras in the Keras.layers.advanced_activations module.

• Initializations

  This is the parameter initialization module, which initializes the call to Init when the layer is added. Keras provides uniform, lecun_uniform, normal, orthogonal, zero, Glorot_normal, He_normal.

• Layers

  The layers module contains layers such as core, convolutional, recurrent, advanced_activations, normalization, and embeddings.

  Where core contains flatten (CNN's full-attached layer needs to be a two-dimensional feature map flatten one-dimensional), reshape (CNN input when the one-dimensional vector into two-dimensional), dense (is the hidden layer, dense is dense meaning), There are others that are not introduced. The convolutional layer is basically the Theano convolution2d package.

• Preprocessing

  This is a preprocessing module, including processing of serial data, processing of text data, processing of image data. Focusing on a data processing, Keras provides imagedatagenerator functions to achieve data augmentation, DataSet amplification, the image to do some elastic transformation, such as horizontal flipping, vertical flipping, rotation and so on.

• Models

  This is the main module, the model. The above defines a variety of basic components, model is to combine them, the following is illustrated by an example.

3. One example: classified by CNN Mnist

• Data download

  Mnist data on its official website is provided, but is not image format, because we are usually directly processing the image, in order to later the program can be reused, I made it into an image format, here can download: http://pan.baidu.com/s/1qCdS6, a total of 42000 pictures.

• Reading picture data

  Keras requires the input data format is the Numpy.array type (NumPy is a python numerical calculation of the library), so you need to write a script to read mnist image, saved as a four-dimensional data, and a one-dimension label, code:

  #coding:utf-8"""Author:weponSource:https://github.com/wepefile:data.py"""import osfrom PIL import Imageimport numpy as np#读取文件夹mnist下的42000张图片，图片为灰度图，所以为1通道，#如果是将彩色图作为输入,则将1替换为3,图像大小28*28def load_data():    data = np.empty((42000,1,28,28),dtype="float32")    label = np.empty((42000,),dtype="uint8")    imgs = os.listdir("./mnist")    num = len(imgs)    for i in range(num):        img = Image.open("./mnist/"+imgs[i])        arr = np.asarray(img,dtype="float32")        data[i,:,:,:] = arr        label[i] = int(imgs[i].split(‘.‘)[0])    return data,label

• Build CNN, Train

  Just 20 lines of code, build a three convolutional layer of CNN, read the following code directly, there are comments, it is easy to read:

  #导入各种用到的模块组件from __future__ Import absolute_importfrom __future__ import Print_functionfrom keras.preprocessing.image Import imagedatageneratorfrom keras.models import sequentialfrom keras.layers.core import dense, dropout, Activation, Flattenfrom keras.layers.advanced_activations Import prelufrom keras.layers.convolutional import convolution2d, Maxpooling2dfrom keras.optimizers Import SGD, Adadelta, adagradfrom keras.utils import np_utils, Generic_utilsfrom Six.moves Import rangefrom Data import load_data# load the database, label = Load_data () print (data.shape[0], ' samples ') #label为0 ~ 9 A total of 10 categories, Keras required format binary class matrices, conversion, directly call Keras provides this function label = np_utils.to_categorical (Label, 10) ############ # # # #开始建立CNN模型 ############### #生成一个modelmodel = sequential () #第一个卷积层, 4 convolution cores per convolutional core size 5*5. 1 represents the channel of the imported picture, and the grayscale image is 1 channels. #border_mode可以是valid或者full, see here for details: http://deeplearning.net/software/theano/library/tensor/nnet/conv.html# theano.tensor.nnet.conv.conv2d# activation function with tanh# You can also add Tanh skills after Model.add (Activation (' Dropout ')): Model. Add (Dropout (0.5)) Model.add (convolution2d (4, 1, 5, 5, border_mode= ' valid ')) Model.add (Activation (' Tanh ')) #第二个卷积层, 8 convolution cores, each convolutional core size 3*3. 4 indicates the number of input features, equal to the number of convolution cores on the previous layer # activation function with tanh# using maxpooling,poolsize for (2,2) Model.add (convolution2d (8,4, 3, 3, border_mode= ' Valid ') Model.add (Activation (' Tanh ')) Model.add (Maxpooling2d (poolsize= (2, 2))) #第三个卷积层, 16 convolution cores per convolutional core size 3*3# The activation function uses the tanh# maxpooling,poolsize for (2,2) Model.add (convolution2d (8, 3, 3, border_mode= ' valid ')) Model.add ( Activation (' Tanh ')) Model.add (Maxpooling2d (poolsize= (2, 2))) #全连接层, first the two-dimensional feature diagram of the previous layer output is flatten to one-dimensional. #Dense就是隐藏层. 16 is the number of feature plots on the previous layer of output. 4 is calculated according to each convolution layer: (28-5+1) get 24, (24-3+1)/2 get 11, (11-3+1)/2 get 4# fully connected with 128 neuron nodes, initialized by Normalmodel.add (Flatten ()) Model.add (Dense (16*4*4, init= ' normal ')) Model.add (Activation (' Tanh ')) #Softmax分类, output is 10 category Model.add (dense (128, init= ' normal ') Model.add (Activation (' Softmax ')) ############# #开始训练模型 ############## #使用SGD + momentum# The parameter loss in Model.compile is the loss function (target function) SGD = SGD (l2=0.0,lr=0.05, decay=1e-6, momentum=0.9, nesterov=true) Model.compile ( LosS= ' categorical_crossentropy ', optimizer=sgd,class_mode= "categorical") #调用fit方法, is a training process. The number of epochs trained is set to 10,batch_size of 100. #数据经过随机打乱shuffle =true. Verbose=1, the information that is output during the training process, 0, 1, 23 ways can, does not matter. Show_accuracy=true, each epoch of the training output accuracy. #validation_split = 0.2, 20% of the data is used as the validation set. Model.fit (data, label, BATCH_SIZE=100, nb_epoch=10,shuffle=true,verbose=1,show_accuracy=true,validation_split=0.2 ) #fit方法在达到设定的nb_epoch时结束, and automatically save the best model, you can then call the Model.evaluate () method to test the test data, #还有model. Predict_classes, Model.predict_proba and other methods, see the documentation for details.

• Code usage and Results

The code is placed in my GitHub machine learning repository: https://github.com/wepe/MachineLearning, non-GitHub users directly point to the lower right of the downloadzip.

In the/deeplearning tutorials/keras_usage directory includes data.py , cnn.py two code, download mnist data after extracting to this directory, run cnn.py this file.

The results are as follows, with the epoch 9 reaching 0.98 of the training set recognition rate and 0.97 of the validation set recognition rate:

Reprint Please specify source: http://blog.csdn.net/u012162613/article/details/45397033

Deeplearning Tutorial (6) Introduction to the easy-to-use deep learning framework Keras