The method of realizing convolutional neural network CNN on Pytorch

This article mainly introduces the method of realizing Convolutional neural Network (CNN) on Pytorch, and now share it with everyone, and give us a reference. Come and see it together.

One, convolutional neural network

The convolutional Neural Network (CONVOLUTIONALNEURALNETWORK,CNN) was originally designed to solve problems such as image recognition, and the current application of CNN is not limited to images and video, but also to time series signals, such as audio signals and text data. As a deep learning architecture, CNN was proposed to reduce the requirement of preprocessing of image data and avoid complex feature engineering. In convolutional neural networks, the first convolution layer will directly accept the image pixel-level input, each layer convolution (filter) will extract the most effective features of the data, this method can be extracted to the most basic features of the image, and then combined and abstract to form higher-order features, so CNN theoretically has the image scaling, Invariance of translation and rotation.

convolutional Neural Network The point of CNN is the local connection (localconnection), weight sharing (weightssharing), and the pooling layer (Pooling) in the drop-down sampling (down-sampling). The local connection and weight sharing reduce the number of parameters, so the training complexity decreases greatly and the overfitting is reduced. At the same time, the weighted value sharing also gives the tolerance of the convolution network to the translation, and the pool layer drop sampling further reduces the output parameter quantity and gives the model tolerance to the light deformation, and improves the generalization ability of the model. Convolution layer convolution can be understood as the process of extracting similar features in multiple locations of images with a few parameters.

Second, the Code implementation

Import Torch Import torch.nn as nn from Torch.autograd import Variable import torch.utils.data as data import torchvision Import Matplotlib.pyplot as Plt torch.manual_seed (1) EPOCH = 1 Batch_size = LR = 0.001 download_mnist = True # Get training Set DataSet Training_data = Torchvision.datasets.MNIST (root= './mnist/', # DataSet storage Path Train=true, # True means Trai        n training set, False indicates test set Transform=torchvision.transforms.totensor (), # normalize the original data to the (0,1) interval download=download_mnist, # Print the training set of the Mnist dataset and the size of the test set print (Training_data.train_data.size ()) print (Training_data.train_labels.size ()) # Torch. Size ([60000, +]) # torch. Size ([60000]) Plt.imshow (Training_data.train_data[0].numpy (), cmap= ' Gray ') plt.title ('%i '% training_data.train_ Labels[0]) Plt.show () # The dataset format obtained through Torchvision.datasets can be directly placed in Dataloader Train_loader = Data.dataloader (dataset= Training_data, Batch_size=batch_size, shuffle=true) # Get test set DataSet Test_data = Torchvision.datasets.MNIST (root= './mnist/', Train=false) # Take the first 2000 test set samples test_x = Variable (Torch.unsqueeze (Test_data.test_data, dim=1), volatile=true) . Type (torch. Floattensor) [: 2000]/255 # ($, 1, +), in range (0,1) test_y = test_data.test_labels[:2000] Class CNN (NN. Module): def __init__ (self): Super (CNN, self). __init__ () Self.conv1 = nn. Sequential (# (1,28,28) nn. Conv2d (In_channels=1, out_channels=16, kernel_size=5, stride=1, padding=2), # (16,28,28) # want to con2d the convolution The picture size does not change, padding= (kernel_size-1)/2 nn. ReLU (), nn. Maxpool2d (kernel_size=2) # (16,14,14)) Self.conv2 = nn. Sequential (# (16,14,14) nn. Conv2d (5, 1, 2), # (32,14,14) nn. ReLU (), nn. Maxpool2d (2) # (32,7,7)) Self.out = nn. Linear (32*7*7) def forward (self, x): x = SELF.CONV1 (x) x = self.conv2 (x) x = X.view (x.size (0),-1) # Will (     batch,32,7,7) flatten to (batch,32*7*7) output = self.out (x)Return Output CNN = CNN (CNN) "CNN" (CONV1): Sequential ((0): conv2d (1, kernel_size= (5, 5), stride= ( 1, 1), padding= (2, 2)) (1): ReLU () (2): maxpool2d (Size= (2, 2), Stride= (2, 2), dilation= (1, 1)) (CONV2): Sequenti Al ((0): conv2d (5, 5), stride= (1, 1), padding= (2, 2)) (1): ReLU () (2): maxpool2d (Size= (2, 2) , Stride= (2, 2), dilation= (1, 1))) (out): Linear (1568)) ' optimizer = Torch.optim.Adam (Cnn.parameters (), L R=LR) loss_function = nn. Crossentropyloss () for the epoch in range (Epoch): For step, (x, Y) in Enumerate (train_loader): b_x = Variable (x) b_      y = Variable (y) output = CNN (b_x) loss = loss_function (output, b_y) Optimizer.zero_grad () Loss.backward () Optimizer.step () if step% = = 0:test_output = CNN (test_x) pred_y = Torch.max (test_output, 1) [1] . Data.squeeze () accuracy = SUM (pred_y = = test_y)/test_y.size (0) print (' Epoch: ', Epoch, ' |     Step: ', step,     ' |train loss:%.4f '%loss.data[0], ' |test accuracy:%.4f '%accuracy) Test_output = CNN (test_x[:10]) pred_y = Torch.max (t Est_output, 1) [1].data.numpy (). Squeeze () print (pred_y, ' prediction number ') print (Test_y[:10].numpy (), ' real number ') "' epoch:0 | step:0 |train loss:2.3145 |test accuracy:0.1040 epoch:0 | step:100 |train loss:0.5857 |test accuracy:0.8865 epoch:0 | step:200 |train loss:0.0600 |test accuracy:0.9380 epoch:0 | step:300 |train loss:0.0996 |test accuracy:0.9345 epoch:0 | step:400 |train loss:0.0381 |test accuracy:0.9645 epoch:0 | step:500 |train loss:0.0266 |test accuracy:0.9620 epoch:0 | step:600 |train loss:0.0973 |test accuracy:0.9685 epoch:0 | step:700 |train loss:0.0421 |test accuracy:0.9725 epoch:0 | step:800 |train loss:0.0654 |test accuracy:0.9710 epoch:0 | step:900 |train loss:0.1333 |test accuracy:0.9740 epoch:0 | step:1000 |train loss:0.0289 |test accuracy:0.9720 epoch:0 | step:1100 |train loss:0.0429 |test accuracy:0.9770 [7 2 1 0 4 1 4 9 5 9]Prediction Number [7 2 1 0 4 1 4 9 5 9] Real number " 

Iii. Analysis and interpretation

By using Torchvision.datasets, you can quickly get data in a dataset format that can be placed directly in Dataloader, control whether to acquire a training dataset or a test data set through the train parameter, or switch directly to the data format required for training at the time of acquisition.

The construction of convolutional neural network is realized by defining a CNN class, and the convolution layer conv1,conv2 and out layer are defined in the form of class attribute, and the cohesion information between each layer is defined in forward, and the number of neurons in each layer should be observed when defining.

The network structure of CNN is as follows:

CNN ((CONV1): Sequential (  (0): conv2d (1, 16,kernel_size= (5, 5), stride= (1, 1), padding= (2, 2))  (1): ReLU ()  (2): maxpool2d (size= (2,2), stride= (2, 2), dilation= (1, 1))) (CONV2): Sequential (  (0): conv2d (+ 32,kernel_size= (5, 5), stride= (1, 1), padding= (2, 2))  (1): ReLU ()  (2): maxpool2d (size= (2,2), stride= (2, 2), dilation= (1, 1)) (OU T): Linear (1568->10))

The experimental results show that the accuracy of the test set can reach 97.7% in the training result of epoch=1.