Pytorch Tutorial Neural Networks

We can pass the torch. NN package constructs a neural network.

Now we've learned that AUTOGRAD,NN defines models based on Autograd and differentiates them.

Onenn.Module模块由如下部分构成：若干层，以及返回output的forward(input)方法。

For example, this diagram depicts a neural network for digital Image classification:

This is a simple feedforward (feed-forward) network that reads input content, each layer accepts inputs from the previous level, and outputs to the next level until the OUTPU results are given.

the training program for a classical neural network is as follows:

1. Define a neural network with learning parameters (or weights)

2. Traversing the Iinput data set

3. Input is processed via neural network to get output results

4. Calculate the loss (how far ouput is from the correct value)

5. Return the gradient to the parameters of the neural network

6. Update the weight of the neural network, usually using a simple update rule:

Weight = weight-learning_rate * Gradient

First, how to define neural networks in Pytorch

To define a neural network:

ImportTorch fromTorch.autogradImportVariableImporttorch.nn as nnImporttorch.nn.functional as FclassNet (NN. Module):#defines the initialization function of NET, this function defines the basic structure of the neural network    def __init__(self):#inherits the initialization method of the parent class, which is to run the nn first. initialization function of moduleSuper (Net,self).__init__()        #define convolutional layers: Input 1-channel (grayscale) picture, output 6 feature graph, convolution core 5x5Self.conv1 = nn. conv2d (1, 6, (5,5))        #define convolutional layers: Input 6 feature graphs, output 16 feature graphs, convolution core 5x5Self.conv2 = nn. Conv2d (6,16,5)        #Define full connection layer: linear connection (y = Wx + b), 16*5*5 nodes connected to 120 nodesSELF.FC1 = nn. Linear (16*5*5,120)        #Define full connection layer: linear connection (y = Wx + b), 120 nodes connected to 84 nodesSELF.FC2 = nn. Linear (120,84)        #Define full connection layer: linear connection (y = Wx + b), 84 nodes connected to 10 nodesSELF.FC3 = nn. Linear (84,10)    #defines a forward propagation function and automatically generates a backward propagation function (Autograd)    defforward (self,x):#Enter the maximum pooling of the x->conv1->relu->2x2 window, update to xx = f.max_pool2d (F.relu (SELF.CONV1 (x)), (2,2))        #If the size is a square, you can specify only one numberx = f.max_pool2d (F.relu (Self.conv2 (x)), 2)        #The view function converts the tensor x into a one-dimensional vector form, with the total number of features unchanged, preparing for the full-attached layerx = X.view (-1, Self.num_flat_features (x))#input X->fc1->relu, update to xx =F.relu (SELF.FC1 (x))#input X->fc2->relu, update to xx =F.relu (SELF.FC2 (x))#input X->FC3, update to xx =self.fc3 (x)returnx#calculates the total characteristic amount of tensor x    defnum_flat_features (selfself,x):#batch culling of the 0th dimension due to default bulk inputSize = X.size () [1:] Num_features= 1 forSinchSize:num_features*=sreturnnum_featuresnet=Net ()Print(NET)

Output Result:

Net (  (CONV1): conv2d (1, 6, kernel_size= (5, 5), stride= (1, 1))  (CONV2): conv2d (6,, Kernel_ Size= (5, 5), stride= (1, 1))  (FC1): Linear (+) (  FC2): Linear (+) 
     (FC3): Linear (+))

Passnet.parameters()可以得到可学习的参数：

params = list (net.parameters ())print(len (params))print(Params[0].size ())  #  conv1 ' s. Weight

Output Result:

TenTorch. Size ([6, 1, 5, 5])

We simulate one-way propagation, where both input and output areautograd.Variable：

input = Variable (TORCH.RANDN (1, 1, +,= net (input)print(out)

Output Result:

Variable containing:-0.0618-0.0648-0.0350  0.0443  0.0633-0.0414  0.0317-0.1100-0.0569-0.0636 [Torch. Floattensor of size 1x10]

Zero the gradient buffers for all parameters and set the random gradient reverse propagation:

Net.zero_grad () Out.backward (Torch.randn (1, 10))

The entire TORCH.NN package only accepts data from small batches of samples, and cannot accept a single sample. For example, nn. Conv2d can build a four-dimensional tensor:nsamples x nchannels x Height x Width.

If you need to operate on a single sample, use Input.unsqueeze (0) to add a dummy dimension.

We look back at the concepts that have emerged:

Torch. Tensor-a multidimensional array
Autograd. Variable-Change the tensor and record the historical operation process. and tensor have the same API, and some APIs for backward (). It also contains gradients related to tensor.
Nn. Module-Neural network modules. Convenient data encapsulation, the ability to move operations to the GPU, but also include some input and output things.
Nn. Parameter-A variable (Variable) that is automatically registered as a parameter when any value is assigned to the module.
Autograd. Function-the definition of feedforward and feedforward using the automatic derivation method is implemented. Each variable operation generates at least one independent function node and records the history operation process after the function that generated the variable is connected.

Second, Loss Function

Pytorch Tutorial Neural Networks