An example of regression and classification of simple neural networks built on Pytorch

This article mainly introduces the pytorch of simple neural network to achieve regression and classification of the example, now share to everyone, but also to make a reference. Come and see it together.

This paper introduces a simple neural network on Pytorch to implement the regression and classification of the example, share to everyone, as follows:

First, Pytorch Introduction

1. Installation method

Login Pytorch official website, http://pytorch.org, you can see the following interface:

By selecting the option you can get the Linux Conda command:

Conda Install Pytorch torchvision-c soumith

Currently Pytorch only supports MacOS and Linux, and Windows is not supported at this time. Install Pytorch will install two modules, one is torch, a torchvision, torch is the main module, used to build a neural network, Torchvision is a secondary module, there are databases, there are some trained neural networks waiting for you to use directly, for example (Vgg, AlexNet, ResNet).

2. NumPy and Torch

Torch_data = Torch.from_numpy (np_data) can convert numpy (array) format to torch (tensor) format; Torch_data.numpy () You can also convert the torch tensor format to the NumPy array format. Note that Torch's tensor and numpy arrays will share their storage space, and modifying one will cause the other one to be modified as well.

For 1-D (1-d) data, NumPy prints the output in the form of a row vector, and torch prints the output as a column vector.

Other functions in numpy such as sin, cos, abs,mean, and so on are the same in torch. It is important to note that the Np.matmul (data, data) and Data.dot (data) matrices in the numpy have the same result, and torch.mm (tensor, tensor) in torch is a matrix multiplication method, which gets a matrix, Tensor.dot (tensor) converts tensor to 1-dimensional tensor, then sums each element by multiplying it with a real number.

Related code:

Import Torch Import NumPy as NP Np_data = Np.arange (6). Reshape ((2, 3)) Torch_data = Torch.from_numpy (np_data) # will NumPy (AR Ray) format converted to torch (tensor) format Tensor2array = Torch_data.numpy () print (' \nnumpy array:\n ', Np_data, ' \ntorch tensor: ', to Rch_data, ' \ntensor to array:\n ', Tensor2array, ') # Torch data format automatically adds line breaks before and after print: # ABS data = [-1,-2, 2, 2] tensor = Torch. Floattensor (data) print (' \nabs ', ' \nnumpy: \ n ', np.abs (data), ' \ntorch: ', Torch.abs (tensor)) # 1-D, NumPy is Line vector form display, torch is a column vector form display # sin print (' \nsin ', ' \nnumpy: \ n ', Np.sin (data), ' \ntorch: ', Torch.sin (tensor)) # Me An print (' \nmean ', ' \nnumpy: ', Np.mean (data), ' \ntorch: ', Torch.mean (tensor)) # matrix multiplied by data = [[+], [3,4]] t Ensor = Torch. Floattensor (data) print (' \nmatrix multiplication (matmul) ', ' \nnumpy: \ n ', Np.matmul (data, data), ' \ntorch: ', TORCH.MM (tensor, tensor)) data = Np.array (data) print (' \nmatrix multiplication (dot) ', ' \nnumpy: \ n ', Data.dot (dat a),     ' \ntorch: ', Tensor.dot (tensor)) 

3. Variable

The neural network in Pytorch comes from the Autograd package, and the Autograd package provides an automatic derivation method for all tensor operations.

Autograd. Variable this is the most core class in the package. Variable can be understood as a container with tensor, which wraps a tensor and almost supports all operations defined on it. Once the operation is complete, you can call. Backward () to automatically calculate all the gradients. In other words, only the tensor is placed in the variable, in order to achieve reverse transfer, automatic derivation and other operations in the neural network.

The original tensor can be accessed through the property. Data, and the gradient about this variable can be viewed through the. Grad property.

Related code:

Import Torch from Torch.autograd import Variable  tensor = torch. Floattensor ([[[1,2],[3,4]]) variable = variable (tensor, requires_grad=true) # print show variable type print (tensor) print ( Variable)  t_out = Torch.mean (tensor*tensor) # each element of ^ 2 V_out = Torch.mean (variable*variable) print (t_out) print (v_ Out)  v_out.backward () # variable error reversal  # Compare variable's prototype and Grad attribute, data property, and corresponding NumPy form print (' variable:\n ', Variable) # v_out = quarter * SUM (variable*variable) This is the V_out calculation step in the calculation diagram # The gradient for the v_out is, D (v_out)/d (variable) = 1/4*2*variab Le = VARIABLE/2 print (' variable.grad:\n ', Variable.grad) # variable gradient print (' variable.data:\n ', Variable.data) # Variable data print (Variable.data.numpy ()) #Variable的数据的numpy形式

Partial output results:

Variable
Variable containing:
1 2
3 4
[Torch. Floattensor of size 2x2]
Variable.grad:
Variable containing:
0.5000 1.0000
1.5000 2.0000
[Torch. Floattensor of size 2x2]
Variable.data:
1 2
3 4
[Torch. Floattensor of size 2x2]
[1.2.]
[3.4.]

4. Excitation function activationfunction

The excitation functions of torch are in Torch.nn.functional, Relu,sigmoid, Tanh, softplus are commonly used as excitation functions.

Related code:

Import Torch Import torch.nn.functional as F from Torch.autograd import Variable import matplotlib.pyplot as plt  x = t Orch.linspace ( -5, 5, x_variable) = variable (x) #将x放入Variable x_np = X_variable.data.numpy ()  # The data result of the NumPy form is obtained by 4 different excitation functions Y_relu = F.relu (x_variable). Data.numpy () y_sigmoid = F.sigmoid (x_variable). Data.numpy () Y_ Tanh = F.tanh (x_variable). Data.numpy () Y_softplus = F.softplus (x_variable). Data.numpy ()  plt.figure (1, figsize= (8 , 6))  Plt.subplot (221) Plt.plot (X_NP, Y_relu, c= ' Red ', label= ' Relu ') Plt.ylim (( -1, 5)) Plt.legend (loc= ' best ')  Plt.subplot (222) Plt.plot (X_NP, y_sigmoid, c= ' Red ', label= ' sigmoid ') Plt.ylim (( -0.2, 1.2)) Plt.legend (loc= ' best ')  Plt.subplot (223) Plt.plot (X_NP, Y_tanh, c= ' Red ', label= ' Tanh ') Plt.ylim (( -1.2, 1.2)) Plt.legend (loc= ' best ')  Plt.subplot (224) Plt.plot (x_np, Y_softplus, c= ' Red ', label= ' Softplus ') Plt.ylim (( -0.2, 6)) Plt.legend (loc= ' best ')  Plt.show ()

Second, the Pytorch realizes the return

Look at the full code first:

Import Torch from Torch.autograd import Variable import torch.nn.functional as F import matplotlib.pyplot as Plt x = Torc H.unsqueeze (Torch.linspace ( -1, 1, y), dim=1) # convert 1-dimensional data to 2-dimensional data y = X.pow (2) + 0.2 * Torch.rand (X.size ()) # Place tensor in Varia ble in x, y = Variable (x), Variable (y) #plt. Scatter (X.data.numpy (), Y.data.numpy ()) #plt. Show () # defines a class of classes Net that constructs a neural network (t  Orch.nn.Module): # Inherits Torch.nn.Module class Def __init__ (self, n_feature, N_hidden, N_output): Super (Net, self). __init__ ()  # Get the construction method of the superclass (parent class) of NET class # define the structure of each layer of the neural network # The information for each layer is the property of the Net class object Self.hidden = Torch.nn.Linear (N_feature, N_hidden) # Hidden layer Linear Output self.predict = Torch.nn.Linear (N_hidden, n_output) # output layer Linear output # The neurons of each layer are built into the forward path of the complete neural Network def forward (self, X): x = F.relu (Self.hidden (x)) # Relu activation of the hidden layer's output x = self.predict (x) return x # defines the neural network net = Net (1, 1) PR The structure of the int (net) # print output net # definition Optimizer and loss function optimizer = Torch.optim.SGD (Net.parameters (), lr=0.5) # Incoming network parameters and learning rate loss_function = Tor Ch.nn.MSELoss () # minimum mean square error  # Neural Network training Process plt.ion () # Dynamic learning process show plt.show () for T in range: prediction = net (x) # feed data x to NET, output predictive value loss = Loss_   function (prediction, y) # Calculates the error of the two, note the order of the two parameters Optimizer.zero_grad () # Clears the update parameter value of the previous step Loss.backward () # Error Inverse propagation, calculates the new update parameter value Optimizer.step () # Assigns the computed update value to the Net.parameters () # Visual training Process if (t+1)% = = 0:plt.cla () plt.scatter (x.data.num PY (), Y.data.numpy ()) Plt.plot (X.data.numpy (), Prediction.data.numpy (), ' R ', lw=5 ' plt.text (0.5, 0, ' l=%.4f '% lo Ss.data[0], fontdict={' size ': +, ' color ': ' Red '}) Plt.pause (0.1)

First, a set of two-time function fitting data with noise is created and placed in variable. Defines a class net that constructs a neural network, inheriting the Torch.nn.Module class. NET class to define the input neurons, hidden layer neurons, the number of output neurons parameters, through the super () method to obtain the construction method of the net parent class, to define the structure of the various layers of net in the way of attributes; forward () of the definition net Methods the neurons of each layer were constructed into a complete neural network forward pathway.

Once the net class is defined, a neural network instance is defined, and the net class instance can directly print the structure information of the output neural network. Then we define the neural network optimizer and the loss function. Once you've defined this, you're ready to train. Optimizer.zero_grad (), Loss.backward (), Optimizer.step () are emptied of the last step of the update parameter values, the reverse propagation of errors and the calculation of new update parameter values, Assigns the computed update value to Net.parameters (). Loop iterative training process.

Operation Result:

Net (

(hidden): Linear (1-10)

(predict): Linear (1)

)

Three, Pytorch realization simple classification

Full code:

Import Torch from Torch.autograd import Variable import torch.nn.functional as F import matplotlib.pyplot as PLT # Generate Data # Generate 2 groups of 100 data points respectively, increase the normal noise, after marking to y0=0 y1=1 two categories of labels, the last cat connected to N_data = Torch.ones (100,2) # torch.normal (means, std=1.0, Out=none)  x0 = Torch.normal (2*n_data, 1) # gives the output tensor the mean value of each element in the form of tensor, sharing the standard deviation y0 = Torch.zeros (+) x1 = Torch.normal ( -2*n_data, 1) y1 = Torch.ones x = Torch.cat ((x0, X1), 0). Type (torch. Floattensor) # assemble (connect) y = Torch.cat ((y0, y1), 0). Type (torch. Longtensor) # Place Variable in x, y = Variable (x), Variable (Y) class Net (torch.nn.Module): Def __init__ (self, n_feature, n_ Hidden, n_output): Super (Net, self). __init__ () Self.hidden = Torch.nn.Linear (n_feature, n_hidden) Self.out = t Orch.nn.Linear (N_hidden, N_output) def forward (self, x): x = F.relu (Self.hidden (x)) x = Self.out (x) return x net = Net (n_feature=2, n_hidden=10, n_output=2) print (net) optimizer = Torch.optim.SGD (Net.parameters (), lr=0.012) Los S_func = Torch.nn.CrossEntropyLoss () plt.ion () plt.show () for T in range (s): out = Net (x) Loss = Loss_func (out, y) # Loss is defined as the difference between the output of the neural network and the sample label y, so take Value before Softmax Optimizer.zero_grad () Loss.backward () Optimizer.step () if t% 2 = = 0:plt.cla () # over a Softma The maximum probability after the excitation function of x is the predicted value # Torch.max returns both the maximum value on a dimension and the index value of the maximum value prediction = Torch.max (F.softmax (out), 1) [1] # takes the maximum value in the 1th dimension and returns Back index value pred_y = Prediction.data.numpy (). Squeeze () target_y = Y.data.numpy () plt.scatter (X.data.numpy () [:, 0], X. Data.numpy () [:, 1], c=pred_y, s=100, lw=0, cmap= ' rdylgn ') accuracy = SUM (pred_y = = target_y)/200 # Predict how much is the same as the real value pl T.text (1.5, -4, ' accu=%.2f '% accuracy, fontdict={' size ': +, ' color ': ' Red '}) Plt.pause (0.1) Plt.ioff () plt.show ()

The net class of the structure part of the neural network is the same as that of the regression part of the previous article.

It is important to note that in the Loop Iteration Training section, out is defined as the output of the neural network, the calculation error loss is not used in one-hot form, loss is defined on the Out and Y Torch.nn.CrossEntropyLoss (), The predicted value, prediction, is defined as the result of out after Softmax (converting the result to a probability value).

Operation Result:

Net (

(hidden): Linear (2-10)

(out): Linear (2)

)

Iv. Supplementary Knowledge

1. Super () function

When defining the construction method of the net class, a super (net,self). __INIT__ () statement is used, and the current class and object are used as arguments to the Super function, and the function of this statement is to make the net class's construction method get its superclass (parent class) constructor method. Does not affect the method of constructing a separate definition of the net class, and you do not have to focus on what the net class's parent class is, but only modify the contents of the class statement if you need to modify the parent class of the net class.

2. Torch.normal ()

Torch.normal () can be divided into three cases: (1) torch.normal (MEANS,STD, out=none) means and STD are tensor, the shape of the two can not be the same, but the number of elements in the tensor must be the same, One by one the corresponding element is the mean and standard deviation of each element of the output, and (2) the mean in Torch.normal (mean=0.0, STD, Out=none) is a definable float, each element shares the mean value, and (3) torch.normal (means , std=1.0, Out=none) STD is a definable float, and each element shares the standard deviation.

3. Torch.cat (seq, dim=0)

Torch.cat can connect several tensor assemblies, and dim specifies which dimension to assemble on.

4. Torch.max ()

(1) Torch.max (input) →float

Input is tensor, which returns the maximum value of float in input.

(2) Torch.max (Input,dim, Keepdim=true, Max=none, Max_indices=none) (Tensor, Longtensor)

Returns the maximum value on the specified dimension =dim and the index value of the maximum value on the dimension.