Learn the model and module of Mxnet (iii) from scratch

After we define the symbol in Mxnet, write the dataiter and prepare the data, we can have fun training. General Training A network has two common strategies, based on model and module-based. Today, I would like to talk about their use.

First, Model

Follow the usual code to take a look directly at the official documents:

　　

# Configure a, layer neuralnetwork    data = mx.symbol.Variable (' data ')    FC1 = mx.symbol.FullyConnected (data, Name= ' FC1 ', num_hidden=128)    Act1 = Mx.symbol.Activation (FC1, name= ' relu1 ', act_type= ' Relu ')    FC2 = Mx.symbol.FullyConnected (Act1, name= ' FC2 ', num_hidden=64)    Softmax = Mx.symbol.SoftmaxOutput (FC2, name= ' SM ') # The Create a model using Sklearn-style two-step way# creates a model    model = Mx.model.FeedForward (         softmax,         Num_epoch =num_epoch,         learning_rate=0.01) #开始训练    model.fit (x=data_set)

The specific API reference http://mxnet.io/api/python/model.html.

And then, this part of the model is finished ... There are two main reasons why this is so fast:

1. Do not have a lot of things, usually check the document can be.

2.model customization is not strong, generally we are rarely used, commonly used or module.

Second, Module

Module really is a great thing, although in-depth understanding, you will feel "wow, good, but feel no bird use" this idea. In fact, I have, now in retrospect, from the point of view of code design and use, module is really a very good thing, it can improve our Network Computing intermediate, advanced interface, so that we can have a lot of personalized configuration let us do it ourselves.

There are four states of the module:

1. The state of initialization is that the memory has not been allocated and basically nothing has been done.

2.binded, after the data and label shape into the bind function and executed, the memory is allocated, you can prepare the computing power.

3. Initialization of parameters. is the initialization parameter.

3.Optimizer installed. is to pass the Sgd,adam into the Optimuzer to train.

Let's start with a simple code:

　　

import mxnet as MX # construct a simple MLP data = mx.symbol.Variable (' data ') FC1 = mx.symbol.FullyConnected (data, name= ' FC1 ', num_hidden=128) Act1 = Mx.symbol.Activation (FC1, name= ' relu1 ', AC  T_type= "Relu") FC2 = mx.symbol.FullyConnected (act1, name = ' FC2 ', Num_hidden = +) Act2 = Mx.symbol.Activation (FC2, Name= ' RELU2 ', act_type= "Relu") FC3 = mx.symbol.FullyConnected (act2, name= ' fc3 ', num_hidden=10) out = Mx.symbol.So Ftmaxoutput (FC3, name = ' Softmax ') # construct the module mod = mx.mod.Module (out) Mod.bind (data_shapes=train _dataiter.provide_data, Label_shapes=train_dataiter.provide_label) mod.init_params () Mod.fit (train_data ITER, Eval_data=eval_dataiter, optimizer_params={' learning_rate ': 0.01, ' momentum ': 0.9}, Num_epoch=n _epoch) 

Analysis: The first is to define a simple Mlp,symbol name is called out, then you can directly use Mx.mod.Module to create a mod. After the mod.bind operation is to allocate the required video memory on the video card, so we need to pass Data_shapehe Label_shape to him, and then initialize the network parameters, and then Mod.fit began to train. Here to Add. Fit this function we have seen twice, in fact it is an integrated function, Mod.fit () in fact its internal core code is this:

　　

for epoch in range (Begin_epoch, num_epoch): Tic = Time.time () Eval_metric.reset () for Nbatch, Data_batch in Enumerate (train_data): If Monitor was not                 None:monitor.tic () Self.forward_backward (Data_batch) #网络进行一次前向传播和后向传播 Self.update () #更新参数 self.update_metric (Eval_metric, Data_batch.label) #更新metric if                      Monitor is not None:monitor.toc_print () if batch_end_callback are not None:                                                       Batch_end_params = Batchendparam (Epoch=epoch, Nbatch=nbatch,                      Eval_metric=eval_metric, Locals=locals ()) For callback in _as_list (Batch_end_callback): Callback (Batch_end_params) 

It is because we can use many intermediate interface in module, so we can make a lot of improvements, for the simplest example: what if our training network is variable in size? We can implement a mutumodule, basically, every time the shape of the data changes, we re-bind the symbol, so that the training can proceed as usual.

　　

Summary: Actually learn a framework of the key or use it, to say the trick is to look at the source and documentation, I write the purpose of these blogs, one is to record some things, the second is to let the latter less go some detours. So there's something that's not going to be very full.

　　

Learn the model and module of Mxnet (iii) from scratch