MXNET: Multilayer Sensing Machine

Start from scratch

Before we understand the principle of multilayer perceptron, we can realize a multi-layer perception machine.

# -*- coding: utf-8 -*-from mxnet import initfrom mxnet import ndarray as ndfrom mxnet.gluon import loss as glossimport gb# 定义数据源batch_size = 256train_iter, test_iter = gb.load_data_fashion_mnist(batch_size)# 定义模型参数num_inputs = 784num_outputs = 10num_hiddens = 256W1 = nd.random.normal(scale=0.01, shape=(num_inputs, num_hiddens))b1 = nd.zeros(num_hiddens)W2 = nd.random.normal(scale=0.01, shape=(num_hiddens, num_outputs))b2 = nd.zeros(num_outputs)params = [W1, b1, W2, b2]for param in params:    param.attach_grad()# 定义激活函数def relu(X):    return nd.maximum(X, 0)# 定义模型def net(X):    X = X.reshape((-1, num_inputs))    H = relu(nd.dot(X, W1) + b1)    return nd.dot(H, W2) + b2# 定义损失函数loss = gloss.SoftmaxCrossEntropyLoss()# 训练模型num_epochs = 5lr = 0.5gb.train_cpu(net, train_iter, test_iter, loss, num_epochs, batch_size,             params, lr)

The performance of the model is greatly improved after the hidden layer is added

# outputepoch 1, loss 0.5029, train acc 0.852, test acc 0.934epoch 2, loss 0.2000, train acc 0.943, test acc 0.956epoch 3, loss 0.1431, train acc 0.959, test acc 0.964epoch 4, loss 0.1138, train acc 0.967, test acc 0.968epoch 5, loss 0.0939, train acc 0.973, test acc 0.973

There is still some complexity in defining model parameters and defining model steps.

Using gluon

# -*- coding: utf-8 -*-from mxnet import initfrom mxnet import ndarray as ndfrom mxnet.gluon import loss as glossimport gb# 定义数据源batch_size = 256train_iter, test_iter = gb.load_data_fashion_mnist(batch_size)# 定义模型from mxnet.gluon import nnnet = nn.Sequential()net.add(nn.Dense(256, activation='relu'))net.add(nn.Dense(10))net.add(nn.Dense(10))net.initialize(init.Normal(sigma=0.01))# 定义损失函数loss = gloss.SoftmaxCrossEntropyLoss()# 训练模型from mxnet import gluontrainer = gluon.Trainer(net.collect_params(), 'sgd', {'learning_rate': 0.5})num_epochs = 5gb.train_cpu(net, train_iter, test_iter, loss, num_epochs, batch_size,             None, None, trainer)# outputepoch 1, loss 1.3065, train acc 0.525, test acc 0.814epoch 2, loss 0.2480, train acc 0.928, test acc 0.950epoch 3, loss 0.1442, train acc 0.958, test acc 0.961epoch 4, loss 0.1060, train acc 0.969, test acc 0.971epoch 5, loss 0.0807, train acc 0.976, test acc 0.973

MXNET: Multilayer Sensing Machine