Mxnet: Basic knowledge and a simple example

Ndarray is similar to a multidimensional array of NumPy, but Ndarray provides more functionality: asynchronous computation of the GPU and CPU, and automatic derivation. This allows Ndarray to better support machine learning.

Initialization

from mxnet import ndarray as ndnd.zeros((3,4))nd.ones((3,4))nd.array([[1,2],[3,4]])out:[[1. 2.][3. 4.]] <NDArray 2x2 @cpu(0)>nd.random_normal(0,1,shape=(3,4)) #标准正态分布# 输出信息y.shapey.size

Operator

operation by corresponding element

x+yx*ynd.exp(x)

Multiplication of matrices

nd.dot(x, y.T)

Broadcast (beoadcasting)

When the two-dollar operator is not the same shape as the left and right sides of the Ndarray, the system tries to extend it to a common shape.

a=nd.arange(3).reshape((3,1))b=nd.arange(2),reshape((1,2))print ('a+b', a+b)out:a+b:[[ 0. 1.] [ 1. 2.] [ 2. 3.]] <NDArray 3x2 @cpu(0)>

The conversion to NumPy

x=np.ones((2,3))y=nd.array(x) # numpy->mxnetz=y.asnumpy() # mxnet->numpy

Replace operation

If we write y=x+y, we will open up new memory to store the calculation results, such as:

x=nd.ones((3,4))y=nd.ones((3,4))before = id(y)y=y+xid(y)==before # False

Can be written into a well-formed array by [:]

z=nd.zeros_like(y)before=id(z)z[:]=x+yid(z)==before # True

Above, the system also created a temporary space for x+y, and then copied it to Z. To avoid this overhead, you can use the full name version of the operator and specify the out parameter

nd.elemwise_add(x,y,out=z)id(z)==before # True

Interception (slicing)

x=nd.arange(0,9).reshape((3,3))x[1:3]out:[[ 3. 4. 5.] [ 6. 7. 8.]] <NDArray 2x3 @cpu(0)>#改变指定位置的值x[1,2]=9.#多维截取x[1:2,1:3]#多维写入x[1:2,1:3]=9.out:[[ 0. 1. 2.] [ 3. 9. 9.] [ 6. 7. 8.]] <NDArray 3x3 @cpu(0)>

Automatic derivation using Autograd

import mxnet.ndarray as ndimport mxnet.autograd as ag

Attaching gradients to variables

x=nd.array([[1,2],[3,4]])x.attach_grad() # ndarray的方法申请相应的空间# 定义函数f=2x*x，显式要求mxnet记录我们要求导的程序with ag.record():    y=x*2    z=y*x# 通过z.backword()来进行求导，如果z不是一个标量，z.backward()等价于nd.sum(z).backward().z.backward()print('x.grad: ',x.grad)x.grad == 4*x# outputx.grad:[[4.,  8.] [12., 16.]]<NDArray 2x2 @cpu(0)>[[ 1.  1.] [ 1.  1.]]<NDArray 2x2 @cpu(0)>

Derivation of control flow

One of the conveniences of imperative programming is the ability to take advantage of almost any derivative of the program, even if it contains the Python control flow. For a graph frame, this corresponds to a dynamic graph, where the structure of the graph changes depending on the input data.

def f(a):     b=a*2    while nd.norm(b).asscalar() < 1000:         b=b*2    if nd.sum(b).asscalar() > 0:         c=b    else:        c = 100 * b     return c

Derivation using record and backward

a = nd.random_normal(shape=3)a.attach_grad()with ag.record():     c = f(a)c.backward()

Head gradient and chain rule

Based on the chain rule:
\[\FRAC{DZ}{DX} = \frac{dz}{dy}\frac{dy}{dx}\]
\ (\frac{dz}{dy}\) is the head gradient of \ (\frac{dy}{dx}\) , while calculation \ (\frac{dz}{dy}\), the head gradient is the default value, that is Nd.ones_ Like (y). We can also specify the head gradient manually.

with ag.record():     y = x * 2     z = y * xhead_gradient = nd.array([[10, 1.], [.1, .01]]) z.backward(head_gradient) print(x.grad)# outx=[[1.,2.] [3.,4.]]<NDArray 2x2 @cpu(0)>x.grad=[[40., 8.] [12., 0.16]]<NDArray 2x2 @cpu(0)>

Mxnet: Basic knowledge and a simple example