Computational Network Toolkit (CNTK) is a Microsoft-produced open-Source Deep learning Toolkit

Computational Network Toolkit (CNTK) is a Microsoft-produced open-Source Deep learning Toolkit

Using CNTK to engage in deep learning (a) Getting Started

Computational Network Toolkit (CNTK) is a Microsoft-produced open-source deep learning toolkit. This article introduces the basic content of CNTK, how to write the CNTK network definition language, and a simple example of running.

According to the description of Microsoft developers, CNTK performance than Caffe,theano, Tensoflow and other mainstream tools are strong. It supports CPU and GPU mode, so there is no GPU, or the neural network is relatively small experiment, directly with the CPU version of the CNTK run on the line. Its open source homepage in Https://github.com/Microsoft/CNTK it describes the neural network as a graph structure, the leaf node represents input or network parameters, and other node calculation steps. It supports convolutional neural networks and recurrent neural networks. Due to the introduction of CNTK soon, the mass tutorials are not much, and the bug is estimated a lot. When I study, the main reference three information:

1 Official Introductory Tutorial Https://github.com/Microsoft/CNTK/wiki/Tutorial This article also mainly takes the tutorial here as an example

2 Official Forum Https://github.com/Microsoft/CNTK/issues

3 Official papers Http://research.microsoft.com/pubs/226641/CNTKBook-20160217..pdf This has 150 pages, I use it as a dictionary, and when I have a problem, I search it inside.

Install cntk:https://github.com/microsoft/cntk/wiki/cntk-binary-download-and-configuration go to this page to find a version that fits your system. I am a Windows user, CNTK has a compiled CPU and GPU version. Because my video card is not Nvidia Tatsu, so helpless can only use the CPU version of the use.  The package that has been compiled is the most convenient, unzip, and then add the directory (like%...%, CNTK-2016-02-08-WINDOWS-64BIT-CPU-ONLY\CNTK\CNTK) to the PATH variable. Conditional people can also compile their own source code, a little trouble, a variety of dependencies, the advantage is the source of the update is relatively fast, cntk a major feature is the current variety of small bugs more, such as I now use the compiled package or two months ago released, has already filled a lot of pits.

After installing the CNTK, running a program is a simple command line: CNTK Configfile=your_config_file, where your_config_file is the definition file for the network, presumably long like this:

command=train:testtrain=[

        action="train"        NDLNetworkBuilder = [        ...        ]        SGD = [        ...        ]        reader = [        ...
　　　　]

]

Test=[...]

The entry is the command command, followed by a module that needs to be run sequentially, separated by a colon. The things that need to be defined within each module are more similar, mainly defining the input format, the network structure, the learning algorithm (currently only SGD) and the parameters. When you define a network structure, you indicate which nodes are the optimization target, which are the evaluation metrics, and which are the points of the output.

As we all know, the hidden layer of the neural network is removed, the input is connected directly to the output layer, so that the line becomes a logistics regression classifier. So https://github.com/Microsoft/CNTK/wiki/Tutorial this tutorial will guide you how to build an LR. I'll change a little bit here and learn how to build a neural network with a layer of hidden layers, such as:

Defining the network structure

CNTK describes a neural network using the Network Description Language (Networks Description language, NDL). Simply put, we want to describe the input feature, the input label, some parameters, the calculation relationship between parameters and input, and what the target node is.

ndlnetworkbuilder=[        Run=ndllr        ndllr=[      # Sample and label Dimensions      sdim= $dimension $      ldim=1          Features=input (Sdim, 1)      labels=input (Ldim, 1)          # parameters to learn      B0 = Parameter (4)       W0 = Parameter (4, Sdim)                  B = Parameter (ldim)      W = Parameter (Ldim, 4)          # operations      t0 = times (W0, features) C14/>z0 = Plus (t0, B0)      s0 = Sigmoid (z0)               t = times (W, s0)      z = Plus (t, B)      s = Sigmoid (z)              LR = Logis Tic (labels, s)      EP = squareerror (labels, s)          # root nodes      featurenodes= (features)      labelnodes= ( Labels)      criterianodes= (LR)      evalnodes= (EP)      outputnodes= (s,t,z,s0,w0)    ]     

Features=input (Sdim, 1)     labels=input (Ldim, 1) and B0 = Parameter (4)  can be imagined as defining variables. Input is a column vector, CNTK inside the operation is all matrix operations, so the input as only one column of the matrix.  T0 = times (W0, features) is a matrix multiplication, t0 multiplies input and weight, Z0 adds a t0 above bias,
S0 represents an activation function. The B0,W0,T0,Z0,S0 constitutes the operation of the hidden layer, where the hidden layer is defined with 4 nodes. T,z,s is the operation of the output layer, and S is the value of the output node. After the framework is defined, it is also necessary to specify a number of root nodes to specify specific tasks, such as featurenodes= (features)  and labelnodes= (labels), respectively specifying the input and output nodes, Criterianodes is the trained
Time to optimize the target, evalnodes is the reference value when doing the evaluation of the output. OUTPUTNODES specifies the node that needs to be output to a file.

Set up training algorithms

SGD = [            epochsize=0     #  number of samples used per round of iterations,  =0 indicates the use of the entire training set        minibatchsize=25  # Training 25 samples to update a parameter        learningratespermb=0.1                # Learning rates per MB        maxepochs=50    #迭代50次    ]    

Currently only SGD (and various variants on SGD), you can set various parameters inside.

Set input format

reader = [        #customDelimiter = ""        readertype = "Ucifastreader"        file = "Train.txt"        minibatchmode = "Partia L "                verbosity = 1        randomize =" None "                features=[            Dim = $dimension $            start = 0        ]            labels=[            start = $dimension $              # Skip $dimension $ elements before reading the label (i.e. the first and dimensions so we h Ave "X1 X2 y" basically)            Dim = 1                          # label has 1 dimension            labeltype=regression            labelmappingfile = "Simp LeMapping.txt "        ]    

This is also a feature of CNTK (spit point), which specifies how the data file is to be read. readerType = "UCIFastReader" 指定用普通的扁平化表格的格式（一行一个样例，同一行内用空格隔开不同的数值），还有别的格式类型，例像格式，文本语料格式等。UCIFastReader 是将被弃用的，而且在目前最新的binary包中是有bug的 （所以说，有条件的同学尽量自己编译最新的源码）。  用官方教程里的设置直接跑回出bug，以上是我修改过的代码。 输入格式主要描述了feature是哪几列，维度是多少，label是哪几列， label的类型等等。

In summary, train This module is to define these things: input format, network content, training mode. This is also the step when running: Read data--SGD training.

Other

In addition to the process of modules other than train, they do not need to define the network structure and training mode, but the input format is still to be specified.    For example, the process of the test module is: Read data, compute network, and get the predicted value, evaluation. The evaluation is targeted at the nodes that are defined in the network structure EvalNodes .SquareError 只是其中的一种评估指标。如果想用别的误差函数，可以去查字典http://research.microsoft.com/pubs/226641/CNTKBook-20160217..pdf 

test=[    action= "test"    reader=[        readertype= "Ucifastreader"        file=            "Test.txt" features=[ dim=2            start=0        ]        labels=[            start= $dimension $            dim=1            labeldim=2        ]    ]

The output module and test process are basically the same, except that the last one is not an evaluation, but rather a value that belongs to Outputnodes to the file. The output module Specifies a directory OutputPath = "LR.txt", and the output file is prefixed with "LR.txt", plus the variable is used as the file name. For example, "LR.txt.W0".

# Output The resultsoutput=[    action= "Write"    reader=[        readertype= "Ucifastreader"        file= " Test.txt "        features=[            dim= $dimension $            start=0        ]        labels=[            start=2            dim=1            Labeltype=regression        ]    ]    outputPath = "LR.txt"       # Dump the output as text]

dumpNodeInfoThe value used to output the parameter. This is useful in debugging, for example, to see how the parameters of the network change:

dumpnodeinfo=[        action=dumpnode        printvalues=true    ]
################################################################### #B =learnableparameter [NeedGradient]   =true  -6.67130613  #################################################################### ep=squareerror ( Labels, s) features=inputvalue [2 x 1 {}] labels=inputvalue [1 x 1 {}] lr=logistic (labels, s) s=sigmoid ( z) t=times (W, features) W=learnableparameter [up]   needgradient=true  1.23924482 1.59913719  ########## ########################################################## z=plus (T, B)

The entire code is as follows. Train file https://github.com/Microsoft/CNTK/wiki/Tutorial/Train-3Classes.txt test file https://github.com/Microsoft/ Cntk/wiki/tutorial/test-3classes.txt. The data is 2-dimensional:

# Copyright (c) Microsoft. All rights reserved.# Licensed under the MIT license.  See LICENSE file in the project root for full LICENSE information.# logistic regression cntk Script--Network Description language# which commands to runcommand=train:output:dumpnodeinfo:test#required...modelpath= "MODELS/LR_REG.DNN" # W Here to write the model Todeviceid=-1 # cpudimension=2 # input data dimensi      ons# training configtrain=[action= "Train" TraceLevel = 1 ndlnetworkbuilder=[run=ndllr ndllr=[  # Sample and Label dimensions sdim= $dimension $ ldim=1 features=input (Sdim, 1) labels=input (Ldim, 1) # parameters to learn B0 = Parameter (4) W0 = Parameter (4, Sdim) B = Parameter (Ldim ) W = Parameter (Ldim, 4) # Operations T0 = times (W0, features) z0 = Plus (t0, B0) S0 = Sigmoid (z0) T = times (W, s0) z = Plus (t, B) s =Sigmoid (z) LR = Logistic (labels, s) EP = Squareerror (labels, s) # root nodes featurenodes=      (features)            labelnodes= (labels) criterianodes= (LR) evalnodes= (EP) outputnodes= (s,t,z,s0,w0)] [SGD = [ Epochsize=0 # =0 means size of the training set minibatchsize=25 Lear ningratespermb=0.1 # Learning rates per MB maxepochs=50] # parameter values for the Reade R reader = [#customDelimiter = "" Readertype = "ucifastreader" file = "Train.txt" Minibat Chmode = "partial" verbosity = 1 randomize = "None" features=[Dim = $dime nsion$ start = 0] labels=[start = $dimension $ # skip $dimension $ el                 Ements before reading the label (i.e. the first and dimensions so we have "x1 x2 y" basically) Dim = 1         # label has 1 dimension labeltype=regression labelmappingfile = "SimpleMapping.txt" ]]]# testtest=[action= "test" reader=[readertype= "ucifastreader" randomize = "None" fi Le= "Test.txt" features=[dim= $dimension $ start=0] labels=[start = $dimension $ # Skip $dimension $ elements before reading the label (i.e. the first and dimensions so we have "X            1 X2 y "basically) Dim = 1 # Label has 1 dimension labeltype=regression Labelmappingfile = "SimpleMapping.txt"]]]# output the resultsoutput=[action= "write" reader=            [readertype= "Ucifastreader" file= "Test.txt" randomize = "None" features=[ Dim= $dimension $ start=0] labels=[start = $dimension $ # Skip $di mension$ elements before Reading the label (i.e. the first and dimensions so we have "x1 x2 y" basically) Dim = 1    # label has 1 dimension labeltype=regression labelmappingfile = "SimpleMapping.txt"] ] OutputPath = "LR.txt" # Dump the output as text]dumpnodeinfo=[Action=dumpnode Printvalues=false]

Next post:

Using CNTK to engage in deep learning (ii) Training natural language models based on RNN (language model)

Http://www.cnblogs.com/sylvanas2012/p/5419477.html

Original blog, without permission, please do not reprint.

Category: Data mining Tags: CNTK, deep learning, neural networks

Computational Network Toolkit (CNTK) is a Microsoft-produced open-Source Deep learning Toolkit