Evaluation and comparison of deep learning framework

Article source:http://www.infoq.com/cn/news/2016/01/evaluation-comparison-deep-learn?utm_campaign=infoq_content& Evaluation and comparison of deep learning framework

Artificial intelligence is undoubtedly the forefront of the computer world, and deep learning is undoubtedly the focus of artificial intelligence, then there are now what open-source deep learning tools, their respective advantages and disadvantages of what? Recently, zer0n and Bamos published an article on GitHub, comparing deep learning tools such as Caffe, CNTK, TensorFlow, Theano, and torch from networks, model capabilities, interfaces, deployments, performance, architecture, ecosystems, and cross-platform.

Network and model capabilities

Caffe is probably the first mainstream industrial-grade deep learning tool that began at the end of 2013 with an excellent convolutional neural network implementation. Caffe is still the most popular toolkit in the field of computer vision, it has many extensions, but it has poor support for recursive networks and language modeling due to some legacy architectural problems. In addition, layers need to be defined in C + + in Caffe, while networks are defined using PROTOBUF.

CNTK was created by the initiator speaker of the deep learning craze and has now developed into a universal, platform-independent deep learning system. In CNTK, the network is designated as a symbolic graph of vector operations, and the combination of operations forms a layer. CNTK enables users to create new, complex layer types without the need for low-level languages by using fine-grained component blocks.

TensorFlow is an ideal RNN (recursive neural network) API and implementation, TensorFlow uses the vector operation of the symbolic graph method, making the designation of the new network quite easy, but TensorFlow does not support bidirectional rnn and 3D convolution, Also, the public version of the diagram definition does not support loop and conditional control, which makes the RNN implementation undesirable because the Python loop must be used and the diagram compilation optimizations are not possible.

Theano supports most advanced networks, and many of the current research ideas are derived from Theano, which leads to the trend of symbolic diagrams used in programming networks. Theano's symbolic API supports loop control, making the RNN implementation easier and more efficient.

Torch Support for convolutional networks is very good. The time domain convolution in tensorflow and Theano can be achieved by conv2d, but this is a bit trickery; The local interface of the time domain convolution makes it very intuitive to use. Torch supports a large number of RNN through many unofficial extensions, and there are many ways to define the network. But torch essentially defines the network as a layer, and this coarse-grained way makes it less supportive of the expansion of the new layer type. Defining new layers in torch is easy compared to Caffe, and does not require C + + programming, with minimal differences between layers and network definitions.

Interface

Caffe supports pycaffe interfaces, but this is only used to assist the command-line interface, and even if it is used, the pycaffe model must be defined using PROTOBUF.

CNTK is used in a way similar to Caffe, by specifying a configuration file and running the command line, but CNTK does not have a python or any other high-level language interface.

TensorFlow supports both Python and C + + two types of interfaces. Users can experiment in a relatively rich high-level environment and deploy models in environments that require local code or low latency.

The Theano supports the Python interface.

Torch runs on Luajit, which is very fast compared to industrial languages like C + +, C #, and Java, where users can write arbitrary types of calculations without worrying about performance, the only problem is that LUA is not the mainstream language.

Model deployment

Caffe is based on C + +, so it can be compiled on a variety of devices and is cross-platform, making it the best choice for deployment.

CNTK and Caffe are also based on C + + and cross-platform, most of the deployment is very simple. But it does not support the ARM architecture, which limits its ability on mobile devices.

TensorFlow supports the C + + interface, and because it uses eigen instead of the Blas class library, it can be compiled and optimized based on the ARM architecture. TensorFlow users can deploy well-trained models to multiple devices, without the need to implement a separate model decoder or load the PYTHON/LUAJIT interpreter. However, TensorFlow does not support windows, so its model cannot be deployed to Windows devices.

Theano lacks the underlying interface, and its Python interpreter is inefficient and unattractive to industrial users. While the python overhead for a large model is not small, its limitations are there, and the only bright spot is that it spans platforms where models can be deployed to Windows environments.

Torch 's model runs require Luajit support, and although this has little impact on performance, it poses a major barrier to integration, making it less attractive than caffe/cntk/tensorflow directly supporting C + + frameworks.

Performance

In a single GPU scenario, all of these toolsets call the CUDNN, so as long as the outer layer's calculations or memory allocations differ little, it behaves as if it were not the same. The performance test in this paper is based on the Convnets benchmark test of [email protected] .

Caffe Simple and fast.

CNTK Simple and fast.

TensorFlow uses only CUDNN v2, but even so its performance is still 1.5 times times slower than V2 with CUDNN torch, and there is a memory overflow problem in training googlenet when the batch size is 128.

Theano performance on large networks is comparable to TORCH7. But its main problem is that the boot time is particularly long because it needs to compile the C/cuda code into binary, and TensorFlow does not have this problem. In addition, the import of Theano consumes time and cannot be freed from pre-configured devices (for example) after importing GPU0 .

Torch very good, no tensorflow and Theano problems.

In addition, in the multi-GPU aspect, CNTK is better than other deep learning toolkits, which implements 1-bit SGD and adaptive minibatching.

Architecture

The Caffe architecture now looks like an average, and its main pain point is that the layers need to be defined using C + +, and the model needs to use the PROTOBUF definition. In addition, if you want to support CPUs and GPUs, users must also implement additional functions, such as and, for Forward_gpu Backward_gpu custom layer types, you must also assign an ID of type int to it and add it to the Proto file.

TensorFlow 's architecture is clear, with a modular design that supports a wide range of front-end and execution platforms.

Theano 's architecture is perverted, its entire code base is Python, and even the C/cuda code is packaged as a python string, making it difficult to navigate, debug, refactor, and maintain.

The Torch7 and nn class libraries have a clear design and modular interface.

Cross-platform

Caffe, CNTK, and Theano are all capable of running on all systems, while TensorFlow and torch do not support Windows.

Evaluation and comparison of deep learning framework