CNN Foundation and Development Environment Construction (comprehensive reference)

CNN Foundation and Environment Construction

Author: Wang Shuai; mail:[email protected]

At present, deep learning has obtained the best known results in solving the problems of image classification and speech recognition, and the series of algorithms have been paid more and more attention by academia and industry. What is deep learning? An intuitive explanation is that if a machine learning algorithm uses multi-layered automatic feature representations in the modeling process, the machine learning algorithm can be called a deep learning algorithm, which means that the machine learning algorithm can automatically calculate feature representations. Convolutional Neural Networks (hereinafter referred to as CNN) are the most basic and effective algorithms in deep learning, although CNN was first proposed by Kunihikofukushima, but it is well known for the improvement of Yannlecun and Yoshua Bengio. It is said that the implementation of the LeNet-5 is still used for handwritten numeral recognition (LeNet-5 's brief introduction please refer to the blog blog.csdn.net/xuanyuansen/article/details/41800721). Why should we mention these two Daniel? Because the two people are currently in the field of deep learning dominate, Yannlecun is currently working in the Facebook AI Lab, and Yoshuabengio is still in academia, since the mention of these two, you can not mention Google's geoffreye. Hinton and Baidu's andrewng, while Andrew Ng and Yoshuabengio are both students of Michael I Jordan, Jordan's great God's laboratory is really a big club in machine learning (see www.cs.berkeley.edu/~jordan/sail2010.jpg for a grand feast).

Now, there's a lot of open source on CNN, and with the advances in hardware technology (this should be a tribute to NVIDIA, the president is Chinese), the use of GPUs for neural Network algorithm training and implementation has become mainstream. This article will briefly describe three GPU-based open source implementations (all running successfully in the ubuntu14.04x64 environment) and a personal simple implementation of LeNet-5. The three GPU-based implementations are Deepnet,caffe and Theano, noting that both deepnet and Theano are Python implementations, and Caffe also exploits a variety of Python libraries (the rest are implemented in C + +). These three open source environments contain many machine learning algorithms (including deep learning algorithms), and CNN is just one. The basis of GPU computing is Nvidia's CUDA environment, so we first introduce CUDA installation in the ubuntu14.04x64 environment.

Cuda in the Https://developer.nvidia.com/cuda-downloads, install the steps please see http://blog.csdn.net/xuanyuansen/article/details/ 43152311, special attention here is to remove the Ubuntu system comes with the open source graphics driver, and to completely remove the original system comes with the Nvidia-related package.

First, deepnet installation

Deepnet is https://github.com/nitishsrivastava/deepnet, the use of GIT commands can be directly checkout down, the source package is the author of Nitishsrivastava (Geoffreye. Hinton Indian students), the installation process according to the source code package in the INSTALL.txt instructions can be noted that the source code package for the presentation of the data can not be used in the original mnist data, and to use the processed mnist data, HTTP// Www.cs.toronto.edu/~nitish/deepnet/mnist.tar.gz. Other things to note are the need to confirm that the Ubuntu system Python environment contains both NumPy and scipy packages, which need to be installed in advance if not. Deepnet implements six models, namely: Feed-forwardneural Nets;restrictedboltzmann machines;deepbelief nets;autoencoders ; Deepboltzmann machines;convolutionalneural Nets.

Second, Caffe installation

The installation of the Caffe is more complex and the pits are much more (OMG). Caffe for Https://github.com/BVLC/caffe, detailed procedures for installation see Caffe's official Installation guide http://caffe.berkeleyvision.org/ Installation.html, and a cow's blog post (very well written) http://www.cnblogs.com/platero/p/3993877.html, the part about Cuda installation can be skipped directly, There are three places to note, one is the installation of INTELMKL, the package is very cumbersome, you can directly use the Openblas or atlas instead (personal use of Openblas), you must remember in the Caffe configuration file to correctly configure the item. The second is OPENCV installation, with the original source package will be error, the solution in: http://code.opencv.org/issues/3814, download ncvpixeloperations.hpp Replace the files within the opencv2.4.9 and rebuild. Third, after installing the Python Anaconda package, Caffe compile error, the workaround is: delete anaconda/lib/below the LIBM, command for sudo rm-rf libm* (see blog link http://blog.csdn.net/ ccemmawatson/article/details/42004105).

Caffe installed, you can follow the development document to build the CNN network, see the official tutorial,http://caffe.berkeleyvision.org/tutorial/.

Third, Theano installation

Theano installation is very simple, easy to use, minutes can be done, documents and http://deeplearning.net/software/theano/. The Theano-based CNN model is conveniently located in the official textbook, with an address of http://www.deeplearning.net/tutorial/lenet.html#lenet.

Iv. LeNet-5 's personal simple Python implementation (non-GPU mode)

Using Python to implement a simple version of LeNet-5, the first is to implement a multilayer neural network, and then implemented on this basis CNN, code see the following link (BMNN2 in the code is the implementation of multilayer neural networks):

1, http://blog.csdn.net/xuanyuansen/article/details/41544153

2, http://blog.csdn.net/xuanyuansen/article/details/41924377

CNN Foundation and Development Environment Construction (comprehensive reference)