Ubuntu14.10+cuda7.0+caffe Configuration

Ubuntu14.10+cuda7.0+caffe Configuration one: Linux installation

Linux installation No, I'm installing ubuntu14.10 here.

II: Installation and commissioning of Nvidia Drive and Cuda Toolkit (*.run method)

1:verify you have a cuda-capable GPU

Do the following, and then verify that the hardware supports GPU CUDA, as long as the model exists in Https://developer.nvidia.com/cuda-gpus, there is no problem.

$ LSPCI | Grep-i nvidia

2:, Verify you have asupported Version of Linux

$ uname-m && cat/etc/*release

The emphasis is on "x86_64", which is guaranteed to be x86 architecture, 64bit system

3:verify the System has GCC installed

$ gcc--version

If not, install it first, this is required to compile Cuda Toolkit, but Ubuntu 14.10 is the default

4:download the NVIDIA CUDA Toolkit

: Https://developer.nvidia.com/cuda-toolkit

Verify Address: Https://developer.nvidia.com/rdp/cuda-rc-checksums

$ md5sum filename

For example: Md5sum Cuda_7.0.28_linux.run, the correct MD5 of this file =312aede1c3d1d3425c8aa67bbb7a55e

5:handle Conflicting Installation Methods

According to the official website, the previously installed version will be suspected of conflict

So, the previously installed toolkit and drievers have to be uninstalled, shielded, and so on (because I am the new system and have not installed the Nvidia driver, so this step can be omitted)

sudo apt-get--purge Remove nvidia*

6:graphical Interface Shutdown

Exit GUI, that is, X-win interface, the operation method is: simultaneously press: CTRL+ALT+F1 (F2-F6), switch to tty1-6 command line mode.

To turn off desktop services:

$ sudo stop LIGHTDM

7:interaction with Nouveau

Nouveau is an open source graphics driver, Ubuntu14.04 is installed by default, but it will affect the installation of Nvidia drivers, so only ask him to go back to his hometown, sorry!

$ sudo vi/etc/modprobe.d/nvidia-graphics-drivers.conf

Write: Blacklist nouveau

Save and exit: wq!

Check:$ cat nvidia-graphics-drivers.conf

$ sudo vi/etc/default/grub

End write:Rdblacklist=nouveau nouveau.modeset=0

Save and exit: wq!

Check:$ cat/etc/default/grub

8:installation CUDA 7.0

Switch to the directory where Cuda_7.0.28_linux.run is located, and then execute the install command:

$ sudo sh cuda_7.0.28_linux.run

Here will ask you all kinds of questions, basically is accept-yes-enter-yes-enter-yes-enter, in fact, let you accept the agreement, and then install the default location confirmation and so on, recruit do not customize the installation location, the default is heaven.

After execution, the following results appear:

Driver:installed Requirereboot

Toolkit:installskip

Samples:installskip

See, this CUDA installation package actually contains the Driver,toolkit and samples three parts, the first time the installation of the driver loaded, but to restart. restarted, or as above, exit the GUI and re-execute the following command:

$ sudo sh cuda_7.0.28_linux.runsu

The results are as follows:

Driver:installed

Toolkit:installed in/usr/local/cuda-7.0

Samples:installed In/home/caffe, butmissing recommended libraries

The installation was successful this time, but some recommended libraries are missing.

9:extra Libraries

Install some necessary library files, such as: OpenGL (e.g, Mesa), GLU, GLUT, and X11 (including Xi, XMU, ANDGLX).

$ sudo apt-get install Freeglut3-dev build-essential libx11-devlibxmu-dev libxi-dev LIBGL1-MESA-GLX Libglu1-mesa Libglu1-mesa-dev

10: The driver is finished, you can go back to the GUI interface, you can continue to play text here ...

$ sudo start LIGHTDM

11:post-installationactions

(1) environment variable configuration

Execute the sudo vi/etc/profile, followed by:

Exportpath=/usr/local/cuda-6.5/bin: $PATH

EXPORTLD_LIBRARY_PATH=/USR/LOCAL/CUDA-6.5/LIB64: $LD _library_path

(2) Verify the installation

A. Verifying the driver version is basically to ensure that the driver is installed properly

$ cat/proc/driver/nvidia/version

B. Compiling the Examples

$ nvcc-v

Once installed here, you can compile and switch directories to ~/nvidia_cuda-7.0_samples:

$ cd/home/username/nvidia_cuda-7.0_samples

$ make

It's going to be a long time.

C. Running the Binaries

Run the compiled file, for example, to see the basic information and bandwidth information for the device:

$ CD Bin/x86_64/linux/release

$./devicequery

$./bandwidthtest

If the test appears to say that the running version of the driver and the actual driver does not match, the reason may be because the installation of Nvidia-cuda-toolkit updated the configuration file, and the original cuda-samples configuration or the driver has changed, so detection can not be compiled through. Consider the following workaround:

A. Uninstalling an existing drive

$ sudo nvidia-installer--uninstall

B. Download the appropriate version of the driver and install:

: Http://www.geforce.cn/drivers

$ sudo sh./nvidia-linux-x86_65*.run

C. Re-install Cuda Toolkit

$ sudo sh cuda-7.0

Nvidia Cuda installation ends (here I recommend not to manually install the appropriate Nvidia driver, preferably uninstall the above process again, because manually installed may cause the wrong version, when the desktop blank problem-----I was re-installed system ..... )

Three: Blas installation and Configuration

1:blas Installation

Here you can choose (Atlas,mkl or Openblas), I use MKL here, first download and install Intel? Math Kernel library for Linux * version mkl, download link is: https://software.intel.com/en-us/intel-education-offerings, please download student version, apply first, Then you will receive an email immediately (there is a serial number installed), open to follow the download on the line. After the download, to extract the files to the home folder (or directly to the tar.gz file to the home folder, in order to save space, remember to delete the compressed file after installation), or other Ext4 file system.

The next step is the installation process, authorizing and then installing:

$ tar zxvf parallel_studio_xe_2015.tar.gz (if you are copying the compressed files directly)

$ chmod a+x parallel_studio_xe_2015-r

$ sudo./install_gui.sh

Environment settings for 2:MKL and Cuda

1. Create a new intel_mkl.conf and edit it:

$ sudo gedit/etc/ld.so.conf.d/intel_mkl.conf

/opt/intel/lib/intel64

/opt/intel/mkl/lib/intel64

2. Create a new cuda.conf and edit it:

$ sudo gedit/etc/ld.so.conf.d/cuda.conf

/usr/local/cuda/lib64

/lib

3. Complete the link operation of Lib file, execute:

$ sudo ldconfig–v

Four OPENCV Installation

1. Download and compile OpenCV (original opencv:http://opencv.org/), or use the modified version of the installation package Install-opencv-master (The following installation method is used to complete the package, The installation package modifies the dependencies.sh file and adds the OpenCV 3.0.0 installation files, while preserving the original 2.3x and 2.4x versions.

2. Switch to the folder where you saved the file, and then install the dependencies:

$ sudo sh ubuntu/dependencies.sh

3. Switch directory ubuntu\3.0\ install OPENCV3.0.0RC1:

$ sudo sh opencv3_0_0-rc1.sh

Ensure the network is unblocked, because the software needs to be networked here for a long time, please wait patiently ... , a long time!

V: Install additional dependencies

1. Google Logging Library (glog),: https://code.google.com/p/google-glog/, then unzip the installation:

$ tar zxvf glog-0.3.3.tar.gz

$./configure

$ make

$ sudo make install

If you do not have permission to chmod a+x glog-0.3.3-r, or simply chmod 777glog-0.3.3-r, after the installation, this folder can kill.

2. Other dependencies to ensure success

$ sudo apt-get install-y libprotobuf-dev libleveldb-devlibsnappy-dev libopencv-dev libboost-all-dev Libhdf5-serial-dev

$ sudo apt-get install-y libgflags-dev libgoogle-glog-devliblmdb-dev Protobuf-compiler

Six: Install Caffe and test

1. Install some of the dependencies that Pycaffe must have:

$ sudo apt-get install-y python-numpy python-scipypython-matplotlib python-sklearn python-skimage python-h5py Python-protobufpython-leveldb python-networkx python-nose Python-pandas python-gflags Cythonipython

$ sudo apt-get install-y protobuf-c-compilerprotobuf-compiler

2. Installation configuration Nvidia CuDNN accelerated Caffe model operation (we did not install it because Install.txt said it was for CUDA6.5)

A. Installing the CUDNN

The version caffe-master default support CUDNN-6.5-LINUX-X64-V2, use CUDNN-6.5-LINUX-R1 will error, before installation please go to the official website to download the latest CUDNN.

$ sudo cp cudnn.h/usr/local/include

$ sudo cp libcudnn.so/usr/local/lib

$ sudo cp libcudnn.so.6.5/usr/local/lib

$ sudo cp libcudnn.so.6.5.48/usr/local/lib

B. Link to a cudnn library file

$ sudo ln-sf/usr/local/lib/libcudnn.so.6.5.48/usr/local/lib/libcudnn.so.6.5

$ sudo ln-sf/usr/local/lib/libcudnn.so.6.5/usr/local/lib/libcudnn.so

$ sudo ldconfig–v

3: Switch to Caffe-master folder, generate Makefile.config configuration file, execute:

$ CP Makefile.config.example Makefile.config

4: Configure the Makefile.config file (only the modification section is listed)

A. Enable Cudnn, remove "#" (currently Caffe-master still only supports R1 version)

USE_CUDNN: = 1

B. Enable the GPU, remove the comment "#"

# cpu_only: = 1

C: Enable Intel Parallelstudio XE Professional Edition for C + + Linux

BLAS: = Mkl

5: Configure the Makefile file (for opencv3.x support)

Look for the section "Derive include and Lib directories" to modify the last line of "LIBRARIES + =" To increase the opencv_imgcodecs

Opencv_core Opencv_highgui opencv_imgproc opencv_imgcodecs

6: Compiling caffe-master!!! "-j8" is compiled using multi-core CPU, which can greatly accelerate the speed of compiling, it is recommended.

$ make All-j8

$ make Test-j8

$ make Runtest-j8

Recommendation: Before make, you can test the driver with the previous./bandwidthtest command.

Seven: Testing with mnist datasets

CAFFE By default will be installed in $caffe_root, is to extract to that directory, for example: $ home/username/caffe-master, so the following work, the default has been switched to the working directory. The following work is primarily to test whether the Caffe is working properly and does not perform a detailed evaluation. For specific settings, please refer to the official website: http://caffe.berkeleyvision.org/gathered/examples/mnist.html

1. Data preprocessing

$ sudo shdata/mnist/get_mnist.sh

2. Rebuild the Lmdb file. Caffe supports three data format input networks, including image (. jpg,. png, etc.), leveldb,lmdb, and chooses different inputs according to your needs.

$sudo shexamples/mnist/create_mnist.sh

Generate Mnist-train-lmdb and Mnist-train-lmdb folders, which contain data sets in Lmdb format

Note: When I execute this command, I report that Not in Gzformat , later found to be networked download mnist There was a problem with the data, and then I went online and downloaded it. mnist data set, after the Ubuntu manually unzip, then execute the following command.

3. Training Mnist

$ sudo shexamples/mnist/train_lenet.sh

At this point, the Caffe installation of all the steps to the end, the following is a simple set of data comparison, experiment from the mnist data set, mainly to examine the performance of the CPU and GPU under different systems. Can see the obvious difference, although the Mnist dataset is very simple, believe that the complex data set, the difference will be greater, UBUNTU+GPU is the only choice.

Test Platform 1 : I7-4770K/16G/GTX 770/cuda 6.5

MNIST Windows8.1 on CPU : 620s

MNIST Windows8.1 on GPU : 190s

MNIST Ubuntu 14.04 on CPU : 270s

MNIST Ubuntu 14.04 on GPU : 160s

MNIST Ubuntu 14.04 on Gpuwith CuDNN : 101

cifar10_full on Gpuwihtout CuDNN : 73m45s = 4428s ( Iteration 70000 )

cifar10_full on GPU withcudnn : 20m7s = 1207s ( Iteration 70000 )

Test Platform 2 : Gigabyte p35x v3 , [Email protected]/16g/nvidiagtx 980 4G

MNIST Ubuntu 15.04 on Gpuwith CuDNN : 33s

Contrast Test 1 : 2*e5-2620 (12CPUs)/128g/tesla K20m/cuda5.5/centos 6.4

MNIST CentOS 6.4 on GPU : 294s

Contrast Test 2 : Tesla K40m/cuda6.5/ubuntu 14.04

MNIST on GPUs with CuDNN : 101

Contrast Test 3 : GTX 660/cuda6.5/ubuntu 14.04

MNIST on GPUs with CuDNN : 49s

Comparison Test 1 is an unfair test, after all, the performance is very poor, it is likely not only by Tesla K20s and GTX 770, may be due to the impact of CentOS or CUDA5.5 (without CuDNN), but the overall conclusion and Caffe official Reference performance numbers consistent, for ordinary users: GTX cost-effective much higher. Contrast Test 2 demonstrates the power of Tesla K40 and believes it should have a stronger performance for complex images. (thank the City University of Hong Kong Ph.D Jingjing, Nanjing University Ph.D Jinlu, Huazhong University of Science and Technology MS Liumaolin to provide test environment and test data. ）

If you don't need to use Caffe in Python and MATLAB, you don't need to install Python and matlab

Eight: Python installation

1. Installing the IDE's operating environment

Choose a suitable IDE for your environment, I am using the Spyder, because it built-in IPython environment, Caffe a lot of programs are based on IPython environment to complete. The installation method is simple and can be installed by searching the "Spyder" directly in Ubuntu Software Center.

2. IPython NoteBook Installation

Another recommended approach is to use the Ipyhthon NoteBook (browser-based Python IDE), especially for teachers who need to do tutorials with Python, which can be exported directly. PY,. IPYNB, HTML format, installation steps are as follows:

$ sudo apt-get install-y ipython-notebook pandoc

Start (automatically open browser):

$ Ipython Nootbook

A simple example of HTML generated using Ipython notebook: examples_notebook.html EXAMPLE_NOTEBOOK.IPYNB

3: Environment configuration

Configuring caffe support for Python interfaces in the Makefile.config file

Python_include: =/usr/include/python2.7 \

/usr/lib/python2.7/dist-packages/numpy/core/include

Python_lib: =/usr/local/lib

Include_dirs: = $ (python_include)/usr/local/include

Library_dirs: = $ (python_lib)/usr/local/lib/usr/lib

4: Execute $ make pycaffe-j8 compile python in Caffe-master directory

5: Join in the/etc/profile

Export Pythonpath=/home/hadoop/caffe/caffe-master/python: $PYTHONPATH

This makes it possible to import caffe in Python.

Nine: Matlab installation

MATLAB installation, I did not use, nor installed, if necessary, can find information on the internet!

Reference documents:

1: Official website http://caffe.berkeleyvision.org/installation.html

2:https://ouxinyu.github.io/blogs/20140723001.html Owang's Blog

3:http://blog.sciencenet.cn/blog-1583812-841855.htmlubuntu+cuda6.5+caffe Installation Configuration Rollup

4:http://www.haodaima.net/art/2823705caffe+ubuntu14.04+cuda6.5 Novice Installation Configuration Guide

5:http://weibo.com/p/2304189db078090102vdvx

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Ubuntu14.10+cuda7.0+caffe Configuration