Ubuntu14.10+cuda7.0+caffe Configuration

Ubuntu14.10+cuda7.0+caffe Configuration one: Linux installation

Linux installation is not said. I'm installing ubuntu14.10 here.

II: Installation and commissioning of Nvidia Drive and Cuda Toolkit (*.run method)

1:verify you have a cuda-capable GPU

Run the following operation, and then verify that the hardware supports GPU CUDA. Only the model exists in the Https://developer.nvidia.com/cuda-gpus. There's no problem.

$ LSPCI | Grep-i nvidia

2:, Verify you have asupported Version of Linux

$ uname-m && cat/etc/*release

The emphasis is on the term "x86_64", which is guaranteed to be x86 architecture. 64bit system

3:verify the System has GCC installed

$ gcc--version

If not, install it first, this is required to compile Cuda Toolkit, but Ubuntu 14.10 is the default

4:download the NVIDIA CUDA Toolkit

: Https://developer.nvidia.com/cuda-toolkit

Verify Address: Https://developer.nvidia.com/rdp/cuda-rc-checksums

$ md5sum filename

For example: Md5sum Cuda_7.0.28_linux.run, the correct MD5 of this file =312aede1c3d1d3425c8aa67bbb7a55e

5:handle Conflicting Installation Methods

According to the official website, the previous installed version number will be suspected of conflict

Therefore, the previously installed toolkit and Drievers will have to be uninstalled, shielded. And so on (since I am the new system.) No NVIDIA drivers have been installed. So this step can be omitted)

sudo apt-get--purge Remove nvidia*

6:graphical Interface Shutdown

Exit the GUI, which is the X-win interface. To do this: at the same time press: Ctrl+alt+f1 (F2-F6), switch to tty1-6 command line mode.

To turn off desktop services:

$ sudo stop LIGHTDM

7:interaction with Nouveau

Nouveau is an open-source graphics driver. Ubuntu14.04 is installed by default, but it will affect the installation of the Nvidia driver, so just invite him back to his hometown, sorry!

$ sudo vi/etc/modprobe.d/nvidia-graphics-drivers.conf

Write: Blacklist nouveau

Save and exit: wq!

Check:$ cat nvidia-graphics-drivers.conf

$ sudo vi/etc/default/grub

End write:Rdblacklist=nouveau nouveau.modeset=0

Save and exit: wq!

Check:$ cat/etc/default/grub

8:installation CUDA 7.0

Switch to the folder where Cuda_7.0.28_linux.run is located. Then run the Install command:

$ sudo sh cuda_7.0.28_linux.run

Here's how to ask you all sorts of questions, basically accept-yes-enter-yes-enter-yes-enter, and in fact, let you accept the agreement. Then install the default location confirmation, and so on. Recruit don't define the installation location yourself. The default is heaven.

After running, the following results appear:

Driver:installed Requirereboot

Toolkit:installskip

Samples:installskip

See, this CUDA installation package actually includes the Driver,toolkit and samples three parts. The driver was mounted on the first installation, just to start again. Start again, or as above, exit the GUI. Run the following command again:

$ sudo sh cuda_7.0.28_linux.runsu

The results are as follows:

Driver:installed

Toolkit:installed in/usr/local/cuda-7.0

Samples:installed In/home/caffe, butmissing recommended libraries

The installation was successful this time, but some of the recommended libraries are missing.

9:extra Libraries

Install some necessary library files, such as: OpenGL (e.g, Mesa), GLU, GLUT, and X11 (including Xi, XMU, ANDGLX).

$ sudo apt-get install Freeglut3-dev build-essential libx11-devlibxmu-dev libxi-dev LIBGL1-MESA-GLX Libglu1-mesa Libglu1-mesa-dev

10: The driver is finished, can return to GUI interface, also can continue to stay here to play text.

。

$ sudo start LIGHTDM

11:post-installationactions

(1) environment variable configuration

Run sudo vi/etc/profile, and add the following:

Exportpath=/usr/local/cuda-6.5/bin: $PATH

EXPORTLD_LIBRARY_PATH=/USR/LOCAL/CUDA-6.5/LIB64: $LD _library_path

(2) Verify the installation

A. Verify the driver version number, in fact, basically ensure that the driver is installed properly

$ cat/proc/driver/nvidia/version

B. Compiling the Examples

$ nvcc-v

The installation is complete here. will be able to compile. Switch folders to ~/nvidia_cuda-7.0_samples:

$ cd/home/username/nvidia_cuda-7.0_samples

$ make

It's going to be a very long time.

。

C. Running the Binaries

Execute compiled files, such as a look at the basic information and bandwidth information of the device:

$ CD Bin/x86_64/linux/release

$./devicequery

$./bandwidthtest

Suppose the test time appears to say that the execution version of the driver and the actual driver does not match, the reason may be due to the subsequent installation of the Nvidia-cuda-toolkit updated configuration files, and the original cuda-samples configuration or the driver has changed, so the detection can not be compiled through. Consider the following workaround:

A. Uninstalling an existing drive

$ sudo nvidia-installer--uninstall

B. Download the driver for the appropriate version number and install:

: Http://www.geforce.cn/drivers

$ sudo sh./nvidia-linux-x86_65*.run

C. Re-install Cuda Toolkit

$ sudo sh cuda-7.0

Nvidia Cuda installation ends (here I recommend not to manually install the appropriate nvidia driver.) It is best to uninstall the above process and reload it again. The version number may be incorrect due to manual installed. By the time the desktop blank question-----I was re-installed system ...

。

。 )

Three: Blas installation and Configuration

1:blas Installation

Here you can choose (ATLAS. Mkl or Openblas), I use MKL here, first download and install Intel? Math Kernel library for Linux * version mkl, the download link is: https://software.intel.com/en-us/intel-education-offerings. Please download the student version. Apply first, and then immediately receive an e-mail (there is a serial number installed), open according to download. After downloading, unzip the file to the home directory (or copy the tar.gz file directly to the home directory to save space. Remember to delete the zip file after installation, or any other Ext4 file system.

The next step is the installation process, authorizing and then installing:

$ tar zxvf parallel_studio_xe_2015.tar.gz (Let's say you copied the compressed file directly.)

$ chmod a+x parallel_studio_xe_2015-r

$ sudo./install_gui.sh

Environment settings for 2:MKL and Cuda

1. Create a new intel_mkl.conf and edit it:

$ sudo gedit/etc/ld.so.conf.d/intel_mkl.conf

/opt/intel/lib/intel64

/opt/intel/mkl/lib/intel64

2. Create a new cuda.conf and edit it:

$ sudo gedit/etc/ld.so.conf.d/cuda.conf

/usr/local/cuda/lib64

/lib

3. Complete the link operation of Lib file, run:

$ sudo ldconfig–v

Four OPENCV Installation

1. Download and compile OpenCV (original opencv:http://opencv.org/), or use the modified version of the installation package Install-opencv-master (the following installation method to use the package is complete, The installation package changed the dependencies.sh file and added the installation files for OpenCV 3.0.0, which retains the original 2.3x and 2.4x versions at the same time.

2. Switch to the directory where the file is saved, and then install the dependencies:

$ sudo sh ubuntu/dependencies.sh

3. Switch folder ubuntu\3.0\ install OPENCV3.0.0RC1:

$ sudo sh opencv3_0_0-rc1.sh

Ensure that the network is unblocked, because the software needs to be networked here for a long time. Please wait patiently.

。。

。 The time is very long!

V: Install additional dependencies

1. Google Logging Library (glog): https://code.google.com/p/google-glog/. Then unzip the installation:

$ tar zxvf glog-0.3.3.tar.gz

$./configure

$ make

$ sudo make install

Assuming that there is no permission on the chmod a+x glog-0.3.3-r, or simply chmod 777glog-0.3.3-r, after loading, this directory will be able to kill.

2. Other dependencies to ensure success

$ sudo apt-get install-y libprotobuf-dev libleveldb-devlibsnappy-dev libopencv-dev libboost-all-dev Libhdf5-serial-dev

$ sudo apt-get install-y libgflags-dev libgoogle-glog-devliblmdb-dev Protobuf-compiler

Six: Install Caffe and test

1. Install some of the dependencies that Pycaffe must have:

$ sudo apt-get install-y python-numpy python-scipypython-matplotlib python-sklearn python-skimage python-h5py Python-protobufpython-leveldb python-networkx python-nose Python-pandas python-gflags Cythonipython

$ sudo apt-get install-y protobuf-c-compilerprotobuf-compiler

2. Installation configuration Nvidia CuDNN accelerated Caffe model operation (we did not install, as Install.txt said to be for CUDA6.5)

A. Installing the CUDNN

The change version number caffe-master default support CUDNN-6.5-LINUX-X64-V2, use CUDNN-6.5-LINUX-R1 will error, before installation please go to the official website to download the latest CUDNN.

$ sudo cp cudnn.h/usr/local/include

$ sudo cp libcudnn.so/usr/local/lib

$ sudo cp libcudnn.so.6.5/usr/local/lib

$ sudo cp libcudnn.so.6.5.48/usr/local/lib

B. Link to a cudnn library file

$ sudo ln-sf/usr/local/lib/libcudnn.so.6.5.48/usr/local/lib/libcudnn.so.6.5

$ sudo ln-sf/usr/local/lib/libcudnn.so.6.5/usr/local/lib/libcudnn.so

$ sudo ldconfig–v

3: Switch to the Caffe-master directory. Generate the Makefile.config configuration file. Run:

$ CP Makefile.config.example Makefile.config

4: Configure the Makefile.config file (list only the changes section)

A. Enable Cudnn, remove "#" (now Caffe-master still only supports R1 version number)

USE_CUDNN: = 1

B. Enable the GPU and remove the gaze "#"

# cpu_only: = 1

C: Enable Intel Parallelstudio XE Professional Edition for C + + Linux

BLAS: = Mkl

5: Configure the Makefile file (for opencv3.x support)

Find the "Derive include and Lib Directories" section and change the last line of "LIBRARIES + =" To add Opencv_imgcodecs

Opencv_core Opencv_highgui opencv_imgproc opencv_imgcodecs

6: Compiling caffe-master!

。！

"-j8" is compiled using multi-core CPU, which can greatly accelerate the speed of compiling, it is recommended.

$ make All-j8

$ make Test-j8

$ make Runtest-j8

Recommendation: Before make. It is possible to test the drive with the previous./bandwidthtest and other commands.

Seven: Test with mnist data set

CAFFE By default will be installed in $caffe_root, that is, to extract to that folder. For example: $ home/username/caffe-master, so the following work. The working folder has been switched to by default. The following work is mainly used to test whether the Caffe is working properly. Do not make a specific assessment.

Specific settings Please refer to the Examiner Net: http://caffe.berkeleyvision.org/gathered/examples/mnist.html

1. Data preprocessing

$ sudo shdata/mnist/get_mnist.sh

2. Rebuild the Lmdb file.

Caffe supports three data format input networks, including image (. jpg,. png, etc.), leveldb. Lmdb Choose different inputs according to your needs.

$sudo shexamples/mnist/create_mnist.sh

Generate Mnist-train-lmdb and Mnist-train-lmdb directories, including data sets in Lmdb format

Note: When I run this command, I report that Not in Gzformat , later found to be networked download mnist There is a problem with the data. And then I went online. Download the mnist DataSet, then manually unzip it on Ubuntu , then run the command later.

3. Training Mnist

$ sudo shexamples/mnist/train_lenet.sh

At this point, all the steps of the Caffe installation are complete, and the following is a simple set of data controls. The experiment originates from the mnist data set, which mainly investigates the performance of CPU and GPU under different systems. Can see the obvious difference. Although the mnist datasets are very easy to believe, the difference is greater and UBUNTU+GPU is the only choice for complex datasets.

Test Platform 1 : I7-4770K/16G/GTX 770/cuda 6.5

MNIST Windows8.1 on CPU : 620s

MNIST Windows8.1 on GPU : 190s

MNIST Ubuntu 14.04 on CPU : 270s

MNIST Ubuntu 14.04 on GPU : 160s

MNIST Ubuntu 14.04 on Gpuwith CuDNN : 30s

cifar10_full on Gpuwihtout CuDNN : 73m45s = 4428s ( Iteration 70000 )

cifar10_full on GPU withcudnn : 20m7s = 1207s ( Iteration 70000 )

Test Platform 2 : Gigabyte p35x v3 , [Email protected]/16g/nvidiagtx 980 4G

MNIST Ubuntu 15.04 on Gpuwith CuDNN : 33s

Control Test 1 : 2*e5-2620 (12CPUs)/128g/tesla K20m/cuda5.5/centos 6.4

MNIST CentOS 6.4 on GPU : 294s

Control Test 2 : Tesla K40m/cuda6.5/ubuntu 14.04

MNIST on GPUs with CuDNN : 30s

Control Test 3 : GTX 660/cuda6.5/ubuntu 14.04

MNIST on GPUs with CuDNN : 49s

Control trial 1 is an unfair test, after all, the performance is very poor, very likely not only by the Tesla K20s and GTX 770 brought. It may also be due to the impact of CentOS or CUDA5.5 (without CuDNN), but the overall conclusion is consistent with reference performance numbers on the Caffe website. For ordinary users: the GTX cost-effective is much higher.

Control Test 2 demonstrates the powerful performance of Tesla K40, believing in complex images. It should have a stronger performance.

(Thanks to the test environment and test data provided by Ph.D Jingjing, Nanjing University Ph.D Jinlu, Huazhong University of Science and Technology MS Liumaolin, Hong Kong.) ）

Assuming you don't need to use Caffe in Python and MATLAB, you don't need to install Python and matlab

Eight: Python installation

1. Installing the IDE Execution Environment

Choose an IDE execution environment that works for you. I am using the Spyder, because it has built-in IPython environment, Caffe a lot of programs are based on IPython environment completed. The installation method is very easy and can be installed by searching the "Spyder" directly in Ubuntu Software Center.

2. IPython NoteBook Installation

Another recommended approach is to use the Ipyhthon NoteBook (browser-based Python IDE). Especially for teachers who need to do tutorials with Python. Ability to export. py,. IPYNB, HTML format directly. The installation process is as follows:

$ sudo apt-get install-y ipython-notebook pandoc

Start (Open your browser yourself):

$ Ipython Nootbook

A simple example of HTML generated using Ipython notebook: examples_notebook.html EXAMPLE_NOTEBOOK.IPYNB

3: Environment configuration

Configuring caffe support for Python interfaces in the Makefile.config file

Python_include: =/usr/include/python2.7 \

/usr/lib/python2.7/dist-packages/numpy/core/include

Python_lib: =/usr/local/lib

Include_dirs: = $ (python_include)/usr/local/include

Library_dirs: = $ (python_lib)/usr/local/lib/usr/lib

4: Run $ make pycaffe-j8 compile python in the Caffe-master folder

5: Increase in/etc/profile

Export Pythonpath=/home/hadoop/caffe/caffe-master/python: $PYTHONPATH

This makes it possible to import caffe in Python.

Nine: Matlab installation

MATLAB installation, I am not practical to. Also not installed. If you need to be able to find information on the internet!

References:

1: Official website http://caffe.berkeleyvision.org/installation.html

2:https://ouxinyu.github.io/blogs/20140723001.html Owang's Blog

3:http://blog.sciencenet.cn/blog-1583812-841855.htmlubuntu+cuda6.5+caffe Installation Configuration Rollup

4:http://www.haodaima.net/art/2823705caffe+ubuntu14.04+cuda6.5 Novice Installation Configuration Guide

5:http://weibo.com/p/2304189db078090102vdvx

Ubuntu14.10+cuda7.0+caffe Configuration