Caffe + Ubuntu 15.04 + CUDA 7.5 Novice Installation Configuration Guide

Caffe + Ubuntu 15.04 + CUDA 7.5 Novice Installation Configuration Guide

Special:

0. Caffe website address: http://caffe.berkeleyvision.org/

1. This article is for the author to complete the experiment, but only for the use of academic exchange, the use of this guide any adverse consequences of the user's own responsibility, not related to the author of this article, thank you! In order to ensure timely updates, reproduced please indicate the source, thank you!

2. This article is intended to provide a reference for beginners, please do not want to spit groove, violent tendencies, please bypass, thank you!

3. This article uses the Caffe-master version, which was downloaded on November 8, 2015, to run the platform: Ubuntu 15.04,cuda7.5,cudnn v4,intel Parallel Studio XE Cluster 2016,OPENCV 3.0.0 (originally ready to change Ubuntu 15.10 but has been an error, there is no careful toss, interested can try. )

4. Installation process, because of different platforms, different equipment, different operators, will encounter a variety of strange problems and error messages, please use Caffe official website of the issues and Caffe-user forum, as well as Google and Baidu. Refer to this guide, please download the latest version of Caffe-master, many of the new version of the file has been changed.

5. Last Updated: March 13, 2016. This update shows you how to install the latest version of Caffe-master in the most streamlined way, if you want to see the older version, please click on the link. Caffe + Ubuntu 15.04 + CUDA 7.0 Novice Installation Configuration Guide

PS: In order to facilitate everyone to use, I provide a Baidu cloud disk, used to share some of the installation process need to use the package and link address (all software packages for academic Exchange use only, please try to download the official website. ）。 Baidu Cloud Disk Link: http://pan.baidu.com/s/1qX1uFHa Password: Wysa

This installation guide, suitable for 0 basis, novice operation, please do not want to spit groove!

A brief introduction: Caffe, a convolutional neural network toolkit, is similar to Alex's cuda-convnet features, but each has its own characteristics. Is the use of C + + Cuda for the bottom-level editing, Python implementation, the original is mainly deployed in Ubuntu, but also the great God released the Windows version, but other relevant information is less, not suitable for novice use, so or Ubuntu is more suitable for beginners. Relatively

This article contains 5 parts, including:

• The first part of Linux installation
• Part II installation of NVidia CUDA Toolkit (*.deb method)
• Part three MATLAB installation and commissioning
• Part IV installation and testing of Caffe-master
• Part V Add a new layer (example of a roipooling layer presented by Fast RCNN)

The first part of Linux installation

Linux installation, if not Linux powder, just must, forced to use it for scientific research, recommended installation into a dual system, many online methods, here I do not elaborate, installation is also a fool-like, and windows, the process is similar to the language, if the difficulty is not big enough, Can be installed in e-version, or even Japanese, German ~ ~ ~, I am installed in Simplified Chinese version, I have a total of 500G space to install Ubuntu 14.04, this version is the latest version, there is a benefit is that Can directly access Windows8.1 NTFS partition, do not do extra work, and support Chinese, for example: $ cd/media/yourname/Partition name/folder name, of course GUI is more convenient.

My partition settings are as follows:

Root partition: \ 100G,

Swap partition: 128G, where the memory is set, it is said that less than 16G of memory, is set to 1.5-twice times the memory

Boot partition: 200M

Home partition: The remaining space, in view of Imagenet,pascal VOC, such as large customers, recommended 500G, at least 300G or more.

PS: Troubleshooting boot partition errors

Basically, reloading will destroy the original boot partition table and a simple way to restore the Windows partition:

$ sudo gedit etc/default/grub

Setting: Grub_default = 2

$ sudo update-grub

This method is suitable for installing dual system, "see Linux, do not see Windows" situation, in turn, please everyone own Baidu Bar.

PS: Special case for my notebook, reference for similar devices only

Notebook configuration: Gigabyte p35x v3,[email protected]/16g/nvidia GTX 980 4g/intel HD 4600/128g ssd*2 + 2T SATA

My two groups of HDD SSD and SATA RAID 0 respectively, the purpose is to merge logical partition, do not consider redundant backup problem, the final state is 2 logical hard disk block 256G SSD + 4T SATA, with the GPA partition, Finally, the Windows partition could not be found by using Ubuntu's Grub boot interface. So the above method fails. However, the F12 and BIOS settings can be used to achieve the choice of boot partition, and I have little chance of using Linux, so I have to deal with it, the Linux masters can toss their own grub boot. Ubuntu 15.04 is installed on the SATA logical partition, SSD partition installed Windows8.1.

Part II: Installation of NVidia CUDA Toolkit (*.deb method)

PS: Especially recommended *.deb method, currently available offline version of the Deb file, the method is relatively simple, do not need to switch to TTY mode, so no longer provide the original *.run installation method, here in Cuda 7.5 for example.

One, CUDA Repository

To obtain CUDA installation package, please go to NVIDIA official website to download the installation package yourself. (https://developer.nvidia.com/cuda-downloads)

$ sudo dpkg-i cuda-repo-ubuntu1504-7-5-local_7.5-18_amd64

$ sudo apt-get update

Two, CUDA Toolkit

$ sudo apt-get install-y cuda

The third part Matlab installs and debugs (takes Matlab 2014a as an example)

1. Download

As the software for commercial software, please look for yourself, install learning, and make sure not to use for commercial purposes, download 24 hours to delete ...

2. Pre-Preparation

Select Mathworks.Matlab.R2014a.Unix.iso-Right-use disk image mount to open, go to the mounted virtual disc, copy all files to the Home/matlab folder (PS: My principle is that GUI on GUI, Like the cmd can be referenced to execute)

Copy Crack/install.jar to home/matlab/java/jar/and overwrite source files

$ sudo cp install.jar/home/matlab/java/jar/

3. Licensing the installation folder

$ chmod a+x matlab-r

4. Installation

$ sudo./install

Option: Do not use the Internet installation

Serial Number: 12345-67890-12345-67890

Default path:/usr/local/matlab/r2014a

Tick create symbolic link from default boot path (implement MATLAB launcher in any location)

Activation file: License_405329_r2014a.lic

Copy libmwservices.so to/usr/local/matlab/r2014a/bin/glnxa64

$ sudo cp libmwservices.so/usr/local/matlab/r2014a/bin/glnxa64/

5. Resolve the compiler gcc/g++ version issue.

Because the gcc/g++ version of Ubuntu 15.04 is 4.9.2, and Matlab 2014a (2015a) version is 4.7.x so when using Matla call Mex file, basically will error, according to the error message, consider the following two-step solution.

A. Downgrade installation gcc/g++ version 4.7.x

(a). Download gcc/g++ 4.7.x

$ sudo apt-get install-y gcc-4.7

$ sudo apt-get install-y g++-4.7

(b). Link gcc/g++ Implementation demotion

$ cd/usr/bin

$ sudo rm gcc

$ sudo ln-s gcc-4.7 gcc

$ sudo rm g++

$ sudo ln-s g++-4.7 g++

B. Violence references new version glibcxx_3.4.20

$ sudo cp/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.20/usr/local/matlab/r2014a/sys/os/glnxa64/libstdc++. so.6.0.20 (libstdc++.so.6.0.20 version, may be different depending on the system, the use of the latest can be.) ）

Directory switch to/usr/local/matlab/r2014a/sys/os/glnxa64/, very important!

$ sudo mv libstdc++.so.6 libstdc++.so.6.backup (just a backup, can not be backed up, directly deleted).

$ sudo ln-s libstdc++.so.6.0.20 libstdc++.so.6

$ sudo ldconfig-v

by command "strings/usr/local/matlab/r2014a/sys/os/glnxa64/libstdc++.so.6 | grep glibcxx_ "Can see if it has successfully included the glibcxx_3.4.21, and if it already exists, it is basically successful."

6. Compiling the caffe file used in MATLAB (see part fifth)

Part V installation and testing of Caffe-master

For the installation of Caffe strictly comply with the official website requirements to: http://caffe.berkeleyvision.org/installation.html

First, install the Blas

Here you can choose (Atlas,mkl or Openblas), where I use MKL to first download and install the intel® Math kernel library for Linux * version mkl (Intel (R) Parallel Studio XE Cluster Edition for Linux 2016), download link is: https://software.intel.com/en-us/intel-education-offerings, use student identity (mail + school) Download student version, fill out all kinds of information, you can download directly, You will also be given an email to tell the serial number. After the download, to extract the files to the home folder (or directly to the tar.gz file to the home folder, in order to save space, remember to delete the compressed file after installation), or other Ext4 file system.

Many people say, download can not, I use this address on November 8, 2105 to download the latest version 2016, still no problem, please carefully operate ^_^.

The next step is the installation process, authorizing and then installing:

$ tar zxvf parallel_studio_xe_2016.tar.gz (if you are directly copying compressed files coming over)

$ chmod a+x parallel_studio_xe_2016-r

$ sh install_gui.sh

PS: When installing, it is recommended to install with root permission, and the root password of Linux will be required during the process. (Set Method: Command line: $ sudo passwd)

II. environment settings for MKL and Cuda

1. Create a new intel_mkl.conf and edit it:

$ sudo gedit/etc/ld.so.conf.d/intel_mkl.conf

/opt/intel/lib/intel64

/opt/intel/mkl/lib/intel64

2. Create a new cuda.conf and edit it:

$ sudo gedit/etc/ld.so.conf.d/cuda.conf

/usr/local/cuda/lib64

/lib

3. Complete the link operation of Lib file, execute:

$ sudo ldconfig-v

Third, install OpenCV 3.0.0

1. Download and compile OpenCV (original opencv:http://opencv.org/), or use the modified version of the installation package I provided (the previous Baidu cloud download) (The following installation method to use the package to complete, The installation package modified the dependencies.sh file and added the installation files for OpenCV 3.0.0)

2. Switch to the folder where you saved the file, and then install the dependencies:

$ sudo sh ubuntu/dependencies.sh

3. Switch directory Ubuntu\3.0\ install OPENCV 3.0.0:

$ sudo sh opencv3_0_0.sh

Ensure the network is unblocked, because the software needs to be networked here for a long time, please wait patiently ... ，

Iv. Installing Additional Dependencies

1. Google Logging Library (glog),: https://code.google.com/p/google-glog/, then unzip the installation:

$ tar zxvf glog-0.3.3.tar.gz

$./configure

$ make

$ sudo make install

If you do not have permission to chmod a+x glog-0.3.3-r, or simply chmod 777 Glog-0.3.3-r, after the installation, this folder can kill.

2. Other dependencies to ensure success

$ sudo apt-get install-y libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev Libhdf5-serial-dev Libgflags-dev libgoogle-glog-dev Liblmdb-dev Protobuf-compiler protobuf-c-compiler Protobuf-compiler

V. Install Caffe and test

1. Install some of the dependencies that Pycaffe must have:

$ sudo apt-get install-y python-numpy python-scipy python-matplotlib python-sklearn python-skimage python-h5py PYTHON-PR Otobuf python-leveldb python-networkx python-nose python-pandas python-gflags Cython Ipython

2. Installation configuration Nvidia CuDNN accelerated Caffe model operation

A. Before installing, please download the latest CUDNN (Cudnn-7.0-linux-x64-v4.0-prod) on the website first.

$ sudo cp include/cudnn.h/usr/local/include

$ sudo cp lib64/libcudnn.*/usr/local/lib

B. Link to a cudnn library file

$ sudo ln-sf/usr/local/lib/libcudnn.so.4.0.7/usr/local/lib/libcudnn.so.4

$ sudo ln-sf/usr/local/lib/libcudnn.so.4/usr/local/lib/libcudnn.so

$ sudo ldconfig-v

3. Switch to the Caffe-master folder, generate the Makefile.config configuration file, and execute:

$ CP Makefile.config.example Makefile.config

4. Configure the Makefile.config file (only the modified parts are listed)

A. Enable Cudnn, remove "#"

USE_CUDNN: = 1

B. Configure some reference files (the additional part is mainly to solve the problem of the HDF5 path under the new version)

Include_dirs: = $ (python_include)/usr/local/include/usr/lib/x86_64-linux-gnu/hdf5/serial/include

Library_dirs: = $ (python_lib)/usr/local/lib/usr/lib/usr/lib/x86_64-linux-gnu/hdf5/serial

C. Enable Intel Parallel Studio XE 2016

BLAS: = Mkl

D. Configuring the path for Caffe support for Python and MATLAB interfaces

Python_lib: =/usr/local/lib

Matlab_dir: =/usr/local/matlab/r2014a

C. Enable OPENCV 3.0, remove "#"

Opencv_version =3

6. Compiling caffe-master!!! "-j16" is compiled using multi-core CPU, which can greatly accelerate the speed of compiling, it is recommended.

$ make All-j16

$ make Test-j16

$ make Runtest-j16

Compile the caffe files used by Python and matlab

$ make Pycaffe-j16

$ make Matcaffe-j16

Vi. testing with the Mnist data set

CAFFE By default will be installed in $caffe_root, is to extract to that directory, for example: $ home/username/caffe-master, so the following work, the default has been switched to the working directory. The following work is primarily to test whether the Caffe is working properly and does not perform a detailed evaluation. For specific settings, please refer to the official website: http://caffe.berkeleyvision.org/gathered/examples/mnist.html

1. Data preprocessing

$ sh data/mnist/get_mnist.sh

2. Rebuild the Lmdb file. Caffe supports a variety of data format input networks, including image (. jpg,. png, etc.), leveldb,lmdb,hdf5 and so on, according to their own needs to choose different input bar.

$ sh examples/mnist/create_mnist.sh

Generate Mnist-train-lmdb and Mnist-train-lmdb folders, which contain data sets in Lmdb format

3. Training Mnist

$ sh examples/mnist/train_lenet.sh

At this point, the Caffe installation of all the steps to the end, the following is a simple set of data comparison, experiment from the mnist data set, mainly to examine the performance of the CPU and GPU under different systems. Can see the obvious difference, although the Mnist dataset is very simple, believe that the complex data set, the difference will be greater, UBUNTU+GPU is the only choice.

Test Platform 1:I7-4770K/16G/GTX 770/cuda 6.5

MNIST Windows8.1 on cpu:620s

MNIST Windows8.1 on gpu:190s

MNIST Ubuntu 14.04 on cpu:270s

MNIST Ubuntu 14.04 on gpu:160s

MNIST Ubuntu 14.04 on GPUs with cudnn:30s

Cifar10_full on GPU wihtout cudnn:73m45s = 4428s (iteration 70000)

Cifar10_full on GPU with cudnn:20m7s = 1207s (iteration 70000)

Test Platform 2: Gigabyte p35x v3,[email protected]/16g/nvidia GTX 980 8G

MNIST Ubuntu 15.04 on GPUs with cudnn:33s

Test platform 3:dell 7910,e5 2623v3 3.0G *2/128g/nvidia Titan X 12G

MNIST Ubuntu 15.04 on GPUs with cudnn:23s (what a real inverse!) ）

Contrast test 1:2*e5-2620 (12CPUs)/128g/tesla K20m/cuda5.5/centos 6.4

MNIST CentOS 6.4 on gpu:294s

Contrast test 2:tesla K40m/cuda6.5/ubuntu 14.04

MNIST on GPUs with cudnn:30s

Contrast test 3:GTX 660/cuda6.5/ubuntu 14.04

MNIST on GPUs with cudnn:49s

Comparison Test 1 is an unfair test, after all, the performance is very poor, it is likely not only by Tesla K20s and GTX 770, may be due to the impact of CentOS or CUDA5.5 (without CuDNN), but the overall conclusion and Caffe official Reference performance numbers consistent, for ordinary users: GTX cost-effective much higher. Contrast Test 2 demonstrates the power of Tesla K40 and believes it should have a stronger performance for complex images. (thank the City University of Hong Kong Ph.D Jingjing, Nanjing University Ph.D Jinlu, Huazhong University of Science and Technology MS Liumaolin to provide test environment and test data. ）

Return

Copyright©xin-yu OU (Owang) | All Rights RESERVEDWebmaster Statistics | Today ip[145] | Today pv[288] | yesterday ip[151] | yesterday pv[317] | currently online [4]

Caffe + Ubuntu 15.04 + CUDA 7.5 Novice Installation Configuration Guide