Ubuntu14.04+cuda7.5+anaconda2-4.0.0+matlab2014b+caffe from Scratch

Ubuntu14.04+cuda7.5+anaconda2-4.0.0+matlab2014b+caffe from scratch Preface

Took three days, from the installation of dual systems to install Caffe and compiled successfully, during the process of countless pits, visited dozens of csdn and blog Park and the Linux Community Technology blog, the ups and downs I am afraid only experienced can experience. Because of the excessive number of blogs, this is not listed here, I would like to thank them!!!

Note: This blog is intended for Cuda-enabled scenarios where all of the following configurations are based on GPU support Cuda.

Hardware configuration:

Notebook: Dell 7447
Graphics: NVIDIA GTX 850M
Memory: 4G
System: win8.1,ubuntu14.04.1 LTS (64-bit)
Memory: 8G

• Ubuntu1404cuda75anaconda2-400matlab2014bcaffe from Scratch
  • Objective
  • Hardware configuration
  • Install Ubuntu under Win81
  • Cuda and CUDNN Installation
  • Installation of Anaconda and MATLAB
  • OPENCV Installation
  • Openblas Installation
  • Caffe Compiling and testing

Install Ubuntu under win8.1

My notebook support uefi boot, if the motherboard does not support UEFI boot, please Baidu or Google.

Pre-Preparation: U disk one, the best capacity is not less than 2g,fat32 format, ubuntu14.04.1 system image (64-bit)

1. Remove an area from your hard drive for Ubuntu
Right-click on "My Computer", select "Properties", click "Disk Management", from the rest of the space to partition a lot of space, specific actions right-click on the corresponding partition, select "Compressed volume" in the Popup dialog box to enter the size of the space to be compressed, here to note:
The compressed partition must be contiguous, so that it can be merged together, but in fact, the compressed space from multiple partitions can not be merged, so when the largest partition remaining space can not meet the needs, can only adjust the partition size, the use of the Partition assistant adjustment, and then the last partition to partition as large as possible, Then compression, it is recommended to compress at least 200G, I allocated 300G.

2, the data in the U disk backup, because in the production system disk will be formatted, download Universal-usb-installer, search engine, there is no post link, please download and install yourself. After installation, plug in the USB stick, open the software, click "I Agree", in step 1 select Ubuntu, step 2 Select the location of the system image and select it, step 3 Select the USB drive system disk, select "Createl", and then wait for the write to complete. The system disk has been successfully created and ejected.
In the selection of the system also stepped on a few pits, I have tried 14.04,14.04.1,14.04.2,14.04.3,14.04.4,15.04 six systems, of which 14.04,14.04.2~14.04.4 installed Cuda drive, restart All Black screen, Installation 15.04 can be successfully restarted, but not the Internet, so the final choice 14.04.1 this version, but this should not be a bug, may be a computer problem, readers can try other versions.

3, close the Quick start, the specific operation please Baidu itself. Open settings, select Update and Recovery-recovery-advanced boot-Restart now-troubleshooting-Advanced Options-uefi firmware Settings-reboot, enter BIOS interface, switch to boot tab, turn off secure boot disabled, save exit. Plug in the U disk, and press and hold the F12 key to enter first setup, different boards may have different keys, please replace the keys themselves, if the failure can be restarted and try again. Select the USB stick to boot, select try without installing, enter the desktop and double-click Install ubuntu14.04 to start the installation.
Some notes:
Installation method Be sure to select something else, and then partition and allocate the free space to its own size. Click "+" to create a new partition, because the GPT disk has no limit on the number of primary partitions, so the primary partition and logical partition when the partition has no effect, the first partition select EFI boot portion, allocate 100-300m space, I assign 200M here, and then create "/" Partition, I assigned about 140G, and then create "/home" partition, I allocated about 150G, in fact, I do not allocate very reasonable here, "/" do not allocate so large, more space should be left to "/home" partition, finally create swap partition, if memory is small, It is recommended to allocate 1.5 times times the size of the memory, it is important to note that when allocating swap space, you should choose end of space to make full use of it. When you're done, click the drop-down menu to select the first EFI boot portion you just created, so don't make a mistake, and there's nothing special to pay attention to next. After the installation is complete reboot, unplug the USB flash drive, when there are four startup items, the first is to start Ubuntu, the third is to start the win8.1. Choose to start Ubuntu, if you can start successfully, the system installation even if it succeeds.

Cuda and CUDNN Installation

1, before the next step, please configure the network to enable the system to the Internet.
2, before downloading the installation package, please confirm whether the computer's video card supports CUDA7.5, if supported please look down, the method is:
In terminal (ctrl+alt+t) input Lspci | Grep-i nvidia view GPU model, if your GPU model appears in the Http://developer.nvidia.com/cuda-gpus list, then congratulations, please look down.
3, because the installation package is large, choose offline installation, to enter the CUDA official website, download ubuntu14.04 corresponding version of the Deb package, registration application Download cuDNN-7.0 v4, the default download path is/home/user/downloads, where user for their own created username Same below
4. After the download is complete, go to the path where the package is installed
CD ~/downloads
Perform a MD5 check to ensure that the downloaded installation package is complete and correct, then enter:
sudo dpkg-i cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
sudo apt-get update
sudo apt-get install Cuda
The name of the installation package can be automatically completed by pressing the TAB key after entering sudo dpkg-i cuda.
5. Configure Environment variables
5.1 First open the profile file with sudo gedit/etc/profile and add the following two paths at the end
Export Path=/usr/local/cuda-7.5/bin: $PATH

Export ld_library_path=/usr/local/cuda-7.5/lib64: $LD _library_path

Exit after saving, terminal input source/etc/profile load path.
5.2 Set the dynamic link library path:
Add the file cuda.conf in/etc/ld.so.conf.d/first:
$ sudo gedit/etc/ld.so.conf.d/cuda.conf

Add the following path within the file
/usr/local/cuda/lib64
Exit after saving, terminal input sudo ldconfig make it effective immediately.
Restart the system!!!
6, if you can successfully enter the desktop, the success is not far away, and then some verification operations.
6.1. Install the Cuda Samples with write permission
In fact, samples is already in the/USR/LOCAL/CUDA-7.5/, but the permission is read-only, the installation here is actually from the above path copy one copy to the path with read and write permission to compile:
$ cuda-install-samples-7.5.sh/home/user

6.2. Verifying the driver version

6.3. Review the version of the NVCC compiler to verify that the Cudatoolkit is successful.
$ nvcc-v

Sample code to compile Cuda:
$ CD ~/nvidia_cuda-7.5_samples

$ sudo make

$ CD ~/nvidia_cuda-7.5_samples/1_utilities/devicequery

$./devicequery

Because I am a dual graphics notebook, so it involves the issue of switching video card, can enter the/usr/share/applications
Double-click the nvidia X Server Settings to switch the video card through the prime profiles, for ease of use, you can copy this icon to the desktop, or you can install a nvidia-prime indicator on your own, with the following commands:
$ sudo add-apt-repository ppa:nilarimogard/webupd8

$ sudo apt-get install Prime-indicator

After rebooting the system, you can see the Nvidia or Intel icon in the upper right tray area and click to switch.
7 Installing CUDNN v4
Go to the file download directory and unzip cudnn-7.0-linux-x64-v4.0-prod.tgz:
$ CD ~/downloads

(ZXVF four-letter exchangeable order)
After decompression in this folder will generate a Cuda folder, into the folder, the header file Cudnn.h added to the system can find the Include folder, the Lib library file to the system can find the Lib folder:
$ sudo cp cuda/include/cudnn.h/usr/local/cuda/include

$ sudo cp-a cuda/lib64/libcudnn*/usr/local/cuda/lib64
At this point, Cuda and CUDNN installation completed, for the convenience of use, or install a sogou Chinese Input method bar.

Installation of Anaconda and MATLAB

1, enter the official website, download the corresponding. SH installation file, I downloaded the anaconda2-4.0.0-linux-x86_64.sh, the terminal into the download folder for installation:
$ CD ~/downloads

$ bash anaconda2-4.0.0-linux-x86_64.sh

Always press the ENTER key to enter Yes, and the software will be installed by default in the user directory. Note at the end of the installation will ask you to add the file to the environment variable, be sure to choose Yes, the default is no, do not shake the default past, otherwise you have to add yourself.
Add Anaconda Library Path
$ sudo gedit/etc/ld.so.conf

Add/home/user/anaconda2/lib, save and exit to make it effective immediately:
$ sudo ldconfig

2. Installing MATLAB
Now the latest Caffe only support to matlab2015a, as far as possible to choose a new version because the Openblas required to install the GCC version can not be less than 4.7, I chose the matlab2014b, because there are different ways to crack, here is not detailed installation process. Faith in the principle of respect for copyright, can buy a genuine buy a genuine bar, cracked version only for non-commercial use, remember!
After the installation is complete, don't forget to add the environment variable:
$ sudo gedit/etc/profile

Add at the end
Export Path=/usr/local/matlab/r2014b/bin: $PATH

Alias matlab= ' Nohup matlab-desktop & '
Exit after saving, load environment variable
$ source/etc/profile
In this case, if you enter MATLAB directly in the terminal or enter the/usr/local/matlab/r2014b/bin directory matlab may appear matlab:command not found, the workaround:
Terminal into the/usr/local/bin directory, enter:
sudo ln-s/usr/local/matlab/r2014b/bin/matlab matlab, at this time in any directory input matlab can start Matlab, have to say that Ubuntu started the speed of matlab seconds to kill Windows.
Well, here, MATLAB is installed successfully.

OPENCV Installation

With automatic installation
Download the source package from Https://github.com/jayrambhia/Install-OpenCV, unzip it, go to Ubuntu folder and install:
$ CD Ubuntu

$ chmod +x *

$./opencv_latest.sh

The default installation in the original path, in order to classify the file, you can create a new OpenCV folder in the/home directory, and then extract the source package to the folder, the latest version is 2.4.13, add and load environment variables after installation is complete:
Set the path to use for dynamic linking
$ sudo gedit/etc/ld.so.conf.d/opnecv.conf

Add inside the opnecv.conf
/usr/local/lib
dir/opencv-2.4.13/build/lib/
#其中dir为你opencv的实际安装路径
Re-execute
$ sudo ldconfig

Finally add a required to environment variable
$ sudo gedit/etc/bash.bashrc

Add content at the end
Export Pkg_config_path= $PKG _config_path:/usr/local/lib/pkgconfig

and source a little bit.
$ SOURCE/ETC/BASH.BASHRC

At this point, the OPENCV installation is complete. Test to see if the installation was successful:
Enter the SAMPLE/C directory under the OpenCV folder
$ CD SAMPLES/C

$./build_all.sh
If it is installed correctly, it will be compiled

$./facedetect lena.jpg

At this point, the OPENCV installation is successful.

Openblas Installation

Here also encountered a few pits, novice recommended installation Ablas:
$ sudo apt-get install Liblapack-dev

$ sudo apt-get install Libatlas-base-dev

$ sudo apt-get install Libatlas-dev

Intel MKL can also be installed, but application and download is also a hassle, so focus on Openblas installation.
1. Download the source package from GitHub and unzip it
$ git clone git://github.com/xianyi/openblas

A Openblas folder is generated in the/home directory and Openblas all related files are extracted here.
2. Open folder:
$ CD ~/openblas

3. Compile and install
$ make Fc=gfortran #如果没有安装gfortran, execute sudo apt-get install Gfortran

4. Execute the following command to complete the installation
$ ln-s/opt/openblas/lib/libopenblas.so/usr/lib/libblas.so.3

$ ln-s/opt/openblas/lib/liblapack.so.3/usr/lib/liblapack.so.3\

5. Add Environment variables
5.1 Creating a dynamic-link library
$ sudo gedit/etc/ld.so.conf
Join:/opt/openblas/lib, exit after saving.
Load to make it effective immediately:
Be sure to add the/include directory to the environment variable, or the subsequent compilation Caffe will say that the cblas.h file cannot be found, but in fact there are cblas.h files in the Include directory.
In Terminal input:
cplus_include_path=/home/user/openblas/include/
Export Cplus_include_path
At this point, Openblas installation is complete.

Caffe Compiling and testing

Well, finally into the final point of the topic, talk not much, open dry!
First, download the source package and unzip it to/home/caffe
Installing dependent libraries
According to the official website:
$ sudo apt-get install Libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev Protobuf-compiler

$ sudo apt-get install–no-install-recommends Libboost-all-dev

After the installation is complete, the Python dependent package is installed, since Anaconda already has a lot of common packages, so do not have to install themselves, first into the/caffe/python/ View the Requirement.txt file, which lists the required dependencies and version requirements, and then in the terminal input Conda lilst to view the installed packages, found that all the required packages have been installed, only python-dateutil This package version is too high to meet the requirements, Go to official website https://pypi.python.org/pypi/python-dateutil/Download a 1.5 version of the package, unzip to annconda2/pkgs, enter the directory, execute:
$ python setup.py Install

Installation is complete.
To add an environment variable:
$ sudo gedit ~/.BASHRC

Add at the end:
Export Pythonpath=/home/xm/caffe/python: $PYTHONPATH

Export Matlabpath=/home/xm/caffe/matlab: $MATLABPATH

After saving, exit, source a bit:
$ source ~/.BASHRC

Before compiling, modify the makefile:
$ CD ~/caffe

$ CP Makefile.config.example Makefile.config

Open the Makefile.config file, modify some of the configuration, mainly whether to use the Cudnn,blas library, Opencv,anaconda some paths, and the path of MATLAB: Modify the following:
USE_CUDNN: = 1

Cuda_dir: =/usr/local/cuda

BLAS: = Open

Blas_include: =/opt/openblas/include
Blas_lib: =/opt/openblas/lib

Matlab_dir: =/usr/local/matlab/r2014b

Anaconda_home: = $ (HOME)/anaconda2

Python_include: = $ (anaconda_home)/include \

     $(ANACONDA_HOME)/include/python2.7      

Python_lib: = $ (anaconda_home)/lib

With_python_layer: = 1

Include_dirs: = $ (python_include)/usr/local/include

Library_dirs: = $ (python_lib)/usr/local/lib/usr/lib

Note: The full PDF version has been uploaded to the resource because the editor does not list the full contents of the modification.

Save it, go to the final build, clap your own drums, and then pray don't make mistakes:)
$ make All-j2

$ make Test-j2

$ make Runtest-j2

The number behind-J is the number of threads that can run concurrently, and the number of CPU cores recommended by the official website.
Compile during the row, the complement, in fact, according to the above steps step by step is no problem (pits are my hard digging added: ().
After compiling the above error, compile Python wrapper:
$ make Pycaffe

When you are done, open Python to see if you can import Caffe successfully:
$ python
Import Caffe

If the error does not indicate that the compilation was successful, then compile MATLAB wrpper:
$ make Matcaffe

OK, done, good excited there are wood!!!
In the final step, add the environment variables:
$ sudo gedit/etc/profile

Add at the end:
Export Caffe_home=/home/xm/caffe
After saving, exit and load:
$ source/etc/profile

Can run a handwritten digital recognition to experience:
$ CD ~/caffe

1. Data preprocessing
$ sudo sh./data/mnist/get_mnist.sh

2, rebuild LDB file, is to process the binary data set for caffe recognition of the data set, all future data, including JPE files to be processed into this format
$ sudo sh./examples/mnist/create_mnist.sh

Generate the mnist-train-leveldb/and mnist-test-leveldb/folders, which contain data sets in LDB format.
3. Training Mnist
$ sudo sh./examples/mnist/train_lenet.sh
Wait a few seconds and find the iteration 1000 times, as well as the output data.

Ubuntu14.04+cuda7.5+anaconda2-4.0.0+matlab2014b+caffe from Scratch