How to configure Caffe in CUDA7.5 in Ubuntu 16.04

How to configure Caffe in CUDA7.5 in Ubuntu 16.04

Due to the recent installation of Ubuntu 16.04, the previous tutorials for configuring Caffe are all about version 14.04, so I had to find it myself, and the configuration was successful. This tutorial does not require you to downgrade the gcc version. After all, cuda7.5 does not support gcc5 or above (not supported by default, actually supported) to avoid a series of messy problems, I met it before.

This article is based on the tutorials on the caffe official website and the experience I have summarized. Thank you for this.

Install Nvidia CUDA7.5 on Ubuntu 14.04 and build the Python Theano Deep Learning Development Environment

Installation Process of CUDA (including GPU driver) in Ubuntu

Caffe + Ubuntu 14.04 64bit + CUDA 6.5 configuration instructions

Install and configure CUDA in Ubuntu 14.04

Install NVIDIA CUDA5.5 in Ubuntu 12.04

Install Theano + CUDA on Ubuntu

For Ubuntu 12.04 CUDA5.5 installation, see the following link Ubuntu 12.04 installation CUDA-5.5

1. download the required file
1.1 Ubuntu16.04 was downloaded from the official website and then made using UltraISO in windows. There is a large article to search for. I will not go into details here.
1.2 download cuda7.5. The downloaded version is the run file of ubuntu15.04, which is convenient for you.
1.3 download cudnn4.0. If you have registered an nvidia account, click download. Otherwise, you need to register an account. After the registration, you can select a few hooks, the next step is to accept the terms and then download them.
1.4 download caffe from the official github.
2. Installation of the video card driver
2.1 The first method is to directly set, update, and software in the ubuntu system. Select the Chinese Server Source to refresh and click the additional driver option, select 361.42 at Nvidia Corporation (the most up-to-date version must be installed for obsessive-compulsive disorder), click application changes, download and install them, and then restart.
2.2 The second method is to download the driver's run file from the official website and select the corresponding video card model for download. Then shut down and plug the monitor into the integrated graphics interface, or under the terminal
Sudo gedit/etc/modprobe. d/blacklist. conf
Enter the password and edit it in the last line.
Blacklist nouveau
Ctrl + C save and enter the terminal
Sudo update-initramfs-u
After the restart, press Ctrl + Alt + F2 on the interface, enter the root account and password, and then
Service lightdm stop
Sh the complete path of your own driver file. The default options are available for installation. Restart after installation.

3. Install Cuda7.5
3.1 take the file name cuda. run as an example. Enter
Sh cuda. run -- override start the installation program. There are a lot of terms here, one by one space to the end, enter accept, enter y in turn, then n (do not install the graphics card driver), then one by one y press enter, you need to enter the password in one place, and check the installation path in two other places. Press enter to complete the installation. The default installation path is/usr/local.
Unzip the downloaded cudnn-7.0-linux-x64-v4.0-prod.tgz, unzip the cuda folder first open the include FOLDER inside, right-click in the terminal open input:
Sudo cp cudnn. h/usr/local/cuda/include/
Cd ~ /Cuda/lib64
Sudo cp lib */usr/local/cuda/lib64/
Continue to update File Link

Cd/usr/local/cuda/lib64/
Sudo rm-rf libcudnn. so libcudnn. so.4
Sudo ln-s libcudnn. so.4.0.7 libcudnn. so.4
Sudo ln-s libcudnn. so.4 libcudnn. so
Then set the environment variable
Sudo gedit/etc/profile
Add at the end
PATH =/usr/local/cuda/bin: $ PATH
Export PATH
Save and create a link file
Sudo vim/etc/ld. so. conf. d/cuda. conf
Press I on the keyboard to enter the editing status and add text
/Usr/local/cuda/lib64
Then Press esc and enter: wq to save and exit.
Enter
Sudo ldconfig makes the link take effect

4. Generate a Cuda Sample Test
First, install all the required dependency packages before making caffe.
Sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
Sudo apt-get install -- no-install-recommends libboost-all-dev
Sudo apt-get install libatlas-base-dev
Sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev
Then start make samples.
Cd/home/gomei/NVIDIA_CUDA-7.5_Samples
Sudo make all-j4
I use j4 as a 4-core computer. Under normal circumstances, the error "unsupported GNU version! Gcc versions later than 4.9 are not supported !", The reason is that this cuda does not support gcc5.0 or above and runs on the terminal.
Cd/usr/local/cuda-7.5/include
Cp host_config.h host_config.h.bak
Sudo gedit host_config.h
Ctrl + F look for a place with "4.9", there should be only one place, above it
# If _ GNUC _> 4 | (_ GNUC _ = 4 & _ GNUC_MINOR _> 9) change two 4 to 5, save and exit, continue
Cd/home/gomei/NVIDIA_CUDA-7.5_Samples
Sudo make all-j4
This should start to make. Here it takes about 5 or 6 minutes. After completion
Cd/home/gomee/NVIDIA_CUDA-7.5_Samples/bin/x86_64/linux
./DeviceQuery
The following information appears:
CUDA Device Query (Runtime API) version (CUDART static linking)
Detected 1 CUDA Capable device (s)
Device 0: "GeForce GTX 750 Ti"
CUDA Driver Version/Runtime Version 8.0/7.5
CUDA Capability Major/Minor version number: 5.0
Total amount of global memory: 2047 MBytes (2146762752 bytes)
(5) Multiprocessors, (128) CUDA Cores/MP: 640 CUDA Cores
GPU Max Clock rate: 1228 MHz (1.23 GHz)
Memory Clock rate: 3004 Mhz
Memory Bus Width: 128-bit
L2 Cache Size: 2097152 bytes
Maximum Texture Dimension Size (x, y, z) 1D = (65536), 2D = (65536,655 36), 3D = (4096,409 6, 4096)
Maximum Layered 1D Texture Size, (num) layers 1D = (16384), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D = (16384,163 84), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x, y, z): (1024,102 4, 64)
Max dimension size of a grid size (x, y, z): (2147483647,655 35, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 1 copy engine (s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Disabled
Device supports Unified Addressing (ultraviolet A): Yes
Device PCI Domain ID/Bus ID/location ID: 0/1/0
Compute Mode:
<Default (multiple host threads can use: cudaSetDevice () with device simultaneously)>
DeviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 8.0, CUDA Runtime Version = 7.5, NumDevs = 1, Device0 = GeForce GTX 750 Ti
Result = PASS
This indicates that the operation is successful.

5. Python Configuration
Decompress the caffe compressed file downloaded from github to any directory, and then install python
There are two ways to install python:
First, install anaconda directly, download it from the official website, and select linux 64bit 2.7 for download and installation. anaconda is easy to install, but you need to change the python inclusion path in the final make configuration file.
The second method is to use the native python2.7 version.
Sudo apt-get install python-pip install pip
Here, we use pip to install some dependent packages required by python. To avoid various problems, we can also install them through apt-get. I have installed both methods (-. -)
Sudo apt-get install python-numpy python-scipy python-matplotlib ipython-notebook python-pandas python-sympy python-nose
Use caffe to decompress the package to/home/user (your user name)/by default. The folder name caffe is used as an example.
Cd/home/user/caffe/python
Sudo su
For req in $ (cat requirements.txt); do pip install $ req; done
It may be slow to install pip. It may take several hours to download it. We recommend that you use the pip source of Tsinghua University for temporary installation. Therefore, the command should be changed to the following:
For req in $ (cat requirements.txt); do pip install-I https://pypi.tuna.tsinghua.edu.cn/simple $ req; done
If there are many red-letter errors for the first time, it is recommended that you run the instructions several times before the installation is successful. You do not need to pay attention to the yellow text prompts. It may be that the pip version needs to be updated.

6. Caffe compilation process
Next we will go to the final step.
Cd/home/user/caffe
Cp Makefile. config. example Makefile. config
Gedit Makefile. config
Uncomment USE_CUDNN: = 1
INCLUDE_DIRS: = $ (PYTHON_INCLUDE) add a space after/usr/local/include and add/usr/include/hdf5/serial. If this sentence does not exist, an error indicating that hdf5.h cannot be found may be reported.
PYTHON_INCLUDE: =/usr/include/python2.7 \
/Usr/lib/python2.7/dist-packages/numpy/core/include are not modified first.
If you need to generate caffe wrapper of matlab, uncomment it and replace it with your own directory.
Let's talk about the problems that may occur in advance:
First, the error such as string. h 'memcy 'was not declared in this scope occurs during the make process because the gcc compiler version is too new. The solution is to enable makefile search and replace
NVCCFLAGS + =-ccbin = $ (CXX)-Xcompiler-fPIC $ (COMMON_FLAGS)
Is
NVCCFLAGS + =-D_FORCE_INLINES-ccbin = $ (CXX)-Xcompiler-fPIC $ (COMMON_FLAGS)
Save and exit
Second, during the make process, we will also report a connection problem where ld cannot find libhdf5 and libhdf5_hl. This may also be caused by hdf5, first, check whether there are libhdf5.so and libhdf5_hl.so in the/usr/lib/x86_64-linux-gnu directory. If so, check whether the property has a correct Link (normally there should be no these two files ), right-click and choose terminal.
Sudo ln libhdf5_serial.so.10.1.0 libhdf5.so
Sudo ln libhdf5_serial_hl.so.10.0.2 libhdf5_hl.so
Note that 10.1.0 and 10.0.2 may be installed on different computers. Check the files in the current directory and
Sudo ldconfig takes effect
Next is the process of directly compiling.
Cd/home/user/caffe
Make all-j4
Make test-j4
Make runtest
Make pycaffe
Make matcaffe
If no error is reported during compilation, the problem will be solved. Open python
Cd/home/user/caffe/python
Python
Import caffe
If no error is reported, the compilation is successful.
Test matlab to open./caffe/matlab/+ caffe/private and check whether a caffe mex file is generated. You can run the program test in the + test folder.
Minor issues:
When using the python interface, what error may be reported (I forgot -!), By the way, the error is 'mean shape incompatible with input shape. '. The processing method is the python/caffe folder. Edit the io. py file and set
If MS! = Self. inputs [in _] [1:]:
Raise ValueError ('mean shape incompatible with input shape .')
Replace
If MS! = Self. inputs [in _] [1:]:
Print (self. inputs [in _])
In_shape = self. inputs [in _] [1:]
M_min, m_max = mean. min (), mean. max ()
Normal_mean = (mean-m_min)/(m_max-m_min)
Mean = resize_image (normal_mean.transpose (1, 2, 0), in_shape [1:]). transpose (2, 0, 1) * (m_max-m_min) + m_min

Then make clean and then make again

7. Summary
Now, the course for compiling Caffe under Ubuntu16.04 is over. The author has installed the system several times in three days and started to use Ubuntu14.04. As a result, the system is faulty and the obsessive-compulsive disorder cannot be used, I tried to dress 16.04 and continue to make the final effort. In the future, the compilation tutorial under python3 may be updated. You need to compile the boost version by yourself, which is also very troublesome.