Caffe Study notes (i)

Caffe is an open source framework for deep learning, developed by Jiayanqing at Berkeley University, and is now maintained by BVLC, the Berkeley Vision and Learning Center, at Berkeley's Visual Learning Centre.
—————————————————————
Before introducing Caffe, we will briefly introduce the next Jiayanqing. http://daggerfs.com/

Jiayanqing is currently working on computer vision at Google, a UC Berkeley, and an undergraduate and master's degree in Tsinghua University. During the study, he worked at the National University of Singapore, Microsoft Research Asia, the NEC laboratory in the United States and the Google Institute.
————————————————————
Caffe's core program is implemented in C + + and provides the interface between Python and matlab, meaning you can invoke Caffe trained models in Python and MATLAB. And, Caffe supports seamless switching between CPU and GPU, which is especially important in today's big data age. At present, most of the people engaged in deep learning are running programs on the GPU, a imagenet dataset has tens of millions of pictures, plus deep learning to use the model layer is more, run a program to spend time is calculated in days, if running on the CPU, it may take a week, But it may only take two or three days on the GPU. But for us normal people, if let us programming on the GPU, afraid to be dumbfounded, then Caffe provides a convenient way to let you ignore the CPU and GPU differences, just one parameter can let your model between the CPU and the GPU to switch freely, Greatly improved the speed of the children's shoes ' hair paper '. (There is also a Python toolkit that allows you to easily switch between the CPU and the GPU, that is, Theano, I was originally because only the Python toolkit has the ability to support GPU programming, so I discard matlab from Python)
According to personal understanding, at present, the deep study area does the image basically uses is Caffe,caffe to its open source high efficiency easy to use characteristic and is famous. The specific advantages of Caffe include the following points:
1. Quick start: Models and corresponding optimizations are given in textual form rather than code.
Caffe gives the model definition, optimization settings and pre-training weights for immediate access.
2. Fast: Able to run the best models and massive amounts of data.
Using Caffe with CUDNN, test the alexnet model, and process each picture on K40 with only 1.17ms.
3. Modularity: Easy to extend to new tasks and settings.
You can use the various layer types provided by Caffe to define your own model.
4. Openness: Open code and reference models are used for reproduction.
5. Good community: can participate in development and discussion through BSD-2.
Seamless switching between 6.GPU and CPU
From the above introduction, the important features of Caffe have two points, one can be run on the GPU, and the second is running fast. So what causes Caffe to run faster than other deep learning frameworks? Personally, the following points are considered:
1. Data storage:
The data generated by Caffe is divided into 2 formats: Lmdb and LEVELDB
They are key/value pairs embedded database management System programming library.
While Lmdb memory consumption is 1.1 times times leveldb, Lmdb is 10% to 15% faster than LEVELDB, and more importantly LMDB allows multiple training models to read the same set of data simultaneously.
As a result, Lmdb replaces leveldb as the default dataset generation format for Caffe. (This is going to be a bit of a notch, and I'm not free to skip this part.) Previously used in the lab server installed on the old version of the CAFFE, in the run mnist and CIFAR-10 examples, CAFFE on the official website of the tutorial or to generate *_LEVELDB format data, but when I ran imagenet instances in a few days, I was prompted to find $caffe_ root/build/tools/directory under the Caffe file, later or through the tools directory with the user to find out, after compiling the old version of the Caffe than the new version of a bunch of things, had to recompile new versions, the compilation process found the lack of lmdb, Check data only found that Caffe has replaced the leveldb with Lmdb, and back to see the mnist and CIFAR-10 examples of the tutorial, found that the tutorial has also been leveldb part of the Lmdb. Alas, this update speed is also very fast, or remind you to upgrade in a timely manner, there are problems to see more than a few times the official website tutorial)
2. Data transfer
When data is transmitted forward and backward, Caffe uses BLOBs to transmit or process the data, which is the standard storage interface used by the framework. Here is a section of the Baidu Encyclopedia on the Blob Interpretation: BLOB (Binary large object), binary large object, is a container that can store the binary files. In computers, blobs are often the type of field used in a database to store binary files. A blob is a large file, a typical blob is a picture or a sound file, and due to their size, it must be handled in a special way (for example: uploading, downloading, or storing to a database, Lmdb or LEVELDB in Caffe)
In Caffe, a BLOB is a four-bit array, but in different situations, the meaning of each dimension is different.
For data: Number*channel*height*width
For convolutional weights: output*input*height*width
For convolution offset: output*1*1*1
3. Other
C + + running efficiency will not need to say, temporarily these bar, if there is a new content to add, but also welcome you to add ...

After the introduction of Caffe, the next step is to install Caffe running example to see the effect of
However, first of all, this series Caffe study notes, do not involve the installation of Caffe, if there is such demand please move elsewhere. I use the laboratory server, Caffe is someone else, I do not have root authority has not actually installed, so the installation part will not be told ~

The above describes some of the caffe on the edge of knowledge, the next run to run the example, change parameters, understand the operation mechanism of the Caffe, Also, by the way, how to add Hinton dropout in the Lenet and CIFAR-10 routines (imagenet layer is included with dropout instance) ...

Reference documents:
Http://www.lupaworld.com/article-250356-1.html