Original address: https://www.zhihu.com/question/27982282 Gein Chen's answer many thanks —————————————————————————————————————————— 1. The first step of learning the program, first let the program run, see the results, so that there will be an intuitive feeling.
Caffe's official Online Caffe | The Deep learning Framework provides a lot of examples, and you can easily start to train some existing classic models, such as lenet. I suggest starting with the LeNet MNIST tutorial, because the dataset is small, the network is small but it's classic, and it takes a little time to get up and running. When you see terminal brushless line output, see the ever-decreasing loss and rising accuracy, the end of the training you get 99+% accuracy rate, feel good and powerful look. You can run a few more examples and familiarize yourself with the environment and interfaces.
2. One-step commissioning, followed by Caffe in the network flow
After playing for a few days, you are a bit familiar with the interface of Caffe, have tired of the existing examples, you begin to want to see how the concrete is realized. I think the best way to do this is to follow the steps of the program in front of the network in a single step, then pass it back as an error message.
Caffe is like a project in your usual programming, you can use the IDE or GDB to debug it, and here we don't dwell on the process of debugging. You can first trace the forward propagation process, nothing more than a high-level to low-order call forward function, solver->net->layer->specific Layer (convolution ...). The back propagation is similar, but because you are not familiar with the various variables in the Caffe, you may have been dizzy when you finished tracking the forward propagation, or take a break to digest the entire forward propagation process.
Just beginning you do not need to each layer of the calculation details are so serious, probably know the procedure of the operation of the process is good, so that you can relatively fast to Caffe have a general grasp.
3. Customized Caffe
Here, you can already say that you have worked caffe, but can not calculate the introduction, because you do not know how to modify the source code, to meet their specific needs. Most of the time we need to define a new layer to complete a particular operation, and you need to add a new layer to the Caffe.
You must not start, your mind is blank. Fortunately, the wiki on GitHub is Caffe development Bvlc/caffe Wiki GitHub has a tutorial, and this is the closest to latest caffe of the source structure of the tutorial, you search the Internet is a lot of blog is a bit outdated, because Caffe recently refactored the code. You can follow it's instructions to add your own layer.
Although you already know where to add your own stuff, the core question you encounter is how to write the following four functions.
- FORWARD_CPU ()
- Forward_gpu ()
- BACKWARD_CPU ()
- Backward_gpu ()
You can first imitate the existing layer to achieve these four functions, and I believe the forward function can be written out soon, but backward or confused.
at this point we need to complement the core content of the neural network--backpropagation.
4. Understand and implement BackPropagation
This I think is platform-independent, whether you are using Caffe, Torch 7, or Theano, you need to understand and grasp deeply. Because I am more stupid, it took a long time to adapt to the various symbols in the derivation. In fact, it is not difficult, is the error along the chain rule rule flow back to the front layer. I'm not going to deduce the post-propagation process myself, because I know I have no way to express it well, and there are a lot of very good tutorials on the web. Here are some of the better learning steps I feel.
- Starting with the back propagation of the shallow neural network (the so-called fully connected layer), because this is relatively simple, and now we often say that the calculation of the gradient of CNN and LSTM will eventually return to here.
- The first must see is ng in the Ufldl tutorial Ufldl Tutorial, and the Chinese version, which is not like to see English classmates is a good news. Of course you do not understand, see again, forget, look, read a few times you will be familiar with the derivation process and mathematical symbols. My mind is not very good, to go back and forth to see a lot of times.
- Of course, UFLDL's tutorial was a little short, and I found a more nuanced tutorial, Michael Nielsen wrote Neural networks and deep learning. It was so good that I got through the two veins of my governor. Reading this on the basis of UFLDL, you should be able to quickly master the reverse propagation of the full connection layer.
- Finally in the Standford Daniel karpathy a blog hacker ' s Guide to Neural Networks, here with specific programming examples to teach you to calculate the gradient, not to deduce the post-propagation formula, is about the general gradient calculation. Try to experience it with your heart.
- At this time you are tempted to go back to see the Caffe source convolution layer of implementation, but found that they do not seem to understand. Though the derivation of the convolution layer and the whole connection layer is similar, there is a gap in the thinking. I suggest you go first to see how Caffe to achieve convolution, Caffe author Jiayanqing Daniel in the answer on the understanding of how to calculate convolution in Caffe? Let me enlightened. Focus on understanding Im2col and Col2im.
- At this point you know the forward propagation of convolution, but also almost can understand how to achieve the latter. I suggest you die. Caffe the calculation process of the convolution layer, make clear every step, after the painful process you will have a new experience of the reverse communication. After that, you should have the ability to add your own layers. Add a complete tutorial for adding a new layer making a Caffe layer computer Vision enthusiast. This tutorial implements a angle to Sine cosine Layer from scratch, contains a gradient derivation, forward and back propagation CPU and GPU functions, a great tutorial.
- Finally, it is recommended to learn basic GPU Cuda programming, although the CUDA function has been encapsulated in Caffe, easy to use, but sometimes need to use the kernel function, such as Cuda interface functions. Here's an introductory video tutorial that speaks very well of the NVIDIA Cuda beginner tutorial video.
Gein Chen
Links: https://www.zhihu.com/question/27982282/answer/80242005
Source: Know
Copyright belongs to the author. Commercial reprint please contact the author for authorization, non-commercial reprint please specify the source.
Reprint: Deep learning Caffe Code how to read
