Image classification Combat (iii)-PYTORCH+SE-RESNET50+ADAM+TOP1-96

TOP1 direct to 96 model:

Pytorch framework, network model SE-RESNET50, optimization algorithm Adam

Pytorch:

Pytorch official documentation, each module function has a GitHub source chain

Links to Tutorials http://pytorch.org/tutorials/

Connection to the official website http://pytorch.org/

Pytorch's GitHub home page Https://github.com/pytorch/pytorch

Pytorch (an elegant frame) https://www.jianshu.com/p/6b96cb2b414e

Pytorch[facebook] is a python-first deep learning framework, a very low-level framework with Tensorflow[google, engineering capabilities],caffe,mxnet[amazon], theano[for research], Its predecessor is torch, the main language interface is LUA, in today's GitHub up to 10 of the Machine learning Project has 9 is the era of Python, has not been too many people use, relatively small audience. Pytorch is now back, rewriting the entire framework with Python and returning to the programmer's line of sight again.

In today's era of machine learning frameworks, no framework is the best, and each framework has its own merits. It's important to have a different framework, not to be proficient in each frame, or at least to look at the code in this framework, because there's a constant number of people on GitHub that reproduce their thesis, and the frames they use are definitely not the same, so you should at least be able to read the code that someone else wrote in each frame.

Advantages and disadvantages of using Keras & Pytorch:

[Keras] A very high-level structure, its back-end support Theano or tensorflow, it is not a framework in essence, just a framework for the operation of a package, you write Keras is actually the back end of the call, Equivalent to you or in TensorFlow or Theano run the program, but you put your language to Keras to deal with the language to become TensorFlow understand, and then to TensorFlow processing, the consequences of course, it is convenient for you to build the network, Easy to define the model to do training, very quickly build your ideas, the project is very strong, but there is also a consequence, that is the details you have no way to control, the training process is highly encapsulated, resulting in you have no way to know the details inside, as well as the specific details of each parameter, making debugging and research becomes very difficult.

[Pytorch] An underlying framework similar to Theano & TensorFlow. Its underlying optimizations are still on the C, but all of its basic frameworks are written in Python.

Se-resnet50_33epoch:

1. SE-RESNET,IMAGENET2017 's Champion

2. The network model, 50 layers, trained 33 epochs.

3. top1-96.

Adam:

1. Learn about the differences between Adam and SGD

80086926

SGD is an initial optimization algorithm, and the Deep learning optimization algorithm has experienced the development of the Nadam such as the SGDM, NAG->adagrad, Adadelta, SGD-A.

In addition, when it comes to optimization algorithms, the entry level must be learned from SGD, and older drivers will tell you that there are better adagrad/adadelta, or no brains for Adam. But look at the latest paper in academia, but found that a great God is still using the entry-level SGD, up to add a Momentum or Nesterov, but also often black Adam

2. Selection and use of optimization algorithms:

In general, use Adam First and then SGD

3. Summary:

Adam converges faster with low time costs

SGD speed is slow, but better, if time is ample use AGD

Image Classification Combat (iii)-PYTORCH+SE-RESNET50+ADAM+TOP1-96