Keras vs. Pytorch

We strongly recommend that you pick either Keras or Pytorch. These is powerful tools that is enjoyable to learn and experiment with. We know them both from the teacher ' s and the student ' s perspective. Piotr have delivered corporate workshops on both, while Rafa? is currently learning them. (see the discussion on Hacker News and Reddit).

Introduction

Keras and Pytorch is Open-source frameworks for deep learning gaining much popularity among data scientists.

• keras  is a high-level API capable of running on top of Tensorflo W, CNTK, Theano, or MXNet (or as Tf.contrib within TensorFlow). Since its initial release in March, it had gained favor for its ease of use and syntactic simplicity, facilitating FA St Development. It ' s supported by Google.
• Pytorch, released in October, was a lower-level API focused on direct work with array expressions. It had gained immense interest in the last year, becoming a preferred solution for academic, and applications of Deep learning requiring optimizing custom expressions. It ' s supported by Facebook.

Before we discuss the nitty-gritty details of both frameworks (well described on this Reddit thread), we want to Preemptiv Ely disappoint You–there ' s no straight answer to the ' Which one is better? '. The choice ultimately comes down to your technical background, needs, and expectations. This article aims to give you a better idea of where each of the both frameworks you should is pick as the first.

TL;DR:

Keras May is easier to get into and experiment with standard layers, in a plug & Play spirit.

Pytorch offers a lower-level approach and more flexibility for the more mathematically-inclined users.

Ok, but why isn't any of the other framework?

TensorFlow is a popular the deep learning framework. Raw TensorFlow, however, abstracts computational graph-building in a-on-the-.-May seem both verbose and not-explicit. Once know the basics of deep learning, which is not a problem. Anyone new to it, the sticking with Keras as its officially-supported interface should is easier and more productive.

[Edit:recently, TensorFlow introduced Eager execution, enabling the execution of any Python code and making the Model training more intuitive for beginners (especially if used with Tf.keras API).

While your may find some Theano tutorials, it's no longer in active development. Caffe lacks flexibility, while Torch uses Lua (though it rewrite is awesome:)). MXNet, Chainer, and CNTK is currently not widely popular.

Keras vs. Pytorch:ease of use and flexibility

Keras and Pytorch differ in terms of the level of abstraction they on.

Keras is a higher-level framework wrapping commonly used deep learning layers and operations into neat, lego-sized Buildin G blocks, abstracting the deep learning complexities away from the precious eyes of a data scientist.

Pytorch offers a comparatively lower-level environment for experimentation, giving the user + freedom to write custom L Ayers and look under the hood of numerical optimization tasks. Development of more complex architectures are more straightforward if you can use the full power of Python and access the Guts of all functions used. This, naturally, comes at the price of verbosity.

Consider this head-to-head comparison of what a simple convolutional network are defined in Keras and Pytorch:

Keras

model=Sequential () Model.add (conv2d(3,3), activation='relu', input_shape= (32,32,3))) Model.add (maxpool2d ()) Model.add (conv2d (3,3), activation='relu') Model.add (maxpool2d ()) Model.add (Flatten ()) Model.add (Dense (10,activation='softmax  '))

Pytorch

classnet (NN. Module):def __init__(self): Super (net,self).__init__() Self.conv1=nn. Conv2d (3,32,3) Self.conv2=nn. Conv2d (32,16,3) Self.fc1=nn. Linear (16*6*6,10) Self.pool=nn. Maxpool2d (2,2)            defforward (self,x): x=Self.pool (F.relu (SELF.CONV1 (x))) x=Self.pool (F.relu (Self.conv2 (x))) x=x.view ( -1,16*6*6) x=f.log_softmax (SELF.FC1 (x), Dim=-1) RETURNX Model=net ()

The code snippets above give a little taste of the differences between the "frameworks". As for the model training itself–it requires around lines of code in Pytorch, compared to a single line in Keras. Enabling GPU acceleration is handled implicitly in Keras, while Pytorch requires us to specify when to transfer data Betwe En the CPU and GPU.

If you ' re a beginner, the high-levelness of Keras may seem like a clear advantage. Keras is indeed more readable and concise, allowing-to-build your first end-to-end deep learning models faster, while Skipping the implementational details. Glossing over these details, however, limits the opportunities for exploration of the inner workings of each computational Block in your deep learning pipeline. Working with Pytorch If you are in the thought regarding the core deep learning concepts, like backpropagation, And the rest of the training process.

That said, Keras, being much simpler than pytorch, are by no means a toy–it ' s a serious deep learning tool used by Beginn ERS, and seasoned data scientists alike.
For instance, in the Dstl satellite Imagery Feature Detection Kaggle Competition, the 3 best teams used Keras in their sol Utions, while we deepsense.ai team (4th place) used a combination of pytorch and (to a lesser extend) K ERAs.

Whether your applications of deep learning would require flexibility beyond what pure Keras have to offer are worth Consideri Ng. Depending on your needs, Keras might just is that sweet spot following the rule of least power.

SUMMARY

• Keras–more Concise, simpler API
• Pytorch–more flexible, encouraging deeper understanding of deep learning concepts

Keras vs. Pytorch:popularity and access to learning resources

A framework ' s popularity is not only a proxy of its usability. It is also important for community support–tutorials, repositories with working code, and discussions groups. As of June 2018, Keras and Pytorch is both enjoying growing popularity, both on GitHub and ARXIV papers (note the most p Apers mentioning Keras mention also its TensorFlow backend). According to a kdnuggets survey, Keras and Pytorch is the fastest growing data science tools.

Unique mentions of deep learning frameworks in arxiv papers (full text) over time, based on 43K ML papers over last 6 year S. So far TF mentioned in 14.3% of all papers, Pytorch 4.7%, Keras 4.0%, Caffe 3.8%, Theano 2.3%, Torch 1.5%, Mxnet/chaine R/CNTK <1%. (CC @fchollet) pic.twitter.com/yoyavc33in

-andrej karpathy (@karpathy) 2018 Marca

While both frameworks has satisfactory documentation, Pytorch enjoys stronger community support–their discussion board is a great place to visit to if you get stuck (you'll get stuck) and the documentation or StackOverflow don ' t provide yo U with the answers need.

Anecdotally, we found well-annotated beginner level deep learning courses on a given network architecture easier to COM E across for Keras than for Pytorch, making the former somewhat more accessible for beginners. The readability of code and the unparalleled ease of experimentation Keras offers may do it the more widely covered by D EEP learning enthusiasts, tutors and hardcore Kaggle winners.

For examples of great Keras resources and deep learning courses, see "Starting deep learning hands-on:image Clas Sification on CIFAR-10 " by Piotr Migda? and ' deep learning with Python ' –a book written by François Chollet, the creator of Keras himself. For Pytorch Resources, we recommend the official tutorials, which offer a slightly more challenging, comprehensive APPROAC H to learning the inner-workings of neural networks. For a concise overview of Pytorch APIs, see this article.

SUMMARY

• Keras–great access to tutorials and reusable code
• Pytorch–excellent community support and active development

Keras vs. Pytorch:debugging and introspection

Keras, which wraps a lot of computational chunks on abstractions, makes it harder to pin down the exact line that causes Y Ou trouble.
Pytorch, being the more verbose framework, allows us to follow the execution of our script, line by line. It's like debugging Numpy–we has easy access to all objects in our code and is able to the use of print statements (or any St Andard pythonic debugging) to see where our recipe failed.
A Keras user creating a standard network have a order of magnitude fewer opportunities to go wrong than does a pytorch use R. But once something goes wrong, it hurts a lot and often it's difficult to locate the actual line of code that breaks. Pytorch offers a more direct, unconvoluted debugging experience regardless of model complexity. Moreover, when in doubt, you can readily the lookup pytorch repo to see its readable code.

SUMMARY

• Pytorch–way Better debugging capabilities
• keras– (potentially) less frequent need to debug simple networks

Keras vs. Pytorch:exporting models and Cross-platform portability

What is the options for exporting and deploying your trained models in production?

Pytorch saves models in Pickles, which is python-based and not portable, whereas Keras takes advantages of a safer Approa Ch with JSON + H5 files (though saving with custom layers in Keras are generally more difficult). There is also Keras in R, in case you need to collaborate with a data analyst team using R.

Running on TensorFlow, Keras enjoys a wider selection of the solid options for deployment to mobile platforms through >tensorflow for Mobile and tensorflow Lite. Your Cool web apps can be deployed with tensorflow.js or keras.js. As an example, see this deep learning-powered browser plugin detecting trypophobia triggers, developed by Piotr A nd his students.

Exporting Pytorch models is + taxing due to its Python code, and currently the widely recommended approach are to start By translating your pytorch model to Caffe2 using onnx.

SUMMARY

• Keras–more deployment options (directly and through the TensorFlow backend), easier model export.

Keras vs. Pytorch:performance

Donald Knuth famously said:

Premature optimization is the root of any evil (or at least most of it) in programming.

In most instances, differences on speed benchmarks should not being the main criterion for choosing a framework, especially W Hen it is being learned. GPU time is much cheaper than a data scientist ' s time. Moreover, while learning, performance bottlenecks would be caused by failed experiments, unoptimized networks, and Data loa Ding Not by the raw framework speed. Yet, for completeness, we feel compelled-to-touch on the this subject. We recommend these-comparisons:

• TensorFlow, Keras and Pytorch comparison by Wojtek Rosiński
• Comparing deep learning frameworks:a Rosetta Stone approach by Microsoft (make sure to check notebooks to get th e Taste of different frameworks). For a detailed explanation of the MULTI-GPU framework comparisons, see this article.

Pytorch is as fast as tensorflow, and potentially faster for recurrent neural Networks. Keras is consistently slower. As the author of the first comparison points out, gains in computational efficiency of higher-performing frameworks (ie. P Ytorch & TensorFlow) 'll in the cases is outweighed by the fast development environment, and the ease of Experimenta tion Keras offers.

SUMMARY:

• As far as training speed is concerned, Pytorch outperforms Keras

Keras vs. Pytorch:conclusion

Keras and Pytorch is both excellent choices for your first deep learning framework to learn.
If you ' re a mathematician, researcher, or otherwise inclined to understand what your model is really doing, consider Choos ing Pytorch. It really shines, where more advanced customization (and debugging thereof) is required (e.g. object Detection WI Th YOLOv3 or Lstms with attention) or if we need to optimize an array expressions other than neural netwo Rks (e.g. matrix decompositions or Word2vec algorithms).

Keras is without a doubt the easier option if you want a plug & Play framework:to quickly build, train, and evaluate A model, without spending much time on mathematical implementation details.

Edit:for side-by-side Code Comparison on a real-life example, see our new Article:keras vs. Pytorch:alien vs. Predator Recognition with transfer learning.

Knowledge of the core concepts of deep learning is transferable. Once you master the basics in one environment, you can apply them elsewhere and hits the ground running as you transition t o New deep learning libraries.

We encourage you-to-try out simple deep learning recipes in both Keras and Pytorch. What is your favourite and least favourite aspects of each? Which framework experience appeals to you? Let us know in the comment section below!

Would you and your team like to learn more about deep learning in Keras, TensorFlow and Pytorch? See our tailored training offers.

Source: https://deepsense.ai/keras-or-pytorch/

From for notes (Wiz)

Keras vs. Pytorch