Practical recommendations for gradient-based training of deep ubuntures-Abstract

Source: Internet
Author: User

Practical suggestions on gradient-based deep architecture training methods

Practical recommendations for gradient-based training of deep ubuntures
Yoshua bengio
Universit é de montr é al

Abstract: learning algorithms related to neural networks, especially deep learning, are designed with many fancy things called hyperparameters. This chapter is a practical guide and provides some suggestions for the most commonly used hyperparameters, especially for reverse propagation learning algorithms that adopt gradient/gradient-based optimization. Many interesting results can be obtained when multiple super parameters are allowed to be adjusted, which is also discussed. In short, it describes the successful and effective training and debugging of a large scale, and it is often used by deep multi-layer neural networks. Finally, we will end with an open question that is difficult to train the deeper architecture.

Abstract: learning algorithms related to artificial neural networks and in particle for deep learning may seem to involve into bells and whistles, called hyper-parameters. this chapter is meant as a practical guide with recommendations for some of
Most commonly used hyperparameters, in participates in the context of Learning Algorithms Based on Back-propagated gradient and gradient-based optimization. it also discusses how to deal with the fact that more interesting results can be obtained when allowing
One to adjust extends hyperparameters. overall, it describes elements of the practice used to successfully and efficiently train and debug large-scale and often deep multi-layer neural networks. it closes with open questions about the training difficulties observed
With deeper ubuntures.

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.