The Promise of deep learning

The Promise of deep learning

by Yoshua Bengio

Humans has a long dreamed of creating machines that think. More than years before the first programmable computer is built, inventors wondered whether devices made of rods and Gears might become intelligent. And when Alan Turing, one of the pioneers of computing in the 1940s, set a goal for computer science, he described a test, Later dubbed the Turing Test, which measured a computer ' s performance against the behavior of humans.

Read:accelerating Cognitive Computing

In the early days of my academic field, artificial intelligence, scientists tackled problems this were difficult for human s but relatively easy for computers–such as large-scale mathematical calculations. In + recent years, we ' re taking on the tasks that is easy for people to perform and hard-describe to a machine–tasks Hu Mans solve "without thinking," such as recognizing spoken words or faces in a crowd.

That's more difficult quest gave rise to the domain of machine learning, the ability of machines to learn. This is the what interests me. It's not a really my goal to make machines that think like humans do. My aim is to understand the fundamental principles, which may enable a entity, machine or living being, to be intelligent. I have a long ago made the bet that this would happen thanks to the ability of such a entity to learn, and my focus was on B Uilding machines that can learn and understand the world by themselves, i.e., learn to make sense of it.

the reason I ' m laying out this chronology was that I believe we ' re at a turning point in the Histo Ry of artificial Intelligence–and, indeed, computing itself. Thanks to more powerful computers, the availability of large and varied datasets, and advances in algorithms, we ' re able t o Cross a threshold that have long held back computer science. Machine learning are shifting from a highly manual process where humans has had to design good representations for each TA SK of interest into a automated process where machines learn more like babies Do-through experience–  building in Ternal representations the world. This is the field of deep learning.

Deep learning isn ' t brand new. Indeed, when I am a student in the 1980s, it is the concept of neural networks, the precursor of deep learning, that got Me interested in pursuing a academic career in computer science. What's new is that the accumulation of many scientific and technical advances have yielded breakthroughs in AI applications Such as speech recognition, computer vision, and natural language processing.  this have brought into the field a Lar GE Group of researchers, mostly graduate students, and we ' re now making progress in deep learning at a gallop.

We ' re able to do, because of advances in creating hierarchies of concepts and representations that computers Discov Er by themselves. The hierarchies allow a computer to learn complicated concepts by building them off of simpler ones.  this is also ho W Humans learn and build their understanding of the world; They gradually refine their model of the world to better fit what they observe and discover new ideas from the composition of older ones, new ideas, them to better fit the evidence, the data.

For example, a deep learning system can represent the concept of an image of a cat by combining simpler concepts, such As corners and contours, which is in turn defined in terms of edges. But we don ' t has to teach it explicitly about these intermediate concepts, it learns them on its own. We don ' t have a to show the system pictures of the possible cat colors, shapes, and behaviors for such object recognitio n systems to correctly identify that it's a Siamese cat that's somersaulting in a photograph. When it "sees" a-cat, it "knows" it is one.

I ' m privileged to being part of a troika of computer scientists who is widely credited with spearheading advances in this Field–along with Geoffrey Hinton and Yann LeCun. We co-authored a paper,  deep learning ,  which is published in the journal Nature in May, where we laid The promise of our branch of A.I. But this isn ' t a field where a few "media stars" is doing all, needs to is done. To produce the advances that is possible and to find applications for them would require thousands of scientists and engin Eers–in academia and in industry.

That's why I ' ve been dedicated to rallying people to our exciting project. I ' m co-authoring a book,  deep learning , with Ian Goodfellow and Aaron Courville. Our core audiences is university students studying machine learning and software engineers working in a wide variety of I Ndustries that is likely to the find important uses for it. This book-in-progress are posted on the Web, and we welcome people to read, learn and give us feedback.

Which brings me to another key Point:i ' m a advocate of open science. Like open source developers, participants in the Open science movement believe that we should share knowledge as soon as W e gain it to increase the pace at which the boundaries of science is pushed, and for the benefit of all. Many of my colleagues and I contribute all of our deep learning inventions to the Theano project and its Derivati Ves on GitHub. There, anybody who are building deep learning systems can use the algorithms and programming tools, and we urge them to con Tribute back to the project:hundreds already does so.

Just as sharing is essential to open science, so is collaboration–the kind that's done transparently. The whole enterprise of science is a giant brainstorm. The Montreal Institute for Learning Algorithms (MILA), with its researchers-including 5 professors, contributes to it Via numerous collaborative the projects with scientists in universities and industry.

The newest of our collaborative-partners is IBM. We look forward to working with scientists and engineers in IBM, and the Watson Group on a very ambitious Agenda, including deep learning for language, speech and vision. We believe that, together, we'll be able to scale up and extend deep learning methods by using powerful computers On very large datasets. It would help machines learn more, across broader domains, faster and from a larger set of data sources, including the vast Amounts of unlabeled data–that has not been curated by humans.

I ' m tremendously excited about the future of deep learning. We ' ve made rapid progress, and while we ' re far from solving the great riddle of what it would take to enable machines to TR Uly understand the world, I ' m very hopeful that we'll crack it.

And then the floodgates would open. Once computers truly understand text, speech, images and sounds, they would become our indispensible assistants. This would revolutionize the the-the-interact with computers, helping us live more conveniently in our day-to-day lives and Perform more effectively at work. It would enable society to take on some of the grand challenges that matter to us–such as curing deadly diseases and spread ing knowledge and wealth more broadly. As importantly, it would help us understand who we is and that part of the WHO we is that have always fascinated me, i.e. Intelligence arises. This have been my dream for more than years, and it's fast becoming our reality.

The Promise of deep learning