Refresh neural Network New depth: Imagenet Computer Vision Challenge Microsoft China researcher wins

Microsoft Research Asia chief researcher Sun Jian

How accurate is the world's best computer vision system? On December 10 9 o'clock in the morning EST, the imagenet Computer Vision Recognition Challenge was announced--Microsoft Research Asia Vichier's researchers, with the latest breakthroughs in deep neural network technology, have won the title of all three major projects with absolute advantage in image classification, image positioning and image detection. At the same time, they were also successful in another image recognition challenge, Ms COCO (Microsoft Common Objects in Context, common object image recognition), and defeated many contestants from academia, businesses and research institutions on image detection and image segmentation projects.

The Imagenet Computer Vision Challenge, organized by researchers from top universities and companies around the world, has become a benchmark in computer vision in recent years, and its results can always be a very intuitive reflection of the research progress and breakthroughs of the research institutes in the field of computer vision. The MS Coco database was founded by Microsoft, and its challenge is currently run by a consortium of academic institutions in academia.

The two challenges have different priorities: ImageNet tends to evaluate the ability to identify significant objects in an image, while Ms Coco tends to evaluate the ability to identify various objects in complex scenes. The ability to win a championship in two world-class competitions is enough to explain that the research team's technological breakthroughs are universal-it can significantly improve research in the field of computer vision and even research outside the field of computer vision, such as speech recognition. So what is the technological breakthrough?

In the field of computer vision, the methods of deep neural networks are often used by researchers to train computers to recognize objects, and Microsoft is no exception. But Microsoft Research Asia's researchers used an unprecedented, deep-layer neural network in the Imagenet Challenge. The number of layers in the network is more than 5 times More than the number of layers in any previously successful neural network.

The challenge behind this technology is enormous. At first, even the researchers themselves were not convinced that a very deep network was possible or useful. "We didn't expect such a simple idea to mean so much. "Microsoft Research Asia chief researcher Sun Jian admits. The team that completed the technology breakthrough was composed of 4 Chinese researchers: Sun Jian and He Cai from Microsoft Research Asia Vichier, and the other two for the Microsoft Research Asia of the joint training doctoral students, respectively, from the Jiaotong University of Zhang Xiangyu and the Chinese Universities of science and technology Ninshoqin.

Microsoft Research Asia, executive researcher He Cai Ming

Of course, this major technological breakthrough is not only shocking to the researchers of this research group. "In a sense, they completely subvert my previous vision of deep neural networks," said Peter Lee, Microsoft's global senior vice president. ”

The Imagenet challenge last year won the system error rate of 6.6%, and this year the Microsoft system error rate has been as low as 3.57%. In fact, the team first achieved a breakthrough in human visual competence as early as this month. At that time, in a paper entitled "Delving deep into rectifiers:surpassing human-level performance on ImageNet Classification", Their system's error rate has been reduced to 4.94%. In the same experiment, the error rate of human eye identification was about 5.1%.

Hollows: This is a story about patience and innovation.

In recent decades, scientists have been training computers to do all sorts of things, such as examples or speech recognition. But for a long time, the errors of these systems are huge and difficult to eliminate.

About five years ago, researchers began to re-use the "neural network" technology and give it new vitality. The revival of neural networks has greatly improved the precision of technology such as image and speech recognition. Thanks to Microsoft's Skype Translator real-time voice translation technology, it is able to better recognize the voice and continuously improve the accuracy of machine translation.

Similar to the human brain, neural networks contain multilevel nonlinear processing layers. Theoretically, the more layers should lead to better learning outcomes. But the biggest challenge in practical experiments is that the amplitude of the anti-wear supervision signal is rapidly attenuated at each level of the anti-pass training, which makes the whole neural network system extremely difficult to train.

Sun Jian recalled: "Three years ago, when the computer vision and the actual field of machine training 8 layers of deep neural network system, the recognition accuracy has a qualitative leap." Last year there was a deep neural network of 20 to 30 layers, and the recognition accuracy was greatly refreshed. ”

Sun Jian and his crew think the network can be deeper. Over the past few months, they have added more layers in a variety of ways, while also guaranteeing the accuracy of the results. They have gone through a lot of wrong attempts and learned a lot of lessons. Finally, a system that they call "deep residual networks" was successfully born in Microsoft Research Asia.

This "deep residual network" is the system they use for the Imagenet Challenge, which achieves an astonishing 152-fold layer, which is 5 times times deeper than any system ever in the world. It also uses a new "residual learning" principle to guide the design of neural network structures. The most important breakthrough in residual learning is to reconstruct the learning process and redirect the information flow in the deep neural network. Residual learning solves the contradiction between the level and accuracy of the deep neural network.

?

Shing: From scientific exploration to intelligent products

Neural networks have a very important advantage, that is, the learning of internal representations or features can be reused in different tasks. Skype Translator is a good example of how the accuracy of translation between English and German can be improved with the increasing translation of English and Chinese.

Sun Jian says their deep residual network is very versatile. After they used the system for the classification of the Imagenet Challenge, they found that the system-learned internal representations or features could significantly improve the other three tasks: detection (detection), positioning (localization), and segmentation (segmentation). "From our deep deep neural network we can see that the deep residual network is powerful and extremely versatile, and it can be predicted to significantly improve other computer vision problems," he added. ”

In fact, the Sun Jian team's years of research in the field of computer vision have been transformed into Microsoft's smart products and services, such as the face recognition and image recognition in Microsoft's Oxford program Api,windows 10, the Windows Hello "brush face" boot feature, Bing's image search, Microsoft Xiaoice multiple image "skills", the picture classification feature in OneDrive, and the widely acclaimed Pocket scanner office lens and many more.

In the case of Microsoft's Oxford program, the program opens up a series of machine learning-related APIs that enable developers without machine learning backgrounds to build their own smart apps. The face recognition API is widely used as the first API to be opened by the Oxford program. The previous how-old.net (Microsoft Genling robot) and twins or not (Microsoft) are based on the human face recognition API and are implemented in a few lines of simple code.

By working closely with Microsoft products, the world's leading computer vision technology from Microsoft Research Asia has been used in hundreds of millions of of people's lives. The findings from Chinese researchers are creating an "invisible revolution" in our lives, providing smarter productivity tools and a more personalized computing experience for users around the world.

"The importance of vision in the human senses is the same, and a major breakthrough in computer vision will undoubtedly provide a powerful impetus for the overall development of AI," said Dr. Wuen Hon, Senior vice president of Microsoft Global and Microsoft Research Asia Dean. To make computers understand the colorful world has been an important force to motivate Microsoft Research and computer colleagues to move forward on this challenging road. The future, there are more breakthroughs waiting for us to challenge! ”

"Microsoft Research Asia has been established for 17 years, and her research environment and atmosphere have cultivated many talents for the Chinese it session." I've been working here for 12 years, and you'll be able to reap exciting discoveries in this environment. Today, I said to my team, please enjoy the feeling of winning the NBA championship one day! "Sun Jian said.

?

related papers:http://arxiv.org/abs/1512.03385

From:http://blog.sina.com.cn/s/blog_4caedc7a0102w2n9.html

Refresh neural Network New depth: Imagenet Computer Vision Challenge Microsoft China researcher wins