A summary of the experts ' outlook on the development trend of deep learning in the next 5 years

Original URL: http://www.iteye.com/news/31270

1. We should see deeper models, which can be learned from fewer training samples compared to today's models, and will make substantial progress in unsupervised learning. We should see more accurate and useful speech and visual recognition systems.

2. I expect deep learning to be increasingly used for multi-mode (multi-modal) problems and to be more structured on the data. This will create new applications for deep learning, such as robotics, data mining and knowledge discovery.

3. The performance of deep learning algorithms and neural networks is far from the theoretical performance. Today, we can design a visual network with five to one-tenth costs, and one-fifteenth of parameters, while performance is better than a network that was designed at an expensive cost a year ago, thanks to improved network architecture and better training methods. I firmly believe that this is only the beginning: deep learning algorithms will be more efficient and able to run on inexpensive mobile devices, even without additional hardware support or excessive memory overhead.

4. One trend I have seen is that architecture is rapidly becoming larger and more complex. We are moving towards building large neural network systems, exchanging input and output of neural components, pre-trained network parts on different datasets, adding new modules, fine-tuning everything, and so on. For example, the Convolutional network was once one of the largest/deepest neural network architectures, but today it is abstracted into a small part of most new architectures. In turn, many of these architectures will be a small part of the future innovation architecture. We are learning how to stack Lego bricks and how to effectively embed them in a large "castle".

5. There are many technologies that are based on deep-supervised learning techniques, and video technology is the same, figuring out how to make deep learning go beyond the present in natural language processing and making significant progress in deep unsupervised learning and deep reinforcement learning.

6. Over the past two years, we have observed that deep learning has been a great success in most areas where deep learning has been used. Even if deep learning does not reach the level of human cognition in the next 5 years (though this is likely to happen in our lifetime), we will also see great improvements in deep learning in many other areas. In particular, I think the most promising area will be unsupervised learning, because most of the data in the world is unmarked, and our brain's new cortex is a good unsupervised learning area.

7. The Big data ecosystem is now focused on collecting, managing, and curatorial large volumes of data. Obviously, there is a lot to be done in terms of analysis and forecasting. Fundamentally, enterprise users don't care about those. Enterprise users only care about the results, i.e. "will this data change the way I behave?" Will it change the choices I make? "We believe these issues are key issues that need to be addressed in the next 5 years." We believe that AI will be a bridge between data and better decisions.

8. Deep learning will play a significant role in evolution, but it needs to be combined with other AI methods. Over the next 5 years, we'll see more and more hybrid systems in which deep learning is used to handle some difficult-to-perceive tasks, while other artificial intelligence and machine learning (ML) techniques are used to deal with other parts of the problem, such as reasoning.

9. Over the next 5 years, increasingly difficult issues such as video recognition, medical imaging or word processing will be successfully solved by deep learning algorithms. We can also see that deep learning algorithms are ported to commercial products, just as the 10-year-old face detection was incorporated into the camera.

A summary of the experts ' outlook on the development trend of deep learning in the next 5 years