Deep learning reproduces the turning point of development

Advances in deep learning and related machine learning have played a key role in the recent achievements of artificial intelligence, which have made it unnecessary for computers to explicitly program, but to “absorb and analyze” large amounts of data to complete self-training.In the past two years, Google's deep learning-based AlphaGo beat the world's top Go players, shocking most AI experts, because in their cognition, this milestone implementation needs at least 5 to 10 Year's time.

Like other major technical accomplishments, deep learning has quickly climbed to the peak of Gartner's hype cycle, but when all the excitement and publicity are accompanied by emerging, promising technology expectations, it often leads to high expectations. That is to say, when technology cannot be realized, people have a serious sense of disillusionment. In the past few decades, artificial intelligence has experienced some such hype cycles, including the so-called artificial intelligence winter in the 1980s, which almost “buried” this field.

In a recent article titled "Deep Learning: A Critical Appraisal," Professor Gary Marcus from New York University conducted a careful assessment of deep learning. He believes that although he has made considerable achievements in the past five years, deep learning may be coming to an end. University of Toronto professor Geoffrey Hinton, who is known as the "father of deep learning," obviously Hold this view.

Although deep learning is avant-garde, it still faces four major problems.

Deep learning is a powerful statistical technique that can be used to classify patterns using large training data sets and multi-layer AI neural networks. Essentially, this is a way for machines to learn from the data, and the data will be modeled in a way that the biological brain learns to solve the problem. Each artificial neural unit is connected to many other similar units and can be statistically increased or decreased depending on the data used to train the system. At the same time, each successive layer in a multi-layer network uses the output of the previous layer as input.

In response, Marcus said, "This technology excels in solving closed classification problems. Given that enough data is available and the test set is very similar to the training set, a variety of potential signals must be mapped to Among the limited categories.” However, it is important to note that deviations from these assumptions may cause problems. Deep learning is just a statistical technique, and all statistical techniques have deviations from the assumptions. Therefore, as with all technologies in the early stages, deep learning must overcome many serious challenges, and there are four main problems.

Question 1: Deep learning faces lack of data

For deep learning, the data aspect is actually very scarce, because the data requirements for deep learning are fundamentally different from other analysis methods in many dimensions. As the size of the data set increases, the performance of traditional analysis tends to be stable. For deep learning techniques, the gradual increase in data sets, coupled with proper training, will help improve performance. Currently, deep learning techniques have become particularly valuable in extracting patterns from complex unstructured data, including audio, speech, images, and video. To do this, they need thousands of data records so that the model can do better in the classification task, and millions of data records can be executed at the human level.

“Humans can learn abstract relationships in a few experiments... And deep learning currently lacks a mechanism for learning abstractions through explicit language definitions. When there are thousands or even billions of training examples, It works best.” Marcus said, “When learning through a clear definition, people rely on an ability to express abstract relationships between algebraic variables. In fact, even 7-month-old babies can To do this, you can get the rules for learning abstract languages ??from a few unmarked examples in just two minutes."

At a recent AI conference, Josh Tenenbaum, a professor of brain cognitive science at the Massachusetts Institute of Technology, expressed his views on the differences between our current state of artificial intelligence and the long-term pursuit of human intelligence. Human intelligence has the ability to transcend data and machine learning algorithms. That is, humans can build models when they perceive the world, including practical common sense, and then use these models to explain their behavior and decisions. According to Tennenbaum, a three-month-old baby has more knowledge of the world around it than any artificial intelligence application. Specifically, artificial intelligence applications start with blanks and then learn from the analyzed data, while babies have genetic and brain structures from the start, so they can learn more than programs.

Research work at the MIT Human Dynamics Lab and the Allen Institute is attempting to supplement statistics by simulating human common sense and/or using logic-based programming tools. Guided AI methods to overcome the limitations of deep learning. Only these research work is still in the early stages.

Question 2: Deep learning is actually superficial

Deep learning is actually very shallow. The "depth" of this technology refers to its highly complex multi-layer statistical properties. However, although some amazing results can be achieved, in the current situation, deep learning is actually very shallow and fragile. “The extraction mode through deep learning is even more superficial than when it first appeared.”

Today's artificial intelligence applications can do a good job with a lot of data and deep learning algorithms, but each application must use its own data set for individual training, even for similar use cases. So far, there is no good way to transfer training from one situation to another. While artificial intelligence does the best in terms of applications and test sets, it is much less effective in trying to generalize or infer its training data set.

Question 3: Deep learning is not transparent enough

Deep learning is not transparent enough. A typical deep learning system has a large number of parameters in its complex neural network. It is very difficult to assess the contribution of each node to decision making in a way that people understand.

In the case of unresolved transparency, the use of deep learning in problem areas such as financial transactions or medical diagnosis may create liability issues, that is, such opacity can lead to serious bias problems.

Question 4: Deep learning is difficult to achieve the desired results

So far, deep learning is very difficult to do. One of its main challenges is the engineering risks inherent in any complex frontier IT system, especially in high-risk applications such as medical, automotive and aircraft, finance and government. While these risks generally apply to increasingly complex artificial intelligence systems, they may present significant problems in deep learning due to their statistical nature, opacity, and difficulties in distinguishing between causality and relevance.

At the same time, we must also ensure that complex artificial intelligence systems can accomplish the tasks we want them to accomplish and act in the way we want to behave. This is a particularly tricky problem in deep learning algorithms, that is, these algorithms are not clear. Data training and learning are accepted in the case of programming.

Of course, we do not have to be pessimistic about the development of deep learning. To address these and other challenges in artificial intelligence and deep learning, several related initiatives and initiatives have been developed to address issues, including Stanford University's One Hundred Year Study of AI and MIT. College's Quest for Intelligence. Hopefully, as with previous powerful technologies, these efforts will help ensure that relevant issues are properly addressed, while making increasingly powerful artificial intelligence systems have a significant positive impact on future economic, social and personal lives.