Application of deep learning in data mining

Deep learning is one of the most important research directions for us, and it is also a necessary tool for the industry to realize a lot of amazing functions and the path to artificial intelligence.

Let's take a look at what deep learning can do, the drone of Google research, whose components are made up of two parts, one eye, one brain, a laser rangefinder and a video camera, and the car is not very well recognized after the video signal is collected, in order for the car to understand that we need a brain, The brain is deep learning, and through deep learning we can tell our computers in the car, what kind of objects are in front of them, and how they are structured and extracted.

For example, this is through the windshield to see the picture, so that the machine understands that the object in the field of view must be moved or stationary, if it is still, can be regarded as a safe object, just to avoid, if it is moving objects, then we also need to judge his speed and direction of travel to the corresponding route planning.

We look at face recognition, we have a lot of technology face recognition, face recognition can do what other things? Deep learning not only tells us where the face is in the picture, even tells me the face of the person's face, is a male, female, how old can be learned, including the important node of the face of the person can guess what the expression of the person, even through the analysis of his lip movements, can say what the person is saying, This includes the color of the hair, what kind of sunglasses to wear, and what kind of lipstick your lips can paint.

To give a well-known example of green deep pupil, for example, we can have security monitoring in important institutions, deep learning training convolutional Neural network CNN, can identify the person being monitored for unusual behavior, there is the pursuit of the vehicle, whether the car has the possibility of escape, speeding, the risk of retrograde lane change.

Let's see Alphago again.

In March 2016, Google DeepMind developed the Alphago 4:1 victory over the world champion Li Shishi. Mark the end of an era and the beginning of an era, the human race in the complete information game lost, the beginning of the first years of the development of artificial intelligence.

Weiqi is difficult to be broken because of the complexity is too high, every move has more than 300 kinds of possible, a game on average more than 200 steps, the total number of States more than the entire universe of all the atoms, can not be searched the whole state, we can only estimate and intuition for the calculation and thinking of go. Chess has been broken long ago, go can persist for so long. Deep learning allows the machine to have human intuition and predict what the person is going to do next, and at the same time analyze and which step to take. So said DeepMind research and development of Alphago, basically do the enemy.

Let's take a look at these two deepmind deep learning networks, which is the strategy network, and I go one step at a time, analyzing the value of each position on the board and giving each location a score. The valuation network below is a neural network for estimating the winning odds between black and white. The combination of these two networks, together with some previous methods of universal search, such as the Monte Carlo search tree, can give the computer a very strong ability to battle. In fact, by the result of the re-disk, Alphago and Li Shishi, alphago from the beginning to think that their winning ratio has more than 60%, to the end of the basic reached 90%, his control over the entire board more than the understanding of human, the situation is not a lot of commentators think may be two or the balance of equilibrium, Li Shishi still have a chance to wait. The overall situation is completely in the grasp of Alphago.

Deep Q Net, intensive learning can teach robots how to flexibly use robotic arms to accomplish tasks. If we had a robot programmed to get him to clip an object, there wouldn't be too much interference, otherwise it wouldn't be able to get an accurate crawl. Now I'm just going to put a box on it. The depth hardening network can automatically train the robot to take what kind of object, while training it how to clip, the first time did not clip to then learn, and then try, until learned. Deep learning, it can be said, gives the robot the ability to pick up objects from a few years old.

Google Deepdream can achieve a fantastic picture production, like a nightmare.

Look at this picture, the bottom of this figure is not a bit abstract, this painting is automatically generated by the deep Learning Network, the basic principle is that when people observe a picture, remember all the details, in our mind when refactoring will use the previous experience and concepts in the brain to shape a new picture, and deep learning is this meaning, In the need of large data volume, accumulated a lot of past experience and data, we gave him a picture of the reconstruction of the time, to create a dream or in the mind when the imagination of the image produced by the understanding. So we can say that it already has the human ability to abstract and reconstruct things.

Easystyle with deep learning, you can combine any image content with another image style

This picture may be very familiar to everyone, the top of this is the United States presidential candidate Trump, the middle of this painting is a famous painter's paintings, through the deep neural network combined we can synthesize the following diagram, without any algorithm tuning, it gets the information on the image above, and then get the information of the middle of this diagram style, The perfect combination is the middle of the picture.

Neural doodle– the graffiti into painting

For example, graffiti a painting, you can get a picture of a smarty pants landscape painting. We can also first parse a picture of the main components, and then adjust the shape of the original diagram, and then reconstruct it, we can draw a real life does not exist in the diagram, this is similar to the human brain to the object of the analysis and reconstruction of the ability.
Image analogies– uses deep learning to deform a picture.

Deep Q Net-enhanced network for AI to play games automatically

Googledeepmind In addition to do go software and to achieve automatic game AI, human learning is not a supervision and non-supervision of the process, is a reward and punishment mechanism, you do the right time there will be good stimulation, for example, I cried, my mother came over to bring the rice, I ate, very happy, I may cry again next time I'm hungry. This is the same system, and then take some strategy to get a higher score, he will remember this strategy. The picture below is a space war game, using the program to play the game has surpassed the world to play this game player's highest level.

A picture question and answer system for dynamic memory network implementation

We can take a look at this picture, and on the left we use a dynamic based on the lstm long short-term memory network to understand a language and answer questions. And the right is directly to the picture to ask questions and let the computer answer, the use of technology is a dynamic memory network.

At present, we can do this degree, ask the top left corner of the bus color is what, and finally converted to a language answer, although the answer is just a simple word, but the fact that deep neural network has understood your problem, at the same time in the picture to understand the relevant elements, and then to resolve, to answer your ability. Several other diagrams are similar concepts.

Visual genome-Next generation image recognition public data set
108,249 Images
4.2 Million Region Descriptions
1.7 Million Visual Question Answers
2.1 Million Object Instances
1.8 Million Attributes
1.8 Million Relationships
Everything Mapped to Wordnet synsets

In fact, the development of deep learning learning is inseparable from the researchers on the data set exploration, before we have a well-known data set called Imagenet, there are millions of pictures to let deep learning network training and testing. Now it's upgraded, called visual genome, not just to classify the pictures, but also to see what's relevant, such as a woman, a hat, a hat, and a relationship. She's holding the guitar and playing it, and we're looking for a relationship between different objects.

This figure is a simple example, the depth of neural network can be resolved this image has two adults and children, children throw Frisbee, adults are looking, we want to relate to the relationship, dynamic relationships are dug out.

Image recognition and NLP, using deep learning to parse structured information in images and generate descriptive language.

Let's see what we can do in this picture, we can let the deep neural network first try to understand the structure of the painting, and then describe the picture in language, such as creating a paragraph: There is a tree to the right of this picture, there is a tower on the left, the tower has a spire and a tower body, there are three windows, a door. There are many people standing in front of the tower.

Word embedding, or distributed representation, a Chinese word vector, is a quantitative representation of words learned using deep learning, with the following characteristics:
King–man? Queen–woman
Words that are similar in meaning at the same time, have similar distances in spatial positions.

Word vectors turn Our common words into a point in space, what are the characteristics of the point: if the lexical meaning is similar, the position in the space should also be similar, the upper left corner can be found many points are the city, although the depth of the neural network do not know Beijing, London in what place, there are no buildings are not known, But through a lot of learning to come out of the city concept, and put them in a very close position in space. We do not have any language and data to teach it, is he through a lot of learning to find themselves.

Construct the depth knowledge Atlas of listed companies in a-share market, and provide relational mining decision analysis. The Knowledge Atlas can relate the investment and financing, the up-down tour, the competition and so on, thus showing the whole picture of an enterprise in the industry.

We learn to build a depth of knowledge atlas can be used in enterprise relationship mining, there is a listed company, such as lithium battery, you can find his investment and financing enterprises, and will go up and down the competition relationship between all together, these enterprises will have information transfer, if the network is built enough to have a model, Analyze and forecast the future trend of the listed company's stock price.

There is also an application is speech recognition, Baidu recently has a speech recognition, called bidirectional cyclic neural network bdrnn, can be identified each syllable, the pronunciation of a slight number of accents and errors, but also to the general meaning of the correct identification. This is a visual image of speech recognition, we reduce the speech signal to a plane map, you will find that the same vowel syllable and consonant syllable in the plane is very similar, are abstracted into a point adjacent to each other, it shows that it really understand the voice of the meaning of the audio signal represents.

Diagnosing heart disease from MRI images and videos, and more scientifically replacing the naked eye to establish a diagnosis model
One of the more social values is that deep learning can be an important basis for medical diagnosis, a particularly well-known heart disease diagnosis competition last year, when the final winning team at the tournament was able to achieve even more accuracy than the level of expertise. Tens of thousands of images are given to the Web learning discipline of deep learning, with the correct answer being discussed by five experts, but the level of the computer exceeds the diagnostic level of a single expert.

Deep learning in Bioinformatics: depth learning in the biomedical field, such as medical image processing, medical signal processing has a very good application base

There is also an application of the field, the analysis of DNA, there are many genetic disease is caused by a genetic mutation, may not be a certain one or two nodes, may be at the same time there are thousands of nodes have a problem, let people judge exactly how the combination will be wrong is impossible. This time deep learning can tell us that after we get a person's DNA, we can automatically analyze how much you are likely to get a disease in the future, and you can prevent it early.

Let's see why deep learning is so powerful. Deep learning is a process of continuous abstraction of features, we give him a picture, deep neural network first extracted points and edges, and then the combination of adult local organs, such as an eye and nose, the local organs can be stitched into one face, face appearance differences, We use the template to match the most similar to see if there is a human face, deep learning is very much like human learning process, you must be a layer of abstraction to understand the deeper concept, the reason is called depth is a multi-layered learning network, each layer is to the characteristics of the abstract higher-order concept, understand very complex things.

This is the result of deep Learning network visualization, we give a recognition of the number of neural network a number ' 8 ' diagram, you can clearly see each layer of neural network on the original image of what characteristics of the transformation.

This is a deep learning common convolution structure, the details do not speak, you can feel, among them the main convolution layer, max-pooling layer, as well as Relu Activation.

Auto-encoder (layer-wise Training), RBM, DBN
? Prelu, Rrelu
? Dropout
? RNN, LSTM
? Max-out
? Highway (residual Net)
? Batch Normalization
? Weight Normalization

As the research progresses, deep learning also has a variety of variants and components, some of which are the latest research on deep learning.

We talk about the application of deep learning in our project, we have a large manufacturing customer, they have a fault prediction project, what can we do? Deep learning in addition to the ability to model more than ordinary strong, can also learn the structure of time series, equipment sensor data is a time series, every second or how many milliseconds product signal, we use the traditional method difficult to deal with such high latitude, so large data volume model France, deep learning can understand the relationship in time, Greatly improve our prediction of fault classification.

The other is in the bank's model of bad customer detection, we have hundreds of-dimensional savings, consumption, credit characteristics if we ask experts to do very difficult, because many times, when you have too many features, it is difficult to think of so many combinations of rules, with deep learning can be automated feature combinations, For example, I found that my bank's savings is very high, but may be suddenly taken out at the end of the month, it may represent that I may just temporary in the inside, with other people borrowed money in the inside, not I have such a high capital to do the mortgage, this time when found, can be ruled out, This may be more than the efficiency of many industry experts.

A manufacturing failure analysis and prediction, millions of times of the sensor signal detection value of the time series analysis, using CNN and RNN modeling, error classification and prediction.

A bank bad customer detection, the customer hundreds of in-line savings, consumption, credit characteristics, as well as dozens of of the characteristics of the off-line label, the use of deep learning to automatic feature combination, extraction of high-level features, the detection accuracy greatly improved

Deep learning for our datainsight data mining platform is the forefront of the important direction, we will introduce a soft and hard integrated solution, will also use the Tesla GPU to do deep learning accelerator.

NVIDIA DGX-1?
The world's first system specifically tailored for deep learning. Its revolutionary performance greatly shortens the training time of the neural network, and the performance is equivalent to a cluster of 250 CPU servers.

NVIDIA Tesla? P100 Accelerator.
First video card with Pascal architecture
Owns 18 billion transistors
Using NVIDIA Nvlink?
Manufacturing process using 16nm FinFET
The Tesla P100 is not only the most powerful GPU accelerator today,
It's also the most technologically advanced GPU chip.
Distributed deep learning system for Datainsight

Based on the TensorFlow distributed version of the scenario, the CPU and GPU of each server in the cluster can be utilized simultaneously
Spark-based distributed scenarios: Elephas (dependent keras), sparknet, Caffeonspark (dependency Caffe)
Distributed version scheme based on CNTK, mxnet, can only use CPU or GPU in the cluster

Datainsight will fully support the above three scenarios, and the non-algorithmic network structure of the whole package, only exposed to the user's modeling helpful parameters, simplifying the work of the user to build a distributed system.

Application of deep learning in data mining