Microsoft Data Mining algorithm: Microsoft Neural Network Analysis Algorithm principle (9)

Objective

This article continues our Microsoft Mining Series algorithm Summary, the previous articles have been related to the main algorithm to do a detailed introduction, I for the convenience of display, specially organized a directory outline: Big Data era: Easy to learn Microsoft Data Mining algorithm summary serial, interested children shoes can be viewed, Before starting the Microsoft Neural Network analysis algorithm, this article first makes a brief introduction to the neural network analysis algorithm, the algorithm because of its own complexity, so I intend to start before the beginning of the algorithm to make a simple summary, because the algorithm itself is subordinate to the research category of higher mathematics, We do not study the inference and validation process of the algorithm, only introduce the characteristics of the algorithm and application scenarios, and the limited personal technical capacity, the wrong place also hope not to spray.

Algorithmic origins

In the thought study, human brain's thinking is divided into: logical thinking, intuitive thinking, and inspiration thinking three basic ways.

The neural network is the second way to use its algorithm to simulate the brain thinking, which is a nonlinear dynamical system, which is characterized by information distributed storage and parallel collaborative processing, although the structure of single neuron and its simple function are limited. But if a large number of neurons constitute the network system can realize the behavior is indeed and colorful. In fact, the simple point is to use this algorithm to simulate the human brain for reasoning and validation.

We first briefly analyzed the human brain work process, I carefully found on the internet a barely look comfortable brain image

Well, look at the point, at least look comfortable, that still at the end of 19th century, there is a "Waldege" Daniel created the neuron study, he said the human complex nervous system is composed of a large number of neurons, said the cerebral cortex includes more than 10 billion neuron, each cubic millimeter source tens of thousands of, sweat .. I'm thinking of a typical big data. They connect with each other to form neural networks that receive information from outside the body through sensory organs and nerves (in neural network algorithms we call training) to transmit the central nervous system, then through the analysis and synthesis of information, and then through the motor nerve to send control information (such as I hit the blog in the text), In turn, to achieve the body and the external environment of the connection.

Neurons, like other cells, include nuclei, cytoplasm, and nuclei, but it's also very special, like a lot of projections, like the one above: cell, axon, and dendritic three. Cells have nuclei, and the role of protrusions is to transmit information. The role of dendrites is as a projection of input information, and the axon is a projection of the output, but it has only one.

That is, a neuron that has n inputs (dendrites), then undergoes information processing (the nucleus), and then only one output (axon). The neuron is connected with the axon of another neuron, and the information is transmitted and processed. I'll go... So complicated ....

Let's take a look at the principle of neural network algorithm formula

The middle one round is the nucleus, X1, X2, X3 ... Xn is the dendrites, and Yi is the axon .... Does that mean something, huh? Our external information is input through the dendrites of the neurons, then after processing in the nucleus, after Yi output, and then output to other neurons ...

But this algorithm has its own characteristics, like human brain neurons, when the input of the external information received every time, the constant stimulation will be based on the different information of their own adjustment, such as: through the constant training of athletes to learn the remote, nonstop training to learn to ride a bicycle .... Wait a minute. The nature of these human acts is formed by the constant training of tens of billions of neurons. And these actions are followed by the correct result-oriented.

The same algorithm will also pass X1, X2, X3 .... Xn These elements are constantly trained to adjust their own parameters to adapt, the same number of training increases to form a correct result-oriented. At this time we can use its own adaptation process to produce the correct results, and through constant training to make it a learning function, of course, the algorithm only reflects a number of basic characteristics of the human brain, but not a lifelike description of the biological system, but a simple imitation, simplification and abstraction.

The algorithm is different from the digital computer, will follow the procedure to perform the operation step by step, but can adapt themselves to the environment, summarize the rules, complete some kind of operation, recognition or control process, and this is the origin of the robot ... The basis of artificial intelligence.

Principle of neural network algorithm

Since the design surface of the neural network algorithm is too large, we will only analyze the principle of the Microsoft Neural Network algorithm here, in the Microsoft Neural Network algorithm, we can simplify the following image:

The Microsoft Neural Network uses a "multilayer perceptron" network of up to three levels of neurons: input layer, optional hidden layer, and output layer.

Input layer: The input neuron defines all the input attribute values and probabilities of the data mining model.

Implicit layer: The hidden neurons accept input from the input neurons and provide output to the output neurons. Hidden layers are locations that assign weights to various input probabilities. Weights describe the relevance or importance of a particular host to a hidden neuron. The greater the weight assigned by the input, the more important it is to enter the value. And this process can be described as the learning process. Weights can be negative, indicating input suppression rather than promoting a particular result.

Output layer: The output neuron represents the predictable attribute value of the data mining model.

Data from the input through the middle of the hidden layer to the output, the entire process is a front-to-back propagation of data and information process, the data value on the next layer of nodes from the front node connected to it, after the data weighted after a certain function operation to obtain a new value, continue to propagate to the next layer of nodes. This process is a forward propagation process.

And when the node output error, that is, and the expected difference, the neural network will automatically "learn", the next layer of nodes on the previous layer of a "trust" level (in fact, the weight of the connection is changed), to take the weight reduction method to punish, if the node output coarse and coarse oh, It is necessary to look at the errors affected by those input nodes, reduce the weight of the node connections that caused the error, punish the nodes, and increase the weights of the connections that make the correct recommended nodes. For those nodes that have been punished, the same method is used to punish the nodes in front of it until the input node stops. This is called: feedback.

And we learn the process is to repeat the above introduction of the process, through forward propagation to get input values, with feedback method to learn. When all the data in the training set has been run over, it is called a training cycle. After training, we get the neural network model, which includes the corresponding value of training concentration and the rule of the influence of predicted value.

In each neuron, there are complex functions in the hidden layer, and these are nonlinear functions, and similar to the basic transmission characteristics of biological neural networks, these functions are called: activation function, that is, the input value of small changes can sometimes produce large output changes.

Of course the function used by the Microsot Neural Network algorithm is this:

Where a is the input value, and O is the output value.

The error function used in the output layer to deal with the inverse propagation, calculate the error, and update the weights is the cross-entropy

In the above formula, Oi is the output of the output neuron I, and ti is the actual value of the output neuron based on the training sample.

The error of the hidden neuron is calculated based on the error and correlation weights of the neurons in the next layer. The formula is:

Where Oi is the output of the output neuron I, the unit has a J to the next layer of output. Erri is the error of neuron I, wij is the weight between the two neurons.

Once the error is calculated for each neuron, the next step is to adjust the weights in the network using the following methods.

Where L is a number in the range of 0-1, it is called the learning function.

In fact, the above function application of the activation function is quite simple. Interested in the detailed study and the calculation of the formula, we are just a brief analysis, listing the characteristics of the algorithm.

Microsoft Neural Network analysis algorithm features

Through the above analysis of the principle, we know that the neural network algorithm is divided into: input layer, hidden layer, output layer three-layer mode connection, wherein the hidden layer is optional, that is, if the Microsoft Neural Network algorithm does not pass through the hidden layer, the input will be directly from the input layer nodes in the output layer.

input Layer Characteristics : If the input layer is discrete, then the input neuron usually represents a single state of the input attribute. If the input data contains null values, the missing values are also included. A discrete input attribute value with more than two states generates an input neuron, which, if there is a null value, automatically regenerates an imported neuron to process the null value, and a continuous input attribute generates two input neurons: one for the missing state, and one for the continuous property itself. Input neurons can provide input to more than one neuron.

hidden layer features : Implicit neurons accept input from input neurons and provide output to the output neurons. There is an activation function for its use to change the threshold.

output Layer Features: output nerve if for discrete input attributes, the output neuron typically represents a single predictive state for predictable, predictable attributes, including missing null values.

If the mining model contains one or more properties that are used only for prediction, the algorithm creates a single network that represents all of these properties, and if the mining model contains one or more properties that are used for both input and prediction, the algorithm provider constructs a network for each of these properties.

For input attributes and predictable attributes with discrete values, each input or output neuron represents a single state. For input and predictable attributes with continuous values, each input or output neuron represents the range and distribution of the attribute value, respectively.

The algorithm provider compares the actual known values of each case in the holdout data to the predictions of the network by accepting the qualitative data set that was previously retained, which is the case collection. That is, through a "batch learning" process to iterate over the entire network of calculations, and change the input weights. After the algorithm processes the entire set of cases, the predicted and actual values of each neuron are examined. The algorithm calculates the degree of error (if error) and adjusts the weights associated with the neural input and returns from the output neuron to the output neuron through a "postback" process. The algorithm then repeats the process for the entire set of cases. After the above layer of precipitation our algorithm even from a non-understanding of "baby" gradually grow into "adult", and this result is our it to explore and predict the tool.

Neural network analysis Algorithm application scenario

Neural network research is extensive, not covered by this article, and it reflects the characteristics of multidisciplinary cross-technical fields. The research work focuses on the following areas:

(1) Biological prototype research. From physiology, Psychology, brain science, pathology and other biological sciences to study nerve cell, neural network, nervous system of biological prototype structure and its functional mechanism.

(2) Establish a theoretical model. Based on the research of Biological prototype, the theoretical model of neuron and neural network is established. These include conceptual models, knowledge models, physical and chemical models, and mathematical models. (3) Network model and algorithm research. On the basis of theoretical model research, a specific neural network model is built to realize computer simulation or preparation of hardware, including the study of network learning algorithm. This work is also known as the technical Model study. (4) Artificial Neural network application system. Based on the research of network model and algorithm, the artificial neural network is used to compose the practical application system, such as the function of signal processing or pattern recognition, the construction expert system, the robot and so on.

I'll go.... That's what I'm talking about. The study of the agricultural level, the above-mentioned all kinds if there are interested in children's shoes can continue in-depth study, any piece of play well can be promoted to the Tagalog, food and clothing without worry ...

To get back to the point, let's take a look at our Microsoft Neural Network analysis algorithms in the field of data mining applications those:

• Marketing and promotional analysis, such as evaluating the success of a direct mail promotion or a radio advertising campaign.

• analysis of manufacturing and industrial processes.

• text mining.

• Any predictive model that analyzes the complex relationships between multiple inputs and relatively few outputs.

In fact, it's the most widely used scenarios, such as when we get a bunch of data, when a goal is no clue, the Microsoft Neural network analysis algorithm is the best scenario for the application, because it uses the "human brain" characteristics to the vast ocean of data to explore useful information. For example: Boss threw the company's database to you ... Let you analyze the company why not make money ... Or what causes the non-profit ... This time the algorithm should be out.

However, from the recent market found that the algorithm in the "text mining" in the special fire, and more in Microsoft's full use, such as the current: Microsoft Xiaoice, various platforms under the speech recognition, and even the proud company such as Apple is slowly surrender to the charm of the algorithm, is interested to view this article: http ://www.yeeworld.com/article/info/aid/4039.html

Conclusion

This article concludes, the full article is just a brief introduction of the neural network algorithm of some basic and principle, of course, because the algorithm of the tall I do not do in-depth analysis, just remember its typical application scenario can be, Next article we will use Microsoft's VS tool to do a detailed data mining application method introduction. Interested can be noticed in advance.

Original address: (original) Big Data era: based on Microsoft case Database Data Mining Knowledge Point Summary (Microsoft Neural Network analysis algorithm principle)

Microsoft Data Mining algorithm: Microsoft Neural Network Analysis Algorithm principle (9)