Bayesian Network Classification algorithm

1: Definition and nature of Bayesian networks

A Bayesian network definition includes a directed acyclic graph (DAG) and a set of conditional probability tables . Each node in the DAG represents a random variable, which can be either a direct or a hidden variable , whereas a forward edge represents a conditional dependency between random variables, and each element in the conditional probabilities table corresponds to the only node in the DAG, storing the joint conditional probabilities of this node for all its direct precursor nodes.

One of the most important properties of Bayesian networks is that we assert that each node is independent of all its non-direct predecessor nodes after the value of its direct precursor node has been established.

This property is similar to the Markov process. In fact, Bayesian networks can be seen as a nonlinear extension of the Markov chain. The significance of this feature is to make it clear that Bayesian networks can easily calculate joint probability distributions. In general, the multivariate non-independent joint conditional probability distribution has the following formula:

In Bayesian networks, the combined conditional probability distributions of arbitrary random variables are reduced due to the existence of the aforementioned properties.

Where parents represents the union of the direct precursor node of XI, the probability value can be found from the corresponding conditional probability table.

Bayesian networks are more complex than naive Bayes, and it is extremely difficult to construct and train a good Bayesian network. But Bayesian network is a model of human cognitive reasoning, which is modeled by a set of conditional probability functions and the causal inference relation of the non-circular graph to the uncertainty, so it has higher practical value.

2: Structure and learning of Bayesian networks

The construction and training of Bayesian networks is divided into the following two steps:

1. Determine the topological relationship between random variables and form a DAG. This step is usually done by a domain expert, and in order to build a good topology, it is usually necessary to iterate and improve.

2, Training Bayesian network. This step is to complete the construction of the conditional probability table, if the value of each random variable can be directly observed, like our example above, then this step of training is intuitive, the method is similar to naive Bayesian classification. But there are usually hidden variable nodes in Bayesian networks, so the training method is more complicated, such as using gradient descent method.

3: Applications and examples

Bayesian network, as an uncertain causal inference model, is widely used in medical diagnosis, information retrieval, electronic technology and industrial engineering, and some problems related to it are also the hot research topics recently. For example, Google uses a Bayesian network in many services.

In terms of using methods, Bayesian networks are mainly used for probabilistic inference and decision-making, in particular, in the case of incomplete information, we can infer the non-observable random variables by observing the random variables, and can not observe the random variables more than one, the general initial phase of the non-observable variable to random values, and then the probabilistic reasoning. Let's give an example.

Example: an example of an unreal account detection in the SNS community, there are four random variables in our model: Account authenticity R, Avatar authenticity h, log density L, friend density F. Where H,l,f is a value that can be observed, and the r we care about most is not directly observable. This problem is classified as probabilistic inference for R through the H,l,f observation value. The inference process can be expressed as follows:

1. Use observation values to instantiate h,l and F and assign random values to R.

2, calculation. The corresponding probability value can be found in the conditional probability table.

Since the above example has only one unknown random variable, no iteration is necessary. More generally, the steps for reasoning using Bayesian networks can be described as follows:

1. Instantiate the observation value for all observable random variable nodes; the non-observable node is instantiated as a random value.

2, the DAG is traversed, for each non-observable node y, the calculation , where WI represents all the nodes except Y, A is a regularization factor, SJ represents the J child node Y.

3, using the third step calculated by each y as the unknown node of the new value of the instantiation, repeat the second step, until the result is fully convergent.

4, the convergence results as inferred values.

Reprinted from: Http://www.cnblogs.com/leoo2sk/archive/2010/09/18/bayes-network.html Zhang Yang

Here are the additions:

4: Other Bayesian network inference algorithms

Bayesian Network Classification algorithm