[PGM] basis of probability Graph Model

I. PGM

Figure from: http://blog.sina.com.cn/s/blog_5980285201011z75.html

 

Ii. Reasoning Based on the probability graph model. Example:

For example, if a message is sent through a noisy channel, the information is composed of 0 and 1. Due to the noisy, each bit has a probability of 20% error. Due to the characteristics of the sent signal, the probability of 1 followed by 1 and 0 followed by a 0 is 0.9. If a bit string "1101" is received, what is the most likely correct information?

The probability dependency in this example can be expressed as the following figure:

The hollow dots S1, S2, S3 ,... indicates the sent bits; R1, R2, R3 ,... indicates the received bits. They are the amount of observed values, so they are represented by solid dots. This is a directed graph. The joint probability distribution of a directed graph is:

P (x1, x2,...) = 1_p (XI | variables represented by all parent nodes of xi)

For example, the joint probability distribution is:

P (S1, S2, s3 ..., r1, R2, R3 ...) = P (S1) P (s2 | S1) P (S3 | S2 )... P (R1 | S1) P (R2 | S2) P (R3 | S3 )...

We also know that:

P (r1 = 1 | S1 = 1) = P (R2 = 1 | S2 = 1) =... = 0.8, P (r1 = 0 | S1 = 0) = P (R2 = 0 | S2 = 0) =... = 0.8,

And P (s2 = 1 | S1 = 1) = P (S3 = 1 | S2 = 1) =... = 0.9, P (s2 = 0 | S1 = 0) = P (S3 = 0 | S2 = 0) =... = 0.9.

To make reasoning and computing easier, we will further convert it into a factor graph ). A factor chart is a undirected two-part graph. Some nodes represent variables and some nodes represent factors. Factor nodes connect nodes that may have probabilistic dependencies, this allows for more intuitive representation of the probabilistic conditional independence relationship. For example, it will be represented as a factor chart (assuming that only the first four bits are considered ):

The entire factor chart has a probability joint distribution P (S1,..., S4, R1,..., R4) = semi fi (XI ),XI is a set of variable nodes connected by factor node fi.

F1 (S1, R1) = P (S1) P (R1 | S1); F3 (S2, R2) = P (R2 | S2), F5, F7 are similar to this; f2 (S1, S2) = P (s2 | S1), F4, F6 is similar to this.

Since this graph is a tree structure, in order to obtain the maximum joint probability distribution, we can break down the problem into the maximum probability of the subtree structure and then combine it. First select a variable node S3 as the root node, and then use the max-product algorithm to compute the data from the leaf node to the root node in the form of message transmission:

The leaf variable node sets the message sent to the factor node to 1, ur1-> F1 (R1) = 1;

The factor node will multiply all input messages by the factors expressed by itself and calculate the maximum value:

Uf1-> S1 (S1) = max {F1 (S1, R1) * ur1-> F1 (R1)} = max {P (S1) * P (r1 = 1 | S1)} = 0.5 * P (r1 = 1 | S1), where P (S1 = 0) = P (S1 = 1) = 0.5.

The variable node multiplied the received message to the factor node: US1-> F2 (S1) = 0.5 * P (r1 = 1 | S1 ).

Similar to the above:

Uf2-> S2 (S2) = 0.5 * maxs1 {P (s2 | S1) * P (r1 = 1 | S1 )},

Ur2-> F3 (R2) = 1,

Uf3-> S2 (S2) = max {F3 (S2, R2)} = P (R2 = 1 | S2 ),

Us2-> F4 (S2) = uf2-> S2 (S2) * uf3-> S2 (S2) = 0.5 * maxs1 {P (s2 | S1) * P (r1 = 1 | S1)} * P (R2 = 1 | S2 ),

UF4-> S3 (S3) = maxs1, S2 {P (S3 | S2) * 0.5 * maxs1 {P (s2 | S1) * P (r1 = 1 | S1 )} * P (R2 = 1 | S2 )}},

Obtain the maximum value when S1 = 1, S2 = 1, that is, UF4-> S3 (S3) = 0.288 * P (S3 | S2 = 1 ),

Ur4-> F7 (R4) = 1,

Uf7-> S4 (S4) = P (r4 = 1 | S4 ),

Us4-> F6 (S4) = P (r4 = 1 | S4 ),

UF6-> S3 (S3) = maxs4 {P (S4 | S3) * P (r4 = 1 | S4 )},

Ur3-> F5 (r3) = 1, UF5-> S3 (S3) = P (R3 = 0 | S3 ),

Therefore, Pmax = max {UF4-> S3 (S3) * UF6-> S3 (S3) * UF5-> S3 (S3 )}

= Max {0.288 * P (S3 | S2 = 1) * maxs4 {P (S4 | S3) * P (r4 = 1 | S4 )} * P (R3 = 0 | S3 )},

In this formula, the maximum value is obtained when S3 = 1, S4 = 1, and the maximum value is Pmax = 0.0373248.

In summary, when S1 = S2 = S3 = S4 = 1, the maximum value is obtained. That is, when the received bits are 1101, the most likely sending string is 1111. We can see that when the probability chart is a relatively simple structure, such as a tree or a chain, we can use the graph structure to simplify the calculation. When the probability chart is a more complex graph structure, direct computing is very difficult. In this case, you can solve the problem by simplifying the problem to an approximate tree or chain structure.

Reference: http://hi.baidu.com/linecong/item/b3e3e8f17d2b72d642c36a8e

 

Iii. Factor