Statistical language model (statistical language models) (conditional probability formula)

Statistical language model (statistical language models)

Conditional probability formula P (S) = P (W1) P (W2 | W1) P (W3 | w1w2)... P (WN | w1w2... wn-1)
P (W1) indicates the probability that the first word W1 appears. P (W2 | W1) indicates the probability that the second word appears when the first word is known. It is not hard to see that the probability of occurrence of the word wn depends on all the words above it. From the computing point of view, there are too many possibilities to achieve. Therefore, we assume that the probability of occurrence of any word WI is only related to the word wi-1 before it (I .e. Markov hypothesis), so the problem becomes very simple. Now, the probability of S appears as follows:

P (S) = P (W1) P (W2 | W1) P (W3 | W2 )... P (WI | wi-1 )...
The next question is how to estimate p (WI | wi-1 ). Now with a lot of machine reads, this problem becomes very simple, as long as the number of times the word (wi-1, WI) appears in the statistical text, and the wi-1 itself in the same text before and after the adjacent appear how many times, and then use two a few division can p (WI | wi-1) = P (wi-1, WI) /P (wi-1)

 

P (x) P (Y | X)
P (x | y) = -----------
P (y)

References: http://googlechinablog.com/2006/04/blog-post.html