Bayesian inference and its application to the Internet (III.) spell check

(The first part of this series introduces the Bayes theorem, and the second part describes how to filter spam, today is part three.) ）

When using Google, if you misspell a word, it will remind you of the correct spelling.

For example, you accidentally entered the seperate.

Google tells you that the word does not exist, the correct spelling is separate.

This is called "Spell check" (Spelling corrector). There are several ways to achieve this, and Google uses a statistical approach based on Bayesian inference. This method is characterized by fast, very short time processing a large number of text, and has a high degree of accuracy (more than 90%). Peter Norvig, Google's research and development director, wrote a famous article explaining how this approach works.

Let's take a look at how to use Bayesian inference to achieve a "spell check". Actually very simple, a small piece of code is enough.

First, the principle

The user entered a word. This is divided into two situations: the spelling is correct, or the spelling is incorrect. We remember the spelling correctly as C (on behalf of correct) and the spelling error in the case of W (representing wrong).

The so-called "spell check", is in the case of W, trying to infer C. From the perspective of probability theory, it is known that W, and then in a number of alternatives, find the most likely C, which is the maximum value of the equation below.

P (C|W)

According to the Bayes theorem:

P (c|w) = P (w|c) * p (c)/P (W)

For all alternative C, the corresponding is the same w, so their P (W) is the same, so what we are asking is actually

P (w|c) * p (c)

The maximum value.

The meaning of P (c) is that a correct word appears "probability" and it can be replaced with "frequency". If we have a large enough text library, the frequency of each word in the text library is equivalent to the probability of its occurrence. The higher the frequency of a word, the greater the P (c).

The meaning of P (w|c) is that the probability of a spelling error w occurs when trying to spell C. This requires statistical support, but to simplify the problem, we assume that the closer the two words are to the glyphs, the more likely they are to be misspelled, P (w| C) is bigger. For example, a spelling that differs by one letter is more likely to occur than a two-letter spell. If you want to spell the word hello, the likelihood of Hallo (a letter) is Haallo high (two letters).

Therefore, we can achieve the maximum value of P (w|c) * p (c) by finding the words closest to the glyph of the input word, and then picking out the one with the highest frequency.