Simple English-Modified Word Clustering Algorithm

Sometimes, simple algorithms are actually useful. Due to the needs of the company's internal search engine optimization, I used the algorithm for searching for English words in programming Pearl River, to implement the search error correction function.

Sometimes words cannot be remembered during a search, and spelling errors may occur. For example, when searching for "height", you may find "heigth" by mistake ", in this case, we need to give a prompt () for the modified word if we cannot find it ().

The algorithm consists of three steps: ① sign the word; ② sort the word according to the Lexicographic Order of the signature; ③ squeeze the word according to the sorting result to aggregate the modified words together;

A) Quick word signature algorithm (currently only used for English words. If it contains other characters, it must be expanded ):

Given that only 26 English letters can be enumerated, the transformed base sorting algorithm is used to implement quick signature:

Step 1: Defines a 26-bit int [] array. The array ranges from 0 ~ 25 represents A, B, C, D ...... X, Y, Z (1), Initialization is all 0:

A	B	C	D	E	F	G
H	I	J	K	L	M	N
O	P	G	R	S	T	U
V		X	Y	Z

 

 

 

 

 

 

(Figure 1)

Step 2: Convert English letters (CASE), such as Jary → jary, and then use the following formula: Save the subscript as the position of the x-97, for example, a ascii code is 97, then it corresponds to the position 97-97 = 0 in the byte [] array, that is, the position increases by 1, and so on.

Stpe 3: Sign a word and scan the array from 0 to 25, for example, what → ahtw, wath → ahtw, system → ems2ty;

In this way, we can obtain the signature of a word by scanning only once. The Code is as follows:

    SignWord(      [] dict =  [       array =     ( a          dict[a - ]++      StringBuilder sb =       ( i = ; i < ; i++          (dict[i] !=               t = dict[i].ToString().Equals() ?              sb.Append(()(i + ) +        }

B) Squeeze words with the same signature. For example, if "what" and "wath" have the same signature, they are put together. We can sort the words by the signature dictionary. However, the Dictnary provided by C # can be used to quickly sort and pressurize. The Code is as follows:

   Dictionary < ,List <  >> SqueezeList(      Dictionary < List <  >> wordDict =  Dictionary < List <  >>     StreamReader fs =            (!.IsNullOrEmpty(line =          key =                   }               list =  List <  >            }

In this way, words are put into the Dictionary using the signature as the Key. Of course, in actual application, we will serialize the Dictionary into a text file, in this way, you do not need to calculate it every time you use it;