Brief discussion on search engine's Word segmentation algorithm understanding full-Text Search technology

Intermediary transaction http://www.aliyun.com/zixun/aggregation/6858.html ">seo diagnose Taobao guest cloud host technology Hall

The rapid development of the Internet in the 21st century to make people's lives more and more convenient, when the increasing proliferation of mass information let us dazzling, search engine appearance can let us quickly find the answer we want. So learn more about search Engine segmentation algorithm, you can let the site in search engines to get a better opportunity to show. Before we explain the Chinese word segmentation technology, first to understand the full text search technology.

Full-Text Search technology

Full-Text search refers to the indexing program scans each word in the article and establishes the corresponding index, recording the location and number of occurrences of the word. When searching through a search engine, the retrieval program searches the index of the record and returns it to the user. Full-text search is divided into Full-text indexing based on words and full-text indexing based on words. A Full-text index based on words indexes and records each word in the content, this method searches for a high rate, but the rate of registration is low, especially for Chinese, and sometimes the search marks will list Marx's results. A word based Full-text index records a word as a unit and can handle synonyms. Search engine has its own thesaurus, when users search, the search engine will extract keywords from the thesaurus as an index, which can greatly improve the accuracy of the retrieval.

Chinese Word segmentation technology

Everyone has been more familiar with Baidu, Baidu has its own Chinese word segmentation technology. General use includes forward maximum matching, reverse maximum matching, best matching method, expert system method, etc. The largest forward matching is the most commonly used word segmentation solution, it uses the mechanical algorithm, through the establishment of dictionaries and the forward maximum matching to Chinese word segmentation. For a simple example, for example, search "where is Peking University", the result is many of the results are included in Peking University, Peking University and other words of the Web page, the search engine is to use the largest matching to judge, the Peking University as a word to index records and return. Of course, the maximum matching also has incompleteness, such as long length of words, search engine sometimes can not be accurate participle, or both before and after the interrelated words can not be accurate participle. For example, "when combined into molecules," will be returned to the binding, the composition, the midnight, and sometimes we want the keyword is "molecule."

Most of the time Baidu will be based on their own words in the vocabulary of the weight of the split, the weight of the calculation based on all aspects of life, more complex, search engine to do is to return the user most want the results, sometimes webmaster do site to stand in the user's point of view to consider the problem, in fact, this is the point of view of the search engine to consider the problem, Whether in the determination of the target keyword or long tail keyword, can be based on the principle of Chinese word segmentation to choose, so that can maximize the reduction of no effort.

The principle of participle constantly changing, constantly in the update, we should continue to learn, only mastered the essence can grasp the essence. This article from the Shenzhen website Construction, the original address: http://www.68160.com, welcome everyone and I exchange, later on more word segmentation technology, especially Chinese word segmentation technology more applications I will continue to share with you.