Search engine--a technical dream of capital game

Intermediary transaction SEO diagnosis Taobao guest Cloud host technology Hall

From Microsoft cost millions of dollars, Yahoo successively buys three manufacturers, to the domestic search engine manufacturer's infighting, all value the search engine latent huge commercial value. However, the madness of capital cannot conceal the light of technology.

"The more invisible the technology, the more profound, because they have been fully integrated into the daily life." ”

In a stream of subway tunnels in Beijing, a row of posters is particularly striking, and this is the movie poster of Lord of the Rings-king Invincible, the 11 grand prize at the Oscars, and Gandalf the Shenfeng and beautiful Liv Taylor are tempted to enter the distant Middle Ages. The poster has a large area for corporate propaganda, and 8848 of the company's logo impressively displays it. This is 8848 companies in order to cooperate with the launch of the business site propaganda, its play is the launch of the "Chinese shopping search engine." In this respect, some people say: "8848 will hold in the hands of the more than 20 million dollars this treasure on the search engine." ”

Such a big money is not only 8848, just from SoftBank and other investors to obtain 82 million U.S. dollars Alibaba also recently officially launched a long rumored search products, and news search and competitive rankings search, Alibaba search target use group is not ordinary netizens, but "network business", Mainly publishes business information and business opportunities. Alibaba CTO Wu said, "The first time we introduced and established a credit certification and security system in the search field." ”

However, these are the industry's search area, based on the whole network of search engine competition between the increasingly popular. HC International in the introduction of the State Council under the Information Office of the Universal Bridge Culture Communication Company's funds, began operating in search of the net, and the registered capital increased to 7.5 million yuan, the company's main business is the search engine.

In the face of these threats, China's largest Chinese search engine Baidu Company is happy to count money, profit nearly billion harvest makes it in the search engine market has achieved absolute advantage. However, in the face of so many eyeing competitors, Baidu also dare not relax, large-scale expansion is underway, including the largest proportion of technical personnel. The original study of natural language graduates find the status of the job is completely changed, once in Microsoft engaged in natural language research Zhang said: "My two younger brother has been Baidu recruit in." ”

International competition is also suffocating, Microsoft to enter every area will make the original manufacturers jittery, Microsoft's search engine is also accompanied by large-scale recruitment, Microsoft also set up a special team, but Google face these challenges are still full of confidence, It is said that a technical master to Microsoft a few days later joined the Google. The good working conditions and the search culture created by Google can be seen in the appeal of technical personnel.

Therefore, although the profit model created by Overture has suddenly made the huge profits of search engines make the capital covet, in the market competition, technology is the most basic guarantee.

The principle of search engine technology

The principle of search engine technology is actually very simple: generally divided into three parts, the first is to use spiders (Spider) for the full web search, automatic crawl Web pages; The next is to crawl the Web page according to the keyword index, but also will record and retrieve related attributes, Chinese search engine also need to first Chinese word Finally, the result is retrieved and returned to the user by retrieving the generated index file and making a complex calculation according to various parameters. Some people think that the search engine interface design can be counted as a new part. This will gradually improve the user experience. In addition, the search engine's ancillary functions include distributed computing module, as well as a set of background monitoring system. In these sections, the core is the ranking of search results, how to put the most appropriate results to the front. Therefore, it can be said that all the other links are prepared for the final calculation.

But the actual retrieval effect is influenced by many factors. Spider stability and grasp the full rate is very important, the first search engine only crawl static pages, now the search engine is required to crawl more dynamic sites, so need to contain script statements of the page to parse, at the same time the wide application of Flash also requires search engines can parse the text and hyperlinks. Massive technology has been studying Chinese word segmentation technology and search engine technology, its chief engineer Wang Dongshe said: "Some sites to prevent the download also did a lot of traps, and sometimes need to analyze the results, although this part of the technical difficulty is not too high, but this is malt." ”

Subsequent format conversions and index creation require a deep technical foundation. The difficulty with indexing is to optimize the storage data structure as much as possible to fit the needs of the search. In this respect, the technology of each search engine company is dissimilar. But it's a common goal to minimize memory, CPU usage, and number of reads. Although some people have very good ideas, but not necessarily with the current technology is easy to achieve. Because of the huge amount of data the search engine needs to deal with, the unrestricted offset and the increase of the attribute may lead to the system's transition expansion and the retrieval speed decreasing. Wang Dongshe said: "The massive in this aspect has developed own independent algorithm, this kind of compression format does not need to decompress, can carry on the computation directly, this can save the resources and the efficient computation." The