Search engine development history

In the early stages of Internet development, there were relatively few websites and it was easier to find information. However, with the explosive development of the Internet, ordinary network users want to find the required information as if they were looking for a needle in a haystack. At this time, professional search websites that meet the needs of the public for information retrieval came into being.
The ancestor of modern search engines was Archie invented by Alan emtage, a student at the University of Montreal in 1990. Although the World Wide Web was not available at the time, the file transmission in the network was quite frequent, and the query was very inconvenient because a large number of files were scattered on various scattered FTP hosts, therefore, Alan emtage thought of developing a system that can search for files by file name, so he had Archie.

The working principle of Archie is very similar to that of the current search engine. It relies on scripts.ProgramAutomatically search for files on the Internet, and then index the relevant information for the user to query with a certain expression. Inspired by Archie's popularity, the University of Nevada system computing services developed another very similar search tool in 1993. However, in addition to index files, you can search for webpages.

At that time, the word "robot" was very popular among programmers. A computer robot is a software program that can continuously execute a task at a rate that cannot be achieved by humans. Because the "robot" program used to search information crawls between networks like a spider, the "robot" program of the search engine is called a "Spider" program.

The world's first robot program used to monitor the scale of Internet development is World Wide Web wander developed by Matthew Gray. At first, it only used to count the number of servers on the Internet, and then developed to be able to retrieve website domain names.

In contrast to wandreer, Martin Koster created aliweb in October 1993, which is the HTTP Version of Archie. Aliweb does not use the "robot" program, but relies on the website to actively submit information to create its own link index, similar to Yahoo, which we are now familiar.

With the rapid development of the Internet, it becomes more and more difficult to search all new web pages, some programmers have improved the working principle of the traditional "Spider" program. The idea is that since all webpages may have links to other websites, it is possible to retrieve the entire Internet from tracking the links of a website. By the end of 1993, some search engines based on this principle began to emerge, with jumpstation and the World Wide Web worm (the predecessor of Goto, that is, overture today ), and repository-based software engineering (rbse) spider.

However, jumpstation and WWW worm only sort the search results by searching tools in the order of matching information in the database, so there is no information correlation degree. Rbse is the first engine to introduce the keyword string matching degree concept in the search result arrangement.

The earliest search engine in the modern sense appeared in July 1994. At that time, Michael mauldin connected John Leavitt's Spider Program to its indexing program and created Lycos, which is now well known. In April of the same year, two doctoral students at Stanford University, David Filo, and Gerry Yang, a US-based Chinese, jointly created a super-Directory Index Yahoo, the concept of search engine is deeply rooted in the hearts of the people. Since then, search engines have entered a period of rapid development. Currently, there are hundreds of search engines with names on the Internet, and the amount of information they retrieve is different from what they used to be. For example, Google, which has recently become popular, has stored 3 billion million webpages in its database!

With the rapid expansion of the Internet scale, a search engine alone cannot adapt to the current market conditions, so there is a division of labor and collaboration between search engines, with professional search engine technology and search database service providers. Like Inktomi outside China, it is not a user-oriented search engine, but overture (formerly goto) other search engines, such as looksmart, MSN, and Hotbot, provide full-text Web search services. Baidu in China also belongs to this category (Note). Sohu and Sina use its technology. Therefore, in this sense, they are the search engines of the search engine.

-- End --