Experiment Analysis of the Relationship between search engine crawling, indexing, and Web page depth

Experiment Analysis of the Relationship between search engine crawling, indexing, and Web page depth

The crawling policy has already been mentioned before. But what is the real relationship between the web page depth and the search engine crawling and indexing? Recently, Seo has analyzed the indexing of a large website in foreign countries, reflecting the relationship between Google indexing and page depth, and reflecting some crawler rules.

• 1. Relationship between webpage depth and crawling and Indexing
  
• 2 Seo Discussion
• 3. More Seo Basics

Relationship between web page depth and crawling and Indexing

1.The layered structure of large websites and the crawling characteristics of search engines

A typical large website generally uses the following hierarchy:

However, the crawler crawling priority follows the principle below. Generally, top-level pages are preferentially crawled, but some final content pages, that is, some underlying pages, it also gets a high crawling priority. According to the crawler policy described above, these pages may obtain a certain number of external links, such:

This is just a theoretical understanding. The Google administrator forum also said that page depth 6 is acceptable, but what is the relationship between page depth and indexing and crawling?

 

2. Web page depth and search engine crawling and index analysis test

The analysis object is a large-scale animal and pet classification information website in Poland. The total number of pages exceeds 0.1 million. The analysis object is based on the paging page of the classification directory of the website, use the SITE Command for analysis, such as site: www. morusek. PL inurl: "/0/" inurl: ogloszenia, indicating that all URLs contain "/0/" pages, this is because the first page of a category directory page of this website is marked with "/0/" in the URL. Similarly, "/1/" indicates the second page, in this way, the number of indexed items on the first page of the classification directory page obtained by the site command does not take into account the inaccuracy of the SITE Command. Similarly, the number of indexed items on the second page is obtained for a clearer expression, the following is a page navigation bar under the category directory page:

The probability of page indexing is analyzed by using the site command, as shown in:

Because the SITE command is not accurate, we can analyze the index probability by using the internal link tool in the Google administrator tool to obtain the index probability chart by page as follows. We can see that the overall trend is similar, from the approximation line, we can see that as the page depth deepens, the index probability decreases from 1.2% to 1.3%:

As you can see, from the analysis results of the internal chain function of the Google administrator tool, the index rate decreases sharply from the fifth page. If you change the page navigation bar above to the following:

After the modification, the effect is very obvious, and the probability of indexing on pages 10th and 15 increases significantly, as shown in:

Seo Discussion

The factors in the above experiment may not be so comprehensive, but some important problems can be reflected from the aspect:

1. As the depth increases, the index probability will be greatly reduced, and the number of pages that are very deep is generally huge. The indexing volume seriously affects the number of internal chains.

2. at the same time, you can modify the pagination navigation bar. Adding a link to the portal page will only increase the probability of the index of the linked page, but will not show the index probability of the page approaching the page, it indicates that the depth of these adjacent pages is still deep, and the index probability is greatly reduced due to the step difference.

Therefore, in case of a large number of pages, we should minimize the depth of the page, so we can only increase the number of single-page links and reduce the number of pages to a minimum, it is enough to put down the paging link on the portal page, which is often one step poor and very different. For more information, see testing how crawl priority works.
.

More about Seo technology and basics

Crazy multi-domain-rich keyword website Optimization Strategy Web Crawler policy Introduction Five most important factors in Google's ranking Optimization Uncover Google News ranking factor Google caffeine search results

Set
Ben
To: 'yahoo ', 'scrollbars = Yes, width = 440, Height = 440, Left = 80, Top = 80, status = Yes, resizable = Yes '); "Style =" color: #1d58d1; text-Decoration: none; "> Void
0 "style =" color: #1d58d1; text-Decoration: none; "> 365key 'Postbookmark', 'scrollbars = No, width = 600, Height = 450, Left = 80, Top = 80, status = No, resizable = yes '); "Style =" color: #1d58d1; text-Decoration: none; ">   (). Text: '') :( D. getselection? D. getselection (): ''); void (vivi = Window. Open ('HTTP: // vivi.sina.com.cn/collect/icollect.php? PID = 2008 & Title = '+ escape (D. title) + '& url =' + escape (D. location. href) + '& DESC =' + escape (t), 'vivi', 'scrollbars = No, width = 480, Height = 480, Left = 75, Top = 20, status = No, resizable = Yes '); Vivi. focus (); "style =" color: #1d58d1; text-Decoration: none; ">