Optimize website structure to attract spiders to crawl site content

Intermediary transaction http://www.aliyun.com/zixun/aggregation/6858.html ">seo diagnose Taobao guest cloud host technology Hall

Spiders come to the site is a good thing, but through the spider access record I found a very big problem, often spiders came to the site after crawling 4, 5 pages left, this is how? believe that a lot of webmaster feel that from the events of June, Baidu Spider is very unstable, my site is the same, Since June, spiders come to crawl a few pages at a time and go, this site included also has not been upgraded up, and finally I have a big check on the site, and then the site's links were collated, not to mention, after finishing the site, spiders each to crawl the page reached 20, 30, let me share my inspection methods and treatment methods.

I think the spider just started not deep in the reason the first one may be the navigation link is not set to, spiders can not access through the navigation links to the depths of your site, then how does it crawl content page? The second may be the spider encountered a dead link, search engine spiders ready to grab a page off the time but can not crawl, Then the site's dead link became the culprit, forcing Baidu Spider left the site. The third may be a long time does not update the content of the site will also cause Baidu spider tired.

First, reset the site navigation links

The navigation link of the website is the user's Guide line, search engine spider is also the guide line, a good navigation route can help search engine spider from the site's homepage step-by-step through the site's entire page, which requires us to set up a site to navigate the time to set up a layer, I would like to talk about setting up the navigation links of several requirements:

1, navigation links to the nearest principle, we set the navigation link when not to search engine spiders too abstruse, in fact, he is a tool to crawl, and the most easy to crawl is what is the most recent things. So we use the navigation link when we can through the navigation links to the next layer of the link, and the next column link to import content page, this is a layer of import methods.

2, the import of the URL is not too complex, I think the site URL can be set up a little bit simpler, the use of a PHP program, a simple directory settings, then spiders crawling crawl is relatively easy.

Second, the dead link must be cleared, leaving is the scourge

Dead links to the site's blocking role is very large, a little attention to dead links may have a fatal impact on the site, check dead links can take advantage of Chinaz tools, and clean up the dead link is relatively difficult. Now I clean the site dead link method is through the browser FTP search function, first in the query tool will be the death of the link to copy the address, copy and then through the browser FTP search function, find it in which file, and finally in the delete, a lot of webmaster will say I this method is very troublesome, But I want to say that this method is really the most effective way, those who clean up the dead link tool I also used, most of them are fake, can not complete the cleanup work.

Third, the article content link carries on the reasonable layout

We are in the content of the site also do not forget the link layout this form, in the regular article update work we can make a link to the key words in the article, point to other article content page or the site's column page, This search engine spiders crawl The content of this article will be next to the content of the crawl, which also indirectly improve the search engine Spiders crawl page progress and strength, we can think about, a central remit out of countless points, spiders can not happy? This means that he can eat more food, This is also the key point of our article content page link layout.

Through the previous three ways to make the structure of the site to show the trend of progressive, through the introduction of layers, spiders crawl methods so that the site's collection has been greatly improved, 8630.html "> sometimes search engine spiders do not want to crawl your site, but your site rubbish too much, Every other week to check the site's links are also necessary, which is to improve the search engine crawling a factor. This article comes from Shanghai Car rental http://www.sqzlsh.cn, reprint please keep the author link, thank you.