How to make the right robots for the webmaster

SEO optimization technology is not only in the content and outside the chain, more importantly, some of the details of the processing, because the content and outside the chain need to master the main points is not many, and easy to operate, easy to understand, while the site optimization of other details of the treatment, relatively less often contact, for these know very little, Really want to deal with a lot of problems, such as SEO optimization of the regular 404-page production, 301 redirect processing, a creation of robots, etc., for the site is an indispensable part.

The following is said that robots, first of all, it is a protocol for the site to grab to take an important role in the creation of a robots, the significance is that it can let spiders obey the orders of the arrangement, to solve some of the site's privacy and do not need to crawl the page, but also can emphasize that a Robots appear to be a very simple operation, but usually if not too much attention or more operation, you write the robots may not be correct, because there are some instructions, there is no writing on the site may affect the crawl.

Baidu Webmaster Platform launched a robots tool (Http://zhanzhang.baidu.com/robots), as long as registered under the certification site on the OK, it can be a good test of your robots write is correct, The tool can detect whether your Web site has robots.txt files, and robots.txt file content, and can generate the correct robots.txt files according to your requirements, for the production of a robots file has been greatly helped.

　One, can detect whether the website has a robots file

After the website is validated, you can test the website arbitrarily, see if there is a production of robots, if it will list your site settings, such as what files have crawled, what files are blocked, at a glance, and prompted whether there is a mistake, but did not do the protocol, he will also be prompted: " Your site does not have a robots.txt file, which means that you allow search engines to crawl the entire contents of your site. As shown in the picture (Chinaz test results):

　　

　Second, you can detect whether the website robots are correct

Many friends take for granted that the writing of robots is very simple, the real test will be found to be wrong. Add site Map In the robots believe that everyone is very familiar with the main purpose is to let search engine spiders smoothly crawl site map, shielding Spider is using disallow, allow crawling with Allow, think of course a lot of webmaster to let spiders crawl site, directly to the site map defined as: allow:http ://www.***.com/sitemap.html, the results of the test is originally wrong, as shown:

　　

　　

Baidu only then found that the original format for sitemap:http://www.***.com/sitemap.html, although this operation is very simple, but without the help of the tools of the robots, you may not find this problem for a long time.

　Iii. generate the correct robots.txt file according to your requirements

If you don't know how to make a robots file, there is a very detailed introduction of the content, including the concept of robots, how to make, using form list, and there are many examples, compared to Baidu encyclopedia to clear more clearly, understand the concept, of course, is manual production, It can be very good to your idea of one by one generated content, such as you do not want to make crawl contact us this page, you can directly on the above operation:

　　

After the completion of only to download the file, put the site to the root directory is done, the feeling is not more than to see Baidu Encyclopedia to clear more clearly? Many other operations of the details of the use of Baidu is no way to solve, after all, the content of the site or a lot of ideas are wrong, can only make good use of the official Baidu tool is the best! Article by the Taobao Landing http://jyxxjc.com Small series of original, reproduced please add the article link, thank you for your cooperation.