Two methods to check whether the site robots.txt reasonable

Read a lot of website optimization articles, rarely mention how to check the site's robots.txt, most of the explanation robots.txt to set. For the master, this is a piece of cake, Can easily judge whether the robots.txt is reasonable, but for some rookie, simply can not judge their own site robots.txt is reasonable, a wrong robots.txt file will make the search engine machine can not dig your site, the result is that your new content can not be enumerated in the search engine index. This is a huge damage to webmaster. If you do something wrong and edit the robots.txt file again, it will take two weeks to see the effect. So it is important to have a correct robots.txt. Today, I would like to introduce two methods of checking the website robots.txt.

First, the use of Google Webmaster tools to check

Google Webmaster tools can be online to verify the files, analysis of the site's robots.txt files, the method is: Login to Google Webmaster tools, in the "console", click on the desired site, click the tool, and then click Analysis Robots.txt.

In addition, if you don't know how to write robots.txt, you can use Google Webmaster tools to provide the test robots.txt file and trial change function to set up a correct robots.txt file, you want to prohibit the file or directory through After the robots.txt format is made, copy it to the Robots section provided by Google, and then you can test the robots.txt file you just wrote in the robots.txt file site area. So you can set up a correct robots.txt.

The specific views are as follows:

　　

Look at the above picture, I believe you can also write a correct robots.txt! hehe

Second, the use of foreign http://tool.motoricerca.info/robots-checker.phtml certification first through the http://tool.motoricerca.info/ robots-checker.phtml login, enter in the box below: your domain name/robots.txt, (for example: Http://www.xxx.com/robots.txt)

(Reminder: The input must before you robots.txt in the Chinese annotation Delete, otherwise cannot this tool does not think you check is robots.txt, also cannot check robots.txt is reasonable. )

　　

When you click Check robots.txt, if robots.txt is correct, the following results are generally present

　　

If there are other results, there must be a problem, if you do not understand, you can use Google translation Ah!

Article Source: Rookie construction Station network www.zqbest.com.