Through Sina "screen door" to learn the correct robots.txt document writing

Intermediary transaction http://www.aliyun.com/zixun/aggregation/6858.html ">seo diagnose Taobao guest cloud host technology Hall

Some time ago due to a part of the webmaster files misreading, led to a false message of the birth-"Sina blocked door", the news that Sina blog began to ban Baidu Spiders, its judgment based on the basis of Sina blog is only a document in a section " User-agent:baiduspider ", this in fact uploaded in 2005, a lot of robots confused webmaster, really think Sina blog began to ban Baidu, a time online is all over the chain is not to do the cry, in fact, if you are familiar with the writing of robots, You can easily identify this false message.

robots.txt files placed in the site root directory, search engine spiders visit a website after the latest look is the file, it is equivalent to your site's "doorman", if the "doorman" refused to search engine spiders, the search engine can not include any of your site interface.

In the Baidu Search Help Center (http://www.baidu.com/search/robots.html),

The definition of how to ban Baidu spider access is as follows:

User-agent:baiduspider

Disallow:/

Only allow Baidu Spider's visit:

User-agent:baiduspider

Disallow:

Baidu spiders are forbidden to access specific directories:

User: *

Disallow: CGI

Disallow:/tmp/

Disallow:/~joe/

Combined with these 3 kinds of grammar, we will take a look at the online spread of the Sina blog:

User-agent:baiduspider

Disallow:/admin/

Disallow:/include/

Disallow:/html/

Disallow:/queue/

Disallow:/config/

Its real purpose is only to prohibit Baidu search engine access to 5 of the background directory, and content page does not matter, so that the robots are not blocked Baidu spiders, but to tell Baidu spiders to "go to the place to change."

A correct and effective robots file can help you better be included in the Baidu, you novice standing in a friend if not familiar with the robots, you can go to the "Baidu Search Help Center" (http://www.baidu.com/search/robots.html) Download the standard case, change the contents of the inside to its own URL.

Above is one of my experience, I hope to have friends reprinted when you can keep my link http://www.jfbest.com more than a guarantee, leaving more than a word of mouth, a lot of luck, a wealth! Thank you!