Robots limit search engine spiders crawl which files

Edit a copy of the Robots file, save as robots.txt, to be present in the root directory of the server

Java code

2. user-agent: *
4. Disallow:/plus/ad_js.php
6. Disallow:/plus/advancedsearch.php
8. Disallow:/plus/car.php
10. Disallow:/plus/carbuyaction.php
12. Disallow:/plus/shops_buyaction.php
14. Disallow:/plus/erraddsave.php
16. Disallow:/plus/posttocar.php
18. Disallow:/plus/disdls.php
20. Disallow:/plus/feedback_js.php
22. Disallow:/plus/mytag_js.php
24. Disallow:/plus/rss.php
26. Disallow:/plus/search.php
28. Disallow:/plus/recommend.php
30. Disallow:/plus/stow.php
32. Disallow:/plus/count.php
34. Disallow:/include
36. Disallow:/templets

：

Open Baidu Webmaster Link: http://zhanzhang.baidu.com/robots/index

Enter a domain name or subdomain

robots.txt File Usage Examples

Robots limit search engine spiders crawl which files