The secret of robots.txt. Guide to Optimizing settings

Intermediary transaction http://www.aliyun.com/zixun/aggregation/6858.html ">seo diagnose Taobao guest cloud host technology Hall

September summed up the seoer most want to know the robots.txt six big points summary.

1. Do you know the meaning of robots.txt Chinese?

2. Do you know that 404 pages should be blocked?

3. Your enterprise station, every page has "about us", and still have to shield it off?

4. Do you know that setting a good robots.txt is beneficial to enhance the overall weight of the site?

5. You want to know the basic robots.txt?

6. Robots.txt how to write, to optimize the best.

1. Robots.txt's Chinese meaning is: Search engine public agreement. Search engine to your site, is to see robots.txt, according to the above written agreement to crawl your Web page.

2. Each site should have 404 pages, if not, will lead to 1 reduce the site PV 2 to reduce traffic 3) search engine on your site is not good, because your site user experience is bad.

Each site has many error pages, each error page should do a 404 page, then each 404 pages are the same. Once you have too many 404 pages, you also let search engines to include, will lead to the site to the weight of these 404 pages, this is worth it? So we have to block out 404 pages.

3. Enterprise station navigation bar above, often there will be "online Money Blog Introduction" "Contact September" "About Us" these pages. These pages, can be said to be full station display, the site almost every page to the weight of these pages, but these pages are worthy of decentralization of the site to them? So we also have to block out these "unimportant content" page, do not let search engines crawl.

4. From 3.4 points, we can draw a conclusion: if the site has too many unrelated pages are included, and you do not want users directly from the search engine to enter these pages, then the site will be the weight of these "meaningless" page, which will reduce the weight of the overall site.

The web site is dragged down by these spam pages.

5. The basic writing of robots.txt.

Create a document called Robots.txt on the desktop

The following are written in order:

User: *

Disallow:/wp-admin/

Disallow:/wp-content/

Disallow:/feed

Disallow:/comments/feed

Disallow:/trackback

Disallow:/guestbook

User: #表示允许的搜索引擎. * representative to allow all, if only allow Baidu Spider, write Baiduspider

Disallow: #是禁止收录的意思

Disallow:/#表示禁止所有收录. Go straight to the disallow: Add a slash in the back.

Disallow:/Website Background directory/#屏蔽目录的格式

Disallow:/404.html #屏蔽404页面

Robots.txt inside, any space, line change is ignored. Robots.txt only look at the order of orders, spaces, lines are for us to see the convenience.

6. Robots.txt how to write, to optimize the best?

After reading the previous 5 points, I believe you have a good understanding of how to optimize the best.

1 Shielding the site from the useless spam page, such as 404

2 Think about what pages you may have to show, but you don't feel the need to let users "look at the search engine". For example, Fuzhou SEO "About us." Then you don't have to be included.

Online money-making methods-September blog original article, reprinted please indicate the source and retain the original link

This article link: http://www.9moom.cn/544.html