Python crawl discuz! user name scripting code _python

Recently learning Python, so I wrote a script to crawl discuz! username in Python, the code is very small but very rubbing. The idea is simply to match the title and then extract the user name to write to the text document. The program to Baidu Webmaster community as an example (a total of more than 400,000 users), hanging on the VPS on the tube, although the use of the delay but later found that only grabbed more than 50,000 user names were sealed ...
The code is as follows:

Copy Code code as follows:

#-*-Coding:utf-8-*-
# Author: Day One
# blog:http://www.90blog.org
# version:1.0
# function: Python crawl Baidu webmaster Platform user name script

Import Urllib
Import Urllib2
Import re
Import time

Def biduspider ():
Pattern = Re.compile (R ' <title> (. *) 's profile Baidu Webmaster Community </title> ')
Uid=1
Thedatas = []
While UID <400000:
theURL = "http://bbs.zhanzhang.baidu.com/home.php?mod=space&uid=" +str (UID)
UID +=1
Theresponse = Urllib2.urlopen (theURL)
thepage = Theresponse.read ()
#正则匹配用户名
Thefindall = Re.findall (pattern,thepage)
#等待0.5 Seconds to prevent frequent access being prohibited
Time.sleep (0.5)
If Thefindall:
#中文编码防止乱码输出
Thedatas = Thefindall[0].decode (' Utf-8 '). Encode (' GBK ')
#写入txt文本文档
f = open (' TheUid.txt ', ' a ')
F.writelines (thedatas+ ' \ n ')
F.close ()

if __name__ = = ' __main__ ':
Biduspider ()

The final results are as follows: