通過python多線程訪問網頁

通過python多線程訪問網頁

@for&ever 2010-07-03

 

 

通過python，採用多線程的方式訪問網頁。
一個簡單的小例子，訪問三個不同的頁面，每個頁面開20個線程並發訪問。

 

python指令碼如下：

 

#!/usr/bin/env python<br /># -*- coding: utf-8 -*-</p><p>'''<br />Created on 2010-7-2</p><p>@author: forever<br />'''</p><p>import sys<br />import threading<br />from urllib import urlopen</p><p>urls = ["/archive/2010/07/03/5711257.aspx",<br />"/archive/2010/06/28/5699311.aspx",<br />"/archive/2010/07/03/5711270.aspx",<br />]</p><p>visitTimesPerPage = 20</p><p>def usage():<br />print 'Usage:', sys.argv[0], 'host'</p><p>def main(argv):<br />host = argv[1]<br />if host == '':<br />usage()<br />sys.exit(2)<br />else:<br />for i in range(visitTimesPerPage):<br />for url in urls:<br />visitPageThread = VisitPageThread(url + str(i), host, url)<br />visitPageThread.start()</p><p>class VisitPageThread(threading.Thread):<br />def __init__(self, threadName, host, url):<br />threading.Thread.__init__(self, name = threadName)<br />self.host = host<br />self.url = url</p><p>def run(self):<br />url = self.host + self.url<br />try:<br />doc = urlopen(url).read()<br /> #print doc<br />except Exception, e:<br />print "urlopen Exception : %s" %e</p><p>if __name__=='__main__':<br />sys.argv.append('http://blog.csdn.net/forandever')<br />main(sys.argv)</p><p>