Python Initial Experience: Write a small crawler with Weibo, blog, and space in three sentences

Target: Use python to write something that automatically accesses the webpage.

Python is really powerful enough. I have improved myself by referring to an excellent python video training program on the Internet. To enable python to open the browser and access the specified URL, you only need the following two sentences:

Import webbrowser as web

Web. open_new_tab ('www .baidu.com ')

If you want to access it for a certain number of times or for a long time, add a while True. The complete program is provided below and then explained:

# Coding = utf-8import webbrowser as webimport timeimport osimport randomM = random. randint (10, 15) print 'random number = ', MN = 10j = 0 while j <M: I = 0 while I <N: web. open_new_tab ('HTTP: // blog.sina.com.cn/s/blog_723395a70101josb.html') I = I + 1 time. sleep (1.5) else: OS. system ('taskkill/F/IM baidubrowser.exe ') j = j + 1 else: print' this python has opened ', M * N,' times 'o = 'C: \ windows \ system32 \ shutdown-s OS. system (o)

Note:

1. encoding is enabled at the beginning, and Chinese characters can be printed in python;

2. random. randint () is to generate a random number between 10 and 15. The number of visits can be controlled here. Then a while is nested in it, so the final number of visits is M * N;

3. Every N times, here N = 10. After opening 10 browser pages, you must use the taskkill/F/IM baidubrowser.exe command to kill the browser process. For other browsers, you can view the browser processes in the task manager. Use OS. system for windows. If you want to switch to the kill command in linux. Why should I close the browser every 10 times? If too many resources are opened and occupied, the computer will crash. So close the browser every 10 times. After each opening, let it sleep for a certain period of time. If there is no time left in the middle, the browser will shut down without requesting data.

4. If you have finished the flash and want to shut down automatically, use the shutdown-s command to enable python to automatically close the browser after running the task in the middle of the night.

Come on:

Conclusion: strictly speaking, this is not a python crawler. It only uses a command to automatically open the browser and parse the content of the website or the browser, not the code written by python. However, this example shows how powerful python is in terms of network. Contribute a real crawler next time.

Disclaimer: This example demonstrates the power and practicality of python. Do not use it for bad purposes !!! (Now, many websites and blogs have anti-crawlers .)