Python to write a crawler applet

We can use Python to implement such a simple reptile function, to crawl the code we want to the local. Let's look at how you can use Python to implement such a feature.

Cause

Late at night suddenly want to download a little ebook to expand the Kindle, think of Python learning too shallow, what "decorator" Ah, "multithreading" Ah have not learned.

The thought of the Python tutorial of the Great God of snow is very classic and famous. Just want to find there is a PDF version of the download, the results did not find!! CSDN has an incomplete also cheated me an integral!! Nima!!

Angry, ready to write a program to climb the snow peaks of the tutorial, and then turn HTML into an ebook.

Process

The process is interesting, with shallow python knowledge, writing Python programs, and climbing the python tutorials to learn python. Think of a little excitement ...

Sure enough Python is very convenient, 50 line or so OK. Directly paste code:

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 This is the

# coding:utf-8 Import urllib   domain = ' http://www.liaoxuefeng.com ' #廖雪峰的域名 path = R ' C:userscyhhao2013desktoptemp ' #html要保存的路径   # An HTML header file input = open (R ' c:userscyhhao2013desktop.html ', ' r ') head = Input.read ()   # Open Python Tutorial Main interface F = Urllib.urlopen ("http://www.liaoxuefeng.com/wiki/001374738125095c955c1e6d8bb493182103fac9270762a000") Home = F.read () f.close ()   # Replaces all space carriage returns (so easy to get URL) Geturl = Home.replace ("n", "") Geturl = Geturl.replace ("", "") &N Bsp # get the string list = Geturl.split (R ' em; ") that contains the URL. ><ahref= "') [1:]   # obsessive-compulsive disorder, make sure to add the first page to the perfect List.insert (0, '/wiki/ 001374738125095c955c1e6d8bb493182103fac9270762a000 ">"   # begins traversing URL List for li in List:url = Li.split (R ' "> ') [ 0] url = domain + URL #拼凑uRL print URL f = urllib.urlopen (URL) html = F.read ()   # get title to write file name title = Html.split ("<title>") [1] title = Title.split ("-Liaoche's official website </title>") [0]   # To turn the code, or add to the path on the tragedy of the title = Title.decode (' utf-8 '). Replace ("/", "") &N Bsp # intercept Body HTML = Html.split (R ' <!--block main--> ') [1] html = html.split (R '

Life is short, I use Python!

The above mentioned is the entire content of this article, I hope you can enjoy.