Python to write a crawler applet

Source: Internet
Author: User
Tags html header split

We can use Python to implement such a simple reptile function, to crawl the code we want to the local. Let's look at how you can use Python to implement such a feature.

Cause

Late at night suddenly want to download a little ebook to expand the Kindle, think of Python learning too shallow, what "decorator" Ah, "multithreading" Ah have not learned.

The thought of the Python tutorial of the Great God of snow is very classic and famous. Just want to find there is a PDF version of the download, the results did not find!! CSDN has an incomplete also cheated me an integral!! Nima!!

Angry, ready to write a program to climb the snow peaks of the tutorial, and then turn HTML into an ebook.

Process

The process is interesting, with shallow python knowledge, writing Python programs, and climbing the python tutorials to learn python. Think of a little excitement ...

Sure enough Python is very convenient, 50 line or so OK. Directly paste code:

?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 This is the # coding:utf-8 Import urllib   domain = ' http://www.liaoxuefeng.com ' #廖雪峰的域名 path = R ' C:userscyhhao2013desktoptemp ' #html要保存的路径   # An HTML header file input = open (R ' c:userscyhhao2013desktop.html ', ' r ') head = Input.read ()   # Open Python Tutorial Main interface F = Urllib.urlopen ("http://www.liaoxuefeng.com/wiki/001374738125095c955c1e6d8bb493182103fac9270762a000") Home = F.read () f.close ()   # Replaces all space carriage returns (so easy to get URL) Geturl = Home.replace ("n", "") Geturl = Geturl.replace ("", "") &N Bsp # get the string list = Geturl.split (R ' em; ") that contains the URL. ><ahref= "') [1:]   # obsessive-compulsive disorder, make sure to add the first page to the perfect List.insert (0, '/wiki/ 001374738125095c955c1e6d8bb493182103fac9270762a000 ">"   # begins traversing URL List for li in List:url = Li.split (R ' "> ') [ 0] url = domain + URL #拼凑uRL print URL f = urllib.urlopen (URL) html = F.read ()   # get title to write file name title = Html.split ("<title>") [1] title = Title.split ("-Liaoche's official website </title>") [0]   # To turn the code, or add to the path on the tragedy of the title = Title.decode (' utf-8 '). Replace ("/", "") &N Bsp # intercept Body HTML = Html.split (R ' <!--block main--> ') [1] html = html.split (R '

Life is short, I use Python!

The above mentioned is the entire content of this article, I hope you can enjoy.

Related Article

E-Commerce Solutions

Leverage the same tools powering the Alibaba Ecosystem

Learn more >

Apsara Conference 2019

The Rise of Data Intelligence, September 25th - 27th, Hangzhou, China

Learn more >

Alibaba Cloud Free Trial

Learn and experience the power of Alibaba Cloud with a free trial worth $300-1200 USD

Learn more >

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.