Some of the Python modules

Urllib and Urllib2

Urllib and URLLIB2 are all related modules that accept URL requests, but provide different functions;

The two modules of Urllib and URLLIB2 are not able to substitute each other;

Urllib2 can accept an instance of the request class to set the URL request header,urllib only the URL can be accepted. This means that you cannot disguise your user agent string and so on.

The UrlEncode method provided by Urllib is used to get the resulting query string, and Urllib2 does not. This is the reason why Urllib and urllib2 are used together.

URLLIB2 has a lot of features that urllib don't have:

1, Urllib does not support caching, URLLIB2 support

2, Urllib does not support the last modification time check, URLLIB2 support

3, Urllib does not support ETAG,URLLIB2 support

4, Urllib does not support compression, URLLIB2 support

5, Urllib all redirects are temporarily redirected, URLLIB2 support distinguishes between temporary redirects and permanent redirects. (redirect: relocate various network requests to another location in various ways )

Anyway, urllib is a basic HTTP library, and Urllib2 's support for the HTTP protocol is more complete.

BeautifulSoup's introduction

Beautiful Soup is a html/xml parser written in Python that can handle nonstandard tags and generate parse tree (s). It provides simple and common navigation (navigating), search and modify the parse tree operation. It can greatly save your programming time

In short, BeautifulSoup is a library of Python, the main function is to fetch data from the Web page, the official explanation is as follows:

BeautifulSoup provides simple, Python-style functions for navigating, searching, and modifying analysis trees. It is a toolbox that provides users with the data they need to crawl by parsing the document, because it is simple, so it does not require much code to write a complete application.

Beautifusoup Automatic input documents are converted to Unicode encoding, and the input document is converted to UTF-8 encoding. You don't have to think about coding, unless the document does not specify an encoding, and BeautifulSoup does not automatically recognize the encoding. Then, you just need to explain the original encoding method.

BeautifulSoup has become as good a Python interpreter as lxml and Html6lib, providing users with the flexibility to provide different analytic strategies or strong speeds

Python Regular expression re module

The regular expression is a small, highly specialized programming language embedded in the Python development language that can be used through the RE module,

A regular expression uses a single character to describe and match a series of strings that conform to a certain syntactic rule.

Requests module

Requests uses URLLIB3, so it inherits all of its features. Requests supports HTTP connection retention and connection pooling. Supports cookie protection sessions, supports file uploads, and supports automatic encoding of response content.

This article is from the "Insane Lunatic Asylum" blog, please make sure to keep this source http://adonislxf.blog.51cto.com/11770740/1886173

Some of the Python modules