Examples of common commands for accessing and capturing web pages using python

This article mainly introduces information about common commands for accessing and capturing web pages using python. For more information about common commands for accessing and capturing web pages using python, see the following article, for more information, see

Common commands for accessing and capturing web pages using python

Simple webpage capturing:

Import urllib. request url = "http://google.cn/" response = urllib. request. urlopen (url) # Return file object page = response. read ()

Directly save the URL as a local file:

Import urllib. request url = "http://google.cn/" response = urllib. request. urlopen (url) # Return file object page = response. read ()

POST method:

Import urllib. parse import urllib. request url = "http://liuxin-blog.appspot.com/messageboard/add" values = {"content": "command line web page request test"} data = urllib. parse. urlencode (values) # Create a request object req = urllib. request. request (url, data) # obtain the data returned by the server response = urllib. request. urlopen (req) # process data page = response. read ()

GET method:

Import urllib. parse import urllib. request url = "http://www.google.cn/webhp" values = {"rls": "ig"} data = urllib. parse. urlencode (values) theurl = url + "? "+ Data # Create a request object req = urllib. request. request (theurl) # get the data returned by the server response = urllib. request. urlopen (req) # process data page = response. read ()

There are 2 common methods, geturl (), info ()

Geturl () is set to identify whether there is server-side URL redirection, while info () contains a series of information.

To solve Chinese problems, encode () encoding dencode () decoding is used:

The above is a detailed description of the examples of common commands for accessing and capturing web pages in python. For more information, see other related articles in the first PHP community!