Python urllib Module Urlopen () and Urlretrieve () detailed _python

1.urlopen () method
Urllib.urlopen (url[, data[, proxies]): Create a class file object that represents a remote URL, and then manipulate the class file object like a local file to get the remote data.
The parameter URL represents the path of the remote data, typically the URL;
Parameter data represents the data that is submitted to the URL by post (the person who has played the web should know two ways to submit the data: Post and get.) If you are not clear, also do not care too much, usually rarely used this parameter;
Parameter proxies is used to set up the proxy.
Urlopen returns a class file object that provides the following methods:
Read (), ReadLine (), ReadLines (), Fileno (), Close (): These methods are used in exactly the same way as the file objects;
info (): Returns a httplib. Httpmessage object that represents the header information returned by the remote server
GetCode (): Returns the HTTP status code. In the case of an HTTP request, 200 indicates that the request completed successfully; 404 indicates that the URL was not found;
Geturl (): Returns the URL of the request;
code example:

Copy Code code as follows:

Import Urllib

url = "http://www.baidu.com/"

#urlopen ()
Sock = Urllib.urlopen (URL)
Htmlcode = Sock.read ()
Sock.close
fp = open ("e:/1.html", "WB")
Fp.write (Htmlcode)
Fp.close

#urlretrieve ()
Urllib.urlretrieve (URL, ' e:/2.html ')

2.urlretrieve method
Download remote data directly to the local.

Copy Code code as follows:

Urllib.urlretrieve (url[, filename[, reporthook[, data]])
Parameter description:
URL: external or local URL
FileName: Specifies the path to be saved to the local (if not specified, Urllib generates a temporary file to hold the data);
Reporthook: is a callback function that triggers the callback when the server is connected and the corresponding block of data is transmitted. We can use this callback function to display the current download progress.
Data: Refers to the post to the server. The method returns a tuple containing two elements (filename, headers), filename is saved to a local path, and the header represents the server's response header.

The following example to illustrate the use of this method, this example will be the first page of Sina crawl to the local, saved in the d:/sina.html file, while showing the progress of the download.

Copy Code code as follows:

Import Urllib
def callbackfunc (Blocknum, BlockSize, totalsize):
"" Callback function
@blocknum: Blocks of data that have been downloaded
@blocksize: Size of data block
@totalsize: Size of remote file
'''
Percent = 100.0 * Blocknum * blocksize/totalsize
If percent > 100:
Percent = 100
Print "%.2f%%"% percent

url = ' http://www.sina.com.cn '
Local = ' d:\\sina.html '
Urllib.urlretrieve (URL, local, callbackfunc)