Python urllib Basic Learning

#-*-Coding:utf-8-*-

# python:2.x

__author__ = ' Administrator '

#使用python创建一个简单的WEB客户端

Import Urllib,urllib2,urlparse

"""

Web address Element

URL Part Description

Prot_sch network protocol or download planning

Nety_loc server location (or user information)

Path slash (/) qualified files or CGI application paths

Query Connector (&) connection key value pair

Params Optional parameters

Frag special anchors in a split document

Network positioning elements

Net_loc

Part description

User username

Password Password

Host Web server run machine name or address (required field)

Port port Number (default is 80)

"""

#urlparse. Urlparse () urlparse (URL, scheme= ", allow_fragments=true)

"""

Resolves a URL to a 6-tuple (Prot_sch,net_loc,path,params,query,frag)

You can use Scheme,allw_fragments to identify whether a URL is allowed to use components when no default network protocol is provided in the URL or when you download a plan

"""

#例如

#print urlparse.urlparse (' http://www.python.org/doc/FAQ.html ')

#ParseResult (scheme= ' http ', netloc= ' www.python.org ', path= '/doc/faq.html ', params= ', query= ', fragment= ')

#urlparse. The Urlunparse () function is the opposite of Urlparse.urlparse (): It is flattening a 6-tuple (Prot_sch,net_loc,path,params,query,frag)

#url1 =urlparse.urlparse (' http://www.163.com ')

#print Urlparse.urlunparse (URL1)

#urlparse. Urljoin () to perform multiple combinations together

# # ' photo.shtml ')

"" "Urllib

Support Web protocol, Http,ftp,gopher protocol, can upload and download, avoid using the previous modules

"""

#urllib. Urlopen () opens a given URL string to the Web connection, returning the file type

#语法urllib. Urlopen (Url,data=none,proxies=none): If there is no given protocol or download, or the file plan is already given, it will open the local file

"""

For all HTTP requests, a common request is get, in which case the Web server sends the request string (encoded key value or reference)

If you require the use of Post method information, see the CGI Application Programming section of the normal document or text.

The F.info () method returns the MIME (multi-target Internet Mail extension, Multipurpose Internet Mail Extension) header file, which notifies the browser that the file type can be opened for that kind of application

The Geturl () method obtains a real URL from the final open file after considering all possible indirect guidance

"""

"""

The Urllilb.urlopen () method is as follows

Object Method Description

F.read ([bytes]) reads all or bytes bytes from f

F.readline () reads a line from F

F.readlines () reads all rows to return a list

F.close () Close the connection

F.fileno () returns the F file handle

F.geturl () returns the true URL opened by f

If you need complex URLs or you want to handle complex situations, such as cookies, we recommend using the URLLIB2 module.

"""

#urllib. Urlretrieve (Url,fulename=none,reporthook=none,data=none) When you need a URL document, you can use it

"""

Reporthook This function is called after each piece of data is downloaded or transmitted, 3 parameters: The number of blocks currently read, the number of bytes in the block, and the number of bytes of files,

Urlretrieve () returns a 2-tuple, (FILENAME,MIME_HDRS), filename containing the download data local file name, Mime_hdrs is a response to the Web server after returning a series of MIME files to judge

More information can be seen in the Mimetools message class, Mime_hdrs is empty for local files

"""

#urllib. QUOTE () and Urllib.quote_plus ()

"""

The quote* () function obtains the URL data, encodes it, and applies it to the URL string, which can be used for some special strings that cannot be printed or are not accepted as valid URLs by the Web server.

Syntax such as

QUOTE (urldata,safe= '/')

"""

#例如

# name= ' Jon Mama '

# number=6

# base= ' http://www/~foo/cgi-bin/s.py '

# final= '%s?name=%s&num=%d '% (base,name,number)

# Print Final

# print Urllib.quote (final)

# print Urllib.quote_plus (final)

#urllib. unquote (), Urllib.unquote_plus () These 2 functions are completely opposite, converting all letters encoded '%xx ' to their ASCII values

The #调用unquote () function decodes all URL-encoded letters in S, returns a string, and the Urllib.unquote_plus () function converts the plus sign to a space character

#urllib. UrlEncode () It is the receive dictionary key-value pair, which compiles it into a CGI request URL string, a key-value pair Format: Key = value, with connectors & divisions, further, keys and their values are passed to Quote_pluis () for proper encoding

#例如

adict={' name ': ' georgion garica ', ' B ': ' C '}

Print Urllib.urlencode (adict)

#urllib方法的例子学习

url1= ' http://cnblogs.com '

#代理服务器

proxies={' http ': ' http://cnblogs.com '}

#使用代理服务器打开

R=urllib.urlopen (url1,proxies=proxies)

Print R.info ()

Print R.getcode ()

Print R.geturl ()

#打开本地文件

F=urllib.urlopen (url= ' file:/f:/from2.html ')

#print F.read ()

#print F.readline ()

Print F.readlines ()

#打开ftp服务器

#f = urllib.urlopen (url = ' ftp://username:[email protected] ')

#保存网页显示进度

def urlabc (a,b,c):

"""

A: Quantity

B: Size

C: Status

"""

Per=100.*a*b/c

If per>100:

per=100

print '%.2f%% '%per

Url= ' http://www.oschina.net/news/55121/tiobe-2014-9 '

Local= ' Oschina.net '

Print Urllib.urlretrieve (URL,LOCAL,URLABC)

#get方法

Params=urllib.urlencode ({' s ': 1, ' B ': 2, ' C ': 3})

F=urllib.urlopen ("http://www.musi-cal.com/cgi-bin/query?%s"% params)

Print F.read ()

print ' * ' *300

#post方法

Params=urllib.urlencode ({' s ': 1, ' B ': 2, ' C ': 3})

F=urllib.urlopen ("http://www.musi-cal.com/cgi-bin/query?%s"% params)

Print F.read ()

print ' * ' *300

#编码解码

Data= ' name=~a+3 '

Data1=urllib.quote (data)

Print Data1

Print Urllib.unquote (data1)

Data2=urllib.quote_plus (data)

Print Data2

Print Urllib.unquote_plus (DATA2)

Data3 = Urllib.urlencode ({' name ': ' Dark-bull ', ' Age ': 200})

Print Data3

Data4=urllib.pathname2url (R ' F:\qtgui\netword1\oschina.net ')

Print Data4

Print Urllib.url2pathname (DATA4)

Python urllib Basic Learning