Python web module learning-urlparse

Python web module learning-urlparse
Continue to learn about the python web module. urlparse is a relatively simple module. It is mainly used to parse the URL string 1 urlparse introduction. python uses urlparse to parse the URL string. The main method is urlprase secondary method: urljoin urlsplit urlunsplit. In addition, you can combine the split parts into a url. Main functions include urljoin, urlsplit, urlunsplit, and urlparse. The URL format includes: Protocol: // user name @ password: subdomain name. Domain Name. top-level domain name: Port/directory/file name. File suffix? Parameter = value # flag 2.1 urlparse

Urlparse. urlparse (urlstring [, scheme [, allow_fragments]) optional parameter: scheme-specifies the default protocol for URLs that do not contain some parts, the default value of this parameter is the Null String allow_fragments to indicate whether the address can be split. The default value of this parameter is True.

 

Parses the urlstring into six parts. It retrieves the URL from the urlstring and returns the tuples (scheme, netloc, path, parameters, query, and fragment), but it is actually based on namedtuple, is a subclass of tuple. It supports some URLs accessed by name attributes or indexes. Each component is a string of characters and may be empty. The component cannot be parsed into a smaller part, and the content after % is not parsed. The separator is not part of the parsing result. Unless it is escaped by a slash, note that the returned tuples are very useful, for example, it can be used to determine the network protocol (HTTP, FTP, etc.), server address, file path, and so on. The urlparse method returns the attributes in the object: attribute Index Value Meaning default value remarks scheme 0 Protocol Null String netloc 1 server address Null String path 2 path Null String parameters 3 parameter Null String query 4 query part null string fragment 5 fragment part Null String username user name None password None hostname host name None port None code:

>>> import urlparse>>> url=urlparse.urlparse('http://www.baidu.com/index.php?username=51cto')>>> urlParseResult(scheme='http',netloc='www.baidu.com', path='/index.php', params='', query='username=51cto ', fragment='')>>> url.scheme'http'>>> url.netloc'www.baidu.com'>>> url.path'/index.php'>>> url.params''>>> url.query'username=51cto'>>> url.fragment''>>> url.geturl()'http://www.baidu.com/index.php?username=51cto'>>> urlParseResult(scheme='http', netloc='', path='www.baidu.com/index.php',params='', query='username=51cto ', fragment='')

 

# Note: When http does not exist in the URL, this parameter 2.2 urljoin1urlparse is automatically used. urljoin (base, url [, allow_fragments]) urljoin is mainly a spliced URL. It uses base as its base address, and then combines it with the relative address in the url to form an absolute URL address. The urljoin function is particularly useful when you append a new file name to the URL base address to process several files at the same location. Note that if the base address does not end with a character or character, the rightmost part of the base address will be replaced by the relative path. If you want to keep the end directory in this path, make sure that the base URL address ends with a character. Code:

>>>url=urlparse.urljoin('http://www.baidu.com','index.html')>>> url'http://www.baidu.com/index.html'

 

Because there are no restrictions on the input parameters, note: 2.2.1 when the input parameters are empty strings, the returned url is also a null string with code:

>>> url = urlparse.urljoin('','')>>> url''

 

2.2.2 If there is a protocol field in the relative url, the protocol in the relative url is used first. Otherwise, the protocol field in the absolute url is used for additional code:

>>> url = urlparse.urljoin('http://www.baidu.com','ftp://www.baidu.com/index.html')>>> url'ftp://www.baidu.com/index.html'>>> url = urlparse.urljoin('http://www.baidu.com','www.baidu.com/index.html')>>> url'http://www.baidu.com/www.baidu.com/index.html'

 

When the relative string does not contain the Protocol, all strings are considered as a path, therefore, the busy string value is combined with the absolute url value 2.2.3 when both the absolute url and the relative url contain the server address and are different, add the following code to the server address and path using the relative url address:

>>> url = urlparse.urljoin('http://www.baidu.com','http://www.baidu.com/index.html')>>> url'http://www.baidu.com/index.html'

 

2.3 urlsplit urlparse. urlsplit (urlstring [, scheme [, allow_fragments]) mainly analyzes urlstring and returns a tuples containing five string items: Protocol, location, path, query, and fragment. When the value of allow_fragments is False, the next item in the group of the tuples is always empty, regardless of whether the urlstring contains fragments. Urlsplit () is similar to urlparse. However, it does not split URL parameters. Applicable to URLs that follow RFC2396, and each path segment supports parameters. In this way, only five elements are returned. Code:

>>> url = urlparse.urlsplit('http://www.baidu.com:80/index.html?src=fie')>>> urlSplitResult(scheme='http',netloc='www.baidu.com:80', path='/index.html', query='src=fie', fragment='')>>> url = urlparse.urlparse('http://www.baidu.com:80/index.html?src=fie')  >>> urlParseResult(scheme='http',netloc='www.baidu.com:80', path='/index.html', params='', query='src=fie',fragment='')

 

2.4 urlunsplit urlparse. urlunsplit (parts) urlunsplit combine values returned by urlsplit () into a url

>>> url = urlparse.urlsplit('http://www.baidu.com:80/index.html?src=fie')>>> urlSplitResult(scheme='http',netloc='www.baidu.com:80', path='/index.html', query='src=fie', fragment='')>>> url=urlparse.urlunsplit(('http','www.baidu.com:80','/index.html','src=fie',''))>>> url'http://www.baidu.com:80/index.html?src=fie'

 

2.4 urlunparse

urlparse.urlunparse(parts)

 

Construct a url from a tuple, which is similar to the URL returned by urlparse. After receiving the tuples (scheme, netloc, path, parameters, query, fragment), it will re-form a url with the correct format, for other HTML parsing modules of Python.

>>> url =urlparse.urlparse('http://www.baidu.com:80/index.html?src=fie')>>> urlParseResult(scheme='http', netloc='www.baidu.com:80',path='/index.html', params='', query='src=fie', fragment='')>>> url = urlparse.urlunparse(url)>>> url