Okay, cut the crap, let's take a look at a few examples first.
One, open a Web page to get all the content
Copy Code code as follows:
From Urllib import Urlopen
doc = Urlopen ("http://www.baidu.com"). Read ()
Print doc
Second, get HTTP headers
Copy Code code as follows:
From Urllib import Urlopen
doc = Urlopen ("http://www.baidu.com")
Print Doc.info ()
Print Doc.info (). GetHeader (' Content-type ')
Third, the use of agents
1. View Environment variables
Copy Code code as follows:
Print "n". Join (["%s=%s"% (k, v) for K, V in Os.environ.items ()])
Print os.getenv ("Http_proxy")
2. Setting Environment variables
Copy Code code as follows:
Import OS
Os.putenv ("Http_proxy", "http://proxyaddr:<port>")
3. Use of Proxy
Copy Code code as follows:
# Use http://www.someproxy.com:3128 for HTTP proxying
Proxies = {' http ': ' http://www.someproxy.com:3128 '}
FileHandle = Urllib.urlopen (Some_url, proxies=proxies)
# Don ' t use any proxies
FileHandle = Urllib.urlopen (Some_url, proxies={})
# Use proxies from Environment-both versions are equivalent
FileHandle = Urllib.urlopen (Some_url, Proxies=none)
FileHandle = Urllib.urlopen (Some_url)