First, open a Web page to get all the content
from Urllib Import Urlopen
Doc = Urlopen ( "Http://www.baidu.com" ) . Read ()
Print Doc
Second, get the HTTP header
from Urllib Import Urlopen
Doc = Urlopen ( "Http://www.baidu.com" )
Print Doc . Info ()
Print Doc . Info () . GetHeader ( ' Content-type ' )
Iii. Use of agents
1. View Environment variables
Print " "N " . Join ([ " %s = %s " % ( k , v ) for k , v inch OS . environ . Items ()])
Print OS . getenv ( "Http_proxy" )
2. Setting Environment variables
Import OS
OS . putenv ( "Http_proxy" , "HTTP://PROXYADDR: " )
3. Using proxies
# Use http://www.someproxy.com:3128 for HTTP proxying
proxies = { ' http ' : ' http://www.someproxy.com:3128 ' }
FileHandle = Urllib . Urlopen ( Some_url , proxies = proxies )
# Don ' t use any proxies
FileHandle = Urllib . Urlopen ( Some_url , proxies = {})
# Use proxies from Environment-both versions is equivalent
FileHandle = Urllib . Urlopen ( Some_url , proxies = None )
FileHandle = Urllib . Urlopen ( Some_url )