Urllib crawler Development in "Python" Python3

Here are three ways

①first Method

The simplest way

② Add Data,http Header

Using the Request object

③cookiejar

Importurllib.request fromhttpImportCookiejarurl='http://www.baidu.com'Print("First Method") Response1=urllib.request.urlopen (URL)#Return Status CodePrint(Response1.getcode ())Print(Len (Response1.read ()))Print("Second Method") Request=urllib.request.Request (URL) request.add_header ("uese-agent","mazilla/5.0") Response2=urllib.request.urlopen (URL)#Return Status CodePrint(Response2.getcode ())Print(Len (Response2.read ()))Print("Third Method")#declaring a Cookiejar object instance to hold a cookieCJ =Cookiejar. Cookiejar ()#using the Httpcookieprocessor object of the Urllib.request library to create a cookie processor, it is also CookiehandlerHandler =Urllib.request.HTTPCookieProcessor (CJ)#Create opener with CookiehandlerOpener =Urllib.request.build_opener (handler)#The Open method here is the same as Urllib.request's Urlopen method, which can also be passed to the requestRESPONSE3 =opener.open (URL)#Return Status CodePrint(Response3.getcode ())Print(Response3.read ())

Urllib crawler Development in "Python" Python3