1, Python crawler request.urlopen request for Web Access to the source code

# Python3 Import Request Package from Urllib ImportRequestImport SYSImport io# If you need print printing, you can set the output environment first if an exception occursSys.StdOut=Io.Textiowrapper (SYS.StdOut.Buffer, encoding=' Utf-8 ')# The URL you need to getUrl= ' http://www.xxx.com/'# header FileHeaders={"User-agent":"mozilla/5.0 (Windows NT 10.0; WOW64) applewebkit/537.36 (khtml, like Gecko) chrome/64.0.3282.186 safari/537.36 "}# Generate Request ObjectReq=Request.Request (URL, headers=Headers# Call the request's Urlopen method to initiate the demand and return the result object, or a GET request if there is no data parameter, or the POST requestResponse=Request.Urlopen (req)# Write the results into an HTML file, with Open(' a.html ',' WB ') asF:f.Write (response.Read ())# Print the returned status codePrint(Response.GetCode ())# Print the returned URL to prevent redirection of URL changesPrint(Response.Url

1, Python crawler request.urlopen request the Web page to obtain the source code