For example, to crawl the problem I have done on the avionics, you can use the following code:
# ubuntu Kylin 14.04
import urllib2
import urllib
url= "http://acm.hdu.edu.cn/userstatus.php?user= 461807914 "
headers = {' user-agent ': ' mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) gecko/20100101 firefox/23.0 '}
request = Urllib2. Request (url,none,headers)
response = Urllib2.urlopen (request)
print (Response.read ())
At the beginning of the writing without that header, give me a pop-up 403 forbidden, is very puzzling. The original site to your request will be judged, if your request information is complete, will accept your request. such as sending the request to use the browser, operating system, hardware platform, and other information, and the lack of such information is often not a normal access, such as reptiles. Now you can add a false header information in the message sent, cheat the other's server on it.