Python uses custom user-agent to crawl Web pages _python

This article illustrates how Python uses custom user-agent to crawl Web pages. Share to everyone for your reference. Specifically as follows:

The following Python code captures the contents of the specified URL by urllib2, and uses a custom user-agent to prevent the site screen collector

Import urllib2
req = urllib2. Request (' http://192.168.1.2/')
req.add_header (' user-agent ', ' Mozilla 5.10 ')
res = Urllib2.urlopen (req)
html = res.read ()

I hope this article will help you with your Python programming.