URLLIB2 Custom Opener Detailed description

The Urllib2.urlopen () function does not support authentication, cookies, or other advanced HTTP features. To support these features, you must create a custom opener object using the Build_opener () function.

Copy the Code code as follows:

Build_opener ([Handler1 [Handler2, ...])

Parameter handler is a handler instance, commonly used are Httpbasicauthhandler, httpcookieprocessor, Proxyhandler and so on.

The object returned by Build_opener () has the open () method, which is the same as the function of the Urlopen () function.

If you want to modify the HTTP header, you can use:

Copy the Code code as follows:

Import Urllib2
Opener = Urllib2.build_opener ()
Opener.addheaders = [(' User-agent ', ' mozilla/5.0 ')]
Opener.open (' http://www.example.com/')

2. Install_opener (opener)

Installs a different opener object as the global opener used by Urlopen ().

3. Password verification (Httpbasicauthhandler)

The Httpbasicauthhandler () handler can use Add_password () to set the password.

Copy the Code code as follows:

H.add_password (REALM,URI,USER,PASSWD)

Realm is the name or description information associated with authentication, depending on the remote server. The URI is the base URL. User and passwd specify a username and password, respectively.

Copy the Code code as follows:

Import Urllib2
Auth=urllib2. Httpbasicauthhandler ()
Auth.add_password (' Administrator ', ' http://www.example.com ', ' Dave ', ' 123456 ')
Opener=urllib2.build_opener (auth)
U=opener.open (' http://www.example.com/evilplan.html ')

4. Cookie processing (httpcookieprocessor)

Copy the Code code as follows:

Import Urllib2,cookielib
Cookie=cookielib. Cookiejar ()
Cookiehand=urllib2. Httpcookieprocessor (Cookie)
Opener=urllib2.build_opener (Cookiehand)

5. Agent (Proxyhandler)

The Proxyhandler (proxies) parameter proxies is a dictionary that maps the protocol name (HTTP,FTP) to the URL of the appropriate proxy server.

Copy the Code code as follows:

Proxy=proxyhandler ({' http ': ' http://someproxy.com:8080 '})
Auth=httpbasicauthhandler ()
Auth.add_password ()
Opener=build_opener (Auth,proxy)

You can also use proxies in Urlopen

Copy the Code code as follows:

Import Urllib2
Proxy = ' http://%s:%s@%s '% (' userName ', ' Password ', ' proxy ')
Information = Urllib2.urlopen ("http://www.example.com", proxies={' http ':p Roxy})

