Copy Code code as follows:
"""
If you only use the normal
Import Urllib.request
html = Urllib.request.urlopen ("http://www.sina.com"). Read ()
Print (Html.decode (' GBK '))
The following error occurred
Builtins. Unicodedecodeerror: ' GBK ' codec can ' t decode byte 0x8b in position 1:illegal multibyte sequence
What to do? It turns out that some websites use gzip to compress the pages.
Take a look at the code below
Suggest you use Python2
Import Urllib2
From Stringio import Stringio
Import gzip
Request = Urllib2. Request (' http://www.sina.com ')
Request.add_header (' accept-encoding ', ' gzip ')
Response = Urllib2.urlopen (Request)
If Response.info (). Get (' content-encoding ') = ' gzip ':
BUF = Stringio (Response.read ())
f = gzip. Gzipfile (FILEOBJ=BUF)
data = F.read ()
Print Data.decode ("GBK"). Encode (' Utf-8 ')
"""
Import IO
Import Urllib.request as R
Import gzip
req = R.request ("http://www.sina.com", headers={"user-agent": "mozilla/5.0" (Macintosh; Intel Mac OS X 10_9_1) applewebkit/537.36 (khtml, like Gecko) chrome/31.0.1650.63 safari/537.36 ", accept-encoding": "gzip "})
BS = R.urlopen (req). Read ()
Bi = io. Bytesio (BS)
GF = gzip. Gzipfile (Fileobj=bi, mode= "RB")
Print (Gf.read (). Decode ("GBK"))