Python obtains the webpage encoding implementation code, and python Encoding

Python obtains the webpage encoding implementation code, and python Encoding

Python code for obtaining webpage Encoding

<Span style = "font-family: Arial, Helvetica, sans-serif; background-color: rgb (255,255,255 ); "> </span> <span style =" font-family: Arial, Helvetica, sans-serif; background-color: rgb (255,255,255); "> python development, the chardet library and Character Set detection are used to automatically obtain webpage encoding. This class is not found in python2.7 and needs to be downloaded from the official website. Now I have downloaded the chardet-2.3.0.tar.gz package file. You only need to put the chardet file decompressed by the package file in python27/lib/site-packages/Under the python installation package. </Span>

Then import chardet

The following describes an automatic detection function for detecting Url connections and then returns the url encoding method of the webpage.

Import chardet # Character Set detection import urllib url = "http://www.jd.com" def automatic_detect (url): content = urllib. urlopen (url ). read () result = chardet. detect (content) encoding = result ['encoding'] return encoding urls = ['HTTP: // www.baidu.com ', 'HTTP: // www.163.com', 'HTTP: // dangdang.com '] for url in urls: print url, automatic_detect (url)

The detect method of the chardet class is used above, the dictionary is returned, and the encoding method encoding is taken out.

Thank you for reading this article. I hope it will help you. Thank you for your support for this site!