Python chardet Simple Application

Python string encoding identification module (third party libraries):

Official address: Http://pypi.python.org/pypi/chardet import chardet import urllib   # 可根据需要，选择不同的数据 TestData = urllib.urlopen(‘http://www.baidu.com/‘).read() print chardet.detect(TestData)   # 运行结果： # {‘confidence‘: 0.99, ‘encoding‘: ‘GB2312‘}The running result indicates that there is a 99% probability that this code is GB2312 encoded. import urllib from chardet.universaldetector import UniversalDetector usock = urllib.urlopen(‘http://www.baidu.com/‘) # 创建一个检测对象 detector = UniversalDetector() for line in usock.readlines(): # 分块进行测试，直到达到阈值 detector.feed(line) if detector.done: break # 关闭检测对象 detector.close() usock.close() # 输出检测结果 print detector.result   # 运行结果： # {‘confidence‘: 0.99, ‘encoding‘: ‘GB2312‘}Application background, if you want to encode a large file, using this advanced method, you can read only one, to identify the encoding method to improve the detection speed. If you want to use a detection object to detect multiple data, be sure to run Detector.reset () after each test. Clears the previous data.

Python chardet Simple Application