Summary of the problem of transcoding in Python3

For this (class) issue:

(1) The problem occurs when the unicodeencodeerror–> description is Unicode encoding;

(2) ' GBK ' codec can ' t encode character–> description is an issue that occurs when encoding Unicode characters as GBK;

At this point, it is often most likely that the character of the Unicode type itself contains some characters that cannot be converted to GBK encoding.

The solution is:

• Scenario 1:

When encoding Unicode characters, add the ignore parameter, ignoring characters that cannot be encoded, so that they can be encoded as GBK normally.

The corresponding code is:

gbktypestr = Unicodetypestr.encode ("GBK", ' ignore ');

• Scenario 2:

Or, convert it to GBK encoded superset GB18030 (that is, GBK is a subset of GB18030):

gb18030typestr = Unicodetypestr.encode ("GB18030");

The corresponding resulting character is the encoding of the GB18030.

"Off-topic"

For the above, it is safer to convert the original utf-8 characters to Unicode, or you can:

Titleuni = Titlehtml.decode ("UTF-8");

To be replaced by:

Titleuni = Titlehtml.decode ("UTF-8", ' ignore ');

This can be achieved, even for those, relatively insignificant some of the special characters, can also be successfully encoded, to avoid coding errors, improve the robustness of the program.

Summary of the problem of transcoding in Python3