Python, Article 4: differences between bytes and string in Python 3, pythonbytes
Original article: http://eli.thegreenplace.net/2012/01/30/the-bytesstr-dichotomy-in-python-3
The most important new feature in python 3 may be to make a clearer distinction between text and binary data. The text is always encoded in unicode and represented in str; the binary data is represented in bytes.
In python3, the str and bytes types cannot be implicitly mixed. You cannot concatenate the str and bytes types. You cannot search for bytes data in str (or vice versa), or use str as a parameter to input functions that require parameters of the bytes type (or vice versa ).
The boundary between strings and byte characters is inevitable. The following illustration should be kept in mind:
Strings can be encoded into bytes and decoded into strings:
>>> '€20'.encode('utf-8')b'\xe2\x82\xac20'>>> b'\xe2\x82\xac20'.decode('utf-8')'€20'
It can be understood as follows:
String is an abstract representation of text. A string is composed of characters. It is also an abstract entity and has nothing to do with any binary representation.
Many details are not required when manipulating strings. You can split, slice, and splice strings to search for strings. However, they do not care about how they are represented internally, nor how many bytes are consumed by a single character at the underlying layer.
Only when you need to encode the string (encode) into a byte, such as transmitting data over the network, or decoding the byte (decode) into a string, we will pay attention to the difference between string and byte.
The input parameters of encode and decode are encoded. Encoding is a way to use binary data to represent abstract characters. There are many types of codes. The UTF-8 given above is one of them, and the following is another:
>>> '€20'.encode('iso-8859-15')b'\xa420'>>> b'\xa420'.decode('iso-8859-15')'€20'
Encoding is a crucial part of the conversion process. If it is not encoded, bytes object B '\ xa420' is just a heap of bits. Encoding. With different codes, the meaning of these bit spaces will be different:
>>> b'\xa420'.decode('windows-1255')'₪20'