Python Standard library notes (6)-struct module

The function of this module is to complete the conversion between Python values and the Python string form of the C language structure. This can be used to process binary data stored in a file or from a network connection, as well as other data sources.

Purpose: To convert between Python basic data types and binary data

structThe module provides a conversion function between a byte string and a Python native data type, such as a number and a string.

module functions and struct classes

In addition to providing a Struct class, there are many module-level functions for working with structured values. Here is the concept of format specifiers, which refers to converting from a string format to a compiled representation, similar to how a regular expression is handled. Typically instantiating a Struct class, invoking a class method to complete the conversion, is more efficient than calling the module function directly. The following examples all use Struct classes.

Packing (Package) and unpacking (unpacking)

StructSupports data packing (packaged) into strings and can be reversed unpacking (decompressed) out of the string.

In this example, the format selector (specifier) requires an integer or long integer, a two-byte string, and a floating-point number. The spaces in the format character are used to separate the individual indicators (indicators) and are ignored when compiling the format.

import  struct import  binasciivalues =  (1 ,  ' AB ' . Encode ( ' Utf-8 ' ), 2.7 ) s =  struct. Struct ( ' I 2s f ' ) Packed_data =  s.pack (*  Values) print  ( ' original value: ' , values) print  (, S.format ) print  (, s.size) print  ( Package Result: ' , Binascii.hexlify (Packed_data)) 

# output(1, b'ab', 2.7)b'I 2s f'12b'0100000061620000cdcc2c40'

This example converts the packaged value to a sequence of hexadecimal bytes and binascii.hexlify() prints it in a method.

Use the unpack() method to unpack the package.

import structimport= binascii.unhexlify(b'0100000061620000cdcc2c40'= struct.Struct('I 2s f'= s.unpack(packed_data)print('解包结果:', unpacked_data)

# output(1, b'ab', 2.700000047683716)

Pass the packaged value unpack() to, basically return the same value (the floating-point number will be different).

BYTE order/Size/alignment

By default, pack is encoded using the byte order of the local C library. The first character of a formatted string can be used to represent the byte order, size, and alignment of the populated data, as described in the following table:

Character	Byte Order	Size	Alignment
@	Local	Local	Local
=	Local	Standard	None
<	Little-endian (small byte order)	Standard	None
>	Big-endian (large byte order)	Standard	None
!	Network (= Big-endian)	Standard	None

If these are not set in the format, they are used by default @ .

The local byte order refers to the byte order that is determined by the current host system. For example: Intel x86 and AMD64 (x86-64) Use small endian, and Motorola 68000 and PowerPC G5 use large endian. Arm and Intel Itanium support switching the byte order. You can use to sys.byteorder view the byte order of the current system.

The local size (size) and alignment (Alignment) are determined by the expression of the C compiler sizeof . It corresponds to the local byte order.

The standard size is determined by the format character, which tells the standard size of each format.

Example:

ImportstructImportBinasciivalues=(1,' AB '. Encode (' Utf-8 '),2.7)Print(' original value: ', values) endianness=[    ('@',' native, native '),    ('=',' native, Standard '),    (' < ',' Little-endian '),    (' > ',' Big-endian '),    ('!',' Network '),] forCode, nameinchEndianness:s=struct. Struct (Code+ ' I 2s f ') Packed_data=S.pack (*ValuesPrint()Print(' format character: ', S.format,' for ', name)Print(' consume bytes: ', s.size)Print(' package result: ', Binascii.hexlify (Packed_data))Print(' unpacking result: ', S.unpack (Packed_data))

# OutputOriginal value: (1B' AB ',2.7) format character: b' @ I 2s f '  forNative, native occupies bytes: APackage Result: b' 0100000061620000cdcc2c40 'Unpacking Result: (1B' AB ',2.700000047683716) format character: b' = I 2s f '  forNative, Standard occupies bytes:TenPackage Result: b' 010000006162cdcc2c40 'Unpacking Result: (1B' AB ',2.700000047683716) format character: b' < I 2s f '  forLittle-Endian occupied bytes:TenPackage Result: b' 010000006162cdcc2c40 'Unpacking Result: (1B' AB ',2.700000047683716) format character: b' > I 2s f '  forBig-Endian occupied bytes:TenPackage Result: b' 000000016162402CCCCD 'Unpacking Result: (1B' AB ',2.700000047683716) format character: b'! I 2s f '  forNetwork Consumption bytes:TenPackage Result: b' 000000016162402CCCCD 'Unpacking Result: (1B' AB ',2.700000047683716)

Format character

The Format character comparison table is as follows:

Standard

Format	C Type	Python Type	size	Notes
x	Pad byte	No value
c	char	bytes of length 1	1
b	signed char	Integer	1	(1), (3)
B	unsigned char	Integer	1	(3)
?	_Bool	bool	1	(1)
h	short	Integer	2	(3)
H	unsigned short	Integer	2	(3)
i	int	Integer	4	(3)
I	unsigned int	Integer	4	(3)
l	long	Integer	4	(3)
L	unsigned long	Integer	4	(3)
q	long long	Integer	8	(2), (3)
Q	unsigned long long	Integer	8	(2), (3)
n	ssize_t	Integer	(4)
N	size_t	Integer	(4)
f	float	Float	4	(5)
d	double	Float	8	(5)
s	char[]	bytes
p	char[]	bytes
P	void *	Integer		(6)

Buffer

Packaging data into binary is often used in scenarios where performance requirements are high.
In such a scenario, you can optimize by avoiding the overhead of allocating a new buffer for each packaging structure.
pack_into()and unpack_from() methods support direct write to pre-allocated buffers.

ImportArrayImportBinasciiImportcTYPESImportStructs=struct. Struct (' I 2s f ') values=(1,' AB '. Encode (' Utf-8 '),2.7)Print(' original value: ', values)Print()Print(' using ctypes module string buffer ') b=Ctypes.create_string_buffer (S.size)Print(' original buffer: ', Binascii.hexlify (B.raw)) S.pack_into (b,0,*ValuesPrint(' package result write: ', Binascii.hexlify (B.raw))Print(' Unpacking: ', S.unpack_from (b,0))Print()Print(' using the array module ') A=Array.array (' B 'B' + ' *S.size)Print(' original value: ', Binascii.hexlify (a)) S.pack_into (A,0,*ValuesPrint(' package write: ', Binascii.hexlify (a))Print(' Unpacking: ', S.unpack_from (A,0))

# output原始值: (1, b'ab'2.7buffer原始buffer  : b'000000000000000000000000'打包结果写入 : b'0100000061620000cdcc2c40'解包        : (1, b'ab'2.700000047683716)使用array模块原始值   : b'000000000000000000000000'打包写入 : b'0100000061620000cdcc2c40'解包     : (1, b'ab'2.700000047683716)

Python Standard library notes (6)-struct module