Tutorials for data compression using the Zlib module in Python

In the Python standard module, there are several modules for data compression and decompression, such as Zipfile,gzip, bz2, and so on. Last introduced the ZipFile module, today to talk about the Zlib module.
zlib.compress (string[, Level])
Zlib.decompress (string[, wbits[, bufsize])

The zlib.compress is used to compress stream data. The parameter string specifies the data stream to compress, and the level of the parameter specifies the degree of compression, which ranges from 1 to 9. The compression speed is inversely proportional to the compression rate, and 1 indicates the fastest compression, while the compression rate is the lowest, while 9 indicates the slowest compression rate with the highest compression. Zlib.decompress is used to decompress data. The parameter string specifies the data to be decompressed, and wbits and bufsize are used to set the system buffer size (window buffer) and the output buffer size, respectively (output buffers). Here's an example of how to use these two methods:

#coding =GBK import zlib, urllib fp = Urllib.urlopen (' http://localhost/default.html ') str = Fp.read () fp.close () #---- Compress the data stream. STR1 = zlib.compress (str, zlib. z_best_compression) str2 = zlib.decompress (str1) print len (str) print len (str1) Print Len (str2) #----Results #5783#1531#5783

We can also use the Compress/decompress object to compress/decompress the data. Zlib.compressobj ([level]) and zlib.decompress (string[, wbits[, bufsize]) Create compress/decompress indent objects, respectively. The use of compression and decompression of data through objects is very similar to the zlib.compress,zlib.decompress described above. But there is a difference between the two on the data compression, which is mainly reflected in the operation of large amounts of data in the case. If you want to compress a very large data file (hundred m) now, if you use zlib.compress to compress, you must first read the data in the file into memory, and then compress the data. This is bound to battle with too much memory. If you use the object to compress, then there is no need to read all the data at once, you can read some of the data into memory for compression, after compression, write to the file, and then read the other parts of the data compression, so the loop repeats, only to compress the entire file. The following example illustrates the difference between:

#coding =GBK import zlib, urllib fp = Urllib.urlopen (' http://localhost/default.html ') # access to the URL. data = Fp.read () fp.close () #----Compressed Stream str1 = zlib.compress (data, zlib. z_best_compression) str2 = zlib.decompress (str1) print ' raw data length: ', len (data) print '-' * 30print ' zlib.compress after compression: ', Len ( STR1) print ' zlib.decompress decompression: ', Len (str2) print '-' * # #----Use compress, decompress object to compress/decompress the data stream com_obj = Zlib.compressobj (zlib. z_best_compression) Decom_obj = Zlib.decompressobj () str_obj = com_obj.compress (data) Str_obj + = Com_obj.flush () print ' Compress.compress compressed: ', Len (str_obj) str_obj1 = decom_obj.decompress (str_obj) str_obj1 + = Decom_obj.flush () print ' Decompress.decompress decompression: ', Len (str_obj1) print '-' * # #----Use compress, decompress object, to block/decompress the data. Com_obj1 = Zlib.compressobj (zlib. z_best_compression) decom_obj1 = Zlib.decompressobj () chunk_size = 30; #原始数据分块str_chunks = [Data[i * chunk_size: (i + 1) * Chunk_size]/For I in range (len (data) + chunk_size)/chunk_size)] s Tr_obj2 = ' for chunk in str_Chunks:str_obj2 + = com_obj1.compress (chunk) Str_obj2 + = Com_obj1.flush () print ' chunked compressed: ', Len (str_obj2) #压缩数据分块解压str_ chunks = [Str_obj2[i * chunk_size: (i + 1) * Chunk_size]/For I in range (len (str_obj2) + chunk_size)/chunk_size)]str_o Bj2 = "for chunk in Str_chunks:str_obj2 + decom_obj1.decompress (chunk) Str_obj2 + = Decom_obj1.flush () print ' sub-block after decompression: ', le N (STR_OBJ2) #----Result------------------------raw data length: 5783------------------------------zlib.compress after compression: 1531zlib.decompress after decompression: 5783------------------------------compress.compress after compression: 1531decompress.decompress decompression: 5783------------------------------block after compression: 1531 sub-block after decompression: 5783

The Python manual describes the zlib module in more detail, more specific applications, and can refer to the Python manual.

