One, compression
(1.1) Use Gzip to package:
# time TAR-ZCF Tar1.tar binlog*
Real 0m48.497s
User 0m38.371s
SYS 0m2.571s
(1.2) Use PIGZ compression while setting the maximum compression speed (-1)
# Time TAR-CV binlog* | Pigz-1-P 24-k >pigz1.tar.gz
Real 0m10.715s
User 0m17.674s
SYS 0m1.699s
(1.3) Using Pigz compression, the default compression ratio
# Time TAR-CV binlog* | Pigz-p 24-k >pigz2.tar.gz
Real 0m22.351s
User 0m39.743s
Sys 0m1.341s
(1.4) Use PIGZ compression while setting the maximum compression ratio (-9)
# Time TAR-CV binlog* | Pigz-9-P 24-k >pigz3.tar.gz
Real 0m46.975s
User 1m26.118s
Sys 0m1.473s
(1.5) Use LZ4 compression while setting the maximum compression speed (-1)
# Time TAR-CV binlog* | Lz4-1 > lz41.tar.gz
Real 0m4.969s
User 0m2.953s
Sys 0m1.199s
(1.6) Using LZ4 compression, using the default settings
# Time TAR-CV binlog* | Lz4 > lz42.tar.gz
Real 0m4.447s
User 0m2.958s
Sys 0m1.185s
(1.7) Use LZ4 compression while setting the maximum compression ratio (-9)
# Time TAR-CV binlog* | lz4-9 > lz43.tar.gz
Real 0m56.144s
User 0m54.300s
Sys 0m1.301s
After packing compressed file size in the above way
127M Tar1.tar (1.1)
152M Pigz1.tar.gz (1.2)
127M Pigz2.tar.gz (1.3)
123M Pigz3.tar.gz (1.4)
216M Lz41.tar.gz (1.5)
216M LZ42.TAR.GZ (1.6)
163M Lz43.tar.gz (1.7)
Second, decompression:
(2.1) Unzip the tar file directly using TAR-XVF
# time TAR-XVF Tar1.tar
Real 0m42.329s
User 0m9.134s
Sys 0m2.704s
(2.2) Extract Pigz packaged files directly using TAR-XVF (Pigz compressed files can be extracted directly using tar)
# time TAR-XVF pigz1.tar.gz
Real 0m42.048s
User 0m9.818s
Sys 0m2.927s
(2.3) Unzip the Pigz compressed file using pigz-d
# File=pigz1.tar && pigz-k-D $file. GZ && Time TAR-XF $file
Real 0m56.706s
User 0m0.074s
Sys 0m2.588s
(2.4) Do not use TAR-XVF to extract Lz4 compressed files
# time TAR-XVF lz41.tar.gz
Gzip:stdin:not in gzip format
Tar:child returned status 1
Tar:error isn't recoverable:exiting now
(2.5) Unzip the LZ4 compressed file using lz4-d
# File=lz41.tar && lz4-d $file. GZ $file && Time TAR-XF $file
Successfully decoded 1203701760 bytes
Real 0m59.851s
User 0m0.064s
Sys 0m2.699s
Iii. Conclusion:
From the above data can be summed up the following conclusions:
1. The use of Pigz is significantly faster than gzip (online parlance is 5 times times), but at the same time CPU consumption is also relatively large (online parlance is about 8 times times), both compressed file size is the same.
2. Using LZ4 can get good compression speed, but the compressed file is slightly larger than gzip and pigz.
3.pigz compressed files can be decompressed directly with the tar-x instruction, while files compressed with LZ4 are not available.
From the implementation of data, if the current system wants to improve the compression decompression rate, you can consider switching to Pigz compression mode, but will increase CPU consumption. It is appropriate to use PIGZ in scenarios where compression efficiency is high, but not affected by high CPU consumption over a short period of time.
Comparison of 3 Lz4,pigz,gzip
