Performance comparison of several common fast lossless compression algorithms

Source: Internet
Author: User
Tags intel core i7

Snappy
Snappy is a compression library used by many projects in Google's in-house production environment, including Bigtable,mapreduce and RPC. Google says the algorithm library has been tuned to the target, not for compression ratios or compatibility with other similar tools. On Intel Core i7 processors, the ability to process data streams on a single core reaches 250m/s-500m/s. The snappy is also optimized for the 64-bit x86 processor, which achieves at least 250mb/s compression performance and 500mb/s decompression performance at the Intel Core i7 processor single cores. Snappy has a compression rate of 1.5-1.7 for plain text, 2-4 for HTML, and 1.0 for JPEG, PNG, and other compressed data. Google strongly touts snappy's robustness, calling it "a design that will not crash even in the face of damage or malicious input," and is stable in Google's production environment, which is tested by petabytes of data compression.
Official website: http://code.google.com/p/snappy/

Fastlz
FASTLZ is an efficient lightweight compression solution Engross, whose official test data is shown in the following table:


1GB Text data test:

Official website: http://www.quicklz.com/

Lzo/minilzo
Lzo is an open source lossless compression C language library, the advantage is that compression and decompression compared to the rapid use of memory small and other characteristics (network transmission hope is the compression and decompression speed is relatively fast, the compression rate is not very high), it provides a relatively full Lzo library and a compact version of the Minilzo library, the online test data are as follows:
Test using BMP and text files on X86 Linux virtual machines (single core 256M memory, Debian 6.0 OS).

Test file

Original size

Size after compression

Compression ratio

Compression time

Decompression time

1.bmp

5292054

159395

3.01%

9.174ms

23.037ms

2.bmp

6912056

33806

0.489%

8.33ms

36.17ms

3.bmp

6220856

5101891

82%

25.78ms

28.43ms

Lzo.tar

6645760

2457890

36.98%

34.68ms

38.62ms

Kdoc.tar

16660480

6987402

41.93%

102.86ms

108.2ms

Kinc.tar

18257920

5684927

31.13%

106.87ms

113.86ms

Official website: http://www.oberhumer.com/opensource/lzo/
A comparison from the Hbase:the definitive guide:

Algorithm

% remaining

Encoding

Decoding

Gzip

13.4%

MB/s

118 MB/s

LZO

20.5%

135 MB/s

410 MB/s

Zippy/snappy

22.2%

172 MB/s

409 MB/s

Performance comparison of several common fast lossless compression algorithms

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.