Google's snappy Compression Algorithm

In many cases, we need to compress a large amount of data quickly, and make a balance between disk Io and CPU so thatProgramIt runs faster. Snappy is such a fast data compressionAlgorithm. For a single core i7 processor (64-Bit mode), the processing speed can reach more than 250 Mb/s.

Generally, snappy is faster than similar compression algorithms, such as lzo, lzf, fastlz, and quicklz. The compression ratio of plain text is about 1.5 ~ 1.7x, which is 2-4x for HTML web pages and does not compress binary data such as images. The value is 1x.

Snappy optimizes the 64-bit and 32-bit processors respectively. Both the 32-bit and 64-bit processors can achieve high efficiency. According to the official introduction, snappy has been tested by Pb-level big data, so it is easy to maintain stability. Many Google frameworks, such as map reduce and RPC, use the snappy compression algorithm.

I currently use the compression algorithm when writing mapreduce programs. The effect is good.

Snappy EnabledSource codePut it on Google Code: http://code.google.com/p/snappy/, new BSD license open source.