Introduction to compression in cassandra

ArticleDirectory

• Start cassandra
• New sstable files need to be written to disk

In "how to install and configure Cassandra", we can configure the data storage location in datafiledirectories.

After Cassandra is started, the data inserted into it will be placed in the directory of datafiledirectories,

There are two column families, standrad1 and super1.

As you can imagine, if the compression function is not available, there will be more and more files in the datafiledirectories directory as the data volume increases.

The compression function in Cassandra avoids the existence of a large number of data files in the datafiledirectories directory.

Set Compression Parameters

After Cassandra is started, we can use sh bin/nodetool-H hostname-P jmxport getcompactionthreshold to obtain the current Cassandra compression parameter: Minimum compression value and maximum compression value.

We can also use the command sh bin/nodetool-H hostname-P jmxport setcompactionthreshold minthreshold maxthreshold to set the minimum and maximum compression values of Cassandra.

Disable compression manually

To test the compression function, run the following command: Sh bin/nodetool-H hostname-P jmxport setcompactionthreshold 0 0

After executing this command, Cassandra's compression function is disabled. When a large amount of data is inserted, you will find that a large number of data files appear in the datafiledirectories directory.

Manual start force Compression

In the previous step, we disabled compression. The advantage of this operation is that it can increase the import speed of large data volumes (this issue will be introduced later ). However, the disadvantage is that a large number of data files appear in the datafiledirectories directory.

If you want to compress these data files. Run the following command: Sh bin/nodetool-H hostname-P jmxport compact

After executing this command, Cassandra combines different column family files into one file. The merging order is based on the time sequence generated by the data files.

Normal compression process

Because Cassandra's compression parameters cannot be set in the configuration file, the default parameter is used each time Cassandra is started: Minimum compression value 4, maximum compression value 32.

The compression operation is used in the following two scenarios:

Start cassandra

When Cassandra is started, it will perform the compression operation: traverse the number of files in each Cf. If it exceeds the minimum value of 4, it will calculate an average value (4 + 32)/2 ), then, sort the data files according to the time sequence generated. Take the average value and a small number of actual files to merge the files.

New sstable files need to be written to disk

When the data in memtable reaches the threshold value (the size of this threshold value can be specified in the configuration file), a new sstable file will be generated. At this time, the system will perform the above "Cassandra start" logic to determine whether compression is required. If compression is required, which files will be merged.

 

These are the compression operations in Cassandra. For more details, refer to org. Apache. Cassandra. DB. compactionmanager.

I hope this article will help you understand the compression operations in Cassandra.

 

More about Cassandra: http://www.cnblogs.com/gpcuster/tag/Cassandra/