Data storage rules in cassandra
- Data: stores real data files. multiple directories can be specified for the sstable file.
- Commitlog: stores data that is not written to sstable (put in the log file before each write ).
- Cache: stores cached data in the system (loads cached data from this directory when the service is restarted ).
Reasonably arrange the positions between the above nodes to improve performance.
CommitlogCommilog consists of two parts: Commilog-XXXX.log and Commilog-XXXX.log.header. The Commitlog-XXXX.log file stores the value of the last update operation, and the Commitlog-XXXX.log.header records the data that has been written to sstable from memtable. Cassandra has two methods to record commitlog: "Cycle" and "batch. When the size of a commitlog file exceeds a threshold, a new commitlog is created. Every update of Cassandra information will be written to commitlog, and files will be synchronized at intervals (the cached data will also be synchronized so that all files will be saved in commitlog ).
MemtableAfter data is written to commitlog, It is cached in memtable (each memtable is a columnfamily service ). When the capacity, number, and interval in memtable exceed the threshold, data is written to the disk to form an sstable file. The data saved in memtable is concurrentskiplistmap <decoratedkey, columnfamily>. The data is written in the key order. The advantage of using memtable is to change random Io into sequential io (which seems to be used in many systems ).
SstableAn sstable file of CF1 consists of the following files:
- Cf1-e-1-Data.db
- Cf1-e-1-Filter.db
- Cf1-e-1-Index.db
- Cf1-e-1-Statistics.db
(File Name)-(Version Number)-(File ID)-(four different parts)
- The filter file is used to quickly locate whether a key is in the sstable file (using a Boolean filter for determination ).
- The index file is used for the index file to save the key and the corresponding location in the data file. Hash and then perform binary search when searching in the memory (there is also a cache optimization, how does this feel like the usual method in the kernel, haha ).
- Data File Stores the index information of some columns corresponding to the data and key (the index is used to quickly locate the value to be searched ).
- The statistics file is used to store the number of columns and the number of rows contained in sstable.
System tablespaceThis part is similar to the practice of many relational databases. In cassandra, in addition to the custom keyspace, there is also a special keyspace: system.
- Manage the system metadata information of Cassandra.
- Cache hint data.
Welcome to shoot bricks.
