LevelDb Analysis of the five (LevelDb of some dynamic operations < two >)

First, read the record according to key:

LEVELDB Read record flow

leveldb First you'll look at the in-memory memtable , If the memtable contains a key and its corresponding value, the value is returned, and if memtable is not read to key, then the immutable Memtable reads, similarly, if read to return, if not read, then can only Helpless from a large number of sstable files in the disk . Because the number of sstable is large and divided into levels, reading the data in sstable is quite a winding journey. The general reading principle is this: first look up from a file belonging to level 0, if found then return the corresponding value value, if not found then to the level 1 in the file to find, so repeated, Until the value corresponding to this key is found in a layer sstable file (or at the highest level, the lookup fails, indicating that the key does not exist in the entire system).

Sstable file a lot, how to quickly find key corresponding value? In Leveldb, Level 0 has always been a specialization, and the process of finding a key in level 0 and Other level is not the same. Because different files under level 0 may overlap the range of keys, a key to be queried may contain multiple files, so the LEVELDB strategy is to first find What files in 0 contain this key (the manifest file contains the scope information of the key in the level and corresponding files and files, leveldb the mapping table in memory), and then sort by the freshness of the file, the new file is in front, After that, the value corresponding to the key is read out. And if it is non-level 0, because the level of the file between the key is not overlapping, so only from a file can find the key corresponding to the value.

From the previous introduction of the LEVELDB and the read operation described here can be seen, relative write operations, read operations to deal with a lot more complex, so the speed of writing must be much higher than the speed of reading data, that is, leveldb more suitable for write operations than read operation of the application. If the application is a large number of read operation types, sequential read efficiency will be higher, because most of the content will be found in the cache, as much as possible to avoid a large number of random read operations.

Second, improve the reading efficiency of the Table cache and Block cache.

For LEVELDB, if the read operation does not find a record in the memory memtable, the disk access operation is performed more than once. Assuming the optimal situation, that is, the first time in level 0 in the most recent file found this key, then also need to read 2 disk, one time is the sstable file in the index portion of the memory read, so according to this index can determine which block in which the key is stored The second time is to read the contents of the block and then find the value corresponding to the key in memory.

Two different cache:table caches and block caches were introduced in LEVELDB to improve the reading efficiency .

Where the block cache is optional, specify whether to turn this feature on in the configuration file.

Table Cache:

For example, in a Get (key) read operation, if LEVELDB determines that the key is within the range of a key range of a file A, it is necessary to determine if the file a really contains the KV. At this point, LEVELDB will first look for the table cache to see if the file is in the cache, and if so, then you can find out which block contains the key based on the index section. If the file is not found in the cache, open the Sstable file, read its index part into memory, and then insert it into the cache to locate which block in index contains the key.

If you determine which block contains this key, you need to read the block content, which is the second read.

Block Cache:

　　This is how the leveldb speeds up reading with two caches. From here, it can be seen that if the data is better local, that is to read the most of the data in the cache can be read, then the reading efficiency should be very high, and if the key is sequential read efficiency should be good, because once read in can be reused multiple times. But if you read randomly, you can infer how efficient it is.

Three

LevelDb Analysis of the five (LevelDb of some dynamic operations < two >)