LevelDB Reading Records

"LevelDB read record "

Leveldb is a stand-alone repository for large-scale key/value data, and LEVELDB is a storage tool from an application perspective. And as a competent storage tool, the common call interface is nothing more than a new KV, delete kv, read kv, update key corresponding to the value of such several operations. Leveldb interface does not directly support the interface of the update operation, if you need to update the value of a key, you can choose to insert the new KV directly vigorous, maintain the same key, so that the system key corresponding to the value will be updated, or you can first delete the old kv, Then insert the new KV, so that the more tactful to complete the update operation of KV.

Assuming the application submits a key value, let's see how leveldb reads its corresponding value value from the stored data. Figure 7-1 is the whole of the LEVELDB reading process.

　　

Leveldb first looks at the in-memory memtable, if the memtable contains the key and its corresponding value, the value is returned, if the memtable is not read to key, then the same in-memory immutable Memtable read, similarly, if read to return, if not read, then can only helpless down from the disk in a large number of sstable files to find. Because the number of sstable is large and divided into levels, reading the data in sstable is quite a winding journey. The general reading principle is this: first look up from a file belonging to level 0, if found then return the corresponding value value, if not found then to the level 1 in the file to find, so repeated, Until the value corresponding to this key is found in a layer sstable file (or at the highest level, the lookup fails, indicating that the key does not exist in the entire system).

So why is it from memtable to immutable memtable, then from immutable memtable to files, and why is this a query path from low to high? What is the reason? This query path is chosen because, from the time of update of information, it is obvious that the Memtable store is the freshest kv pair; The immutable of the KV data pairs stored in the memtable , and the KV data in all sstable files must not be as fresh as in-memory memtable and immutable memtable. For sstable files, if the same key,level L information is found at level L and level l+1, it must be newer than level l+1. In other words, the search path listed above is sorted by the freshness of the data, and the more fresh it is, the more it will be searched first.

Why would you prefer to find fresh data? This is a truism, for instance. For example, we first insert a data into the leveldb {key= "www.samecity.com" value= "we"}, after a few days, the Samecity website renamed: 69 The same city, at this time we insert data {key= "www.samecity.com "Value=" 69 with the same city "}, the same key, different value, logically understood as if there is only one storage record in Leveldb, that is, the second record, but there is likely to be two records in Leveldb, that is, the above two records are stored in Leveldb, At this point, if the user queries key= "www.samecity.com", we certainly want to find the latest update record, that is, the second record is returned, which is why you should first look for fresh data.

The previous article said: for sstable files, if both level L and level l+1 to find the same key,level l information must be newer than the level l+1. This is a conclusion that theoretically requires a process of proving otherwise it will incur the following problems: For God's horse? From the truth, it is clear: because the level of l+1 data is not from the stone seam, nor dream of dreaming, then it is from where? Level l+1 data is obtained from level L after compaction (if you don't know what compaction is, then ...). Perhaps later will know), that is, you see the current level l+1 layer of sstable data is from the original levels L, and now I am more than the original to be fresh, level l data, so can be certified, now the level L + now 1 of the data should be fresh.

Sstable file a lot, how to quickly find key corresponding value? In Leveldb, Level 0 has always been a specialization, and the process of finding a key in level 0 and Other level is not the same. Because different files under level 0 may overlap the range of keys, a key to be queried may contain multiple files, so the LEVELDB strategy is to first find What files in 0 contain this key (the manifest file contains the scope information of the key in the level and corresponding files and files, leveldb the mapping table in memory), sorted by the freshness of the file, the new file is in front, and then in turn, Read the value corresponding to key. And if it is non-level 0, because the level of the file between the key is not overlapping, so only from a file can find the key corresponding to the value.

The last question, if given a key to query and a key range contains the sstable file of this key, then how does leveldb do the specific search process? LEVELDB will typically look in the in-memory cache for a cached record of the file, if it is included, read from the cache, or if not, open the Sstable file and load the index portion of the file into memory and into the cache. In this case, the cache has this sstable, but only the index part in memory, then LEVELDB according to the index can be located to which content block will contain this key, read the contents of this block from the file, in accordance with the records of the comparison, If found, the result is returned, if not found, then the sstable file of this level does not contain this key, so go to the next sstable to find.

From the previous introduction of the LEVELDB and the read operation described here can be seen, relative write operations, read operations to deal with a lot more complex, so the speed of writing must be much higher than the speed of reading data, that is, leveldb more suitable for write operations than read operation of the application. If the application is a large number of read operation types, sequential read efficiency will be higher, because most of the content will be found in the cache, as much as possible to avoid a large number of random read operations.

Reference: http://www.cnblogs.com/haippy/archive/2011/12/04/2276064.html

LevelDB Reading Records