LevelDb Principle Analysis

Before talking about LevelDb, I first met two Daniel, Jeff Dean and Sanjay Ghemawat. These two are heavy-weight Google engineers and few Google Fellow engineers.

Jeff Dean: Jeff Dean.

Sanjay Ghemawat: Taobao.

LevelDb is an open-source project initiated by these two great-level engineers. In short,LevelDb is a C ++ library that can process Key-Value data persistence storage of billions of data records.. As mentioned above, these two are the design and implementers of Bigtable. If you know about Bigtable, you should know that there are two core parts in this influential distributed storage system: master Server and Tablet Server. The Master Server manages data storage and distributed scheduling. The actual distributed data storage and read/write operations are performed by the Tablet Server, levelDb can be understood as a simplified Tablet Server.

LevelDb has the following features:

First, LevelDb is a KV System with persistent storage. Unlike the memory-type KV system such as Redis, LevelDb does not eat as much memory as Redis, but stores most of the data on disks.

Secondly, LevleDb stores data in sequence based on the recorded key values, that is, the adjacent key values are stored sequentially in the stored files, applications can customize key size comparison functions. LevleDb stores these records in sequence according to user-defined comparison functions.

Again, like most KV systems, the operation interface of LevelDb is very simple. Basic operations include writing records, reading records, and deleting records. It also supports atomic batch operations for multiple operations.

In addition, LevelDb supports the snapshot function, so that the read operation is not affected by the write operation, and consistent data can always be seen during the reading operation.

In addition, LevelDb also supports data compression and other operations, which directly helps reduce storage space and increase IO efficiency.

LevelDb has outstanding performance. The official website reports that its random write performance reaches 0.4 million records per second, while the random read performance reaches 60 thousand records per second. In general, write operations of LevelDb are much faster than read operations, while sequential read/write operations are much faster than random read/write operations. As for the reason, we have read our subsequent LevelDb daily records, and you will probably understand the internal reasons.

Record 2 of LevelDb: overall architecture

LevelDb is essentially a set of storage systems and some operation interfaces provided on this set of storage systems. To facilitate understanding of the entire system and its processing process, we can look at LevleDb from two different perspectives: static and dynamic. From the static point of view, we can assume that the entire system is running (constantly inserting and deleting data to read). At this point, we will take a photo of LevelDb, from the photos, we can see how the system data is distributed in memory and disk and in what status. from a dynamic perspective, we mainly know how the system writes a record, reads a record, deletes a record, and also includes internal operations in addition to these interface operations, such as compaction, how to restore the system after the system crashes during runtime, and so on.

The overall architecture described in this section is mainly described from the static perspective. The subsequent sections will detail the file or memory data structures involved in the static structure, the second half of the LevelDb daily records mainly describes the LevelDb in a dynamic perspective, that is, how the entire system runs.

As a storage system, LevelDb stores data records in memory and disk files. As mentioned above, when LevelDb has been running for a while, let's take a perspective photo of LevelDb, you will see the following:

 

 

Figure 1.1: LevelDb Structure

It can be seen that the LevelDb static structure consists of six main parts: MemTable and Immutable MemTable in the memory and several main files on the disk: Current file, Manifest file, log file and SSTable file. Of course, in addition to these six main parts, LevelDb also has some auxiliary files, but the above six files and data structures are the main components of LevelDb.

LevelDb's Log file and Memtable are consistent with those described in the Bigtable paper. When an application writes a Key: Value record, LevelDb writes the record to the log file first, after successful writing, the record is inserted into the Memtable, which basically completes the write operation, because one write operation only involves one disk sequential write and one memory write, that's why LevelDb writes extremely fast.

The role of Log files in the system is mainly used for system crash recovery without data loss. If there is no Log file, because the written records are saved in the memory at the beginning, if the system crashes, data in the memory has not been dumped to the disk, so data will be lost (this problem exists in Redis ). To avoid this situation, LevelDb first records the operations in the Log file before writing the data into the memory, and then records them in the memory, so that even if the system crashes, you can also recover the Memtable in the memory from the Log file without causing data loss.

When the memory occupied by the data inserted by Memtable reaches a limit, the memory records need to be exported to the external storage file. LevleDb will generate a new Log file and Memtable, the original Memtable becomes an Immutable Memtable. As the name suggests, the content of this Memtable cannot be changed and can only be read or deleted. The new data is recorded in the new Log file and Memtable. The LevelDb background scheduling will export Immutable Memtable data to the disk to form a new SSTable file. SSTable is formed by constantly exporting data in the memory and performing Compaction operations. All files of SSTable are in a hierarchical structure. The first layer is Level 0, and the second layer is Level 1, and so on, the level gradually increases, which is why LevelDb is called.

The files in SSTable are Key-ordered. That is to say, the SSTable of each Level is the same when the Small and Medium key records are before the large Key records. However, note that: level 0 SSTable file (Suffix. sst) is more special than other Level files: Within this Level. sst file. Two files may overlap keys. For example, two levels 0 sst files exist, file A and file B. The key range of file A is {bar, car }, if the Key range of file B is {blue, samecity}, both files may have a key = "blood" record. For other levels of SSTable files, it will not appear in the same Level. the key of the sst file overlaps, that is, any two of Level L. sst files, so that their key values do not overlap. Pay special attention to this. Later, you will see that many operations are different for this reason.

A file in SSTable belongs to a specific level, and its stored records are sorted by keys. Therefore, the minimum key and maximum key in the file must exist, which is very important information, levelDb should write down this information. Manifest records the management information of each SSTable file, such as the Level of the file, the file name, and the minimum key and maximum key respectively. Is the description of the content stored by Manifest:

 

 

Figure 2.1: Manifest Storage

The figure shows only two files (manifest records the information of all SSTable files), that is, test of Level 0. sst1 and test. the sst2 file also records the key ranges corresponding to these files, such as test. the key range of sstt1 is "an" to "banana", and the file test. the key range of sst2 is "baby" to "samecity". We can see that the key ranges of sst2 overlap.

What is the Current file? This file contains only one information, that is, the current manifest file name. In the running process of LevleDb, with Compaction, The SSTable file will change, new files will be generated, old files will be discarded, and Manifest will also reflect this change, at this time, a new Manifest file is generated to record this change, while Current is used to indicate which Manifest file is the one we care about.

The content described above constitutes the overall static structure of LevelDb. In the subsequent content of LevelDb's Daily recording, we will first introduce the specific data layout and structure of important files or memory data.

LevelDb log 3: log File

In the previous section, the main function of log files in LevelDb is to ensure that data is not lost during system fault recovery. Because before writing records to the Memtable in the memory, the Log file is written, so that even if the system fails, the data in the Memtable cannot be dumped to the SSTable file on the disk, levelDB can also restore the Memtable data structure content in the memory based on the log file, without causing system data loss. LevelDb and Bigtable are consistent in this regard.

Let's take a look at the specific physical and logical layout of log files. LevelDb will cut a log file into a physical Block in 32 K, each read unit uses a Block as the basic read unit. The displayed log file consists of three blocks. Therefore, in terms of physical layout, A log file is composed of consecutive 32 K blocks.

 

 

Figure 3.1 log File Layout

These blocks are not visible in the application's field of view. The application sees a series of Key: Value pairs. Inside LevelDb, a Key: Value Pair is considered as a record data, in addition, a record header is added before the data to record some management information for internal processing. Figure 3.2 shows how a record is represented inside LevelDb.

 

 

Figure 3.2 record Structure

The record header contains three fields. ChechSum is the verification code for the "type" and "data" fields. To avoid incomplete or damaged data, when LevelDb reads record data, it will verify the data. If it finds that it is the same as the stored CheckSum, it means that the data is complete and intact, and you can continue the subsequent process. The "record length" records the data size. The "data" is the Key: Value pair described above, the "type" field indicates the relationship between the logical structure of each record and the physical block structure of the log file. Specifically, there are four types: FULL/FIRST/MIDDLE/LAST.

If the record type is FULL, it indicates that the current record content is completely stored in a physical Block and is not cut by different physical blocks. If the record is cut by adjacent physical blocks, the data type is one of the other three types. We will illustrate this in the example shown in Figure 3.1.

Assume that three records exist: Record A, Record B, and Record C. The size of Record A is 10 K, Record B is 80 K, and Record C is 12 K, the logical layout in the log file is shown in Figure 3.1. Record A is shown in the blue area in the figure. Because the size is 10 K <32 K, it can be placed in A physical Block, so its type is FULL; Record B is 80 K, because Block 1 is put into Record A, there is still 22 K left, which is not enough to put down Record B. Therefore, the rest of Block 1 is put into the FIRST part of Record B, and the type is marked as FIRST, it indicates that it is the starting part of a Record. There are still 58K records not stored in Record B, which can only be placed in the subsequent physical blocks in sequence, because Block 2 is only 32 K in size, the remaining part of Record B is not stored, so Block 2 is used to put Record B and the ID type is MIDDLE, which means this is a piece of data in the MIDDLE of Record B; the rest of Record B can be fully placed in Block 3. The type ID is LAST, which indicates the end data of Record B. The yellow Record C in the figure is 12 K in size, block 3 has enough space for all. Put it down, so its type is identified as FULL.

From this small example, we can see the relationship between the logical record and the physical Block. LevelDb reads the logical record as a Block at a time, and then Concatenates the logical Record Based on the type for subsequent processing.

Record 4 of LevelDb: SSTable File

SSTable is a crucial part of Bigtable. It is also true for LevelDb. Understanding the SSTable Implementation Details of LevelDb also helps you understand some implementation details in Bigtable.

This section describes the static layout structure of SSTable. We once said in "LevelDb daily Note II: overall architecture" That SSTable files form a hierarchical structure of different levels, as for how this hierarchical structure is formed, we will discuss it in the Compaction section below. This section describes the physical layout and logical layout structure of a file in SSTable, which is helpful for understanding the running process of LevelDb.

Different levels of LevelDb have many SSTable files (characterized by. sst later). The internal layout of all. sst files is the same. In the previous section, we introduced that Log files are physically segmented, and SSTable will also divide the files into physical storage blocks of a fixed size, but the logical layout of the two is very different, the root cause is: the records in the Log file are unordered, that is, the Key size of the record is not explicitly related to the size,. the sst file is arranged from small to large according to the record Key. From the SSTable layout described below, we can see why Key order is designed. the key to the sst file structure.

 

 

Figure 4.1. Structure of sst file blocks

Figure 4.1 shows. the physical division structure of sst files, like Log files, is also a storage Block of fixed size. Each Block is divided into three parts, and the red part is the data storage area, the blue Type area is used to identify whether the data storage area adopts the Data Compression Algorithm (Snappy compression or no compression), and the CRC part is the data verification code, used to identify whether data is generated and transmitted incorrectly.

The above is. the physical layout of sst is described below. the logical layout of sst files, the so-called logical layout, that is to say, although everyone is a physical block, what content is stored in each block, what internal structure is, and so on. Figure 4.2 shows the internal logic of the. sst file.

 

 

Figure 4.2 logical layout

As shown in Figure 4.2. sst files are divided into data storage areas and data management areas. The data storage area stores the actual Key: Value data. The data management area provides some index pointers and other management data, the goal is to quickly and conveniently search for corresponding records. Both regions are based on the above blocks, that is, the first several blocks of the file actually store KV data, and the subsequent data management areas store and manage data. Management data is divided into four different types: Purple Meta Block, red MetaBlock index, blue data index Block, and a file tail Block.

LevelDb 1.2 is not actually used for Meta Block, but only retains an interface. It is estimated that content will be added to later versions. Let's look at the internal structure of the Data Index Area and the Footer at the end of the file.

 

 

Figure 4.3 Data Index

Figure 4.3 shows the internal structure of the data index. Once again, The KV records in the Data Block are arranged in ascending order of keys. Each record in the Data index area is the index information created for a Data Block, each Index contains three items, for example, Index I of data block I shown in Figure 4.3: the first field in the red part records the Key that is greater than or equal to the maximum Key value in data block I, and the second field indicates that the data block I is in. start position in the sst file. The third field indicates the size of Data Block I (sometimes there is Data compression ). The following two fields are used to locate the position of the data block in the file. The first field needs to be explained in detail, the Key value stored in the index may not necessarily be the Key of a record. For example, in Figure 4.3, assume that the minimum Key of data block I is "samecity ", maximum Key = "the best"; minimum Key of data block I + 1 = "the fox", and maximum Key = "zoo". For Index I of data block I, the first field records the minimum Key ("the best") that is greater than or equal to the maximum Key ("the best") of block I + 1 and smaller than that of block I + 1 ("the fox "), in this example, the first field of Index I is "the c", which meets the requirements, while the first field of Index I + 1 is "zoo ", that is, the maximum Key of the data block I + 1.

The internal structure of the Footer block at the end of the file is shown in Figure 4.4. metaindex_handle indicates the start position and size of the metaindex Block. inex_handle indicates the start address and size of the index block; these two fields can be understood as the index, which is set up to correctly read the index value, followed by a filling area and magic number.

Figure 4.4 Footer

The above section mainly describes the internal structure of the data management area. Let's take a look at how the data part of a Block in the data area is internally arranged (the red part in Figure 4.1 ), figure 4.5 shows its internal layout.

 

 

Figure 4.5 internal structure of data Block

It can be seen that it is also divided into two parts, the first is a KV record, the order is based on the Key value from small to large, at the end of a Block is a Restart Point, which is actually a pointer pointing to some recorded locations in the Block content.

What is "Restarting? We have repeatedly stressed that the KV records in the Block content are sorted by the Key size. In this case, the adjacent two records may overlap the Key part, for example, key I = "the Car ", key I + 1 = "the color", there is an overlap between the two "the c". To reduce the storage of the Key, key I + 1 can only store the part "olor" that is different from the previous Key, and the common part of the two can be obtained from Key I. The record Key is stored in the Block content to reduce storage overhead. The "restart point" means that at the beginning of this record, all Key values are rerecorded instead of recording only different Key parts. Assume that Key I + 1 is a restart point, the "the color" is fully stored in the Key, instead of using the simple "olor" method. The end of the Block indicates which records are the key points.

 

 

Figure 4.6 record format

In the Block content area, what is the internal structure of each KV record? Figure 4.6 shows its detailed structure. Each record contains five fields: key share length, for example, the above "olor" record, the length of the key shared with the previous record is the length of "the c", that is, 5; the non-shared length of the Key, which is 4 for "olor"; the length of the value indicates the key: the length of Value in Value. The actual Value is stored in the Value content field. The key string "olor" is actually stored in non-shared content.

These are all internal mysteries of the. sst file.

Record 5 of LevelDb: MemTable details

The preceding section of the LevelDb daily notice describes important static structures related to disk files. This section describes the data structure in memory Memtable, and its important position in the entire system is self-evident. In general, all KV data is stored in Memtable, Immutable Memtable, and SSTable. Immutable Memtable is exactly the same as Memtable in terms of structure. The difference is that it is read-only, write operations are not allowed, while Memtable allows writing and reading. When the memory occupied by data written by Memtable reaches a specified amount, it is automatically converted to Immutable Memtable. when the data is dumped to the disk, the system will automatically generate a new Memtable for write operations to write new data, if you understand Memtable, Immutable Memtable is easy to understand.

The MemTable of LevelDb provides interfaces for writing, deleting, and reading KV records. However, in fact, there is no real delete operation in Memtable, deleting the Value of a Key is implemented as inserting a record in Memtable, but it is marked with a Key. The true delete operation is Lazy, this KV will be removed in the Compaction process in the future.

Note that KV pairs in the Memtable of LevelDb are stored in sequence based on the Key size. When the system inserts a new KV, levelDb needs to insert this KV to a proper location to maintain this Key order. In fact, the Memtable class of LevelDb is only an interface class. The real operation is done through the SkipList behind it, including the insert and read operations. Therefore, the core data structure of Memtable is a SkipList.

SkipList was invented by William put. He published Skip lists: a probabilistic alternative to balanced trees in Communications of the ACM June 1990, 33 (6) 668-676, the data structure and insert/delete operations of SkipList are explained in detail in this paper.

SkipList is an alternative to the data structure of the Balance Tree. But unlike the red and black trees, SkipList balances trees based on a randomization algorithm, in this way, it is relatively simple to insert and delete the SkipList.

For a detailed introduction to the SkipList, refer to this article.

SkipList is not only a simple implementation of maintaining ordered data, but also avoids frequent tree node adjustment operations when inserting data compared with the Balance Tree. Therefore, the write efficiency is very high, levelDb is a high-Write System, and SkipList should also play an important role in it. To speed up the insert operation, Redis also uses the SkipList as the internal data structure.

Write and delete records of LevelDb daily records

In the previous five sections of LevelDb daily records, we introduced some static files of LevelDb and their detailed layout. Starting from this section, let's take a look at some dynamic operations of LevelDb, such as reading and writing records, Compaction, restore errors.

This section describes how to update records in levelDb, that is, insert a KV record or delete a KV record. The update operation speed of levelDb is very fast, because its internal mechanism determines the simplicity of this update operation.

 

 

Figure 6.1 LevelDb write records

Figure 6.1 shows how levelDb updates KV data. For an insert operation Put (Key, Value), the insert operation includes two steps: the first step is to append the KV record to the end of the previously introduced log file in the form of sequential writing, because although this is a disk read/write operation, the efficiency of sequential file append writing is very high, therefore, the write speed does not decrease. The second step is: if the log file is successfully written, insert the KV record into the Memtable in the memory. As mentioned earlier, Memtable is only a layer of encapsulation, it is actually a Key-ordered SkipList, and the process of inserting a new record is also very simple, that is, first find the appropriate insert location, and then modify the corresponding link pointer to insert the new record. After this step is completed, the write record is completed. Therefore, an insert record operation involves a disk file append write operation and a memory SkipList insert operation. This is the root cause of the high write speed of levelDb.

From the above introduction, we can also see that keys in log files are unordered, while keys in Memtable are ordered. What if a KV record is deleted? For levelDb, there is no immediate delete operation, but it is the same as the insert operation. The difference is that the insert operation inserts the Key: Value, and the delete operation inserts the "Key: the delete tag does not actually Delete the record, but does the true delete operation only when Compaction is performed in the background.

The write operation of levelDb is so simple. The real trouble lies in the read operation to be introduced later.

RECORD 7: Read records

LevelDb is a standalone database for large-scale Key/Value data. From the perspective of applications, LevelDb is a storage tool. As a competent storage tool, the common calling interface is nothing more than adding KV, deleting KV, reading KV, and updating the Value corresponding to the Key. The LevelDb interface does not directly support update operations. To update the Value of a Key, you can insert a new KV directly to keep the Key identical, in this way, the value corresponding to the key in the system will be updated; or you can delete the old KV first, and then insert the new KV, which is more euphemistic to complete the KV update operation.

Assuming that the application submits a Key Value, let's see how LevelDb reads the corresponding Value from the stored data. Figure 7-1 shows the entire LevelDb read process.

 

 

Figure 7-1 LevelDb read record Process

LevelDb first checks the Memtable in memory. If Memtable contains the key and its corresponding value, the value is returned. If the key is not read in Memtable, next, read the Immutable Memtable, which is also in the memory. Similarly, if read is returned, if it is not read, it can only be searched from a large number of SSTable files on the disk. Because SSTable has a large number and is divided into multiple levels, reading data in SSTable is a rather winding journey. The General read principle is as follows: first, search for files belonging to level 0. If yes, the corresponding value is returned. If no value is found, search for the files in level 1, this repeats until the value corresponding to this key is found in the SSTable file of a layer (or the highest level is found. If the search fails, the Key does not exist in the system ).

So why is the query path from Memtable to Immutable Memtable and Immutable Memtable to the file? Why? The reason for selecting such a query path is that Memtable stores the most fresh KV pairs in terms of Information Update time, And the freshness of the KV data pairs stored in Immutable Memtable follows the times; the KV data in all SSTable files is not as fresh as Memtable and Immutable Memtable in memory. For the SSTable file, if the same key is found in both level L and Level L + 1, the level L information must be newer than level L + 1. That is to say, the search paths listed above are arranged according to the degree of freshness of the data. The fresher the, the first, the first.

Why should we prioritize the search for fresh data? The truth is self-evident. For example. For example, we first insert a data {key = "www.samecity.com" value = "value"} into levelDb. After a few days, the samecity website was renamed as: 69 in the same city, in this case, we insert the data {key = "www.samecity.com" value = "69 City"}, the same key, different values. Logically, it seems that levelDb has only one storage record, that is, the second record, but there may be two records in levelDb, that is, the above two records are stored in levelDb. If the user queries key = "www.samecity.com ", of course, we want to find the latest update record, that is, the second record, which is why we need to first find fresh data.

As mentioned above: For SSTable files, if the same key is found in both level L and Level L + 1, the level L information must be newer than level L + 1. This is a conclusion. Theoretically, a proof process is required. Otherwise, the following problems may occur: What about Shenma? In principle, it is clear that Level L + 1 data is not from the cracks of the stone, nor from the dream. Where did it come from? Level L + 1 data is obtained from Level L after Compaction (if you do not know what Compaction is, then ........ that is to say, the current Level L + layer 1 SSTable data you see comes from the original Level L, the current Level L is fresher than the original Level L data, so it can be proved that the current Level L is fresher than the current Level L + 1 data.

There are many SSTable files. How can I quickly find the value corresponding to the key? In LevelDb, level 0 has always been specialized. The process of searching for a key in level 0 is different from that in other levels. Because different files under level 0 may have overlapping key ranges, a key to be queried may contain multiple files, in this case, LevelDb's policy is to first find out which files in level 0 contain this key (the manifest file records the level and the corresponding file and the key range information in the file, levelDb retains this ing table in the memory), which is sorted by the freshness of the files. The new files are listed first, and then searched in sequence to read the value corresponding to the key. If the value is not level 0, the key of the level file does not overlap, so the value corresponding to the key can be found only from one file.

In the last question, if a key to be queried and an SSTable file containing the key range are given, how does levelDb perform the specific search process? Generally, levelDb first searches for the Cache records that contain this file in the memory. If it contains, it reads the records from the Cache. If it does not contain the records, it opens the SSTable file, at the same time, load the index part of this file into the memory and put it into the Cache. In this way, the Cache contains the SSTable Cache item, but only the index part is in the memory. Then levelDb can locate the content Block that will contain this key based on the index, read the contents of this Block from the file and compare them one by one based on records. If the Block is found, the result is returned. If the Block is not found, it indicates that the SSTable file of this level does not contain this key, so go to the next level of SSTable to search.

From the write operations of LevelDb and the read Operations described here, we can see that the read operations are much more complicated than the write operations, therefore, the write speed must be much higher than the speed of reading data. That is to say, LevelDb is suitable for applications with more write operations than read operations. If the application has a lot of read operations, the sequential Read efficiency will be relatively high, because most of the content will be found in the cache, as much as possible to avoid a large number of random read operations.

Record 8: Compaction

As mentioned above, for LevelDb, the write record operation is very simple. Even if only one Delete mark is written to the delete record, reading the record is complicated, it takes a lot of time to search for memory and hierarchical files based on their freshness. To speed up reading, levelDb adopts the compaction method to compress existing records. This method deletes KV data that is no longer valid and reduces the data size, reduce the number of files.

The compaction mechanism and process of levelDb are basically the same as those described in Bigtable. In Bigtable, three types of compaction are described: minor, major, and full. The so-called minor Compaction is to export data from memtable to the SSTable file; major compaction is to merge SSTable files of different levels, and full compaction is to merge all SSTable files.

LevelDb contains two types: minor and major.

We will give a detailed description of its mechanism.

Let's take a look at the minor Compaction process. The purpose of Minor compaction is to save the content to a disk file when the memtable size in the memory reaches a certain value. Figure 8.1 shows its mechanism.

 

 

Figure 8.1 minor compaction

As can be seen from 8.1, when the number of memtables reaches a certain level, it will be converted to immutable memtable. At this time, records cannot be written to it, but KV content can only be read from it. Previously, immutable memtable is actually a multi-level queue SkipList, where records are sorted according to keys. Therefore, this minor compaction is easy to implement, that is, to traverse the records in immutable memtable from small to large, and write them into a new SSTable file at level 0 in sequence, after writing the data, create the index data of the file. This completes a minor compaction. It can also be seen that the deleted record does not really delete this record in the minor compaction process, and the reason is also very simple. Here we only know where to delete the key record, but where is the KV data? Complex search is required. Therefore, this key is not deleted during minor compaction, but is written into the file as a record. As for the true delete operation, it will be done in a higher level of compaction in the future.

When the number of SSTable files under a certain level exceeds a set value, levelDb selects a file (level> 0) from the SSTable at this level ), merge it with the SSTable file of level + 1 of level 1, which is major compaction.

We know that keys in each SSTable file are stored in an ascending order at a level greater than 0, in addition, the key ranges of different files (the minimum and maximum keys in the file) do not overlap. The SSTable file of Level 0 is somewhat special. Although each file is arranged in ascending order of keys, the file of level 0 is directly generated through minor compaction, therefore, two sstable files under any two levels 0 may overlap in the key range. Therefore, when performing major compaction, you can select a file for a level greater than level 0, but for level 0, after specifying a file, in this level, it is very likely that the key range of other SSTable files overlaps with this file. In this case, you need to find all the files that overlap with level 1 for merging, that is, when level 0 selects a file, multiple files may be involved in major compaction.

After selecting A level for compaction, levelDb also needs to select the specific file for compaction. Here, levelDb has A small trick, that is, taking turns. For example, this is the compaction of File, next time, compaction is performed on file B of file A in key range, so that each file will have the opportunity to combine with high-level level files in turn.

If the files of level l a and level L + 1 are merged, the problem arises again. Which files of level L + 1 should be merged? LevelDb Selects all files with overlapping key range in the L + 1 layer and file A to merge with file.

That is to say, if file A of level L is selected, all files B, C, D to be merged are found in level L + 1 ..... And so on. The remaining question is how to merge major? That is to say, given a series of files, each file contains keys in order. How can we combine these files to make the newly generated files still have keys in order, and at the same time, discard KV data that is no longer valuable.

Figure 8.2 illustrates this process.

 

 

Figure 8.2 SSTable Compaction

The Major compaction process is as follows: multiple files are sorted by means of multi-channel merge, and the smallest Key record is found in sequence, that is, all records in multiple files are re-sorted. Then, take a certain standard to determine whether the Key still needs to be saved. if it determines that there is no saved value, it will be discarded directly. If it feels that it still needs to be saved, write it to an SSTable file generated in level L + layer 1. In this way, KV data is processed one by one to form a series of new L + 1 layer data files, the previous L-layer files and L + 1-layer file data involved in compaction have no meaning at this time, so they are all deleted. In this way, the merging process of L-layer and L + 1-layer file records is completed.

In the major compaction process, what are the criteria for determining whether a KV record is discarded? One criterion is that if a key is smaller than the Key in the L layer, the KV can be discarded during the major compaction process. As we have analyzed earlier, if a record with the same Key exists in a file with a lower level than L, it indicates that there are newer values for the Key, the Value in the past is meaningless, so it can be deleted.

Cache in LevelDb

As mentioned above, for levelDb, if the read operation does not find a record in the memory memtable, it requires multiple disk access operations. Assuming that the key is found in the latest file in level 0 for the first time, you also need to read the disk twice, and read the index part of the SSTable file into the memory at one time, in this way, the index can be used to determine the block in which the key is stored; the second is to read the content of the block, and then find the value corresponding to the key in the memory.

Two different caches are introduced in levelDb: Table Cache and Block Cache. Block Cache is optional for configuration, that is, whether to enable this function in the configuration file.

 

 

Figure 9.1 table cache

Figure 9.1 shows the table cache structure. In the Cache, the key Value is the name of the SSTable file, and the Value part contains two parts. One is the file pointer to the SSTable file opened on the disk, which is used to facilitate reading the content; another is to point to the Table structure pointer corresponding to the SSTable file in the memory. The table structure is in the memory and stores the index content of the SSTable and the cache_id used to indicate the block cache, of course, there are also some other content.

For example, in the get (key) read operation, if levelDb determines that the key is within the key range of A file A under A certain level, it is necessary to determine whether file A actually contains the KV. At this point, levelDb first looks for the Table Cache to see if the file is in the Cache. If yes, you can find the block containing the key based on the index. If the file is not found in the Cache, open the SSTable file, read its index part into the memory, insert it into the Cache, and locate the block containing the Key in the index. If you determine which block of the file contains this key, you need to read the block content, which is the second read.

 

 

Figure 9.2 block cache

Block Cache is used to speed up this process. Figure 9.2 shows its structure. The key is the cache_id of the file, and the starting position of the block in the file is block_offset. The value is the content of the Block.

If levelDb finds this block in the block cache, it can avoid reading data and simply find the key value in the block content in the cache. What if it does not find it? Read the block content and insert it into the block cache. LevelDb uses two caches to speed up reading. It can be seen from this that if the data to be read is locally better, that is to say, most of the data to be read can be read in the cache, the reading efficiency should be very high, in addition, the efficiency of sequential reading of keys should be good, because it can be reused multiple times after one read. However, for random reading, you can infer the efficiency.

Ten versions, VersionEdit, and VersionSet of LevelDb daily records

Version stores information about the current disk and all files in the memory. Generally, only one Version is called "current" version (the current Version ). Leveldb also saves a series of historical versions. What are the functions of these historical versions?

After an Iterator is created, the Iterator references the current version. As long as the Iterator is not deleted, the version referenced by the Iterator will survive. This means that when you use an Iterator, You need to delete it in time.

After a Compaction is completed (a new file is generated and the file before merging needs to be deleted), Leveldb creates a new version as the current version, the current version will change to the previous version.

VersionSet is a collection of all versions and manages all surviving versions.

VersionEdit indicates changes between versions, which is equivalent to delta increments, indicating how many files are added and deleted. The relationship between them.

Version0 + VersionEdit --> Version1

VersionEdit will be saved to the MANIFEST file, and the data will be read from the MANIFEST file as the data recovery.

The version control of leveldb reminds me of dual buffer switching. Dual buffer switching comes from graphics. It is used to solve the problem of flashing screen during screen painting and is also useful in Server programming.

For example, there is a dictionary database on our server. Every day we need to update this dictionary database. We can open a new buffer, load the new dictionary library into this new buffer, and wait until the loading is complete, point the dictionary pointer to the new dictionary library.

The version management of leveldb is similar to that of dual buffer switching. However, if the original version is referenced by an iterator, the version will remain unchanged until it is not referenced by any iterator, you can delete this version.

Note: original address: http://www.samecity.com/blog/Index.asp? SortID = 12

Reference: 1. Wikipedia: http://zh.wikipedia.org/wiki/LevelDB

2. google code: http://code.google.com/p/leveldb/