LevelDB sstable File

"LevelDB sstable file "

Leveldb There are many sstable files at different levels (the prefix. SST is characteristic), all the. sst files have the same internal layout. The previous section describes the log file is a physical block, sstable will also divide the file into a fixed-size physical storage block , but the logical layout of the two is very different, the root cause is: Log file is the key unordered, that is, the key size of the record has no definite size relationship, The interior of the. sst file is arranged from small to large according to the key of the record, from the sstable layout described below can be realized that key order is why so design. SST file Structure key.

　　

Figure 4.1 shows the physical partitioning structure of a. sst file, like a log file, divided into fixed-size storage blocks, each block is divided into three parts, and the red part is the data storage area. The blue type area is used to identify whether the data store uses a data compression algorithm (snappy or uncompressed two), the CRC part is the data check code, to determine whether the data in the generation and transmission error.

The above is the physical layout of the. sst file, which describes the logical layout of the. sst files, the so-called logical layout, that is, although everyone is a physical block, but what the content of each piece of storage, the internal structure and so on. Figure 4.2 shows the internal logical interpretation of the. sst file.

　　

As can be seen from Figure 4.2, from a large aspect, the. sst file can be divided into data storage area and data management area, the data store holds the actual key:value data, and the data Zone provides some index pointers and other administrative data to find the corresponding records more quickly and conveniently. Two regions are based on the above-mentioned block, that is, the file in front of a number of blocks actually store the KV data, the data management area behind the storage administration data. The management data is divided into four different types: the purple meta block, the Red Metablock Index and the Blue Data Index block, and a file trailing block.

LevelDb version 1.2 for the meta block is not actually used, just reserved an interface, it is estimated to include in the subsequent version of the content, the following we look at the data index area and the file tail footer internal structure.

　　

Figure 4.3 is the internal structure of the data index. Again, the KV record in data block is sorted by key from small to large, each record in the data index area is the index information that is established for a certain data block, each index information contains three contents, and the index of the block I is shown in Figure 4.3. I: The first field in the Red Section records the key that is greater than or equal to the largest key value in Block I, the second field indicates the starting position of block I in the. sst file, and the third field indicates the size of the data block I (sometimes with data compression). The following two fields are good to understand, is used to locate the data block in the file position, the first field needs to explain in detail, the key value stored in the index is not necessarily a key of a record, in the example of Figure 4.3, assume the minimum key= "samecity" of the Block I, the maximum key= " The best "i+1", the smallest key= "The Fox", the largest key= "zoo", then for the index of the data block I, the first field record is greater than or equal to block I of the maximum key ("The best") is smaller than the data block i+ 1 of the minimum key ("The Fox"), so the first field of index I in the example is: "The C", this is to meet the requirements, and the first field of index i+1 is "Zoo", that is, the largest key of the block i+1.

The internal structure of the footer block at the end of the file is shown in Figure 4.4,metaindex_handle, which indicates the starting position and size of the Metaindex block; Inex_handle indicates the starting address and size of the index block. These two fields can be understood as indexed indexes, which are set up to correctly read the index values, followed by a fill area and magic number.

　　

The above is mainly about the internal structure of the data management area, let us look at the data section of a block of the internal layout of the data part (Figure 4.1 in the Red section), Figure 4.5 is its internal layout.

　　

LevelDB sstable file