LEVELDB source analysis of the eight: memtable

To read this article, refer to:

LEVELDB Source Analysis One: coding

LEVELDB Source Analysis of the second: comparator

LEVELDB source Analysis of the third: Arena

LEVELDB source analysis of the four: Atomicpointer

LEVELDB source analysis of the Five: skiplist (1)

LEVELDB source analysis of the Six: Skiplist (2)

LEVELDB source analysis of the seven: Random

All KV data in Leveldb is stored in memtable,immutable memtable and sstable, immutable memtable is structurally and memtable exactly the same, except that it is read-only, Write operations are not allowed, while Memtable is allowed to write and read. When the data written by memtable reaches a specified amount of memory, it is automatically converted to immutable memtable, waits for dump to disk, the system automatically generates a new memtable for write operations to write new data, and understands memtable, then immutable         Memtable Nature is a cinch. LEVELDB's memtable provides an operational interface for writing KV data, deleting and reading KV records, but in fact memtable does not have a real delete operation, and the value of deleting a key is implemented as a record insertion in memtable, However, the deletion of the key will be hit, the actual deletion is deferred, will be removed in the subsequent compaction process of this kv. It is important to note that the KV pair in the Leveldb memtable is stored sequentially according to the key size, and when the system is plugged into a new kv, the LEVELDB is to plug the kv into the appropriate position to maintain the key ordering. In fact, Leveldb's memtable class is just an interface class, the real operation is done through the skiplist behind, including insert operations and read operations, so memtable core data structure is a skiplist.

Memtable main function is to Skiplist, arena, comparator combination and management, interface function shielding the bottom operation, the user more elegant.

I. Constructors

Memtable::memtable (const internalkeycomparator& CMP)
    : COMPARATOR_ (CMP),
      refs_ (0),
      table_ ( Comparator_, &arena_) {
}

The constructor initializes the private member variable, Table_ is the skiplist type, &aerna_ is passed as key, and Arena_ is the arena type.

two. Memory Estimation function

size_t Memtable::approximatememoryusage () {return Arena_. Memoryusage (); }

The direct call here is the Memoryusage method of the Arena class, which returns the total size (imprecise) of memory used by the entire memory pool.

three. Add a function

void Memtable::add (SequenceNumber s, ValueType type, const slice& key, const SLI ce& value) {//Format of an entry is concatenation of://Key_size:varint32 of Internal_key.size ()//K EY bytes:char[internal_key.size ()]//Value_size:varint32 of Value.size ()//value Bytes:char[value.size
  ()] size_t key_size = Key.size ();
  size_t val_size = Value.size (); Reference LEVELDB Source Analysis II: Comparator about internal key,//Because internal key consists of User_key, sequence and type three fields, User_key//That is the key here
  , sequence and type are packaged into a uint64_t type of data,//So the length here is key_size+8 size_t internal_key_size = key_size + 8; Reference Leveldb One of the source code analysis: coding, in order to save space, the numbers are encoded storage,//Varintlength method to find out the length of the encoding.
  The composition of Encoded_len is described in the following figure. Const size_t Encoded_len = Varintlength (internal_key_size) + internal_key_size + varintlength (val_size) + Val_
  Size Allocate memory char* BUF = Arena_.
  Allocate (Encoded_len); Encoded internal_key_size, encoded and stored in BUF, p points to internal_key_sizeEnd char* p = EncodeVarint32 (buf, internal_key_size);
  Copy key to BUF, occupy key_size size memcpy (p, Key.data (), key_size);
  p + = key_size;
  The sequence and type are packaged and stored in the BUF, with a size of 8 bytes, and the ENCODEFIXED64 is simply copied (considering the big or small end).
  ENCODEFIXED64 (P, (S << 8) | type);
  p + = 8;
  Encoding val_size, encoded and stored in BUF, p points to the end of val_size p = EncodeVarint32 (P, val_size);
  Copy value to buf, occupying val_size size memcpy (p, Value.data (), val_size);
  Determines whether the amount of memory after storage is equal to the size of the initial calculation assert ((P + val_size)-buf = = Encoded_len); Insert into skiplist in Table_.
Insert (BUF); }

A complete BUF content is shown in the following figure.

four. Get the function

If you can find the value corresponding to key, store the value in the *value parameter, and the return value is true.
If there is a delete identity in this key, a notfound () error is stored in the *status parameter, and the return value is true. Otherwise the return value is false bool Memtable::get (const lookupkey& key, std::string* value, status* s) {//Get Memkey,memkey actually contains KL
  Ength|userkey|tag, which means it contains internal_key_size//and Internal_key Slice Memkey = Key.memtable_key ();
  Table::iterator iter (&table_); Find the node ITER in skiplist greater than or equal to Memkey.
  Seek (Memkey.data ()); If you find this node if (ITER.    Valid ()) {//The structure of a node is as follows//entry format is:///Klength Varint32//UserKey Char[klength]// Tag UInt64//Vlength Varint32//value char[vlength]//Check that it belongs to same user key  . We don't check the//sequence number since the Seek () call above should has skipped//all entries with overly
    Large sequence numbers.
    Const char* Entry = Iter.key ();
	uint32_t key_length; Remove the klength, and point key_ptr to klength after//why add 5. Reference Leveldb One of the source code analysis: Coding const char* KEY_PTR = GeTvarint32ptr (Entry, entry+5, &key_length);
    Compares the UserKey in the UserKey and LookupKey in the node, and if it is equal, the node is found. if (Comparator_.comparator.user_comparator ()->compare (Slice (Key_ptr, key_length-8), Key.user  _key ()) = = 0) {//Get Tag,tag equals (sequence<<8) |type const uint64_t tag = DECODEFIXED64 (Key_ptr + key_length
	  -8); 
          Remove type and determine switch (static_cast<valuetype> (tag & 0xff)) {case Ktypevalue: {//Remove the size and contents of value
          Slice v = getlengthprefixedslice (key_ptr + key_length);
          Value->assign (V.data (), v.size ());
        return true;
          } case ktypedeletion: *s = Status::notfound (Slice ());
      return true;
}}} return false; }

}

The first parameter to get a function is the LookupKey type, and LookupKey is a helper class that makes it easier to manipulate memtable. Since LookupKey's official notes are particularly detailed, this is not an analysis.

Reference Link: http://www.360doc.com/content/14/0325/16/15064667_363619194.shtml