LEVELDB source Analysis of the third: Arena

I. Principle

Arena is a memory pool implemented internally by LEVELDB. We know that the use of memory is important for a high-performance server-side program. C + + provides new/delete to manage memory application and release, but for small objects, the direct use of new/delete costs relatively large, to pay extra space and time, price is not high. In addition, we also want to avoid multiple applications and releases caused by memory fragmentation. Once the fragments reach a certain level, even if the total amount of memory is sufficient, the illusion of insufficient memory is caused by lack of enough contiguous free space.
C + + STL in order to avoid memory fragmentation, implement a complex memory pool, LEVELDB is not so complex, but implemented a "one-time" memory pool arena. In Leveldb, not all places use this memory pool, mainly memtable use, mainly for temporary storage of user update data, because the updated data may be very small, so the use of memory pool here is appropriate.

In order to avoid the frequent allocation of small objects, you need to reduce the call to new, the simplest way is to request large chunks of memory, multiple points to the customer. Leveldb with a Vector<char *> to save all memory allocation records, the default per request 4k of memory block, record the current memory block remaining pointer and the number of memory bytes remaining, whenever there is a new request, if the current remaining bytes can meet the needs, then directly returned to the user. If not, for requests exceeding 1k, the new block of memory is directly new and returned, less than 1K of the request, then an additional 4k memory block is requested, from which a portion is allocated to the user. When the memory pool object is refactored, the allocated memory is freed, ensuring that the memory is not compromised.

two. header Files

Class Arena {Public:arena ();

  ~arena ();
  Return A pointer to a newly allocated memory block of "bytes" bytes.

  Allocates a memory block of bytes size, returning a pointer to that memory block char* Allocate (size_t bytes); Allocate memory with the normal alignment guarantees provided by malloc//malloc-based byte-aligned RAM allocation char* allocatealigned (

  size_t bytes); Returns an estimate of the total memory usage of the data allocated//by the arena (including space allocated and not ye
  T used for user//allocations). Returns the total size (imprecise) of memory used by the entire memory pool, where only the total size of the allocated memory blocks and/or the space used to store the individual memory block pointers is calculated.
  The size of data members such as ALLOC_PTR_ and alloc_bytes_remaining_//are not calculated.
  size_t memoryusage () const {return blocks_memory_ + blocks_.capacity () * sizeof (char*);
  } private:char* Allocatefallback (size_t bytes);

  char* Allocatenewblock (size_t block_bytes);
  Allocation state//Current memory block (block) offset pointer, which is the first address of unused memory char* Alloc_ptr_;

  Represents the unused space size in the current memory block (block) size_t Alloc_bytes_remaining_; Array of new[] allocated memory blocks//used to store every request to the systemA pointer to the allocated memory block std::vector<char*> Blocks_;

  Bytes of memory in blocks allocated so far//the total size of RAM allocated to date size_t Blocks_memory_;
  No copying allowed Arena (const arena&);
void operator= (const arena&); };

three. source Files

Inline char* arena::allocate (size_t bytes) {
  //The semantics of what to return is a bit messy if we allow
  //0-b Yte allocations, so we disallow them here (we don ' t need
  //them for our internal use).
  If you allow 0 bytes of memory to be allocated, then the return value is semantically more difficult to understand, so it is forbidden to bytes=0
  //If the required memory is less than the memory remaining in the current memory block, then get the
  assert (bytes > 0) directly from the current memory quickly;
  if (bytes <= alloc_bytes_remaining_) {
    char* result = alloc_ptr_;
    Alloc_ptr_ + = bytes;
    Alloc_bytes_remaining_-= bytes;
    return result;
  }
  Because Alloc_bytes_remaining_ is initially 0, the first call to allocate is actually directly called Allocatefallback
  //If the required memory is greater than the memory remaining in the memory block, Also calls Allocatefallback
  return Allocatefallback (bytes);
}

char* arena::allocatefallback (size_t bytes) {//If the required memory is greater than the memory remaining in the memory block and is greater than 1K, the memory is allocated a piece of bytes size separately. This avoids wasting too much space (because if bytes is more than 1K from the 4K memory block to fetch, then if the current memory block is just the remaining//1K, you can only create a new 4K memory block, and take the bytes. The newly created memory block is the current memory block, and subsequent operations are based on the current inside//block, then the original memory block of 1K space is wasted) if (bytes > KBLOCKSIZE/4) {//Object is more than a quar  ter of our block size.
    Allocate it separately//to avoid wasting too much space in leftover bytes.
    char* result = Allocatenewblock (bytes);
  return result; }//If the required memory is greater than the memory remaining in the block, and is less than 1K, reallocate a memory block, the default size of 4 K,//The memory remaining in the memory block is wasted (this can be wasted, but the wasted space is less than 1K).
  And in the new memory block//To take the bytes size of memory.
  We waste The remaining space in the current block.
  Alloc_ptr_ = Allocatenewblock (kblocksize);

  Alloc_bytes_remaining_ = kblocksize;
  char* result = Alloc_ptr_;
  Alloc_ptr_ + = bytes;
  Alloc_bytes_remaining_-= bytes;
return result; }

Provides byte-aligned memory allocations, typically 4-byte or 8-byte alignment allocations, and//The benefits of aligning memory simply means accelerating memory access.  First get a pointer to the size of a const int align = sizeof (void*),//Obviously, under 32-bit systems is 4, 64-bit system is 8, for ease of expression, we assume that is 32-bit system, that is align = 4,//And then will we use char *  The pointer address is translated into an unsigned integer (reinterpret_cast<uintptr_t> (Result)://It's an unsigned int, that's guaranteed to being the same size As a pointer.), through with the operation to//get size_t Current_mod = reinterpret_cast<uintptr_t> (alloc_ptr_) & (align-1); current pointer mode 4// Value, with this value, it is easy to know that the slop = Align-current_mod multiple bytes, memory is aligned,//so there is result = Alloc_ptr + slop.
Then get the bytes size of the memory, which actually requires a size of needed = bytes + slop.    char* arena::allocatealigned (size_t bytes) {const int align = sizeof (void*);   We ' ll align to pointer size assert ((Align & (align-1)) = = 0); Pointer size should be a power of 2 size_t current_mod = reinterpret_cast<uintptr_t> (alloc_ptr_) & (align-1
  );
  size_t slop = (Current_mod = = 0? 0:align-current_mod);
  size_t needed = bytes + slop;
  char* result; if (needed <= alloc_bytes_remaining_) {
    result = Alloc_ptr_ + slop;
    Alloc_ptr_ + = needed;
  Alloc_bytes_remaining_-= needed;
  } else {//Allocatefallback always returned aligned memory result = Allocatefallback (bytes);
  } Assert ((Reinterpret_cast<uintptr_t> (Result) & (align-1)) = = 0);
return result; }

For memory alignment refer to: Memory alignment what's going on.

Allocate a new memory block
char* Arena::allocatenewblock (size_t block_bytes) {
  char* result = new Char[block_bytes];
  Blocks_memory_ + = block_bytes;
  Blocks_.push_back (result);
  return result;
}

Frees the entire memory pool occupied by memory
Arena::~arena () {for
  (size_t i = 0; i < blocks_.size (); i++) {
    delete[] blocks_[i];
  }
}

Reference Link: http://www.cnblogs.com/shenzhaohai1989/p/3904808.html