Memory pool (mem-pool) of GlusterFS uses instance analysis

My Sina Weibo: http://weibo.com/freshairbrucewoo.

You are welcome to exchange ideas and improve your technology together.

 

The previous blog analyzed in detail the implementation technology of the memory pool of GlusterFS. Today we will look at how GlusterFS uses this technology.

Step 1: allocate and initialize:

The cli process will involve the establishment and initialization of the memory pool during initialization. The Code involved in the memory pool Initialization is as follows (the glusterfs_ctx_defaults_init function in the cli. c file ):

1/* frame_mem_pool size 112*64 */2 pool-> frame_mem_pool = mem_pool_new (call_frame_t, 32); // allocate a memory pool object for the call needle object. The object type is call_frame_t, 32 such memory blocks 3 if (! Pool-> frame_mem_pool) 4 return-1; 5 6/* stack_mem_pool size 256*128 */7 pool-> stack_mem_pool = mem_pool_new (call_stack_t, 16 ); // allocate a memory pool object for the call stack object. The object type is call_stack_t, and 16 such memory blocks are 8 9 if (! Pool-> stack_mem_pool) 10 return-1; 11 12 ctx-> stub_mem_pool = mem_pool_new (call_stub_t, 16); 13 if (! Ctx-> stub_mem_pool) 14 return-1; 15 16 ctx-> dict_pool = mem_pool_new (dict_t, 32); 17 if (! Ctx-> dict_pool) 18 return-1; 19 20 ctx-> dict_pair_pool = mem_pool_new (data_pair_t, 512); 21 if (! Ctx-> dict_pair_pool) 22 return-1; 23 24 ctx-> dict_data_pool = mem_pool_new (data_t, 512); 25 if (! Ctx-> dict_data_pool) 26 return-1;

 

The code above shows that the number of struct objects in the collection system may be used in advance, when the object memory is really needed, it can be directly obtained from these memory pools. After the memory is used up, it is put back into the memory pool, which reduces the additional system overhead for allocating and releasing the memory, memory Allocation usually requires switching from the user State to the kernel state, which is time-consuming. Of course, the same object also reduces the initialization time.

 

The mem_pool_new function is called for memory allocation in the code, instead of the mem_pool_new_fn function ended in the previous blog. It is because mem_pool_new is a defined macro function, that is, mem_pool_new_fn function is called, function parameters indicate the memory size, quantity, and name of the object, respectively (name the allocated memory, that is, the object name );

 

1 #define mem_pool_new(type,count) mem_pool_new_fn (sizeof(type), count, #type)  

 

Step 2: retrieve an object memory block from the memory pool:

 

1 call_stub_t * new = NULL; 2 3 GF_VALIDATE_OR_GOTO ("call-stub", frame, out); 4 5 new = mem_get0 (frame-> this-> ctx-> stub_mem_pool ); // extract an object memory block from the memory pool

 

For example, the following code retrieves the object memory block (call_stub_t) of the call stub ):

 

 

The same use of the function is not our introduction of mem_get, but mem_get0 function, the mem-get0 encapsulates mem_get, the parameter judgment and the need to use the memory initialization is 0, the Code is as follows:

 

1 void * 2 mem_get0 (struct mem_pool * mem_pool) 3 {4 void * ptr = NULL; 5 6 if (! Mem_pool) {7 gf_log_callingfn ("mem-pool", GF_LOG_ERROR, "invalid argument"); 8 return NULL; 9} 10 11 ptr = mem_get (mem_pool ); // obtain a memory object Block 12 13 if (ptr) 14 memset (ptr, 0, mem_pool-> real_sizeof_type); // initialize 0 15 16 return ptr; 17}

 

 

Step 3: return the object memory block to the memory pool:

After an object is used, it will be re-stored in the memory pool, for example, calling the stub object (call_stub_t)

 

1 void 2 call_stub_destroy (call_stub_t * stub) 3 {4 GF_VALIDATE_OR_GOTO ("call-stub", stub, out); 5 6 if (stub-> wind) {7 call_stub_destroy_wind (stub); 8} else {9 call_stub_destroy_unwind (stub); 10} 11 12 stub-> stub_mem_pool = NULL; 13 mem_put (stub ); // put back the object memory block to the memory pool 14 out: 15 return; 16}

 

Step 4: destroy the memory pool:

If the entire memory pool object is no longer needed, destroy the memory pool. The function to implement this function is mem_pool_destroy:

 

1 void 2 mem_pool_destroy (struct mem_pool * pool) 3 {4 if (! Pool) 5 return; 6 7 gf_log (THIS-> name, GF_LOG_INFO, "size = % lu max = % d total = %" PRIu64, 8 pool-> padded_sizeof_type, pool-> max_alloc, pool-> alloc_count); 9 10 list_del (& pool-> global_list ); // from the global memory pool object, towline 11 12 LOCK_DESTROY (& pool-> lock); // destroy the lock 13 GF_FREE (pool-> name ); // release the memory 14 GF_FREE (pool-> pool) occupied by the name; // release the memory allocated by the memory pool, that is, the memory 15 GF_FREE (pool) provided to the user ); // release the memory occupied by the memory pool object 16 17 return; 18}

 

 

In general, the memory pool objects will be released and destroyed only when the program exits. In another case, the temporarily allocated memory pool may also be released and destroyed during system running, because it is not guaranteed that a pre-allocated memory pool can meet the memory required by the object during the entire system operation, this object may be used a lot at each stage, so that the memory pool pre-allocated object memory block is used up, then you need to temporarily allocate the memory pool object, after which the number of this object may be reduced, in this case, you need to release the temporary allocation and return it to the system memory.

 

OK! The memory pool management technology is an important means to provide memory usage and efficiency. The memory pool technology used by Glusterfs uses the slab Algorithm for Linux kernel to manage small memory blocks, it is the technology of allocating Memory Based on objects. You can familiarize yourself with the slab principles to better understand the memory pool technology of Glusterfs!