------- Tor source code analysis part 3-Log facility and smart linked list, ------- tor

------- Tor source code analysis part 3-Log facility and smart linked list, ------- tor

Bytes ------------------------------------------------------------------------------------

Shows the internal logic of init_logging () (\ tor-0.3.1.8 \ src \ common \ log. c:

 

Its task is to initialize the global Log facility used by the tor. It first detects and initializes the mutex lock (log_mutex) used to protect log information and log files. It is a tor_mutex_t object.

-- Review the previous discussion. The specific method is as follows:

Determine the value of the global variable log_mutex_initialized in the same source file -- 0 indicates that the log mutex lock has not been initialized, so it calls tor_mutex_init ()

Obviously, what is actually responsible for initializing log_mutex is the pthread library routine pthread_mutex_init (), and sets the status mark log_mutex_initialized to 1.

Global variables involved include:

 

 

Next, it checks a global smartlist_t Pointer -- pending_cb_messages (CallBack-based Log message queue) -- if the pointer is NULL

Call smartlist_new () (\ tor-0.3.1.8 \ src \ common \ container. c) to initialize it.

Then check the passed real parameter -- if it is 1, it indicates that the start message queue is disabled, and it changes the global variable queue_startup_messages to 0, meaning that it is not guaranteed during the process of waiting for the configuration log

Store the message. In fact, 0 is input for this real parameter in tor_main (), indicating that the message should be recorded in the early stage, for example:

 

In this case, it calls smartlist_new () again to allocate and construct a smartlist_t structure pointer and assign it to the global variable pending_startup_messages

 

Process the log message queue at startup time. Pending_startup_messages indicates various startup messages (to be output) that will be played back after the log system initialization is complete.

Global variables involved include:

 

If you are dizzy, let's sort out the global variables and usage discussed so far:

Log_mutex -- protects the mutex lock between log files and messages;

Log_mutex_initialized -- whether log_mutex is initialized;

Pending_cb_messages -- A smartlist_t pointer (smartlist_t *) that stores CallBack-based Log message queues and can be initialized using smartlist_new;

Pending_startup_messages -- A smartlist_t pointer that stores the log message queue at startup time and can be initialized using smartlist_new;

Queue_startup_messages -- whether to record log messages at startup time;

Disable_startup_queue -- the parameter of init_logging (). The caller can use it to enable or disable the Message Queue function at the start time;

Shows the internal logic of init_logging:

 

Tor implements a smart linked list-smartlist, which can be adjusted by capacity and store any data type. It consists of the smartlist_t struct.

 

(\ Tor-0.3.1.8 \ src \ common \ container. h. The "container" is a container. This module abstracts many data structures as containers,

Smartlist is only one of them.

Smartlist_t has a pointer array (list). Each element (void *) can point to any type of data and the number of elements can be adjusted. This is why it is called a smart linked list.

Such:

 

Note: the size of the list array can be adjusted. After adjustment, you need to update the capacity field (the current capacity limit );

The num_used field records the number of current elements. These elements (void *) point to valid data. You need to be aware that when num_used is less than or equal to capacity,

The list size remains unchanged. When num_used is greater than capacity, list will be dynamically expanded, and num_used and capacity will be updated at the same time.

Operations on smartlist_t are implemented by some well-written routines with the prefix of smartlist _. Other module functions do not need to know the internal details of the smart linked list,

You only need to call these routines to ensure safe and correct use of the smart linked list, which again reflects data abstraction and encapsulation, and interface exposure... And other advanced C programming skills!

As shown in the following figure, analyzing the internal logic of these smartlist routines helps you understand the design concept of the smart linked list:

Smartlist_new () allocates a piece of memory in the heap for the smartlist_t structure, and returns a pointer to the memory block, as shown in:

 

 

We can see that the actual allocation function is tor_malloc (), which encapsulates the system library function malloc () and implements the tor's own security allocation algorithm; the size of the allocated memory

That is, the size of the struct smartlist_t (which should be 12 bytes) is calculated during compilation. Before returning a smartlist_t pointer, it will set the num_used field to zero,

Indicates that no element has been added. Setting the capacity field to the constant value (16) defined by the SMARTLIST_DEFAULT_CAPACITY macro indicates that the initial capacity is limited to 16

Void pointer (total size: 16*4 = 64 bytes); next it will call tor_calloc () to allocate another 64-byte heap memory

Store the void pointer and let the list field hold the address of the memory block. Figure shows the heap memory layout after smartlist_new () is returned:

When I analyze other smartlist _ * () helper routines later, I will extend this graph to explain their functions.

Smartlist_free () destroys a smart linked list, but it does not actually release the memory occupied by the smartlist_t structure and void * array allocated in the heap, for example:

Smartlist_free () by callingTor_free ()-> raw_free ()-> free ()Sets the passed smartlist_t pointer to NULL and returns the result.

Related heap memory, as shown in:

 

 

Smartlist_add () adds new elements to the smart linked list (void * array.

It first calls smartlist_ensure_capacity () to check whether the current capacity limit is allowed. Otherwise, it will dynamically increase the capacity of the void * array and then add new elements.

Append to the end. As shown in:

 

Note: When smartlist_ensure_capacity () is called, it uses a smartlist_t pointer to increment the num_used field value (indicating the value of the appended

In smartlist_ensure_capacity (), check whether the number of appended elements exceeds the current capacity limit and take corresponding measures!

After the smartlist_ensure_capacity () process is completed, you can ensure that the target element accessed as the expression sl-> list [sl-> num_used ++] under the array is

It is initialized as a new void pointer within the permitted range. Syntax Parsing is shown in:

 

Assuming that the number of elements has reached the initial limit (num_used = capacity = 16), the related heap memory smart linked list layout is as follows:

The heap memory smart linked list layout after the call of smartlist_add () is as follows:

It can be seen that smartlist_ensure_capacity () will expand the current capacity limit of the void * array according to the power of two times, while smartlist_add () is in the extended part

And the rest of the extended part are NULL, which will be used later.

Now let's go deep into smartlist_ensure_capacity () to study its heap memory allocation algorithm, which is similar to a 2-power extension mechanism to some extent.

OS kernel memory allocation algorithm simplified version! As shown in:

Smartlist_ensure_capacity () logic points:

① It is an inline routine, which means that during the compilation phase, the compiler will directly insert it into the caller's function. In other words, when we disassemble

During smartlist_add (), commands like "call smartlist_ensure_capacity" are not displayed, because the logic of the latter has been hard-coded to the former,

This reduces the performance overhead of function calls, stack frame creation, and destruction when returned!

 

② Its second parameter type is size_t (that is, unsigned int), which is a type that safely represents the length and size (no negative value), as described earlier,

Smartlist_add () will pass in the value after increasing 1 for this parameter, while smartlist_ensure_capacity () will judge whether the updated value exceeds the void * Array

The current upper limit;

 

③ Some conditions at the beginning of the block computing OS/hardware platform support the maximum void * array upper limit-SIZE_MAX is defined in the platform-related limits. h header file,

As shown in, this header file is located in the include sub-path of the Visual Studio installation directory. Its value depends on the compiler. For example, here 0 xFFFFFFFF,

It is 4,294,967,295 in decimal format. Similarly, the INT_MAX value is calculated as 2,147,483,647;

The value of SIZEOF_VOID_P is 4 on a 32-bit platform.

 

After a series of pre-calculation, the maximum value of MAX_CAPACITY (indicating the void * array, in the unit of "number of elements") is obtained.

1,073,741,823 elements (SIZE_MAX/(sizeof (void *), that is, the void * array is up to 4 GB.

 

④ Tor_assert () is used to check the number of appended elements. When the maximum capacity is reached, the abort () function of the library is called to terminate the program;

Tor_assert () has been mentioned in the previous article.

 

⑤ If the number of elements appended is within the limit of the current capacity, smartlist_ensure_capacity () will return directly without doing anything. Its caller

You can perform subsequent operations with peace of mind. If the number of appended elements exceeds the current upper limit, the current upper limit is increased by a multiple of the power of 2 each time until it can accommodate

The number of appended elements, and the memory of the extended part is initialized to zero for later use, and the capacity field must be updated as the new upper limit.

When analyzing source code algorithms, verbal descriptions are often pale and powerless. Let's look at the internal logic of my smartlist_ensure_capacity,

Is it a bit of OS kernel memory distributor shadow?

 

Finally, let's analyze the tor_reallocarray () in smartlist_ensure_capacity (),

And the memset () function call, which is responsible for memory resetting, end with this article, as shown in:

 

From this we can infer that init_logging () and related components use smartlist to centrally manage log messages.

The next article will analyze the function call before the second condition compilation block-monotime_init ().

------------------------------------------------------