--NDPI Source code Analysis of Protocol and maintenance

In the previous article, we did a source code analysis of the example in ndpi. This time we will learn as much as possible about the structure and operation of the ndpi interior. We will take the following three goals (questions) to read the source code of NDPI.

1, ndpi internal is how to register and maintain the need to detect the agreement?

2, ndpi in the process of initialization. What kind of work did you do?

3, ndpi in the bottom of the implementation of detail is how to use the data structure?

Note: This is limited to space. This article is a pointer to the initialization section in use for source code analysis. The analysis function of the main body and the detailed protocols will be introduced in the following article. If there is something wrong or the understanding is not in place, you are welcome to discuss together.

First, the index

In the example (Pcapreader) described above, we have given us a very useful navigation.

is the Setupdetection function, which basically includes the regular use functions we used to initialize ndpi.

In the next source code analysis, we use this function as an index. Gradually spied on the interior of the ndpi. Details such as the following:

in the function above. The main is the previous 11 row operation. The operation behind the 11 line is mainly for the application of the Pcapreader itself (in fact we have already marked the simple gaze above). Next we will be divided into two stages to introduce.

Note: Two or three the title is chosen only because of the specific gravity relationship. Macros in the above source code and so on we will be described below

Second, Ndpi_init_detection_module

In this section we will introduce the initialization function Ndpi_init_detection_module in line 8th, the definition in line 3rd above, and so on in the third part of the system. The implementation of Ndpi_init_detection_module is in the Ndpi_main.c file. This function does not work as a maintenance protocol. It is the initialization of a lower number of parameters. The ndpi_detection_module_struct type pointer is returned when the function ends. This type will run through the initialization process. Provides the container for holding the parameters and the data set. We will introduce each of the following:

Function Prototypes:

struct Ndpi_detection_module_struct *ndpi_init_detection_module (u_int32_t Ticks_per_second,
void* (*__ndpi_malloc) (unsigned long size),
void (*__ndpi_free) (void *ptr),
Ndpi_debug_function_ptr ndpi_debug_printf)

1. Initialization of memory management functions

_ndpi_malloc = __ndpi_malloc;
_ndpi_free = __ndpi_free;

The __ndpi_malloc and __ndpi_free Here are our custom functions. The _ndpi_malloc and _ndpi_free on the left side of the equals sign do not refer to variables. Here is a function pointer that is encapsulated inside the NDI.

defined in NDPI_MAIN.C, details such as the following:

2, initialization of the debug function

Here are two places to use the ndpi_debug_printf function, the first is the error handling when applying for memory in 3 steps downstairs. The second place is the incoming in the debug function. Similar to the 1th step. However, the debug information output function here is not declared in Ndpi_main.c, but inside the ndpi_detection_module_struct structure. The NDPI_DEBUG_PRINTF function pointer is declared inside the structure. See the source code, no longer listed here.

3. Creation and initialization ofndpi_detection_module_struct pointers

This is actually the declaration of the ndpi_detection_module_struct pointer and the memory request through malloc.

The ndpi_detection_module_struct here will return at the end of the function.

4. Initialization of protocol mapping diagram

Ndpi_bitmask_reset (Ndpi_str->detection_bitmask);

This part is mainly through the macro Ndpi_bitmask_reset and the structure (ndpi_detection_module_struct) internal detection_bitmask To achieve together. Detection_bitmask is actually an array of u_int32_t, andNdpi_bitmask_reset is one of the macros defined in the NDPI_MACROS.H header file to maintain the registration protocol. The underlying fact is that the detection_bitmask array is 0 implemented by Menset.

This is just like our big question. Is the core content of the article. will be part of the third. The system is described in detail.

5. Redis Initialization

Redis is a key-value storage system. Similar to memcached, it supports storage of value types relative to many others, including string (string), list (linked list), set (set), Zset (sorted set-ordered set), and hash (hash type). a simple initialization of the Redis variables in the function ( ndpi_detection_module_struct) through the struct ( Ndpi_str->redis = NULL;). There is no specific introduction here. This section is defined in Ndpi_credis.h and NDPI_CREDIS.C.

6, initialization of the Protocol timeout value

In this function, a larger space is used to initialize the timeout. There's really nothing to explain here, the timeout value for each application-level software.

7, the initialization of the AC algorithm

AC algorithm refers to Aho-corasick's own active machine algorithm. Included in the Ndpi_main.h header file #include<ahocorasick.h>. In fact, the Ahocorasick is encapsulated and implemented in the ndpi in Ahocorasick. C.

Return to here, mainly through the Ac_automata_init function to initialize. The initialization here also does not involve the algorithm itself, simply creating and initializing the struct ac_autimata_t and passing it back to the struct (ndpi_detection_module_struct).

8. The initialization of LRU memory management

Ndpi_init_lru_cache (&ndpi_str->skypecache, 4096);

Here is also a deep pit, memory management. But the functionality that Ndpi_init_lru_cache implements is not very complex either. We can go and see. Implemented in NDPI_CACHE.C.

9, the initialization of multi-threaded

Here is the initialization of multithreading. The Pthread_mutex_init () function creates a mutually exclusive lock in a dynamic manner. Parameter attr Specifies the properties of the new mutually exclusive lock.

Assuming that the parameter attr is empty, the default mutually exclusive lock property is used. The default properties are high-speed mutually exclusive locks. The properties of the mutually exclusive lock are specified when the lock is created, there is only one lock type attribute in the Linuxthreads implementation, and the different lock types behave differently when attempting to lock a locked mutually exclusive lock.

the Pthread_mutexattr_init () function returns zero after it completes successfully. Any other return value indicates that an error has occurred. after the function runs successfully. Mutually exclusive locks are initialized to an unlocked state.

10. Initialization of the default port

Ndpi_init_protocol_defaults (NDPI_STR);

Here ndpi_str is the struct (ndpi_detection_module_struct)that we initialized after the first few steps. The main function of the ndpi_init_protocol_defaults function is to maintain a binary tree structure that records the default port for each protocol.

The repetition in this function is relatively high, we stick to a relatively representative program section to explain together:

There are two functions involved. is Ndpi_build_default_ports and ndpi_set_proto_defaults. Ndpi_build_default_ports operation is relatively simpleWithout a specific introduction, he simply passes the port number in the number of ports_a/ports_b and returns. Here is the main introductionNdpi_set_proto_defaultsThe implementation process of the function.

NOTE: The functions described above (10) are implemented in Ndpi_main.h

Third, Ndpi_set_protocol_detection_bitmask2

Before we talk about this function, let's start by describing the origins of the parameters used by this function.

That is, the preceding macro definitions and variables.

1,ndpi_protocol_bitmask All

in fact, the first time you see this statement, the first reaction is also a macro definition. But in fact here ndpi_protocol_bitmask represents a variable type, and all is an instance (variable) that defines the processing. The specific definitions are in ndpi_macros.h, such as the following:

After reading the above definition, it is not difficult to findNdpi_protocol_bitmaskThe type is plainly a u_int32_t array.

We continue to look at the size of the array ndpi_num_fds_bits and how it is calculated:

A series of questions are raised here? Why should(((x) + ((y)-1))/(y) )？ And what is the purpose of this definition?

According to my understanding. The data structure for maintaining the protocol mappings here is the ndpi_protocol_bitmask_struct(u_int32_t array ) mentioned above. For each position of the array, for example fds_bits[1], this u_int32_t a co-owned 4 bytes.

Also ratione materiae 32 bits. Each person represents the mapping for this one protocol. This can be seen not only from the definition above. In the next 2nd part it will be more obvious to see that this is a hash-like mapping structure. Then go back to the question of why ((( x) + ((y)-1))/(y) ), where y is actually 32. So the array is computed in order to arrive at an array size that exactly satisfies the protocol mapping (the number of bits in the array is not all applications and mappings.) After all, there will be a bit of wasted space)

2. Ndpi_bitmask_set_all (all)

The main function of this macro is very obvious, which is to set the application in the map. But this is only a superficial understanding.

In Ndpi_macros.h, NDPI provides a very robust mapping setup function.

For example, the following:

Note: What should we do when we use the hypothesis that we only want to register the tests of individual applications? To passNdpi_bitmask_addMacro to join. The corresponding hash number of the protocol is defined in ndpi_protocols_osdpi.h.

The space is no longer listed here.

3, Ndpi_set_protocol_detection_bitmask2 (ndpi_struct,&all)

This can be said to be the core function of the test protocol. And the 2nd part of the 101, where the repetition rate is also relatively high. But there is a dependency between some agreements. So let's take one of the following typical code snippets. Let's see what kind of work it's finished.

So far, the initialization analysis is almost complete.

Reprint please: http://blog.csdn.net/grublinux/article/details/37670619

In addition, the sub-sister of this blog post pcapreader--source code analysis is very honored to participate in the blog contest, if you think it is helpful to you. Please support me. Voting address:http://vote. Blog.csdn.net/article/details?articleid=31603915

--NDPI Source code Analysis of Protocol and maintenance