ndpi Work Flow

Here is an example of ndpi's routine ndpireader.c, which describes the process of ndpi from grabbing the packet to the final analysis of the specific protocol. In short, ndpi is the analysis of the packets from the lower layer to the level.
First, draw your own flowchart.
This picture is I at the beginning to see ndpi source of the time to do the flowchart, is not very clear and correct, even the function of the call relationship is just in order to draw, now looks really a little low, but also roughly explain some of the problems, it is too lazy to modify. I'll explain it in the next article. Here if you want to really understand their workflow, it is best to debug through the GdB tool, go to the example folder, run the Gdb-tui ndpireader command, Locate the. pcap file under the Test folder or grab a packet on your internet, then set a breakpoint in the GDB command line, run R-i *.pcap to enter debug mode ~ ~ can Baidu debugging related knowledge.

The first step is to initialize the program, call the Setupdetection () function, and do a lot of work here, and intend to write a new article to specifically describe this function.
Next, the thread calls the LIBPCAP library function to fetch the packets through the computer's network card, or to read the incoming. pcap files (how to run simple operations such as the official documentation, under the Doc folder)
Next to each packet (here we need to clear two concepts, packet (packet) and data flow (flow), a data flow may have a lot of packets, as we request a Web request, because the page information is very large, so it will be divided into a number of packets to transport, However, these packets belong to a data stream, first, the data link layer and the IP layer of the Packet Analysis Pcap_packet_callback () function, determine whether it is based on IP protocol, etc., and obtain its source destination IP, protocol type and so on.
Next Call the Packet_processing () function to perform the Transport layer analysis. The Get_ndpi_flow () function is called when the Transport Layer analysis is performed, and the function returns the structure of Ndpi_flow (it is important to note the difference between Ndpi_flow and NDPI_FLOW_STRUCT two structures). In the Get_ndpi_flow () function, get information about the transport layer, such as the source destination port. The IDX is then computed based on the five elements (source destination IP, source destination port, protocol type (TCP\UDP)).

IDX = (vlan_id + lower_ip + upper_ip + iph->protocol + lower_port + upper_port)% Num_roots;
  ret = Ndpi_tfind (&flow, &ndpi_thread_info[thread_id].ndpi_flows_root[idx], node_cmp);

This is where I first started the open source code when the place that bothered me, began to never know the role of IDX, and later found that the program maintains an array to record all the data flow, and IDX is used to identify the different data streams, based on the previous resolution of the packet of five tuples to calculate the IDX, and then query Ndpi_flows_ Root[] This array is indexed to the IDX location if there is already a record. Generally, for a data flow, the first packet query of the stream Ndpi_flows_root[idx] is empty, a new Ndpi_flow object is created and saved to that location, and so on when the subsequent packets of the data stream are caught, because they belong to the same stream (that is, the IDX is the same), So Ndpi_flows_root[idx] is not empty, then directly return the existing ndpi_flow can be. At this point, we get the ndpi_flow structure, which is also the meaning of the Get_ndpi_flow () function.

The

Next function calls the Ndpi_detection_process_packet () function for application layer analysis. This is also the principal function of application protocol analysis. Note that the parameter passed in by this function is ndpi_flow_struct (flow, as shown below), and the function first initializes the Flow->packet, which is the packet structure. Because for the same flow, some variables in the struct have been initialized in the first packet, and these variables may change in certain circumstances, such as the detection of protocols, and for each packet, flow must change the information in the Flow->packet. Next will call the Ndpi_connection_tracking () function, the main function of this function is to determine the ' location ' of the package, familiar with the TCP protocol is known, a TCP after three handshake to establish a connection Bababababa .... It is important to know the function and function of the Syn,ack,seq,ack_seq four variables. This function plays an important role in the function of packet reorganization. Some code is posted here

    if (Tcph->syn! = 0 && Tcph->ack = = 0 && Flow->l4.tcp.seen_syn = 0 && flow->l4.tcp.
    Seen_syn_ack = = 0 && Flow->l4.tcp.seen_ack = = 0) {Flow->l4.tcp.seen_syn = 1; }//first time if (Tcph->syn! = 0 && Tcph->ack! = 0 && Flow->l4.tcp.seen_syn = 1 && flow->
    L4.tcp.seen_syn_ack = = 0 && Flow->l4.tcp.seen_ack = = 0) {flow->l4.tcp.seen_syn_ack = 1; }//Second if (Tcph->syn = = 0 && Tcph->ack = = 1 && Flow->l4.tcp.seen_syn = 1 && flow->
    L4.tcp.seen_syn_ack = = 1 && flow->l4.tcp.seen_ack = = 0) {flow->l4.tcp.seen_ack = 1;
       }//Third//above three sentences is three times handshake corresponding judgment statement if ((flow->next_tcp_seq_nr[0] = = 0 && flow->next_tcp_seq_nr[1] = = 0) || (Proxy_enabled && (flow->next_tcp_seq_nr[0] = = 0 | | flow->next_tcp_seq_nr[1] = = 0))) {if (Tcph->ack! = 0) {//packet_direction indicates that the direction is from the source Ip to destination ip\ from destination IP to source IP flow->next_tcp_seq_nr[flow->packet.packet_direction] = Ntohl (TCPH->SEQ) + (Tcph->sy N?
    1:packet->payload_packet_len);
    if (!proxy_enabled) {flow->next_tcp_seq_nr[1-flow->packet.packet_direction] = Ntohl (TCPH->ACK_SEQ); }}} else if (Packet->payload_packet_len > 0) {/* Check TCP sequence counters */if ((U_int32 _t) (Ntohl (TCPH->SEQ)-flow->next_tcp_seq_nr[packet->packet_direction]) > ndpi_struct->
    tcp_max_retransmission_window_size) {packet->tcp_retransmission = 1; }

The Ndpi_selection_packet is then set, which mainly records the next four layers of information for each packet. The variable is the ndpi_selection_bitmask_protocol_size type, which is probably a 10101011 such thing, as shown in the code below, to take these variables with or manipulate the resulting value, For example, 110111101 of the two 0 means that it is not IPV6 and not TCP.

#define NDPI_SELECTION_BITMASK_PROTOCOL_SIZE            u_int32_t
#define NDPI_SELECTION_BITMASK_PROTOCOL_IP          (1 <<0)
#define NDPI_SELECTION_BITMASK_PROTOCOL_INT_TCP         (1<<1)
#define Ndpi_selection_ BITMASK_PROTOCOL_INT_UDP         (1<<2)//shift Operation
#define NDPI_SELECTION_BITMASK_PROTOCOL_INT_TCP_OR_UDP      (1<<3)
#define Ndpi_selection_bitmask_protocol_has_payload     (1<<4)
#define Ndpi_selection_bitmask_ Protocol_no_tcp_retransmission   (1<<5)
#define Ndpi_selection_bitmask_protocol_ipv6            (1< <6)
#define Ndpi_selection_bitmask_protocol_ipv4_or_ipv6        (1<<7)
#define Ndpi_selection_ Bitmask_protocol_complete_traffic    (1<<8)

In the next call to the following code, here guessed_protocol_id I have not figured out what to do with, then use to see it again

flow->guessed_protocol_id = (int16_t) ndpi_guess_protocol_id (ndpi_struct, protocol,
                                Sport, dport);
    flow->protocol_id_already_guessed = 1;

Finally, call the Check_ndpi_flow_func () function for specific application protocol detection, which will not enter a different interface depending on the tcp\udp\. There are more things here, for the time being to write an article about the type of HTTP protocol, so it is not described in detail. After this function, if the protocol type is still not detected, continue to detect the next packet until the protocol type of the data stream is detected.