(1) Understand Netfilter & iptables in linux: What is Netfilter?

I have been studying the linux Firewall System for some time. Due to the complicated work recently, I have been afraid of getting familiar with it for a long time. Take time to sum up these things. One is to make a deposit for yourself, and the other is to welcome the elders who are more advanced in this field to give advice to the younger brother, learn together, and make progress together.
This series of blog posts focuses on analyzing the implementation mechanism, principles, and design concepts of Netfilter, at the same time, from the user-state iptables to the kernel-state Netfilter, the interaction process and communication means. As for the entry-level usage of iptables, there will be a lot of information on the Internet, so I will not waste any time here.
Many people will feel like this after getting started with iptables: How does one take effect for every rule I configure with the iptables command? How does the kernel perform matching of these rules? If iptables cannot meet my current needs, can I scale it? These questions are the topics I will share with you in the following blog. It should be pointed out that Netfilter and IP protocol stack are seamlessly compatible, so if you have the foundation of the protocol stack, you will feel familiar when reading this article. Of course, it doesn't matter if it doesn't, because I will popularize the introduction of protocol stack at key points. It's just a matter of popularity. I won't go into details, because there are too many things involved, and I am still researching and exploring it. Okay. Let's get started.
Note: The kernel version I have studied is 2.6.21 and iptables version 1.4.0.
 
What is Netfilter?
To illustrate this problem, first look at a basic network communication model:
 
In the process of sending data, the top-to-bottom is the process of adding a header. the header of this layer is added for each data arriving at a layer. At the same time, the receiver is a process of "stripping the header". After packets are collected from the network adapter, the header of each layer is removed from the upper layer of the protocol stack, raw data is what eventually reaches the user.
The underlying mechanism of the "stack" mode is basically like the following:
 
For each packet received, it comes in from "A" and passes the route decision. If it is sent to the local machine, it goes through "B" and then continues to pass to the upper layer of the protocol stack; otherwise, if the destination of the data packet is not local, it goes through "C" and forwards the packet along "E.
For each packet sent, there is also a route decision to determine the interface from which the packet is sent, and then go through the "D" point, finally, the package is sent along with "E.
The five key points of the protocol stack Are A, B, C, D, and E.
Netfilter is a subsystem introduced by Linux 2.4.x. As a general and abstract framework, Netfilter provides a complete set of hook function management mechanisms, this makes it possible to filter data packets, convert network addresses (NAT), and trace protocol-based connections. The position of Netfilter in the kernel is shown in:
 
This figure intuitively reflects the relationship between iptables in the user space and the Netfilter-based ip_tables module in the kernel space and their communication methods, as well as the role of Netfilter in this figure.
Go back to the five key points "ABCDE" discussed earlier on the protocol stack. Netfilter re-names these five points in netfilter_00004.h, as shown in, meaning I will not explain it anymore. The cat is called Mimi:
 
At each key point, there are many callback functions that have been pre-registered by priority (What are these functions and what are they used later. Some people like to call these functions "Hook Functions" and talk about the same thing.) These are embedded in these key points, forming a chain. For each incoming packet, it will be "played" by those callback functions in turn, and then, depending on the situation, whether to let it go, discard it, or how to drop it. However, in any case, these callback functions must finally report the health and death of the data packet to Netfilter, because after all, every data packet is lent by Netfilter from the protocol stack to Happy, if you don't know how to get rid of others, you have to "see the dead. Each hook function must return one of the following values to the Netfilter framework:
N NF_ACCEPT continues normal data transmission. The returned value tells Netfilter that the packet is still accepted so far and should be submitted to the next stage of the network protocol stack.
N NF_DROP discards the datagram and no longer transmits it.
The n NF_STOLEN module takes over the datagram and tells Netfilter to "forget" the datagram. The callback function starts to process the data packet, and Netfilter should discard any processing of the data packet. However, this does not mean that the resources of the data packet have been released. This data packet and its own sk_buff data structure are still valid, but the callback function obtains the ownership of this data packet from Netfilter.
N NF_QUEUE queues the datagram (usually used to process the datagram to the user space)
N NF_REPEAT: this value should be used with caution when calling the callback function again to avoid endless loops.
To make us more professional, we began to make some conventions: the five key points mentioned above are called hook points, the callback functions registered by each hook point are called hook functions.
Linux 2.6 kernel Netfilter currently supports IPv4, IPv6, DECnet and other protocol stacks. Here we mainly study IPv4 protocol. The following figure shows the protocol type, hook Point, hook function, and priority:
 
 
For each type of protocol, data packets are transmitted sequentially in the direction of the hook point, and Netfilter on each hook point hangs many hook functions according to the priority. These hook functions are used to process data packets.
Netfilter uses the NF_HOOK (include/linux/netfilter. h) macro to switch to the Netfilter framework in the protocol stack. Compared with version 2.4, the kernel version 2.6 is more flexible in the definition of the macro. The definition is as follows:
# Define NF_HOOK (pf, hook, skb, indev, outdev, okfn )\
NF_HOOK_THRESH (pf, hook, skb, indev, outdev, okfn, INT_MIN)
Description of macro NF_HOOK parameters:
1) pf: protocol family name. The Netfilter architecture can also be used outside the IP layer. Therefore, this variable can also have names such as PF_INET6 and PF_DECnet.
2) hook: the name of the HOOK point. For the IP layer, obtain the preceding five values;
3) skb: not explained;
4) indev: the device that receives the data packet, represented in the struct net_device structure;
5) outdev: indicates the device that sends data packets, which is in the struct net_device structure;
(As you can see later, the above five parameters will be passed to the handler function registered in nf_register_hook .)
6) okfn: it is a function pointer. When all the registration functions of the HOOK point are called, this process is followed.
NF_HOOK_THRESH is a macro:
# Define NF_HOOK_THRESH (pf, hook, skb, indev, outdev, okfn, thresh )\
({Int _ ret ;\
If (_ ret = nf_hook_thresh (pf, hook, & (skb), indev, outdev, okfn, thresh, 1) = 1 )\
_ Ret = (okfn) (skb );\
_ Ret ;})
We found that the NF_HOOK_THRESH macro only adds one thresh parameter, which is used to specify the priority when the macro is used to traverse the hook function. At the same time, the macro calls the nf_hook_thresh function internally:
Static inline int nf_hook_thresh (int pf, unsigned int hook,
Struct sk_buff ** pskb,
Struct net_device * indev,
Struct net_device * outdev,
Int (* okfn) (struct sk_buff *), int thresh,
Int cond)
{
If (! Cond)
Return 1;
# Ifndef CONFIG_NETFILTER_DEBUG
If (list_empty (& nf_hooks [pf] [hook])
Return 1;
# Endif
Return nf_hook_slow (pf, hook, pskb, indev, outdev, okfn, thresh );
}
This function adds only one cond parameter. If this parameter is set to 0, traversal is abandoned and the okfn function is not executed; if the value is 1, run nf_hook_slow to complete the sequential traversal of the hook function okfn (which is executed in ascending order of priority ).
A two-dimensional struct array is defined in the net/netfilter/core. h file to store callback processing functions for Hook points of different protocol stacks.
Struct list_head nf_hooks [NPROTO] [NF_MAX_HOOKS];
The number of lines NPROTO is 32, which is the maximum protocol cluster currently supported by the kernel. The number of columns NF_MAX_HOOKS is the number of Mount Points. Currently, this value is 8 in the 2.6 kernel. Shows the final structure of the nf_hooks array.
 
In include/linux/socket. h, the serial number of the IP protocol AF_INET (PF_INET) is 2, so we can get the mount point of the hook function of the TCP/IP protocol family:
PRE_ROUTING: nf_hooks [2] [0]
LOCAL_IN: nf_hooks [2] [1]
FORWARD: nf_hooks [2] [2]
LOCAL_OUT: nf_hooks [2] [3]
POST_ROUTING: nf_hooks [2] [4]
At the same time, we can see that in the 2.6 kernel IP protocol stack, the normal process of the Protocol Stack is switched to the Netfilter framework, then, the following operations are performed to call all HOOK functions of each HOOK point sequentially:
1). ip_rcv function in net/ipv4/ip_input.c. This function is mainly used to process IP packets at the network layer. The entry point from Netfilter framework is:
NF_HOOK (PF_INET, NF_IP_PRE_ROUTING, skb, dev, NULL, ip_rcv_finish)
According to the previous understanding, the meaning of the Code has been very intuitive and clear. That is: if the protocol stack receives an IP packet (PF_INET), the packet is sent to the NF_IP_PRE_ROUTING filter point of Netfilter, check whether [R] has registered the related hook function for processing data packets at the filter point (nf_hooks [2] [0. If yes, the nf_hooks [2] [0] In the traversal chain table will be used to find the matching match and the corresponding target, determine how to process the data packet based on the value returned to the Netfilter framework (by the hook module or by the ip_rcv_finish function ).
[R]: the so-called "check" just now ". Its core is the nf_hook_slow () function. In essence, this function is very simple. You can find the two-way linked list nf_hooks [] [] based on the priority and find the corresponding callback function to process data packets:
Struct list_head ** I;
List_for_each_continue_rcu (* I, head ){
Struct nf_hook_ops * elem = (struct nf_hook_ops *) * I;
If (hook_thresh> elem-> priority)
Continue;
Verdict = elem-> hook (hook, skb, indev, outdev, okfn );
If (verdict! = NF_ACCEPT ){... ... }
Return NF_ACCEPT;
}
The above code is part of the core code of the nf_iterate () function in net/netfilter/core. c. The function is called by the nf_hook_slow function and further processed according to the returned value.
2). The ip_forward function in net/ipv4/ip_forward.c. Its entry point is:
NF_HOOK (PF_INET, NF_IP_FORWARD, skb, skb-> dev, rt-> u. dst. dev, ip_forward_finish );
After route selection, all packets that need to be forwarded by the local machine will be processed by the ip_forward function. Here, the function is switched from the NF_IP_FOWARD filter point to the Netfilter framework, and the matching search is performed on the nf_hooks [2] [2] filter point. Finally, the execution of ip_forward_finish function is determined based on the returned value.
3). In net/ipv4/ip_output.c, The ip_output function is used in the form of the Netfilter framework:
NF_HOOK_COND (PF_INET, NF_IP_POST_ROUTING, skb, NULL, dev, ip_finish_output,
! (IPCB (skb)-> flags & IPSKB_REROUTED ));
Here we can see that the starting point is changed from the unconditional macro NF_HOOK to the conditional macro NF_HOOK_COND. The condition for calling this macro is: if the protocol stack currently processes the data packet skb without the re-routing flag, the data packet enters the Netfilter framework. Otherwise, the ip_finish_output function is called directly to process the data through the protocol stack. In addition, conditional macros and unconditional macros have no difference.
If you need to fall into the Netfilter framework, the data packet will be matched at the nf_hooks [2] [4] filtering point.
4). It is also the ip_local_deliver function in net/ipv4/ip_input.c. This function processes all data packets whose destination address is local. The cut-in function is as follows:
NF_HOOK (PF_INET, NF_IP_LOCAL_IN, skb, skb-> dev, NULL, ip_local_deliver_finish );
First, all data packets sent to the local machine will go to the nf_hooks [2] [1] filtering point to check whether there is a callback processing function for related data packets. If yes, the system will perform matching and action, finally, execute the ip_local_deliver_finish function based on the returned value.
5). The ip_push_pending_frames function in net/ipv4/ip_output.c. This function is used to re-constitute a complete IP packet and then send it out. The entry point for accessing the Netfilter framework is:
NF_HOOK (PF_INET, NF_IP_LOCAL_OUT, skb, NULL, skb-> dst-> dev, dst_output );
All packets sent from the local machine are first filtered by the nf_hooks [2] [3] filter point of Netfilter. In general, there are few limits on the packets sent from their machines, whether on a vro or a PC. Because the potential risks of this operation are obvious, some services may fail due to some improper settings, so there are very few scenarios for intercepting data packets on this filtering point. Of course, it is not ruled out that there are special requirements.
 
Section: The hook mechanism of the Netfilter framework in the Linux kernel can be summarized as follows:
During the entire process of data packets flowing through the kernel protocol stack, on some predefined key points, PRE_ROUTING, LOCAL_IN, FORWARD, LOCAL_OUT, and POST_ROUTING will find whether hook functions are registered based on the protocol cluster PF_INET of data packets. If no, the function pointed to by the okfn function pointer is directly returned to continue to go through the Protocol Stack. If yes, the nf_hook_slow function is called, so as to enter the Netfilter framework to further call the hook function registered under the filter point, and then determine whether to continue executing the function directed by the function pointer okfn based on its return value.
Unfinished, To be continued...

From Xinyi Fang's blog