The next generation packet filtering framework after iptables is Nftables

The night has been deep, ready to tell the story about NetFilter, writing a little loose, because no draft, a bit with the consciousness of the flow, one go do not know is to boast or self-deprecating, right when a child wrote diary, childhood like to write a diary every day, high school is the name of the withdrawal of a few boxes of the Codex, A few years ago as a result of drinking to Zhou Kee, now realize that life is short, time is not enough, not in a daze, ready to know about the Linux network things bit by bit recorded, would like to continue writing on paper, but found in the era of personal computer smartphones, Many words would not have been written ... The last not to finish the story about iptables, this article continues ...

I. Nftables of the-iptables of pre-transmission

Iptables almost nobody knows, people are trapped in the box and feel that everything is taken for granted, but I am the exception, and many other things, in this field, I still do and happy to do that "excluded people."
Many of the shortcomings of iptables can no longer be ignored, but only a few people see these, most people as users, just use it. I'm not going to be so noisy here. The following drawbacks come from Nftables's propaganda documents, but even in foreign countries, there is a huge controversy:
The 1.iptables framework knows so much about the kernel that it generates a lot of code redundancy.
This is obvious, for example, for TCP and UDP, take Sport,dport is not different, but iptables use two sets of code, this is just an example, there are many similar.
The rule structure of the 2.iptables is not reasonable in design.
This is to be highlighted.

Structure of the 1.iptables

Iptables consists of a table, a chain, and a rule, in which the rules are composed of match,target. As shown in the following structure:
table{
chain[
Rule
(Match,match,match,...)
->target,
Rule
(Match,match,match,...)
->target,
...
],
chain[
...
],
...
}

2.iptables rule Matching Execution process

The rules of the

Iptables are matched in the order of configuration, matching each rule on each chain of each table, sequentially matching each match in each rule, and all matching matches executing the target of the rule, as determined by target:
A. Continue to match next rule
B. Make some changes to the packet
C. Jump to another chain (that is, start by matching each rule in the chain sequentially)
D. Returns the chain that raised the jump (that is, the next rule that continues to match the chain before the jump)
E. Drop packet
F. Receive packets (i.e. no longer continue to match down, return directly)
G. Logging
H ...
The entire Iptables framework executes the following process:

 loop 1:static breakrule = 0; traverse a chain for each rule {nomatch = 0; Loop 2: Traverse each of the match {result = rule->match[curr of a rule] (skb, info); if (Result != match)  {nomatch = 1;break;}} if  (nomatch == 1)  {continue the next rule of the chain;} Result = rule->target (Skb, info);if  (result == drop)  {break drop Packet}  else if  (result == accept)  {break accepts packet} else if  (result ==  goto)  {breakrule = rule; jump to the corresponding chain, perform loop 1} else if  (Result == return)  {break returns a call to chain, executes its breakrule next rule}  ...} 

Read the above code basically know the iptables command implementation, the programmer can do is to expand the function of iptables, there are two specific practices: write a match and write a target. In addition, the programmer is not a thing, the rest of the user's imagination to see ...
Through the above process, it can be found that the process of packet filtering will eventually be implemented to match the rules, and the filtered action is finally implemented to the target of the rule, the previous match matches the return result is 0 or not 0 to indicate whether the match, only all match matches, will execute the target. This determines a few things:
A. If you want to achieve multiple target, you have to write more than one rule.
For example, to implement log and DROP, then write two rules, or expand a log_and_drop target, the former affects efficiency, the latter needs programming. You really care about performance, and you are not programmers do not understand programming, you are crazy ...
B. You can write a match and sneak a little thing in there, but the outside doesn't know
All this is too irregular, you can in a match inside a check code to get rid of a packet, you can also do log in the inside, do NAT, but the original intention of the Iptables framework does not allow you to do so but did not stop your behavior.
We can see another detail in the details of the iptables execution flow (which is not drawn in the process above), namely that iptables is really not enough to just determine "match" in match, even the code is very reluctant to design. If you look at the core function of ipt_do_table, you will find a control variable called Hotdrop, what does this variable do? As the note means:

@hotdrop: Drop Packet If we had inspection problems

This hotdrop is passed in as an outgoing parameter to each match callback function, which indicates that a packet is discarded within match. This exposes the lack of design, discarding a packet is not the target to do? The job of a match is to decide if the packet matches, why do you want to instruct it to discard it? Isn't that a leapfrog? It's just a detail, you can say 1000 reasons that it's reasonable, but it's ugly!

Two. A little bit of history

It's definitely a truth to know that history always understands more, but it's the specialization that prevents most programmers from reading history, even it's history. The best historical material is the original, NetFilter history is not long, from the Linux 2.3.15 kernel version was introduced so far, will not be tampered with like Kozoki.
We certainly want to see the history that Iptables was introduced to.
Iptables was introduced to replace IPChains, because at the time IPChains maintainer Rusty Russell realized that it had many drawbacks. In general, there are two drawbacks, and the others are made up of these two:
A. The firewall framework of the kernel only has 3 checkpoints, i.e., input,forward,output, and the control of the loopback packet and the Indev,outdev is weak;
B. Code write dead, match fixed, no extensibility.
The problem is here in B. For the problem A,rusty Russell proposed NetFilter design, carefully designed 5 hook points, to solve the problem of almost all control points, especially the output point design top-notch, it was placed after the route, The reason is that the Linux protocol stack after the routing operation will not give a complete filter match, such as the source IP address, export device, and so on, the output after the route to the caller again to route the permissions. Forward and input as a two-point route, while maintaining minimal effort, because if you do not open the Ip_forward option, even if the packet is not input will not enter the forward, if there is no route found, you will not reach input, Nor will it reach forward. For prerouting, it can differentiate local loopback traffic and network card traffic via Conntrack ... Anyway, this is the kernel work, this netfilter design is very perfect, still be used today.
For the problem B,rusty Russell put forward iptables, it is a highly extensible framework, that is, from now on, Iptables has a match/target pairing extension, every time you need to expand, each match/ In addition to the user-state LIB, Target also supports kernel-state support, which turns the fixed-match pattern of the ipchains era into one that can be programmed to expand itself.
Rusty Russell is the perfect solution for ipchains, but that's it! Any new framework from the same author is almost to solve the disadvantage of the previous framework, iptables as a rookie, in the time of cheering, no one will consider its drawbacks, everything is like this, right?
Iptables's malpractice is gradually discovered, Rusty Russell as co-author of IPChains and iptables, its attitude towards the latter to replace the former is always conservative, a new framework needs another person or team to present, and not likely to appear in the Rusty Russell himself and the interior of the Iptables team.
A large number of documents have been written for netfilter,rusty, which can be found on the NetFilter website: http://people.netfilter.org/rusty/unreliable-guides/Undeniable, These are precious first-hand information, for we understand netfilter, probably no better than these. Anyone can find clues to the "Why XX is so" from these original books, and they are also a beacon of guidance on how to improve the existing framework.

Three. Nftables debut

Given the many drawbacks of iptables (it is not a disadvantage, but the extension of Match/target pairing leads to developers extending the disadvantage and eventually identifying it as a disadvantage), Nftables aims to improve them individually. First, there are two issues:
A. How to parse a packet in a uniform way
in solving this problem, u32 match gives the author the idea of
B. How to execute multiple action
In fact, Iptables's matches/target pairing approach limits the developer's mindset. Why do we have to differentiate between match and target? The Iptables framework's execution process limits the result of match as a Boolean, and all actions are performed in target, and if you remove this restriction and open the entire process to the developer, it's much more flexible. Nftables in the background. It turns out that nftables do more than fix the drawbacks and go farther.
       First Nftables uses the "virtual machine to interpret bytecode" way, so that a rule is really "do something for a packet" such a flexible command, and removed the " Match all of the match after executing a target "such a restriction. The way virtual machines execute bytecode has long been adopted by BPF, and our well-known tcpdump capture program is used to filter packets. Let's take a look at the execution flow of the Nftables framework:

 loop 1:static breakrule = 0; traverse a chain for each rule {nomatch = 0;reg[ MAX] Loop 2: Iterates through each of the expression {void rule->expression[curr]->operations->expr of a rule (skb,  Info, reg) if (reg[verdict] != continue)  {break;}} if  (reg[verdict] == continue)  {continue the next rule of the chain;}  else if  (reg[verdict] == drop)  {break drop packet} else if  (reg[verdict ] == accept)  {break Accept Packet} else if  (Reg[verdict] == goto)  { breakrule = rule; jumps to the corresponding chain, executes the loop 1} else if  (Reg[verdict] == return)  { Break calls chain, executes its breakrule next rule}  ...} 

Light from this process, it has been able to iptables with the victory. As you can see, Nftables does not have match and target, but instead abstracts a rule into a number of expressions, that is, expression, which is a subject plus predicate, it is "executable", it can "do anything", Instead of just calculating a matching result. In addition, the nftables contains a set of registers, one of which is the verdict register, which indicates "what to do next". After each expression is executed, the register is removed and the value of the register is taken to take the next action. This verdict register replaces the target return value in Iptables, which allows multiple actions to be taken in a rule, each of which can be parsed into an expression, After each expression is executed, set the verdict register to continue!
In addition to the significant differences in the execution process, Nftables's greatest significance is that it abstracts expression, and the Nftables kernel framework can register many kinds of expression, each with a set of operations, Where the expr callback function executes the concrete expression expressions. Typical expression is:
Payload expression:
A piece of data from a packet is copied to a buffer indicated by a nftables register, except for an error verdict register is continue.
Compare expression:
Compares a piece of data to the buffer indicated by the Nftables register, and sets the verdict register to break if it is not equal.
Counter expression:
Increments the byte counter and the value of the packet counter according to the size of the packet, keeping the verdict register as continue.
Log expression:
Logs the packet, keeping the verdict register as continue.
NAT expression:
According to the value of the Nftables register set the NAT,VERDICT register of the packet to the result of the NAT operation, note that the NAT reference data are from the Nftables register.
compat expression:
Maintain compatibility with iptables. It is subdivided into the match register and the target register, where the match expression calls the match of iptables rule, and if the match setting verdict Register is continue, the break;target expression is called iptables The rule's target and sets the verdict register based on the result of target.
...
Go back to Nftables's execution process and combine the expressions above to see what it all looks like?
Isn't that just an interpreter? Like a high-level language such as the Python interpreter, each expression is interpreted to execute, we can break a nftables rule into a series of expressions, that's all, as follows:

Expr1:reg[verdict] = continue;reg[0] = skb[m...n];expr2:info[0] = something from Userspace;ret = Compare (Reg[0], info[0]) ; if (ret = = true); Then reg[verdict] = CONTINUE; else reg[verdict] = break; Break Fiexpr3:log_packet (SKB); Expr4:ret = Do_nat_packet (SKB, reg[i]/*address to trans*/...); if (ret = = true); Then reg[verdict] = CONTINUE; else reg[verdict] = break; Break Fi ...

Look, how many things a rule has done ah Ah! The interpreter executes expression in the order of EXPR1 to EXPR4, checking the verdict register before each execution of the next expression, then who is the interpreter and, of course, the Nftables execution process above!
Nftables, what the hell is this thing? Actually a virtual machine! So where do the instructions from this virtual machine come from? Configuration from the user state. What is the configuration of user mode? Use the NFT command, of course. What is the order of NFT? is a command similar to iptables. Can the NFT command give an example? Yes:
nft Add rule IP filter input IP saddr 1.1.1.1 drop
How does this command correspond to many expressions? The answer is that the NFT command-line tool contains a "compiler" that compiles a human readable command into a single expression code. What are the details of the compilation? Can write a book, but the way to understand the tcpdump should be understood. The tcpdump command will eventually inject the compiled instructions into the kernel's BPF system, and here is a very common tcpdump command:
tcpdump-i eth0 DST 1.1.1.1
What code does it translate into? Follow the-DD parameter to see the following:
[email protected]:/usr/local/etc/nftables# tcpdump-i eth0 DST 1.1.1.1-dd
{0x28, 0, 0, 0x0000000c},
{0x15, 0, 2, 0x00000800},
{0x20, 0, 0, 0x0000001e},
{0x15, 4, 5, 0x01010101},
{0x15, 1, 0, 0x00000806},
{0x15, 0, 3, 0x00008035},
{0x20, 0, 0, 0x00000026},
{0x15, 0, 1, 0x01010101},
{0x6, 0, 0, 0x0000ffff},
{0x6, 0, 0, 0x00000000},
Please refer to the BPF's manual for details on what it means. The rules set by Nftables will eventually be "compiled" into similar "directives" injected into the kernel's nftables system, forming a form of expression. It is important to note that not all rule directives can be compiled, such as iptables compatible instructions, and log instructions cannot be compiled.
Nftables is such a beautiful package filter framework, understand how it works, you can expand it, and iptables extension match/target different, for nftables, you just need to expand expression, That means you have to write your own expression, and then let the nftables virtual machine (that is, the execution process above) execute it. Finally, let's look at the structure of the Nftables framework:
table{
chain[
Rule
(Expression,expression,expression,...)
Rule
(Expression,expression,expression,...)
...
],
chain[
...
],
...
}
Expression: = Expression | DataType | Operation | Expression | DataType
Operation: = + | - | * | / | memcpy | Contain | ...
DataType: = U8 | U16 | U16 | ... | Container | ...
Container: = Hash | Map | Tree | Set | List | Array | ...

Four. What happened to all this?

It is worth noting that although Nftables iptables in an aesthetic sense, its performance is not very efficient as a framework. Compared with NF-HIPAC, iptables is no worse than nftables. In fact, like Nftables and iptables, for all the rule on a chain, it is also a one-way walk, except that different ways of performing a specific match when traversing each rule. So, compared with Nf-hipac, why did Nftables succeed?
In fact, Nftables is still far from success, its resistance is not from performance, but from the Iptables camp! Nftables as a graceful framework, considering not only performance, but actually performance is only a small aspect of its consideration. As a framework, it is first to consider extensibility and compatibility with iptables. The voice of the opposition in minutes resounded through the ear, iptables is not wrong, match/target pairing way is not wrong, match is to return true or false, the end result is to target to execute, in short, to distinguish between match and Target, and my duties!!
Think Iptables can't perform multiple actions why don't you write a target that can perform multiple actions?
Think that iptables one-by-one execution and can not complete NF-HIPAC pioneering why not the NF-HIPAC package into a separate match ah ah?!
The NF-HIPAC that are packaged as separate match will look like this:
Iptables-a input-m Hipac--match-hipac hipac_test-j Nothing
NF-HIPAC Create Hipac_test
Nf-hipac Add hipac_test-s 1.1.1.1-j DROP
...
See the advantages of iptables, people are not to come and Nf-hipac to spell performance, others are the sea of hundred rivers, others have tolerance is big! Is the man surnamed Mao's chair going on the battlefield? No! No! No! Let's not talk about Nf-hipac, and similar to the above also ipset,ipset is encapsulated into a match and iptables linkage? Iptables is not bad, wrong in people simply should not directly expand each simple function into a set of Match/target consortium, the final form of disgusting code! Is that right?
All right! I admit that the above yy are all right! But look at Nftables, can it play like this and play more?! Oh, yes! Right! Nftables internal directly built up a lot of container class data types, such as Rbtree,hash, as a composite container, you put things inside, and write match? I say this means, written match/target know, to routinely how much business ah, you want to register match or target, but also copy a lot of management code, see Xtables-addons know the pain. With Nftables, if you want to expand a function module for iptables, a lot of work can be done in the user state, in other words, if the content is only based on SKB (that is, the packet), then Nftables is protocol independent, because no matter what protocol, You can filter expression with payload,compare,bit and other simple expression open, the work of the Protocol parsing in the user-compiled Nftables instructions when the completion can be, to the kernel state, nftables virtual machine Only execute expression, no matter the protocol!
The world is moving forward, we must look ahead! Look at the Linux packet filtering framework, from the initial implementation of the transplant BSD, to the present nftables, the middle experienced a lot of bumpy twists and turns, every time there are new things come in there will always be a retro curse! Well, here it is, the Linux kernel has built-in nftables support directly on the trunk, just as it did when NetFilter entered the trunk.
Just do it, epoch-making nftables, I'm not saying how fierce it is, but that it's really clean.
Tables,chains, that's a bad name, but it's just a name anyway. Iptables not because of the name, I do not want to understand it tangled so long, and now a nftables ... Cisco Tube-like things called list, ACL, is just a name, if the tables is not good, list is not more low-level it? Whether it's low-level or not, Huawei continues to be called Cisco. So the next generation of packet-filtering frameworks, also called tables, are expected to show that cultural identity is more useful than its substance, especially when it's about names. Iptables has been deeply rooted, nftables the name will make people more receptive. The original iptables replaced IPChains, which is a revolutionary replacement, and this time, more shows a mature Linux mechanism of a natural evolution, or evolution is more appropriate.

Five. Nftables use up

My first thought was to run Nftables on 2.6.32, but failed, there was no way to compile. Then here is a way to see a lot of propaganda documents and howto and nftables of the main station, spent a long time to clone the Git image, compiled through, and finally ran up.
Then I simply download the 3.17 version of the kernel directly on the kernel.org, in the Make config when the NFT related things are selected, and then compile the update kernel. At the same time to download the user-nftables-0.3 version of the utils, compile, the process of what is missing what, and finally very smooth. After make install, first execute:
nft-f/usr/local/etc/nftables/ipv4-filter
This command loads the filter table in the kernel, and you can manually load the table in addition to the preconfigured files.
After Table,chain is ready, you add rule on the chain you want:
nft Add rule IP filter output IP daddr 1.2.3.4 drop
Other usage please man NFT, very detailed but not exhaustive documentation, additional good information in Https://home.regit.org/netfilter-en/nftables-quick-howto and https:// home.regit.org/2014/01/why-you-will-love-nftables!
Why didn't I transplant it to a lower version of the kernel? Because I think it's too simple, why this raving? Because Nftables only and NetFilter Nf_register/unregister_hooks interface, the others are inside their framework, the complexity is nft_expr_ops, and this is very independent, and the existing kernel does not have any relationship. For the user-state utils, there is a nftables project in itself, which is a compilation problem, and this compilation is already possible by compiling.

Six. Configure the firewall to become a programming

At the end of this article, let me look at the final difference between Nftables and iptables from a holistic perspective.
I have analyzed the changes brought about by Nftables from the perspective of internal principles, so how much benefit can this give the user? If there is no benefit, there will be no power switching to nftables in the user base. The benefits are not much, only one, but this is enough, that is: Nftables allows users to follow the idea of programming to organize their own configuration logic.
What do you say? Let's look at an example on a wiki:
Nft Add rule IP filter input IP protocol vmap {tcp:jump tcp-chain, udp:jump udp-chain, Icmp:jump Icmp-chain}
What is it? This is a "programming statement" that has a simple if-else if-else if logic, or you can think of it as switch-case. Note that this is done in a single rule! If you use Iptables, you have to write more than one rule on your own. How much of the above statement looks like this:

proto = skb->net_hdr->proto;if  (proto == tcp)  {     tcp_chain (SKB);}  else if  (PROTO == UDP)  {    udp_chain (SKB);}  else if  (proto == icmp)  {    icmp_chain (SKB);} 

Nftables has become a real programming language! Since it became a programming language, if the support variable will be how flexible one thing ah, fortunately, oh, no, not to say lucky, but nftables native nature, nftables support "variable"! Note the following command:
NFT Add set filter blackhole {type ipv4_addr\;}
NFT Add rule IP Input IP saddr @blackhole drop
NFT add element filter Blackhole {192.168.1.4, 192.168.1.5}
Although Iptables's Ipset match can do this, it's just a match, nftables native support for this syntax! Even, you can even use a dictionary to map the syntax of a policy:
NFT Add Map filter mydict {type ipv4_addr:verdict\;}
NFT Add rule filter input IP saddr vmap @mydict
NFT add element filter mydict {192.168.0.10:drop, 192.168.0.11:accept}
As a result, administrators will be as flexible as programmers to organize their own logic.

Seven. A bit of a pessimistic fact

To search the Ipchain documents, few can open, and then to search iptables, the current popularity is very prosperous, nftables? I can find the results, but it takes a bit of effort to use them. This is the process of fought, you can imagine the future someday, Nftables will be like ipchain as gradually cold, fade out of sight of people ... Is this the fate of the Linux environment packet filtering framework?
This is not the inevitable development of technology, the old UNIX tools Vi,emacs until now remains the weapon of hackers, network Tools Netcat is also a compact portable enduring ... Linux's packet-filtering framework has been upgraded several times in just 15 years. It is hopeful that the underlying framework of NetFilter is basically stable, whether it is iptables or Nftables, which is based on NetFilter, and the early IPFW is not, then (before the Linux 2.3.15 kernel) The bottom packet filtering framework and its rudimentary, so NetFilter first appeared on the upper. It is worth noting that this contains too much is the developer Rusty Russell personal style, NetFilter is he finished, IPChains is also him, this can't help but reminiscent of the broken rank Ingo Molnar, introduced the O (1) Scheduler, And then replaced it with a better CFS scheduler ... UNIX is very different, personal factors are few ...

The next generation packet filtering framework after iptables is Nftables