Configuring the Linux firewall with Ipset

Iptables is a user-space tool that configures firewall rules in the Linux kernel, which is actually part of the NetFilter framework. Probably because Iptables is the most common part of the NetFilter framework, this framework is often called iptables, Iptables is a firewall solution introduced by Linux from version 2.4.

Ipset is an extension of iptables, which allows you to create rules that match the entire address sets (address collection). Unlike ordinary iptables chains, which are linear storage and filtering, IP collections are stored in indexed data structures, which can be found in an instant collection that is larger and more efficient.

In addition to some common situations, such as preventing some dangerous hosts from accessing the machine, which reduces system resource usage or network congestion, Ipsets also has some new firewall design methods and simplifies configuration.

In this article, after a quick discussion of the installation requirements for ipsets, I'll take a moment to introduce the core mechanisms and basic concepts of iptables. Then I'll introduce ipset usage and syntax, and demonstrate how ipset can be combined with iptables to accomplish a variety of configurations. Finally, I'll provide some details and a higher-level example to illustrate how to solve real-world problems.

Ipset has significant performance improvements and extended features over traditional iptables, such as applying a single firewall rule to the same group and network as the entire host.

Because Ipset is only an extension of iptables, Iptables is also described.

In many Linux releases, Ipset is a simple installation package that you can install through the package management tools provided by your Linux distribution.

The important point to understand is that, like Iptables, Ipset is made up of both the tool of the user space and the module of the kernel space, so you need to have both parts ready. You also need to "Ipset-aware" this iptables module, which is used to add rules that match against sets. （...... ）

First we use our own Linux distribution Package management tool to search for Ipset. Install Ipset and Xtables-addons-source packages on Ubuntu, then run Module-assistant auto-install xtables-addons and wait about 30 seconds for Ipset to be ready to use.

If your Linux distribution is not supported, you will need to build the source code and patch the kernel based on the installation steps in the Ipset home page.

This article uses Ipset v4.3 and Iptables v1.4.9.

Iptables Overview

In simple terms, the Iptables firewall configuration consists of a collection of rule chains, each containing a rule. A packet that, during each processing phase, the kernel talks about the appropriate rules to determine the fate of the datagram.

The rule chain is matched sequentially, based on the flow direction of the packet (remote-to-local, remote-to-remote or Local-to-remote) and the current processing stage (before or after "routing").

When a rule chain needs to be matched, the packet needs to be aligned in order with each rule in the chain, and the matching rule is found straight ahead. Once a matching rule is found, the target rule is called. If the last rule does not match the packet, the default rule is used.
A rule chain is that many rules are arranged in order, and a rule is a combination of match/target. A simple match example is "TCP destination port is 80". Target's example is "accept this package". Target can also redirect packets to other user-defined chains, and the user-defined chain provides mechanisms, including grouping and subdivision rules, to cascade multiple links to complete a function.
Each iptables command used to define a rule, whether for simple rules or complex rules, consists of three basic parts, including specifying Table/chain (and order), match, and Target.

Figure 2. Parsing the iptables command
To configure all of these options and create a complete firewall, you need to run a series of iptables commands in a specific order.
The iptables is very powerful and extensible. In addition to many internal features, Iptables provides an API to extend match and target.
Ipset
Ipset is the match extension for iptables. If you want to use it, you need to use the Ipset command-line tool to create a set merge that specifies a unique set and name, and then index those collections separately in the match section of the iptables rule.
A collection is a list of addresses that are convenient and efficient for quick querying.
There are two common iptables commands that block packets from 1.1.1.1 and 2.2.2.2 into the host:
Iptables-a input-s 1.1.1.1-j DROP
Iptables-a input-s 2.2.2.2-j DROP
The match section syntax-S 1.1.1.1 indicates that the match source address is a 1.1.1.1 packet.
The following Ipset/iptables command can also achieve the above purpose:
Ipset-n MySet Iphash
Ipset-a MySet 1.1.1.1
Ipset-a MySet 2.2.2.2
Iptables-a input-m set--set myset src-j DROP
The Ipset command above creates a collection (MySet of type Iphash) that contains two addresses (1.1.1.1 and 2.2.2.2).
The iptables command then uses the match option through the-m set--set myset src, which means "match the source address contained in the collection MySet the packet"
SRC represents the source address, and DST represents the destination address. If both SRC and DST are used, both the source address and the destination address must be matched.
In the second example, only one iptables command is required, regardless of how many IP addresses in the collection need to be added. Although only two addresses are used in this example, you can simply define 1000 addresses based on this example, and still require only one iptables statement. If you use the first example method and do not use Ipset, you need 1000 iptables rules.
Set Types
Each collection is of a specific type, and it defines not only what type of value can be stored inside (IP addresses, networks, ports and so), but also how the packet is matched (in other words, the part of the packet needs to be checked and checked). In addition to some of the most common collection types, such as checking IP addresses, some other collection types are provided, such as checking ports, address and port checks simultaneously, MAC address and IP address checking, etc.
Each collection type has its own rules that represent the type of collection, the scope, and what it contains that is worth distributing. Different collection types use different types of indexes and are optimized under different conditions. The collection type needs to be selected according to different realities.
The most flexible collection type is Iphash, which can store arbitrary IP addresses and Nethash (Ip/mask). Please refer to Ipset's Man manual for all collection types.
Setlist is a special collection type that allows you to organize multiple collections inside a collection. For example, you need a separate collection that contains both an IP address and network information.
Advantages of Ipset
In addition to the performance benefits, in some cases ipset allows for a more straightforward configuration method.
If you want to define a firewall environment, the environment does not process packages from 1.1.1.1 and 2.2.2.2, and the processing is included in Mychain, note that the following methods are not valid:
Iptables-a input-s! 1.1.1.1-g Mychain
Iptables-a input-s! 2.2.2.2-g Mychain
If the packet is from 1.1.1.1, it fails to match the first rule, but succeeds when the second rule is matched. If the packet is from 2.2.2.2, matching the first rule succeeds.
Although there are some other methods that can not be applied to Ipset can meet the specified requirements, but Ipset is the most direct.
Ipset-n MySet Iphash
Ipset-a MySet 1.1.1.1
Ipset-a MySet 2.2.2.2
Iptables-a input-m set! --set MySet src-g Mychain
With the above method, if the packet comes from 1.1.1.1, it does not match the rule (because the source address 1.1.1.1 does match the set MySet). If the packet is from 2.2.2.2, it does not match the rule.
This is a simple example of the basic advantages of matching a complete condition in a rule. In other respects, each iptables rule is independent from other rules, and it is difficult to connect the rule logic, especially if it contains mixed normal and reverse tests. Ipset simply makes the configuration easier in these cases.
Another advantage of Ipset is that the collection can be dynamically modified, even if the iptables rule is using the collection. adding/modifying/deleting interfaces is simple and order-independent. On the other hand, each rule in the iptables is complex, and the order of the rules is an important element, so it is difficult to modify the internal rules and there are potential problems.

Excluding WAN, VPN and other Routed Networks from the nat-the right

Outbound Nat (SNAT or IP spoofing) allows host access within a private LAN Internet.iptables NAT rules to match packets within the private network to access Internat, replacing the source address of the package with the gateway address (making the packet look like it was sent from the gateway, To hide the host behind the Gateway).

NAT automatically tracks the active connection, so it can send the returned packets to the correct intranet host (by modifying the destination address of the packet to the internal host address).

The following is a simple outbound NAT rule, 10.0.0.0/24 is the internal LAN:

Iptables-t nat-a postrouting \
-S 10.0.0.0/24-j Masquerade

This rule matches all packages from the intranet and disguises them. If only one route is connected to Internat this method is very efficient, and all traffic through that road is a public network traffic. However, if there are routes connected to other private networks, such as VPN or inability to WAN connections, you may not use address spoofing.

One simple way to overcome this limitation is to establish NAT rules based on the physical interface, rather than using a network address-based approach.

Iptables-t nat-a postrouting \
-O Eth0-j Masquerade

The rule assumes that eth0 is an external interface that matches all packages that leave this interface. Unlike the previous rules, other intranet-based packets do not match this rule (such as OpenVPN connections) when accessing the public network through other interfaces.

Although many connections are routed through different interfaces, it is not possible to assume that all links are like this. An example is an Kame-based IPSec VPN connection (such as Openswan) that does not use a virtual interface.

Another case where the above interface matching technique is not applicable is if the outward interface (an interface connected to the Internet) is routed to an intermediary network of other private networks, rather than to the Internet.

Firewall rules that are designed by matching physical interfaces can be used in some human constraints and rely on network topologies.

It was later discovered that Ipset had another application. Suppose there is a local LAN (10.0.0.0/24) that needs to be connected to the Internet, in addition to three local networks (10.30.30.0/24, 10.40.40.0/24, 192.168.4.0/23, and 172.22.0.0/22), Execute the following command:


Ipset-n routed_nets Nethash
Ipset-a routed_nets 10.30.30.0/24
Ipset-a routed_nets 10.40.40.0/24
Ipset-a routed_nets 192.168.4.0/23
Ipset-a routed_nets 172.22.0.0/22
Iptables-t nat-a postrouting \
-S 10.0.0.0/24 \
-M set! --set routed_nets DST \
-j Masquerade

As we can see, the ipset simple implementation of exact matching. The rule disguises all packets that come from (10.0.0.0/24), not the other packages that are in the network in the Routed_nets collection. Because the configuration is completely network-based, you don't have to worry about other special network connections (such as VPNs), and you don't have to worry about physical interfaces and network topologies.

Limiting Certain PCs to has Access only to Certain public Hosts

If your boss is more concerned about working hours on the Internet, please restrict your employees ' PCs to only a few designated websites, but don't want all your internal PCs to be restricted.

Restricting 3 PCs (10.0.0.5, 10.0.0.6 and 10.0.0.7) can only access worksite1.com,worksite2.com and worksite3.com. Execute the following command:
Ipset-n limited_hosts Iphash
Ipset-a limited_hosts 10.0.0.5
Ipset-a limited_hosts 10.0.0.6
Ipset-a limited_hosts 10.0.0.7
Ipset-n allowed_sites Iphash
Ipset-a allowed_sites worksite1.com
Ipset-a allowed_sites worksite2.com
Ipset-a allowed_sites worksite3.com
Iptables-i FORWARD \
-M set--set limited_hosts src \
-M set! --set allowed_sites DST \
-j DROP

This example uses two sets in a rule. If the source address matches the Limited_hosts destination address does not match the allowed_sites, the packet is discarded.

Note that the rule is added to the forward chain, and it does not affect the firewall host's own communications.

Blocking Access to the Hosts for all but Certain PCs (inverse Scenario)

Suppose the boss wants to prevent employees from visiting several specific websites, but does not block his own PC and his assistant's PC. In this example, we can match the MAC address of the boss and the assistant's PC, not the IP address. Assuming that their Mac is 11:11:11:11:11:11 and 22:22:22:22:22:22, the sites that need to organize employee visits are badsite1.com, badsite2.com, and badsite3.com.

This time we do not use the second set to match the MAC address, but instead use multiple iptables commands, mark the packet with Mark Target, and use the following rules to process the tagged packet.

Ipset-n blocked_sites Iphash
Ipset-a blocked_sites badsite1.com
Ipset-a blocked_sites badsite2.com
Ipset-a blocked_sites badsite3.com
Iptables-i forward-m Mark--mark 0x187-j DROP
Iptables-i FORWARD \
-m mark--mark 0x187 \
-M Mac--mac-source 11:11:11:11:11:11 \
-j MARK--set-mark 0x0
Iptables-i FORWARD \
-m mark--mark 0x187 \
-M Mac--mac-source 22:22:22:22:22:22 \
-j MARK--set-mark 0x0
Iptables-i FORWARD \
-M set--set blocked_sites DST \
-j MARK--set-mark 0x187

In the above example, the commands used are more and more complex because they do not use Ipset to do all the matching work. Because of the use of multiple iptables commands, the order of each command is very important.

Note These rules use the-I (INSERT) option instead of the-a (append) option. When a rule is inserted, he will be added to the top of the chain, while the previous rules are automatically moved down. Because every rule is inserted in Germany, the actual order of validity is reversed.

The last Iptables command is actually at the top of the forward chain. The rule matches all destination addresses that match the Blocked_sites collection and then marks the data as 0x187. The following two rules match packets from a specific MAC address and have been marked as 0x187, and then mark them as 0.

Finally, the final iptables rule discards all packets marked as 0x187. Except that the source is a packet of two specific MAC addresses, he will match all the packets in the Blocked_sites collection with the target address.

This is one way to solve the problem. There are other ways to use a user-defined chain, in addition to using the second Ipset collection method.

It is not possible to use the second Ipset collection instead of the markup, because Ipset does not have a Machash collection type, only the collection type, but he requires matching both the IP and Mac, not just the MAC address.

Warning: In most real-world environments, this approach may not work, and most of the sites you need to block have multiple IP addresses (such as Facebook, MySpace, and so on), and these IPs are frequently replaced. One limitation of the iptables/ipset is that the hostname can only be used if it is resolved to a single IP address.

Also, the hostname lookup occurs only when the command executes, so if the IP address changes, the firewall is unaware, but the previous IP address is still used. For this reason, a better way to complete Web access restrictions is to use an HTTP proxy, such as squid.

Automatically Ban Hosts this attempt to Access Invalid Services

Ipset provides iptables with the goal Extension feature, which provides a mechanism for dynamically adding and removing targets to a collection. Instead of manually adding a target using the Ipset command, it is automatically added at run time through Iptables.

For example, if the remote host tries to connect to port 25, but you are not running the SMTP service, we suspect that the other person is hostile, so we organize his other attempts before the other party has done something bad, using the following rules:

Ipset-n banned_hosts Iphash
Iptables-a INPUT \
-p TCP--dport 25 \
-j SET--add-set banned_hosts src
Iptables-a INPUT \
-M set--set banned_hosts src \
-j DROP

If a packet is received from port 25, assuming that the source address is 1.1.1.1, then the address is immediately added to the Banned_hosts collection, and the following example is equivalent:

Ipset-a Banned_hosts 1.1.1.1

All 1.1.1.1 connections will be blocked.

He will also prevent other hosts from scanning the device, unless he does not scan Port 25th.
Clearing the Running Config

If you want to clear the configuration of Ipset and iptables, set the firewall reset to run the following command:

Iptables-p INPUT ACCEPT
Iptables-p OUTPUT ACCEPT
Iptables-p FORWARD ACCEPT
Iptables-t filter-f
Iptables-t raw-f
Iptables-t nat-f
Iptables-t mangle-f
Ipset-f
Ipset-x

If the collection is being used, it means that other iptables rules are referencing the collection, and the collection cannot be destroyed (ipset-x), so the reset,iptables chain must be cleared first in order to complete any state.

Conclusion

Ipset adds a lot of useful features and functionality to Netfilter/iptables, as described in this article, Ipset not only provides the possibility of a new firewall configuration, but also reduces the difficulty of using iptables to configure firewalls.

At any time, if you want to apply firewall rules to a group, you should use Ipset. As in the previous example, you can accomplish a wide variety of network configurations and strategies by combining ipset with other features of Iptables.

Configuring the Linux firewall with Ipset