The method of implementing batch shielding IP address under Linux

The problem of Netfilter/iptables

In Linux, it is easy to use the Netfilter/iptables framework to prohibit IP addresses:

The code is as follows:

$ sudo iptables-a input-s 1.1.1.1-p tcp-j DROP

If you want to completely block an IP address segment, you can do so simply by using the following command:

The code is as follows:

$ sudo iptables-a input-s 1.1.2.0/24-p tcp-j DROP

However, what do you do when you have 1000 separate IP addresses with no CIDR (Classless Inter-domain routing) prefix? You have to have 1000 iptable rules! This is obviously not suitable for large-scale shielding.

The code is as follows:

$ sudo iptables-a input-s 1.1.1.1-p tcp-j DROP

$ sudo iptables-a input-s 2.2.2.2-p tcp-j DROP

$ sudo iptables-a input-s 3.3.3.3-p tcp-j DROP

. . . .

What is an IP set?

This is the IP set debut. An IP set is a kernel feature that allows multiple (independent) IP addresses, MAC addresses, or even port numbers to be encoded and effectively stored in bitmap/hash kernel data structures. Once the IP set is created, you can create a iptables rule to match the collection.

You'll soon see the benefits of the IP collection, which allows you to match multiple IP addresses with a iptable rule! You can construct IP sets with multiple IP addresses and port numbers, and you can dynamically update rules without performance impact.

Installing the Ipset tool in Linux

To create and manage IP sets, you need to use a user space tool called Ipset.

To install on Debian, Ubuntu, or Linux Mint:

The code is as follows:

$ sudo apt-get install Ipset

Fedora or Centos/rhel 7 installed:

The code is as follows:

$ sudo yum install Ipset

Using the Ipset command to disable IP

Let me show you how to use the Ipset command with a simple example.

First, let's create a new IP set named Banthis (name arbitrary):

The code is as follows:

$ sudo ipset create Banthis hash:net

The second parameter (hash:net) is required and represents the type of the collection. There are multiple types of IP sets. The IP set of the Hash:net type uses hash to store multiple CIDR blocks. If you want to store a separate IP address in a collection, you can use the Hash:ip type.

Once you have created an IP set, you can check it with the following command:

The code is as follows:

$ sudo ipset list

This shows a list of available IP collections with detailed information that includes members of the collection. By default, each IP collection can contain 65,536 elements (this is a CIDR block). You can increase the limit by appending the "Maxelem N" option.

The code is as follows:

$ sudo ipset create banthis hash:net maxelem 1000000

Now let's add the IP block to this collection:

The code is as follows:

$ sudo ipset add banthis 1.1.1.1/32

$ sudo ipset add banthis 1.1.2.0/24

$ sudo ipset add banthis 1.1.3.0/24

$ sudo ipset add banthis 1.1.4.10/24

You will see that the members of the collection have changed.

The code is as follows:

$ sudo ipset list

It's time to create a iptables rule that uses IP sets. The key here is to use the "-M set--match-set" option.

Now let's create a iptable rule that allows those IP blocks to access Web services through port 80. You can use the following command:

The code is as follows:

$ sudo iptables-i input-m set--match-set banthis src-p tcp--destination-port 80-j DROP

If you want, you can save a specific IP set to a file that can be restored from the file later:

The code is as follows:

$ sudo ipset save banthis-f banthis.txt

$ sudo ipset destroy Banthis

$ sudo ipset restore-f banthis.txt

In the above command, I use the destory option to remove an existing IP set to see if I can restore it.

Automatic IP address disabled

Now you should see the power of the IP set. Maintaining an IP blacklist is a tedious and time-consuming task. In fact, there are a lot of free or fee-based services to help you do this. As an added bonus, let's look at how to automatically add IP blacklists to IP sets.

First let's get a free blacklist from iblocklist.com, this site has a different free and charged list. The free version is a peer-to-peer format.

Next I'm going to use an open source Python tool called Iblocklist2ipset to turn the Peer-to-peer format blacklist into an IP set.

First, you need to install the PIP (refer to this guide to install the PIP).

Use the following command to install Iblocklist2ipset.

The code is as follows:

$ sudo pip install Iblocklist2ipset

In some distributions such as fedora, you may need to run:

The code is as follows:

$ sudo python-pip install Iblocklist2ipset

Now go to iblocklist.com and crawl any URL to a peer-to-peer list (such as the "Level1" list).

Paste the URL into the following command.

The code is as follows:

$ iblocklist2ipset Generate

--ipset banthis "Http://list.iblocklist.com/?list=ydxerpxkpcfqjaybcssw&fileformat=p2p&archiveformat=gz"

> Banthis.txt

After the above command runs, you will get a file called Banthis.txt. If you look at its contents, you'll see something like these:

The code is as follows:

Create Banthis hash:net family inet hashsize 131072 Maxelem 237302

Add Banthis 1.2.4.0/24

Add Banthis 1.2.8.0/24

Add Banthis 1.9.75.8/32

Add Banthis 1.9.96.105/32

Add Banthis 1.9.102.251/32

Add Banthis 1.9.189.65/32

Add Banthis 1.16.0.0/14

You can use the following Ipset command to load this file:

The code is as follows:

$ sudo ipset restore-f banthis.txt

You can now view an automatically created IP set:

The code is as follows:

$ sudo ipset list banthis

In writing this article, the "Level1" Class table contains 237,000 screened IP lists. You can see that a lot of IP addresses have been added to the IP set.

Finally, create a iptables command to block these villains!

Summarize

In this article, I describe how you can use powerful Ipset to mask unwanted IP addresses. Combined with Third-party tool iblocklist2ipset, you can smoothly maintain your IP mask list. Those who are curious about the performance improvement of Ipset, the following figure shows the benchmark results for iptables using and not using Ipset (note the time axis).