Tang Zhiming said Sun, New Day, New day, new day
Kang Yi and Zuo Xinmin
Shiyue Zhou, although the old state has its life Restoration
This is why a gentleman has no need of him.
-- Liji · Emy
In the previous article "anti-DDoS (II): CC attack tool implementation and defense theory", the author elaborated a defense state machine that can be used to defend against DDoS attacks from the application layer, however, this state machine relies on an ip address blacklist and whitelist firewall that can quickly delete large numbers of entries. Currently, we have not found any good open source implementation for our use.
· Implementation solution selection:
Hardware Implementation or software implementation?
In the face of attacks such as a large number of malformed packets, the hardware implementation will be a very good choice, because during this type of packet filtering, the system has very few States to remember (for FPGA and ASIC Hardware Implementation Solutions, the cost of memory elements can never be ignored, and registers and static RAM are very expensive, therefore, when there is little information to be remembered, the speed advantage of the Pure Hardware Solution makes it win over the software solution ).
However, when the state machine needs to process a large amount of memory information, we need to choose a cheap storage-dynamic random memory (such as DDR3 in SDRAM) as the storage medium of the system state machine, to reduce the cost and complexity of the system. At this time, the software implementation is superior. Although the hardware-only implementation speed is much higher than the software method, we also see from the lvs performance test results in the first article "DDoS Attack and Defense (I): Overview, the lvs implemented by the software and used as the server front-end balanced scheduler has excellent performance and is competent for handling massive user requests in the actual production environment. It can be seen that if the design is reasonable, there is no need to worry too much about the performance of the software.
Finally, we decided to use the software method to implement the required ip address blacklist and whitelist module.
· Final system aerial view:
I spent about 20 days using the C language to implement this module. Among them, there are about 2300 lines of core code in the kernel space, and the total number of lines of code in the user space management tool is about 700 lines. The following is a bird's eye view of the system:
· Fripadm, a user space management tool, communicates with frdev modules working in kernel state through ioctl
· Frdev maintains two double_hash_table instances and provides a hook function hanging on NF_INET_PRE_ROUTING. It operates the two double_hash_table instances to implement the ip blacklist and whitelist functions respectively.
· Frdev provides the operation functions of these two double_hash_table instances to the user space through the device-driven ioctl mechanism in the kernel. Our user space management tool fripadm is implemented based on this.
The main data structure of the kernel state and its corresponding operation functions are as follows:
Features of struct fr_ip_hash_array: precise ip address query, fuzzy ip address query, custom hash table length, custom hash function, and rnd as the random number of hash columns input; maintenance of accurate ip address hash table; maintain the linked list of fuzzy ip addresses, and maintain statistics on precise ip addresses and fuzzy ip addresses. ip character description Syntax:/* ips_syntax: RE digit =: [0-9] num =: (digit) {1, 3} atom =: num | (num'-'num) | '*' ip =: atom '. 'Atom '. 'Atom '. 'Atom ips =: (ip '') + * // ret 0 success, otherwise syntax error/" 1-220. *. 100.33 1-220. *. 100.33 1-220. *. 100.33 "struct fr_ip_hash_array method: fr_ip_hash_array_malloc/route records: add an exact ip record; add a fuzzy ip record (in Pointer Reference Mode) fr_ip_hash_array_delete_ip record comment: delete a specified number of precise ip addresses randomly: add an exact ip address or fuzzy ip address fr_ip_hash_array_delete_ip_bystrings fr_ip_hash_array_find_bool to fr_ip_hash_array_find_bool through string expression: check whether the given ip address is within the range of the stored fuzzy ip address or in the hash table of the precise ip address. fr_ip_hash_array_find_ip_bool: Check whether the given ip address is in the hash table of the precise ip address.
· Why is dual hash table buffer used?
Consider the following scenarios:
Scenario 1: DDoS attacks from the application layer often flood into a large number of illegal ip requests in an instant, such as tens of thousands of illegal ip addresses. Therefore, the firewall black/white list function must have at least the following requirements: A large number of data items can be updated in a short period of time, and the system service cannot be paused.
Analysis: if only one global hash table is used, when a large number of data items are added or deleted in a short period of time, for example, thousands of data items, even if multiple read/write locks are used to separate hash tables, competition for shared resources will seriously affect the system response speed. In severe cases, the system may pause or worse, this is intolerable for High-load servers in the production environment.
Solution: Change Time by Space
The dual-hash Table buffer policy is used to compress the competition hotspots of system shared resources to the very short time of master-slave switchover of two hash table pointers, this method can significantly reduce the competition for sharing resources when a large number of data items are updated.
The system query will directly access the fr_ip_hash_array pointed to by the master, and the user's update operation will directly target another fr_ip_hash_array instance pointed to by mirror until the switch_assist_update operation is executed, the master will be "updated" instantly ". This is the main work feature.
For SMP and multi-queue Nic systems, the following policies can be adopted: Most cpu cores are responsible for Processing Kernel softirq tasks, the remaining few CPUs are responsible for updating, switching, and rebuilding the dual-hash table. In this way, the system can quickly defend against attacks.
However, this scheme will make the system need to maintain two hash tables with mutual images, which will increase the read/write burden on the system memory.
The implementation details are as follows:
Struct fr_ip_double_hash members: struct fr_ip_hash_array * master_ptr; rwlock_t master_lock; struct fr_ip_hash_array * pai_ptr; rwlock_t pai_lock;
Struct fr_ip_double_hash method: fr_ip_double_hash_malloc/fr_ip_double_hash_destroy ignore: only insert ip operations targeting mirror cannot exceed limit fr_ip_double _ Hash_mirror_delete_all: delete all ip records in mirror, that is, all precise ip and fuzzy ip records fr_ip_double_hash_switch_mirror_update: replace mirror with the master, update the master to mirror (at this time, the mirror is the previous master) fr_ip_double_hash_rebuild: the dual hash table is rebuilt, and the length of the new hash function, rnd, and hash table can be specified, this will solve the problem of low query efficiency of hash tables and prevent external attacks against hash tables. Of course, after reconstruction, many original ip entries will not be lost. Fr_ip_double_hash_find_bool fr_ip_double_hash_find_bystrings_bool
Hook Function mounted to the Protocol Stack:
At the end of the module initialization function fr_ip_dev_init, that is, when two dual-Hash Table instances (used as blacklists and whitelists respectively) are initialized successfully and the fedev device is registered successfully, it will execute nf_register_hook, mount the specified hook function fr_nf_hook_sample to NF_INET_PRE_ROUTING.
The main processing code of fr_nf_hook_sample is as follows:
if(fr_ip_double_hash_find_bool(double_hash_white_ptr,sip))
return NF_ACCEPT; else if(fr_ip_double_hash_find_bool(double_hash_ptr,sip))
return NF_DROP; else
return NF_ACCEPT;
Among them, double_hash_white_ptr points to the White List fr_ip_double_hash instance, and double_hash_ptr points to the black list fr_ip_double_hash instance. Because fuzzy ip matching is supported, the code above makes the "pass" and "Block" policies for source ip address filtering available.
Ioctl processing function of the kernel space frdev:
Currently, ioctl supports the following operations from the user space:
// Blacklist operation/* in-black ip */# define FR_IP_IOCTL_TYPE_FIND 1 # define FR_IP_IOCTL_TYPE_FIND_BYSTRINGS 2 // * enter the ip character search # define limit 3 # define limit 4 // * # define limit limit 5 // * # define limit 6 // * # define limit 7 // * # define FR_IP_IOCTL_TYPE_MIRROR_DELETE_ALL 8 // * # define limit 9 // * # define FR_IP_IOCTL_TYPE_REBUILD 10 //* # define FR_IP_IOCTL_TYPE_COPY_HASH_STRUCT 11 // * # define FR_IP_IOCTL_TYPE_DUMP 12 // * output distribution and statistical information of the dual hash table # define limit 13 // * a large number of precise ip addresses are added at a time, in binary format, enter # define limit 14 // * // white List Operation/* in-white ip */# define FR_IP_IOCTL_TYPE_WHITE_FIND 101 # define FR_IP_IOCTL_TYPE_WHITE_FIND_BYSTRINGS 102 // * # define limit 103 # define limit 104 // * # define limit 105 # define limit 106 // * # define limit 107 // * # define fr_ip_ioctl_type_white_1__delete_all 108 // * # define limit 109 //*# define FR_IP_IOCTL_TYPE_WHITE_REBUILD 110 // * # define crop 111 // * # define FR_IP_IOCTL_TYPE_WHITE_DUMP 112 // * # define crop 113 // * # define crop 114 //*
For more information about the above functions, see the fr_ip_dev_ioctl_routine function in frdev source code.
User space management tool: fripadm
The first step is to implement fripadm_black_in_exe and fripadm_white_in_exe, which are the tools for managing blacklists and whitelists separately, use shell scripts to perform secondary encapsulation to obtain the user-friendly tools fripadm_black_in.sh and fripadm_white_in.sh.
Fripadm provides the following APIs for C language developers in the user space:
// Double_hash_find_bystrings (int fd, char * str, unsigned int size) for ip blacklist operations // fd where fd is frdev -- file descriptor double_hash_mirror_insert_bystrings (int fd, char * str, unsigned int size) random (int fd, char * str, unsigned int size) double_hash_mirror_delete_bystrings (int fd, char * str, unsigned int size) double_hash_mirror_delete_bins (int fd, char * str, unsigned int size) random (int fd, unsigned int size) double_hash_mirror_delete_all (int fd) random (int fd) double_hash_rebuild (int fd, unsigned int modular, unsigned int rnd) double_hash_dump (int fd) // double_hash_white_find_bystrings (int fd, char * str, unsigned int size) for ip address whitelist operations) // fd where fd is frdev -- file descriptor double_hash_white_1__insert_bystrings (int fd, char * str, unsigned int size) double_hash_white_1__insert_bins (int fd, char * str, unsigned int size) round (int fd, char * str, unsigned int size) double_hash_white_mirror_delete_bins (int fd, char * str, unsigned int size) round (int fd, unsigned int size) double_hash_white_mirror_delete_all (int fd) double_hash_white_switch_mirror_update (int fd) double_hash_white_rebuild (int fd, unsigned int modular, unsigned int rnd) double_hash_white_dump (int fd)
For specific implementation of fripadm_black_in_exe, fripadm_white_in_exe, fripadm_black_in.sh, and fripadm_white_in.sh, see frdev source code.
Final system test:
After the frdev device is compiled and installed according to the process described in README, you can perform the following tests:
The data packet that was originally dropped by the black, after the white ip entry is updated, it is ACCEPT by the white, and the red line shows the icmp_seq range of the data packet to be truncated.
The frdev statement ends here.
Recently, when I read the white hat Web security book, I found Yahoo's System yahoo Detecting System Abuse used to defend against DDoS attacks at the application layer. Yahoo applied for patent protection for this System. The following describes the system:
(Patent n0.: US 7,533,414 B1Source http://patentimages.storage.googleapis.com/pdfs/US7533414.pdf)
A system continually monitors service requests and detects service abuses.
First, a screening list is created to identify potential abuse events. A screening list contains des event IDs and associated count values. A pointer implements ically selects entries in the table, cing advanas events are already ed.
An incoming event ID is compared with the event IDs in the table. if the incoming event ID matches an event ID in the Screening list, the associated count is incremented. otherwise, the count of a selected table entry is decremented. if the count value of the selected entry falls to Zero, it is replaced With the incoming event.
Event IDs can be based on properties of service users, such as user identifications, or of service request contents, such as a search term or message content. the screening list is analyzed to determine whether actual abuse is occurring.
The general idea is as follows:
The system maintains a filter table to obtain the user's request frequency to determine whether the table has a service abuse, and then takes relevant measures, such as BLOCK.
This idea of defense is similar to the defense state machine we previously proposed. I think this is inevitable.
As mentioned in the previous article, one of the main causes of DDoS attacks is the openness of network services. We cannot solve this problem from the lower layer (because service availability is the first requirement ), it can only be analyzed from the upper layer.
The application layer is already at the top of the protocol stack. Therefore, to defend against DDoS attacks at the application layer, you can only search for solutions at the application layer. Therefore, in this case, in addition to statistical analysis, is there a better way?
In the process of implementing frdev, many problems are solved through the Internet,If you need the source code of frdev, leave the email below :)
If a partner wants to enhance the performance togetherFrdev features are welcomed :)
Mailing: yunthanatos@163.com