Linux protocol stack Search Algorithm Optimization

Linux protocol stack Search Algorithm Optimization
The implementation of Linux's network protocol stack is accurate but not exquisite. You don't have to mention Netfilter. TC alone is enough, but there are several serious injuries. This article makes an incomplete record, which should be a breeze, do not take it seriously. 0. The Linux protocol stack of the search type is implemented as a pure software and the hardware interface is retained, but hardware is not involved in this article. 0. 1. No creation is found. If no route entry is found, a failure is returned and the packet is discarded. For such searches, the creation and deletion of table items are triggered by specific events (such as manual configuration and nic up/down), rather than automatically. The success and failure of the search results consume the same performance. Different protocol stacks treat the Success and Failure Methods Differently. Therefore, this article does not focus on such searches. 0. 2. if the query fails, a new table item is created, such as conntrack and neighbor query, therefore, the impact of search result Success and Failure on performance is completely asymmetrical. If the search fails, the performance loss is huge. Even for an efficient hash algorithm, At least you need to traverse the conflicting linked list specified by a specific hash value to find the failure, on average, this is already a huge overhead, and the failure is discovered. This is the beginning. Next we need to allocate memory and create table items, which is a huge expense, it consumes both time and space. Although space consumption is inevitable, I want to find the failure at the fastest speed before I have to allocate memory to create table items. 0. 3. the TCP socket lookup between 0.1 and 0.2 is between 0.1 and 0.2. For the Listen status socket lookup, the goal is to create a customer socket, but first, it must ensure that the specific TCP tuples are not found in the socket in the ESTABLISHED status or TW status. If a large number of TW sockets exist, it will take a lot of time to prove that "no one of so many TW sockets matches it ". It would be nice to explain this quickly. Next, I will analyze the search methods in Several Linux kernel protocol stacks in less detail. 2. nf_conntrack provides a lot of space for Linux nf_conntrack optimization. tests show that the kernel protocol stack added to the conntrack will be halved in the case of full load PPS (Packet Per Second, the maximum number of persistent connections dropped by half. For short connections, even if the time of each timeout is set very short, the performance will also decline significantly. 3. the same is true for query similar to the cache. For example, the query for the routing cache, we know that the routing cache has an expiration time. If a router has too many transit traffic, there will be a large number of routing items to be cached, and searching for the cache itself is a huge overhead. There is a high possibility of hash conflicts, and it has not been found after such a great deal of effort, you have to go to the slow path! 4. ipset lookup is similar to the lookup of table items in ipset. Today, the gap between seeing a doctor in the hospital suddenly found that ipset of version 6.23 has the timeout parameter, and supports the timeout time itself to do a lot of things, logic processing is a lot automated, but the protocol stack does not know whether a table item has been deleted due to expiration. In my opinion, even if the timeout parameter is not carried, it would be good if we can quickly determine that "not in set". Of course, we cannot clearly determine that "not in set", and then perform searches for specific data structures, such as hash and tree searches. 5. in the above Article, the Bloom filter eventually expressed a desire, that is, to find a failure in the search as soon as possible, so that you can directly do the right thing, rather than spending time on something that will inevitably fail, this price may be sufficient for chimney Spam like OpenWRT, but it is absolutely not enough for Linux to be on the elegant hall. Of course, almost all operating systems implement the same protocol stack as Linux. 6. Is it better to apply the Bloom filter to deploy a Bloom filter before the search algorithm described in the preceding section 2-5? If the Bloom algorithm is well designed, in most cases, if 0 is returned, I can directly jump to the logic of the Creation operation, saving a lot of Traversal Time. If 1 is returned, exact matching is still required. This is equivalent to adding a Bloom filter on the basis of an algorithm that I thought was not good at, and the situation deteriorates. But this is the price! This is an adventure! I can push the responsibility to "This Bloom algorithm is not well designed"! On the other hand, we need to weigh the overhead of computing N hash and the overhead of adding traversal to computing 1 hash. At this time, we should not simply analyze the time complexity, because for Bloom, if N is determined, so the time complexity is undoubtedly O (1). Is it better than the hash table? In fact, we should take the weighted statistical value, which relies on rigorous performance stress testing. 7. Hierarchical hash Lookup is similar to the idea of page table lookup in MMU and the idea of the routing lookup algorithm in BSD. It uses multi-layer hash Lookup instead of a single hash plus conflicting linked list traversal.