Identification of Layer7 data streams (Connection Tracking) in Linux Streaming Server Load balancer

Identification of Layer7 data streams (Connection Tracking) in Linux Streaming Server Load balancer
1. the nf_conntrack that supports Layer7 does not really need to be destroyed. After that, you will feel that you need to change "data streams based on quintuple" to "data streams with fixed offsets based on application layer protocols, the sooner the better! Therefore, this person added several fields for nf_conn in the Linux 3.17 kernel that supports zone conntrack: bool l7; // Boolean, indicating whether to perform layer7 matching. U32 offset; // The offset u32 offlen of the application layer Stream ID; // the three fields above the length of the application layer Stream ID are set in the CT target, and zones are also set, it indicates that all data packets belonging to the zone $ id use the fixed-length stream identification defined by the application layer fixed offset to identify a stream, instead of using the traditional quintuple to identify a stream. Redefine tuple and add a bool l7 to indicate whether it is the stream ID at the application layer. At the same time, add an array sid of MAX_IDLEN length, which means that the maximum Stream ID is MAX_IDLEN bytes. The above is the basic data definition, so it is not difficult to modify the code logic. It is mainly to modify the resolve_normal_ct function and retrieve the l7 in the tmpl template. If it is not 0, it indicates that the "Application Layer Stream ID" is required to identify the stream. In this case, the [iphdr + iphdrlen + transphdrlen] location is located based on the offset and offlen fields to retrieve the offlen bytes of data, calculate the hash value as the key of the hash calculation. Before _ nf_conntrack_find_get, tuple is filled into the sid of the application layer and the l7 of tuple is set. This means that when you find conntrack, compare the tuple sid value instead of the quintuple. Finally, at the time of conn confirm, the sid indicated by the payload information indicated by the conntrack according to its offset and offlen (it has been put into the tuple structure and its char sid [MAX_IDLEN]; field. It took less than two hours to modify, compile, and test (the iMac I bought was so powerful !!). Casual, play, eat something, drink tea, and get started. This person is me! After thinking about the meaning of what you do, it is also a reflection process! I suddenly found that everything I did was meaningless. The conntrack struct does not store any information for the application layer. Although I have extended it, it can store many things, such as routing, socket, etc, but there is actually nothing to use, that is, these are all things that you have nothing to play. The most important information stored in conntrack is NAT information, that is, tuple information. This tuple is based on the traditional 5-tuples, if I use the application layer Information Based on sessionID to identify a tuple, what should I do with NAT? If the IP address of the client changes, even if the sessionID remains unchanged, NAT still needs to be re-implemented, but it still has no benefit. My intention is to save the series of re-operations due to changes in the IP address and port, but it is still not saved because the IP address and port are changed, the IP address and port information must be modified or modified again. If the above Code is written on paper, it is clear that I will tear it apart and then throw it into the trash can... 2. the reuseport supporting Layer7 arbitrary payload hash calculation is a powerful Linux kernel that supports UDP reuseport. This mechanism can be used to provide UDP Load Balancing services, if you do not know about it, You can bing it. It can perform load balancing, that is, it calculates a fixed hash through a fixed 5-tuples, and then distributes a packet to a fixed socket based on this fixed hash, if the IP address does not change, everything will be fine, but the IP address will change in the mobile environment, which means the 5-tuples information has changed, the re-calculated hash will also change (if it is not changed, it will be a collision !), This means that the next UDP packet sent by the IP Client may be distributed to another socket, which is not expected in the UDP-based persistent connection service. The core code of _ udp4_lib_lookup is as follows:

Begin: result = NULL; badness =-1; sk_nulls_for_each_rcu (sk, node, & hslot-> head) {// hash calculation of sessionID, the server should not authenticate sport/saddr! Score = compute_score (sk, net, saddr, hnum, sport, daddr, dport, dif); if (score> badness) {result = sk; badness = score; reuseport = sk-> sk_reuseport; if (reuseport) {// stream version of 5 tuples, calculate a hash value based on 4 tuples // hash = inet_ehashfn (net, daddr, hnum, saddr, htons (sport); // sid stream version, which is calculated based on sessionID. // The problem is how to upload this sid... overhaul hash = sid_based_hash (sid,); matches = 1 ;}} else if (score = badness & reuseport) {matches ++; // check whether the last matched sk is replaced by this sk. The hash value affects if (u64) hash * matches)> 32 = 0) {result = sk;} hash = hash * 1664525 + 1013904223;}/** if the nulls value we got at the end of this lookup is * not the expected one, we must restart lookup. * We probably met an item that was moved To another chain. */if (get_nulls_value (node )! = Slot) goto begin;

 

The comment mentions overhaul, which means that I must upload a skb here to obtain the sid Based on the reuseport flag of setsockopt, the sid offset, and the sid offlen, and then calculate the hash, but this repair is very easy. Just re-compile the kernel. It is nice to use sessionID in UDP reuseport to identify a stream, because at this time the data has been transferred to the transport layer, in addition to the re-encapsulated data packets, basically all of them reach a UDP Service on the local machine, and the data packet has arrived here, which means that the identification of 5-tuples, such as NAT, has passed completely. The next step is to send data to the application layer, at this time, we can identify a stream based on the sid of the application layer to ensure that even if the client IP address changes, the request can reach the same UDP Service thread... this also provides a good real-world solution for the mobile age. In the age of frequent replacement of quintuple groups, how can we keep the application layer open...