Network troubleshooting: Consider the Server Load balancer System

Network troubleshooting: Consider the Server Load balancer System

It may be difficult to eliminate network problems. If a Server Load balancer system is added to the network, it will pose another challenge. It may not be easy to identify whether the Server Load balancer system is losing data packets, changing data packets in some way, or adding more latencies. However, you can use some tips to make it easier for you to discover problems.

The first step in troubleshooting is to check the statistics of the corresponding network components. However, if the statistics show that everything is fine, but network problems still occur, you have to introduce the Swiss army knife in the troubleshooting field-data packet analysis. Although there are a lot of outstanding paid products on the market can be used to analyze data packets, but I still prefer the Open Source Tool Wireshark (https://www.wireshark.org/download.html ).

When analyzing questions related to the Server Load balancer system, the first question to be answered is whether the Server Load balancer system is in transparent mode. In transparent mode, the Server Load balancer system transmits the IP address of the original client as the source IP address. In non-transparent mode, the Server Load balancer system uses the virtual IP address (VIP) of the Server Load balancer system to perform Network Address Translation (NAT) for server requests. Non-transparent mode is the most common implementation mode.

Now you are ready to get the tracking file. Ideally, a shard (tap) is inserted for each vertex ). If you do not have a shard, you can use the SPAN (Switch Port Analyzer) or the mirror port on the switch to capture traffic. Alternatively, you can run the tcpdump command on the inbound and outbound ports of the firewall and Server Load balancer system. The key is to capture all the data packets in four locations at the same time and analyze sessions from four different advantageous locations.

After capturing the data, you must find a single session that appears in all four trace files. Generally, you only need to filter the corresponding two IP addresses. However, remember that the Server Load balancer system executes NAT on the server side, so filtering the Client IP address does not apply to traces on the server side.

This problem can be solved by entering Layer 3. You can filter according to the serial number in the TCP header. However, be careful. Wireshark displays the relative serial number by default. In the end, you will encounter hundreds of packets with a serial number of 1. The key is to disable the relative serial number in the TCP Parameter option. As long as you deselect this option, the actual decimal number is displayed, rather than the sequence column number starting from the session. Once you filter out the same serial number in all four trace files, you should have a packet in each file.

If your server Load balancer system creates its own data packets on the NAT end and sends them to the server, the problem arises. The sequence fields are no longer the same from start to end. The best field used in this scenario is the field specific to the application layer. For HTTP, we recommend that you use the Cookie field. For HTTPS, we recommend that you use the Random Bytes field in Client Hello.

Finally, you can analyze the traces of a single session captured in multiple places. First, find the data packet loss phenomenon. In Wireshark's expert analysis language, the lost data packets are labeled as "Previous segment not captured" (uncaptured Previous segments ). This will appear in one or more trace files, but not all trace files. For example, if you see a response from the server on the outbound end (rather than the inbound end) of the firewall Trace, you will know that the firewall has lost data packets.

After packet loss is analyzed, check the TCP handshake mechanism to ensure that TCP options are not tampered with along the way. Windows scaling and Selective Acknowledgements often disappear when a device along the way creates its own data packets instead of transmitting them transparently. The two options are important for throughput and should not be removed.

The last question to be concerned about in the traces is the very high incremental time (delta time ). If the data is captured in four different locations, you can actually view what is being added with latency (if there is something ). Let's take a look at the handshake mechanism. Use the interval between synchronous request (SYN) and Synchronous Response (SYN/ACK) as the benchmark. Let's take a look at the remaining requests and responses left by the firewall inbound end closest to the client.

For request/response combinations with an increment of one second or longer, check each trace step until you find the port that is adding latency. Is it a firewall with a surging processor usage? Or is the Server Load balancer system that is faulty with the NAT table tracked? It may be a server with too many concurrent connections. Check the trace carefully to tell you where there is a problem and where there is no problem.

Setting a packet capture mechanism may take valuable time in network fire suppression, but it can save a lot of time in the long run.

English: Network Troubleshooting: Consider The Load Balancer

This article permanently updates the link address:

1 2 Next Page