Analog Millions TCP concurrency

Http://www.d1net.com/server/news/451583.html

What is TCP concurrency

TCP concurrency refers to the number of connections that a server "hold" at the same time, specifically the number of TCP connections that the server sees in the "established" state. The current server TCP status statistics report is available through Netstat-n|grep ^tcp|awk ' {print $NF} ' |sort-nr|uniq-c, and the following figure is my execution result (I'm connecting this machine via SSH so there's a " Established "State of TCP connections)

To test TCP concurrency is to reach the peak of this value, to achieve this must meet two points: in a short period of time to construct millions connection server side hold live millions connection

It should be noted that the above "test" does not include "connected interactions" simply refers to "hold connection".

Why can't traditional tools meet

Many servers are TCP structures such as MySQL, Tomcat, Nginx, these tools also have a corresponding pressure testing tools, more famous include: Jmeter, Tsung. The implementation of these tools is basically consistent while starting multiple tasks each task opens a socket to connect to the server

This test method is limited to the number of tasks (number of threads or processes) that can be started by three resources (a file descriptor) that is limited to the maximum number of ports available on the native port--65535

The first restriction can be solved by means of a "coprocessor" technique; the second limit can be resolved by adjusting the system parameters when memory is satisfied (refer to my "you really Know" Too many open files); The third limit is almost fatal-- Traditionally, only multiple servers can be used together.

Even if the above three problems have been solved, it is difficult to create a huge pressure in the "short time", a large number of sockets will eat up the memory, multiple server synergy is necessarily a distributed problem (think about the U-turn hair).

New Way of thinking

The feeling of TCP connection is a "channel", which is actually an "illusion". All networks are basically based on "store-and-forward" passes. The TCP connection after the three handshake reaches the "established" state, and the server reserves the resource for it-even if the client has stopped ignoring the connection. So can we not go through the TCP/IP protocol stack directly through the raw socket construction three times handshake it? As long as we build a large number of three handshake can be a huge pressure on the server.

We focus on the two arrows of Client->server. The first packet is a SYN packet, a seq= random number, and the second packet is an ACK packet, ack= receives the packet's ACK from the seq+1,seq= receiving the packet. In fact, there is no direct relationship between TCP packets, we received a packet can directly calculate the reply packet ack, seq

The above idea basically can prove our method is theoretically feasible, in practice we also need to overcome some problems how to obtain server to client's Syn+ack (three times handshake the second arrow); After all, we are not using the socket to open the TCP connection directly ( This does not require you to construct TCP three-time handshake. Server will attempt to ARP address client after receiving client request, if found unable to resolve it is considered an illegal packet send RST packet to close TCP connection

The first problem is that we get the raw packets directly via the Libpcap "bypass" kernel. The following diagram is the principle of libpcap

The BPF (Berkeley Packet Filter) drive is used at the bottom of the libpcap, and this is the driver specifically used to "debug" in kernel, which is now standard for all operating systems ( As long as the tcpdump is supported then the underlying must have implemented this drive model. It is independent of other protocol stacks in kernel directly and reads data link layer packets.

By Libpcap we can get all the packets (even if the operating system does not) and then construct our own packets to be written directly to the data link layer through the raw socket. The entire "->"-> "package" does not require kernel participation at all.

The second problem is that I've already given the answer in the previous article--Construct and reply to the ARP packet ("Deep understanding ARP Attack"). In short, it is through the libpcap to get ARP request, through the raw socket reply ARP response.

Do it

I'm trying to get rid of all the irrelevant stuff and keep only the most streamlined parts, less than 300 lines of code. The code is divided into two parts, "Initiating TCP SYN Packets" and "replying to syn+ack packets, ARP request packets." The specific content can be seen here

This time I put the CMake file deliberately, execute the following cmake can be compiled.

Https://github.com/fireflyc/million-tcp-client

Simulate multiple IP addresses

Only 65,535 TCP connections can be emulated by a single server, limited to the port limit. But I provide the demo program is can specify the IP address, this IP address only need and target IP in the same network can be. For example, my test environment:

The test machine has its own IP address 172.16.46.128, but this IP address is not used, just for the convenience of my SSH connection. The IP address of the server is 172.16.46.133, I start three tcp-client to bind 200, 201, 202 respectively.

Each tcp-client process can simulate 65,535 TCP connections. (This actually has room for improvement)

Correct the "Do you really know" Too many open files? "One of the flaws in this article is that the limit of TCP connections that a server can reach is not limited to memory but--1048576. The upper limit of the soft nofile is 2^20=1048576, so the maximum TCP concurrency that a server can theoretically achieve is--1048576.