Technical Essentials of Network programming __ programming

First Salute to Cloud wind, his Skynet gave me a lot of inspiration. The core of Skynet is message management in a multithreaded environment, how to get messages to be safely and efficiently delivered from one service to another.  Service runs in a thread pool. Skynet implements a lightweight and efficient network module. I rewrote this section in C + + and made some changes to make it clearer and easier to use.  This article describes the implementation of this network module. It is based on Skynet v1.0.0-alpha10 (2015-8-17). I try not to stick the code, just to describe it in words.
Overview The network module does not start a new thread inside, it runs in the main thread of the program. Since network IO is usually not a hot spot for performance, a thread is feasible. It should be noted that if all features are handled in one thread, there are some pitfalls (as mentioned below), and usually we need to have a single thread for the network layer. The network module provides several interfaces for listening to network addresses, initiating connections, and disconnecting. Send data. These interfaces are non-blocking. They are also thread-safe and can be called with multiple threads.  The network module provides a message pump that pops up the status of the network layer and receives the data. The user can process the data in the main thread according to the specific situation, or it can be handled with a new worker thread.
Epoll uses Epoll to do multiple IO interfaces. Epoll uses the default horizontal trigger mode.  Use the Epoll class to encapsulate epoll operations. A Epoll object has only one data member, that is Epoll FD. It has these several interfaces: Create, destroy, delete fd add need to monitor FD: Add while listening to this FD read event. Modify the type of event that an FD needs to monitor: Usually we always need to listen for read events, and read events to determine whether or not to listen for parameters. Get event information: Returns an array of events, each of which is a struct whose structure contains information about which FD has read/write events.
The simple socket uses a socket class to encapsulate the socket FD. Each connection corresponds to a socket object. The socket object must contain an FD. Because we require non-blocking write data, the socket object also has a writebuffer. This is the simplest case.
Socketmanager in practice, network connections are frequently established or disconnected. It is obviously inefficient to instantiate and then destroy the socket object. An improved approach is to put unwanted socket objects into the pool, take them out of the pool when needed, and continue to use them after initializing the internal data. This method causes the number of socket objects to gradually increase, and then stabilize in a certain number. Another method is to statically instantiate a large enough array of socket objects at the beginning. The size of the array can be easily estimated.   We swapped some memory for a stable structure. I'm using the second method.
The implementation of Socketmanager is simple. It provides an interface to return a socket that is not being used. It is important to note that this interface must be thread-safe.
Improved sockets to support Socketmanager, a simple socket class requires some extra information. ID: Used to differentiate between other socket objects, actually it equals the object's subscript in the array. State: The status of being used, there are actually many other state user data: The user can save some information in it. For example, which module (service) The connection belongs to.
Socket Write Cache This cache is used to temporarily record the data that needs to be sent, and when FD is writable, send them. To avoid the overhead of string copying, we agreed that the application tier is responsible for allocating memory for the data that needs to be sent, and that the caching of the network layer only holds the pointer passed over. After the data is sent, the network layer releases the memory.  This is an important design. The cached internal data structure is an advanced first out queue (linked list implementation). Each piece of data corresponds to a node.
Record three data per node: Data address, data length, original data address
The process of sending data is this: the application layer needs to send data, socket ID, buffer, size network layer to find this socket, if its write cache is empty, send directly (non-blocking).  If the data is all sent out, the operation completes. If the data is not sent out, or the cache is not empty, the buffer, size, offset (possibly some of the data has been sent) is logged to the cache. The network layer is asynchronous, when FD is writable, sends the header node's data, according to the sending size, maintains the offset. If a node's data is sent out, release the buffer of the node. Then delete the node.
The read cache Skynet for the socket does not implement read caching for the socket.  Whenever the socket receives data, the network layer's message pump transmits the data. This is because the network layer does not involve logic, and it does not know how to parse the data. Just as sending the data, the network layer simply sends a block of memory, which does not know what data is in the memory block, and how the data is organized. Allowing the logical layer to parse and process the data is a broader design for the application surface.
Using pipes to sync root real-world situations, you can allow all tasks in the server to be Chengri in one line. But more often, in order to reduce coupling, using multiple CPUs, the network module runs in a separate thread.  Users of the network, however, may come from multiple threads. Therefore, we need to synchronize the application layer to the network layer operation. A thread-safe message queue can perform this task well. There is also a more concise approach, that is, the use of pipelines.  Use pipelines as a bridge between the application layer and the network layer. The user writes to the pipe, the pipeline guarantees that the operation is synchronized, and the network thread reads the operation from the pipe and executes it. The pipeline greatly reduces the complexity of the code.
The message instructions at the network layer are very brief, and the vast majority of instructions are only more than 10 bytes. The instruction that sends the data is the data pointer, very short. The pipeline guarantees that 4096 bytes of data are written to the pipe one at a time, all atomic operations. We don't have to worry about sync. On the other hand, the pipeline's cache space is 64K. Because the network layer can quickly process instructions, this capacity is sufficient. Considering the worst-case scenario, the pipe cache is full, and the operation of the write pipe blocks. Wait until the instructions in the pipe are processed to make room for the data to be written again. So the instructions are not lost.
Consider a special case: the write pipe operation and the network thread are the same thread, and the pipe is full.  Because the write operation is blocked, the network thread cannot process the instruction, and the system falls into a deadlock. Therefore, the application layer and the network layer preferably belong to different threads.
Listening process I use the logic flow of the listening operation to illustrate how the application layer and the network layer fit together. Operations that begin with use belong to the application thread (code may be at the network layer), and operations that begin with net belong to the network thread.
Use. Call the Listen function. Use. Create socket FD, bind network address, start listening. Use.  To reserve a socket object that has a status of free. Use. Write a listening instruction to the pipeline, including socket FD, Socket ID. Use. The Listen function returns the Socket ID.
Net. Read the listener instruction from the pipe. Net. According to the socket ID, locate the socket object, initialize it, and place its state as Plisten.
Use. Call the start function, and the socket ID that is returned by the Listen function is use. The start instruction writes to the pipeline.
Net. Read the start command from the pipe. Net. Locate the socket object and add the socket FD to the epoll. The socket state is placed to listen. Net. Handle connection events.
Process end.
The initial state of the socket's state socket free, indicating that it was not in use. If you want to use a socket, the state changes from free to reserve, indicating that it has been booked, do not know what to do.
If you want to listen with a socket, the state changes from reserve to Plisten, and it is not added to Epoll at this time. After the system initialization completes, the network starts, the socket FD adds to Epoll, the state changes from Plisten to listen. It can handle the connection request.
If there is a connection request, the state of a socket is changed from free to reserve, then to Paccept, and it is not added to Epoll at this time. If the application layer decides to accept this connection, add the socket FD to Epoll, and the state changes from paccept to connected. It can send and receive data.
If you want to use a socket to initiate a connection. Change the state of a socket from free to reserve, initiate the connection and add FD to the Epoll. The state is connecting, the connection succeeds (three times handshake completes), the state becomes connected. It can send and receive data.
If you want to close a socket, the cache has data, the state first set to Halfclose, after the data sent, set to free. The halfclose socket will not read the data again while the state is in, but will continue to send the cached data out.
Message pump like pumping water from the pool, the application layer loops call the Msgloop function to extract information from the network layer. If the loop code executes in a separate thread, the thread is a network thread. The message pump throws these messages: Data:socket receives data, returns data pointers and lengths, and releases the memory that the pointer points to when the application layer is used. Close: Connect off Open: Start listening or connection setup. Accept: There is a connection request. Exit: Network layer exits.
When exception handling initiates a connection, we set the socket to non-blocking and call the Connect method, which returns an error code immediately. If the error code is einprogress, the connection is in progress (three times handshake). We added FD to the Epoll. When the connection is complete, epoll emits an event. If it is another error code, the connection fails. Terminates the connection operation.
When reading and writing data to the pipe, if return eintr, the operation is interrupted by the system, need to write again. If it is a different error, print an error log.
When writing data to the socket, if return eintr (interrupted), you can immediately rewrite, if returned Eagain (blocked), write later. If it is a different error, print an error log, and then disconnect.
When reading data to the socket, if return eintr (interrupted), reread, if returned Eagain (blocked), print an error log, reread. Disconnect if it is a different error.
Network constants These constants can be adjusted according to the specific situation.

When creating a epoll, tell the kernel how much the number of listening, corresponding Epoll_fd_max = 1024;
When the socket listens on the network address, it needs to specify the size of the half connection queue, corresponding to Listen_backlog = 32;
When reading data from the socket FD, because we do not know beforehand how much data will be read, in the dynamic application of memory, the initial application size is a constant: Min_read_buffer = 64; The read_buffer_size of each socket is independent and dynamically adjusted based on the size of the previous data: one or two times larger.
Because the socket objects are statically initialized, they are larger in number than Epoll_fd_max, and we want to quickly find a free State socket object. So they have a lot of numbers: Max_socket = 65536;