The first tribute to the cloud, his Skynet gave me a lot of inspiration. The core of Skynet is message management in multi-threaded environment, how to make the message safely and efficiently pass from one service to another service. Service runs in the thread pool. The Skynet implements a lightweight and efficient network module. I rewrote this section in C + + and made some changes to make it more structured and easier to use. This article describes the implementation of this network module. It is based onSkynetv1.0.0-alpha10 (2015-8-17).I try not to stick to the code, only to describe it in words.
OverviewA new thread is not started inside the network module and it runs in the main thread of the program. Because network IO is usually not a hot spot for performance, a thread is feasible. It is important to note that if all features are handled in one thread, there are some pitfalls (mentioned below), and usually we need a single thread for the network layer. The network module provides several interfaces for listening to network addresses, initiating connections, and disconnecting. Send data. These interfaces are non-blocking. They are also thread-safe and can be invoked with multiple threads. The network module provides a message pump that pops up the status of the network layer and receives the data.The user can process the data in the main thread depending on the situation, or it can be processed with a new worker thread.
EpollUse Epoll to multiplex IO interfaces. The epoll uses the default horizontal trigger mode. Use the Epoll class to encapsulate the epoll operation. A Epoll object has only one data member, which is Epoll FD. It has these several interfaces: Create,destroyed,Delete FDAdd an FD that needs to be monitored: Add a read event that listens to this FD at the same time. Modifies the type of event that an FD needs to monitor: Usually we always need to listen for read events, and read events to determine whether listening is required based on parameters. Gets event information: Returns an array of events, each of which is a struct containing such Info: Which FD has a read/write event.
a simple SocketUse a socket class to encapsulate the socket FD. Each connection corresponds to a socket object. The socket object must contain a FD. Because we require non-blocking write data, the socket object also has a writebuffer. This is the simplest case.
SocketmanagerIn practice, network connections are frequently created or disconnected. It is obviously inefficient to instantiate and then destroy the socket object. An improved method is to put the unwanted socket object into the pool, take it out of the pool when needed, and then continue to use it after initializing the internal data. This method increases the number of socket objects gradually, Then stabilize in a certain quantity. The other approach is to statically instantiate an array of socket objects that are large enough to begin with. The size of the array can be easily estimated. We replace the stable structure by wasting some memory. I'm using the second method.
The implementation of Socketmanager is simple. It provides an interface to return an unused socket. It is important to note that this interface must be thread-safe.
an improved SocketTo support Socketmanager, a simple socket class requires some extra information. ID: Used to differentiate the other socket object, which is actually equal to the object's subscript in the array. State: Whether it is used or not, there are actually many other state user data: The user can save some information inside. For example, which module (service) The connection belongs to.
write cache for socketsThis cache is used to temporarily record the data that needs to be sent, and to send them when FD is writable. To avoid the overhead of string copying, we agreed that the application layer is responsible for allocating memory for the data that needs to be sent, and that the cache of the network layer only holds pointers that are passed over. After the data is sent, the network layer frees the memory. This is an important design. The cache internal data structure is a FIFO queue (linked list implementation). Each data corresponds to one node.
Each node records three data: data address, data length, original data address
The process of sending data is this: the application layer needs to send data, socket ID, buffer, size network layer to find this socket, if its write cache is empty, send directly (non-blocking). If the data is all sent out, the operation is complete. If the data is not sent out, or if the cache is not empty, the buffer, size, offset (possibly a portion of the data has been sent) is recorded in the cache. The network layer asynchronously, when the FD is writable, sends the data of the head node, maintaining the offset according to the size of the send. If the data for a node is sent , release the buffer of this node. Then delete the node.
read cache for socketsSkynet does not implement read caching for sockets. Each time the socket receives data, the network layer's message pump transmits the data. This is because the network layer does not involve logic, and it does not know how to parse the data. Just as the data is sent, the network layer simply sends a block of memory that does not know what data is in the memory block and how the data is organized. The logic layer to parse and process the data, is a more applicable surface design.
to synchronize with a pipeThe root of the actual situation, you can let the server all the tasks in a line thread processing. But more often, in order to reduce the coupling, the use of multi-CPU,The network module runs in a separate thread. Users of the network may come from multiple threads.Therefore, we need to synchronize the operation of the application layer to the network layer. A thread-safe message queue can perform this task very well. There is also a more concise way to use pipelines.use pipelines as a bridge between the application layer and the network layer.The user writes to the pipeline, the pipeline guarantees that the operation is synchronized, and the network thread reads the operation from the pipe and executes it. The pipeline greatly reduces the complexity of the code.
The message commands on the network layer are very brief, with the vast majority of instructions only more than 10 bytes. The instruction that sends the data passes the data pointer, which is very short. The pipeline guarantees that the data that is written to the pipe within 4096 bytes at a time are atomic operations. We don't have to worry about syncing. On the other hand, the buffer space for the pipeline is 64K. Because the network layer can quickly handle the instructions, this capacity is sufficient. Consider the worst case scenario where the pipeline cache is full, and the operation of the write pipeline is blocked. Wait until the instructions in the pipeline are processed, make room, and the data is written again. So the instructions are not lost.
Consider a special case: The Write pipeline operation is the same thread as the network thread, and the pipeline is full. Because the write operation is blocked, the network thread cannot process the instruction, and the system is locked into a deadlock. Therefore, the application layer and the network layer are best to belong to different threads.
monitoring ProcessI use the logic flow of the listening operation to illustrate how the application and network layers work together. Operations that begin with use belong to the application thread (the code may be at the network layer), and the operations that begin with net belong to the network thread.
Use. Call the Listen function. Use. Create a socket FD, bind the network address, and start listening. Use. Set a socket object with a state of free to be a mutex. Use. Writes a listening instruction to the pipeline, and the command parameters include the socket FD, Socket id.use. The Listen function returns the Socket ID.
Net. Read the listening instruction from the pipeline. Net. Based on the socket ID, locate the socket object, initialize it, and set its status to Plisten.
Use.call the start function, passing in the socket ID returned by the Listen functionUse . Start the command write pipeline.
Net. Read the start command from the pipeline.Net. Locate the Socket object and add the socket FD to the epoll. Socket status is reset tolisten.Net. Handles connection events.
The process is over.
the status of the Socketthe initial state of the socket is free, indicating that it is not being used.If you want to use a socket, the state is changed from free to reserve, indicating that it has been scheduled and does not know what to do.
If you want to listen with a socket, the state changes from reserve to Plisten, and it is not added to Epoll at this time.after the system initialization is complete, the network starts, the socket FD is added to the Epoll, and the state changes from Plisten to listen. It can handle connection requests.
If there is a connection request, the state of a socket is changed from free to reserve and then to Paccept, when it is not added to Epoll.If the application layer decides to accept the connection, add the socket FD to the Epoll, and the state changes from paccept to connected. It can send and receive data.
If you want to use a socket to initiate a connection. Change the state of a socket from free to reserve, initiate a connection and add the FD to the Epoll.The status is connecting, after the connection is successful (three handshake complete), the status becomesconnected. It can send and receive data .
If you want to close a socket, the cache has data, the status is first set to Halfclose, after the data is sent, set to free.state, the halfclose socket will not read the data again, but will continue to send the cached data out.
message Pumplike a pump pumping water from a pool, the application layer loops through the Msgloop function to extract information from the network layer.if the loop code executes in a separate thread, the thread is the network thread.the message pump throws these messages:Data:socket receives data, returns the data pointer and length, and releases the memory that the pointer points to when the application layer is finished.Close : Connection closedOpen: Start listening or connection setup.Accept: There is a connection request.Exit : The network layer exits.
Exception HandlingWhen the connection is initiated, we set the socket to non-blocking and then call the Connect method, which immediately returns an error code. If the error code iseinprogress, indicating that the connection is in progress (three handshake). We added FD to Epoll. When the connection is complete, epoll issues an event.if it is a different error code, the connection failed. Terminates the connection operation.
when you read and write data to the pipeline, if you returnEintr, indicating that the operation was interrupted by the system and needs to be re-written.if it is a different error, print an error log.
when you write data to the socket, if you returneintr (interrupted), can be rewritten immediately,If you returnEagain (blocked), write again after a while.if it is a different error, print an error log, and then disconnect.
when reading data to the socket, if return eintr (interrupted), re-read,If you return Eagain (blocked), print an error log and read it again.If it is a different error, disconnect.
Network Constants These constants can be adjusted according to the specific situation.
When creating the Epoll, tell the kernel how many listeners are listening, and the corresponding Epoll_fd_max = 1024;
When the socket listens to the network address, it needs to specify the size of the half-connection queue, correspondingListen_backlog = +;
When reading data from the socket FD, since we do not know how much data will be read, in the dynamic application of memory, the size of the initial application is a constant:Min_read_buffer = 64; The read_buffer_size of each socket is independent and dynamically adjusts to the size of the previous data: increase by one or two times.
Because the socket object is statically initialized, the number of them is greater than Epoll_fd_max, and we want to be able to quickly find a socket object with a free status. So they have a lot of numbers:Max_socket =65536;
Technical Essentials of Network programming
