BT source code learning experience (5): Unified Network Service Interface-rawserver

BT source code learning experience (5): Unified Network Service Interface-rawserver
Author: Wolfenstein

Network services will be required in the future (the generation of seed files can be completed locally, but the actual content downloaded from these seed files and the network is required to provide the tracker service ), in the BT program design, a unified interface is provided for the network service. In this way, when other parts of the program need to open a network service, you only need to register with this interface, and provide the corresponding processing object (handler). When a network event occurs, it will automatically process the related functions in the processing object.
This unified network service interface is defined in BitTorrent/rawserver. in py, it calls the library related to the network socket. In addition, rawserver also provides the add_task function, which allows some tasks to be postponed.
During initialization, rawserver can pass in a doneflag parameter from outside, which is the data type of an event and can be triggered elsewhere, in this way, the master loop (in listen_forever) in rawserver can be interrupted at any time ). In addition, some internal variables are also set. Finally, it adds a task to itself, scan_for_timeouts, which regularly checks the network connections that have timed out and closes them.
We can see that add_task calculates the actual execution time of the task to be delayed and adds it to a sorted list (funcs ), and keep the list in order. The list is in the order of actual execution time.
When other modules want to provide network services, it first calls the create_serversocket function of rawserver. This function will return a socket object and the socket is already in the listen state. Of course, if an external network connection comes in at this time, there will still be no action, because the corresponding processing object has not been registered.
Next, call the start_listening function. The function adds the network plug-in and its corresponding processing object to a dictionary. The dictionary uses the network plug-in Descriptor (FD) is the primary key. It is worth noting that this function name contains the words listen, but the socket. Listen function is not called here, but has been called in create_serversocket. There is no limit on the type of the passed object to be processed. The only requirement is that it must contain the external_connection_made function, so that when the external network is connected, this function will be called. The data_came_in function should also be provided to process network data and the connection_flushed function should be provided to process the data that has been officially issued (as opposed to the buffer, the following two functions can also not be provided, because in the external_connection_made function, you can relocate the newly connected network data processing object to an object containing the data_came_in function and connection_flushed function.
After the start_listening function is processed, the network plug-in already exists in the serversockets dictionary.
When other modules need to connect to the external network, the start_connection function should be called. This function will add the network plug-in to the other dictionary single_sockets. Of course, the singlesocket object is used for packaging to a certain extent. The following analysis shows that the main function of the singlesocket object is to buffer the output data and write the data to the socket without blocking.
The processing object to be passed in for start_connection is an object that must contain data_came_in but can not contain external_connection_made.
Poll objects are used in start_listening and start_connection. This is a module provided by the system that provides the polling mechanism and uses file descriptors as parameters, you can get the corresponding event (that is, the plug-in corresponding to the file descriptor has data inflows or settings), and both functions call the register function of poll, this facilitates subsequent poll round-robin operations.
Note that after these functions are executed, the network connection will not be processed, because although the corresponding network plug-in is opened, the corresponding processing object is also registered, however, the entire round-robin mechanism has not yet been established. The listen_forever function can be established only after it is called. The body of this function is an infinite while loop. Only the doneflag event can be used to stop this loop. The first thing it does is to find the time of the last task to be executed from the added task funcs, subtract from the current time, calculate the period, and then poll the time with poll, in this way, the external tasks will not be delayed after the round robin ends. The Round-Robin result is returned in events. This is a list, and its elements are binary groups composed of file descriptors and events. The next step is to execute all the external tasks that need to be executed immediately according to the time. The main function of _ make_wrapped_call is to execute the external tasks, but to add some protection code for unexpected processing to them. After executing these external tasks, call _ close_dead to disable inactive network connections. Then, use _ handle_events to process the network events collected by the previous poll.
The main body of _ handle_events is a for loop, which checks every sock and its corresponding event. First, check whether it is in the serversockets dictionary or in the single_sockets dictionary. If it is the former, it is a plug-in the listener, and then check the network event. If it is not an error event, it means that there are external connections, and anyone familiar with socket programming should know that the correct processing method is to create a new socket, and then let the interport In the listener go to accept it, in the future, data should be read and written in the new socket. This is also the case. The new socket is packaged with singlesocket and stored in the single_sockets dictionary because it is the same as the socket created with start_connection, there may be data inflows, and the listening plug-in only needs to process network connections. Next, the external_connection_made function in the previously registered processing object is called and some other related operations are allowed. We noticed that the processing object is transferred to the new singlesocket, of course, the singlesocket processing object can be redirected to other objects in the external_connection_made function.
The following else statement indicates that sock has only one exception in the single_sockets dictionary, namely OS. Pipe. In this case, you do not need to process this event. You can directly continue to process the next event. Then check the event. If an error occurs, close the plug-in. Otherwise, data flow exists, and data flow is nothing more than inbound and outbound. If it is inbound, read the data into a buffer zone and call the data_came_in provided in the processing object for processing. The parameter obtained by data_came_in is the data in the buffer zone, and it does not need
Handle socket and consider possible blocking issues. In addition, because the write operation is encapsulated in singlesocket, that is, if the network is blocked, data will be written to the buffer first, so that S. Write can be called in data_came_in. Finally, if it is data outflow, call S. try_write. This function is also safe to implement.
Finally, check whether the data has been actually sent (flushed). If yes, call the connection_flushed function provided in the processing object to finish the work.
In the future, we can see that various objects have been created in the BT implementation, and these objects have various complex relationships, but all network services, all are implemented through rawserver. Specifically, the rawserver object will only be created, all modules that require network services will register the processing objects of network services to this rawserver for convenient and unified management.
Finally, I found that someone analyzed the BT source code last year using Google search today. I not only lamented my ignorance, but found the current version (4.0.3) there are already some differences with the current version, and I can continue with the idea of reading the source code to provide you with a different perspective, therefore, I decided to continue writing my learning experience, and hope everyone can support it.