EMule source code parsing (2)

(From: http://hi.baidu.com/elseif/blog/item/d94998f9d977e258252df2c1.html)

1. Block Mechanism-ensure the correct transmission of resources

Continue cknowfile Parsing

To speed up content delivery, multipart processing is a simple and effective method. EMule processes each file in blocks. In addition, if the hash value of each part is retained, the validity of the downloaded content can be determined only when a part of the file is downloaded.

When obtaining information about each shared file, eMule performs multipart processing on it. Therefore, if you need to know the multipart processing and Restoration Mechanisms in eMule, check cknownfile :: just implement the createfromfile function. The classes involved in this function and related to block processing and hash computing are all in shahashset. cpp and shahashset. h.

(Note the two member variables in cknowfile: caichhashset * m_paichhashset and carray <uchar *, uchar *> hashlist)

The following describes several main classes:

The caichhash class is only responsible for one hash value and provides basic operations such as direct assignment and comparison between two caichhash classes. Caichhashalgo is a common interface of the hash algorithm. Other hash algorithms can be used as long as they are implemented. This makes it easy to use different hash algorithms to calculate hash values. Caichhashtree is a tree-based hash value organization. It has a member variable of the left and right subtree, and its type is a pointer to caichhashtree. This is a typical method to implement the tree structure.Caichhashset contains a variable of the caichhashtree type, which is directly responsible to cknownfile, representing the block information of a file..The comments section at the beginning of the shahashset. h file explains the way it is segmented.. Here two constants 9728000 and 184320 are used, which are 180 K and K respectively. This is two different partitioning methods in eMule. That is, a large file is first divided into several k blocks and these blocks are organized into a tree structure, then each such block is broken down into several k blocks (52 blocks, plus a K block), which are still organized according to the tree structure. The final structure is still a tree.Cknownfile: The createfromfile method is used to gradually create such a tree when reading the content of the target file.. Caichhashtree: findhash can locate the corresponding branches (a pointer to caichhashtree) based on the offset of the read target file and the size of the next part ). If necessary, the branches are automatically created. Therefore, during the multipart operation, the entire caichhashtree is created when the file is read from start to end, and the corresponding hash value of the multipart is also assigned.

At last, we need to pay attention to the hashlist variable in the cknownfile class. That is to say, it also separately retains the hash value of the md4 Algorithm for all blocks in 9728000 bytes. In this way, there are two sets of block verification mechanisms for a file, which can adapt to the network infrastructure in different scenarios. The network infrastructure Infrastructure (MFC) already has some network infrastructure classification classes, for example, casyncsocket. However, in the design of eMule, in order to develop network-related code more efficiently, other classes are constructed as the infrastructure, the Code of these basic classes also has a high reuse value (note some classes that can be reused here ).

(Some reusable classes for network programming)

The first is the casyncsocketex class. The features of this class have been described in asyncsocketex. h. It
Fully compatible with the casyncsocket class, that is, to replace all casyncsocket in the application with casyncsocketex, the program can still be the same as the original function,
Therefore, it is more convenient to use. However, on this basis, it is more efficient, mainly in the message distribution mechanism, that is, it processes socket-related messages more efficiently than the original MFC casyncsocket class.

In addition, the casyncsocketex class supports dividing a socket into several layers by implementing the casyncsocketexlayer class, which makes it easy to implement many network functions, such as setting proxy, or use SSL for encryption.

In addition, the throttledcontrolsocket class and
Throttledfilesocket class. These two classes only define two interfaces. If you want to implement the speed limit function for any other network socket class, you only need to use the default sending function (such
Instead of sending data, you can cache the data, and then implement throttledcontrolsocket or
When the sendfileandcontroldata or sendcontroldata method in the throttledfilesocket interface is used, the data is actually sent.
Upload speed limit, which requires the uploadbandwidththrottler class for configuration.
Uploadbandwidththrottler is a subclass of winthread, and runs a thread separately. Next time, we will detail how it controls global uploads.
Speed.

Network Infrastructure-Global speed limiter uploadbandwidththrottler

Uploadbandwidththrottler is a global upload speed limiter used in eMule. It inherits
Cwinthread class. When this class is created, a new thread is created and runs independently. When the class is destructed, the corresponding thread is automatically stopped. The target function of this thread is
Runproc, and then to prevent the runproc function from using the this pointer, it uses runinternal to actually complete the work of the working thread. In eMule
There is another class lastcommonroutefinder with a similar structure.
Several socket queues are stored in uploadbandwidththrottler. The processing methods of these queues are slightly different. In standard queue
(M_standardorder_list) All queuing classes implement the throttledfilesocket interface. These classes can also transmit file content.
Transmission control information.

The other four queues are queues that implement the throttledcontrolsocket interface class. The classes in these queues mainly focus on transmission control information.

These four queues have a temporary high priority, a temporary general priority, a formal high priority, and a formal general priority. Unlike addtostandardlist,

Queueforsendingcontrolpacket adds all the sockets to be added to the queue to two temporary queues. Add them to normal temporary queues based on their priorities. In the large loop of runinternal, items in the temporary queue are first moved to the normal queue, and then processed.

Uploadbandwidththrottler uses two critical partitions and two events.
Pauseevent is used to pause the entire large loop. Threadendedevent indicates that the entire thread is stopped. Sendlocker is used in a Large Loop
And tempqueuelocker is an additional lock for two temporary queues. This allows you to send the data to be sent from the existing queues, while adding the new socket
To the queue.

The large loop in runinternal of uploadbandwidththrottler is the daily operation of the worker thread. This
The following is done in a Large Loop: Calculate the quota, that is, the number of bytes that can be sent in this cycle, arrange the scheduling, calculate the amount of time that should be sleep in this cycle, and then perform the corresponding sleep, to speed up.
The operation control information queue sends data in this queue. Note that the sockets (m_controlqueuefirst_list and
M_controlqueue_list. The socket in the standard queue does not. If the quota for sending data is not used up after the end of a loop,
Some quotas will be saved to the next round.

Network Infrastructure-eMule socket cemsocket

Cemsocket is a subclass of casyncsocketex and throttledfilesocket. It provides several functions
So it can be used as a convenient socket for eMule. For example, you can easily specify a proxy, and create a new proxy layer in casyncsocketex and
The function added to the List is blocked. In addition, it can be divided into states, such as whether the current sending control information is in progress.

In cemsocket, The sendcontroldata and
Sendfileandcontroldata method. As mentioned above, these methods are used to work with uploadbandwidththrottler to complete
Local speed limit function. Its function should comply with uploadbandwidththrottler's requirements, and send a specified number of bytes when it sends data this time. Therefore, the application
When using cemsocket, other parts of the program should not directly call the standard send or sendto method, but call
Sendpacket. Here we have another structure packet, which usually contains a complete package in the eMule protocol, such as header data with a protocol, and also has a built-in
Packpacket and unpackpacket can be compressed and decompressed by yourself. Sendpacket puts the packet to be sent in its own queue.
Two queues are also available to control the information packet queue and standard information packet queue. Add yourself to the uploadbandwidththrottler queue if necessary.
We have noticed that the sendcontroldata and sendfileandcontroldata methods of cemsocket call another overload of their own.
Send method. We also know that this method is called in a large loop in the uploadbandwidththrottler worker thread, and the send Method
The content itself is also a big loop, but it makes a clear sense that, without exceeding the quota of the current sending, the package in the package queue will be taken out and sent out. Similarly, a temporary
It is used to ensure that the operations of pulling packets from the package queue to send and placing the packets in the queue are mutually exclusive. Therefore, if you combine it with uploadbandwidththrottler
We can see a two-tier queue, that is, all sockets constitute a sending queue, which guarantees the speed limit under the control of uploadbandwidththrottler,
The packet that each socket sends will form a queue, ensuring that each data transmission meets the requirements of uploadbandwidththrottler.