Use scapy to send icmp packets. With 200 threads enabled, each thread stops sending a package. I tested the speed of only 300 packets of sec. Is it a problem of low efficiency of python multithreading? --------------------------------------- It was not a multi-thread problem that scapy was slow when sending packets. A single thread scapy has several packets per second, and multiple threads have a maximum of 60 packets per second. If it is written in C, it sends an icmp packet every second. With 200 threads enabled, each thread stops sending a package. I tested the speed of only 300 packets/sec. Is it a problem with low efficiency of python multithreading?
---------------------------------------
Later I found that it was not a multi-thread problem, but scapy was slow when it was originally launched. A single thread scapy has several packets per second, and multiple threads have a maximum of 60 packets per second. if it is written in C, it is 3 W per second .. reply: Because Python has a full lock Board (only one thread can be executed at a time), it cannot take advantage of multiple cores. Therefore, if your multi-threaded process is CPU-intensive, the multi-threaded process cannot improve the efficiency. On the contrary, the efficiency may also decrease due to frequent thread switching; in IO-intensive scenarios, multi-threaded processes can execute other threads in idle time during IO blocking to improve efficiency. With 200 threads enabled, each thread stops sending a package.
==========
Can I change my posture to a black Python? For example, the for loop is tens of thousands of times slower than the C language. When different threads access resources at the same time, a protection mechanism is required. GIL (Interpreter global lock) is used in Python ). Intuitively, this is a global lock applied to the interpreter (from the interpreter's perspective. This means that for any Python program, no matter how many processors there are, there is always only one thread running at all times. Therefore, without IO operations, multithreading in python is less efficient than single-threaded operations.
You can see the most difficult problem with Python.
After reading the above comments, it is difficult for me to understand why my python program is full of 12 cores, whether it is python2.6 or 2.7 or pypy. I really don't know that the rumor that python programs can only run on one core is that these half-hanging sub-scripts are so cruel. A program written by a half-hanging python programmer is likely to have a higher performance than a half-hanging c programmer. Gil is for a python interpreter process. This is the truth. If the interpreter can interpret and execute multiple processes, there will be no gil problem. Similarly, it will not cause multiple interpreters to run on the same core. The libcap library used by scapy is very slow. Try sendpfast of scapy.
The search keyword is Global Interpreter Lock.
Although I haven't used Py to write projects for many years, I 'd like to ask:
> With 200 threads enabled, each thread stops sending a package.
This is really good... Is the thread really free of money... It is found that when the speed of a multi-threaded Program reaches a certain value, increasing the number of threads does not increase the speed. Later, we simply used multiprocess to try to use the invariant as shared data, which can be mitigated, in addition, pypy is much faster, and pypy is equipped with JIT. Try Jython?
