Python multithreaded-thread module

Both the thread and threading modules are capable of implementing multi-threading in Python, and generally it is easier to use threading because thread has a number of drawbacks, such as when the main thread ends, so the child threads are forced to terminate, no warnings, and no normal cleanup work. Therefore, the use of threading modules is generally more recommended. But for the purposes of learning, we have two modules to look at.

Before we learn the code, we need to understand the Python GIL, the global interpreter lock. This lock guarantees that only one thread can run at the same time.

Wait a minute...... I clearly want to use multi-threading, why this lock is guaranteed that only one thread to run, so it will not be able to concurrency it? Isn't this a pit-daddy?

In fact, we know that Python is a general explanatory language, that is, our code will be interpreted by the interpreter before it can be run, and a Python process has only one Python interpreter. We know that CPU scheduling of threads is unordered and random, so we cannot guarantee the order in which the code is executed. For example, I declare a variable in the previous, use this variable, but when the CPU is dispatched, the code behind us runs first, and at this point our variables are not declared, which will certainly lead to a variety of bugs. The GIL exists to ensure that code execution is not confusing. In a multithreaded environment, the Python virtual machine executes as follows:

1. Set GIL

2. Switch to a thread to run

3. Run:

A. Specify the number of bytecode directives, or

B. Thread active Surrender control (can call Time.sleep (0))

4. Set the thread to sleep

5. Unlock GIL

6. Repeat all of the above steps again

That is to say, I have the code chunked, put the related code in one piece, so that the dependent code can be executed together, and the non-relational code can be executed separately. In general, however, there is only one interpreter running, and the code that can run at the same time is only a single piece, but it is separated.

So there are a lot of people say that Python multithreading is the chicken, the role is not too big. But I can't say it all.

For example, for all I/O-oriented programs that invoke the built-in operating system C code, the GIL is freed before the I/O call to allow other threads to run while the thread waits for I/O. If a thread does not use many I/O operations, it will occupy the processor (and GIL) on its own time slice. In other words, I/O intensive Python programs take advantage of multithreaded environments more than computationally intensive programs.

In the file read and write operation, or the crawler is waiting for the page to download, it is more appropriate to use multi-threading.

After talking about how Python realizes multithreading and the use of multithreading above, the following is the formal beginning of the module learning.

Thread module

To install my previous discussion, before learning any Python content, call the Help () function first to see its built-in assistance documentation. There is a lot of content in the help documentation, so let's look at the FUNCTIONS section first.

1. allocate () / allocate_lock ()

We can see that the instructions are the same. Allocate_lock (), lock object

The function is to create a lock object. And the details need to see the LockType part, now put aside.

At the same time, there is a sentence: allocate () is a obsolete synonym, that is, the previous method is a past syntax, is obsolete, so we use Allocate_lock () is good.

2. exit () / exit_thread ()

Exit_thread () is a obsolete synonym, likewise, we use exit () just fine.

Synonymous with raise systemexit , a thread is exited unless an exception is caught.

At the same time, here is the way that Python threads exit:

1. Call the exit function such as Thread.exit ()

2. Using Python's standard method of exiting a process, sys.exit (), even the process exits, and the thread disappears.

3. Throw a Systemexit exception

Of course, we have a way to set the daemon thread, but this is the content in the threading module.

3. get_ident (), Integer

Returns a non-zero integer that uniquely identifies the current thread and other threads that exist at the same time.

This can be used to identify each thread resource. Assigning threads on some platforms is a sequential number starting with 1, but this is not absolute, and the identity of one thread may be reused by another thread after exiting.

4. Interrupt_main ()

Throws a Keyboardinterrupt exception in the main thread.

A child thread can use this method to interrupt the main thread.

5. stack_size ([size]), size

Returns the size of the thread stack used when creating a new thread.

You can use the size parameter to specify the stack size in bytes, and then create the appropriate thread. and must be 0 (using platform or configuration default) or a positive integer whose value is at least 32768 (+ K). If changing the thread stack size is not supported, the THREADERROR exception is triggered. If the specified size is not valid, ValueError is triggered and its size is not modified. The K-byte is currently supported with a minimum stack size value, which is intended to ensure sufficient stack space for the translation itself. Note that some platforms may have special stack size limitations, such as requiring a minimum heap greater than KB or a multiple of the system memory page size, details to view the system platform is the documentation. Usually 4 KB per page is the most common, you can try to use a multiple of 4kb when there is no document to explain.

Generally not used, the method is too low ground.

　　

6. start_new (function, args[, Kwargs])/start_new_thread (function, args[, Kwargs])

Start_new () is an obsolete synonym, so using Start_new_thread is OK.

Begins a new thread and returns its identifier. The thread invokes the corresponding function functions and passes the elements of the args Yongzu as positional arguments, and the dictionary Kwargs is passed in as a keyword argument. When the function returns, the thread ends, but its return value is ignored. When an unhandled exception is triggered inside the function, the thread ends, and the stack trace information is printed, and of course the trigger systemexit exits without a trace.

The first two parameters must be required. So, even if the function we want to run doesn't have arguments, we're going to pass an empty tuple.

As can be seen, here is a function as a whole, that is, a block of code to execute. What we need to do is to encapsulate our business logic in a function, and the function can invoke other functions, create instances of the class, and so on.

summed up in a picture:

A code example that is keyed from somewhere:

defLoop1 ():Print 'Child thread 1 starts:', CTime () sleep (4)    Print 'end of Child thread 1:', CTime ()defloop2 ():Print 'Child thread 2 starts:', CTime () sleep (2)    Print 'End of Child thread 2:', CTime ()defMain ():Print 'start of main thread:', CTime () Thread.start_new_thread (LOOP1, ()) Thread.start_new_thread (Loop2, ()) Sleep (6)#to prevent the main thread from stopping, and deliberately waiting for 6 seconds, is the function of two function execution time and: 4+2=6    Print 'main thread End:', CTime ()if __name__=='__main__': Main ()

Output:

It can be seen that the results of multithreading are actually achieved, but we added sleep (6) to prevent the main thread from exiting, and the result is this:

The main thread does not wait for the child thread to execute, and when the main thread finishes executing, the process exits and the child thread disappears. To prevent this, we let the main thread wait, but, here, because I know that the sum of time for two threads does not exceed 6 seconds, in fact, it is enough to calculate the longest function, greater than 4 seconds, but when we change to sleep (4) After that, the result is this:

Successful execution is complete.

Main line enters upgradeable exit.

In other words, the time card is too tight, the probability of success is not necessarily, depending on whether the CPU is the first to dispatch the child or the main thread, but the CPU scheduling is not known.

However, usually we need to execute the function of the time is not determined, and the performance of the CPU is different, the time required is different, what should be done this time?

Using the thread lock, that is, using the Allocate_lock () method. This method returns a LockType type lock object. Let's learn what LockType is.

LockType

Its help document is still visible in the thread module, after removing several built-in methods and repeating methods, actually only 3 methods are available, let's take a look.

1. acquire ([wait]), BOOL

Lock the code block. Locking if the thread runs the code block, the thread will be blocked if it is not unlocked.

When there is no parameter, it is locked, waits for another line to threads unlocked, and returns true if it is already locked.

When a given parameter is true, only if the Boolean value of the parameter is real is it locked, and a numeric response is successfully added to the lock.

Blocking is non-disruptive.

2. release ()

Release the lock so that another blocked thread acquires the thread lock. The lock object must be in a locked state, but it does not have to be threads unlocked by the same line.

3. locked (), BOOL

Returns a Boolean value that determines whether the lock object is in a locked state.

A code example that is keyed somewhere:

loops = [4, 2]defLoop (Nloop, Nsec, lock):Print 'Child Threads', Nloop,'Start:', CTime () sleep (nsec)Print 'Child Threads', Nloop,'End:', CTime () lock.release ()#UnlockdefMain ():Print 'start of main thread:', CTime () locks= []#a list to hold the lock objectNloops =Range (len (loops)) forIinchNloops:lock= Thread.allocate_lock ()#Create a process lock objectLock.acquire ()#try to get this lock objectLocks.append (Lock)#Place the acquired lock object in a list     forIinchNloops:thread.start_new_thread (Loop, (I, loops[i], locks[i]))#each cycle starts a new thread     forIinchNloops: whileLocks[i].locked ():#Check to see if all the lock locks are released, otherwise loop all the way to prevent the main thread from stopping causing the child threads to exit            Pass    Print 'main thread End:', CTime ()if __name__=='__main__': Main ()

Since two threads are set to start at the same time, the line printed at the beginning looks a bit messy.

About Threading's pit, fill in the next article.

Python multithreaded-thread module