Python Twisted Learning Series 1 (reprinted stulife The best Twisted Introductory tutorial)

Part I: Theoretical basis of twisted

author :[email protected]http://krondo.com/?p=1209 Translator : Yang Xiaowei ( with free translation )

Preface :

someone has recentlyTwisteda list of messages such as "provide a copy of the task emergencyTwistedThe requirements of the introduction ". It's worth revealing that this sequence is not what they wanted..especially the introductionTwistedframework and based onPython asynchronous programming, it may not be possible to speak clearly for a short time. So,If your time is urgent, this is not the information you are looking for.

I believe that if you know nothing about the asynchronous programming model, a quick introduction will also not allow you to understand it, at least you need to understand a little bit about the basics. I have usedTwistedFramework for a few years, so think about how I learned it(It's slow to learn.)and found that the greatest difficulty in learning it was notTwisteditself,Instead of understanding the model, you can better write and understand the code of the async program, only by understanding the model. MostTwistedcode is clearly written, and its online documentation is great (at least at the level of open source software). But if you don't understand the model, whether it's readingTwistedthe source or useTwistedthe code is more or more relevant, you will find it very nerve-racking.

So, I'll use the previous sections to introduce this model to let you master its mechanism, and later, to explainTwistedthe characteristics. In fact, at first, we don't useTwisted, instead, you will use a simplePythonto illustrate how an asynchronous model works. We are studying for the first timeTwisted, it starts from the bottom-up implementation that you don't normally use directly. Twistedis a highly abstract system, so when you use it, you will experience its multi-layered nature. But when you go to study, especially when trying to understand how it works, this multi-layered nature of pumping is a great way to understand. So, we're going to go from the inside out and start learning it from the lower level.

Model:

To better understand the features of the asynchronous programming model, let's review the two familiar models. In the process of elaboration, we assume a program that contains three separate tasks for each other. Here, in addition to the provisions of these tasks to complete their own work, we do not make a specific explanation, we will slowly understand them later. Note: Here I use the word "task", which means it needs to do something.

The first model is a single-threaded synchronization model, 1 as shown:

Figure 1 synchronization model

This is the simplest way to program. At one point, only one task can be executed, and a task can begin after the previous one has ended. If the task can be executed in a predetermined order, the completion of the last task means that all previous tasks have been completed without any errors and output the results available to them-how simple the logic is.

Let's present a second model,2 :

Figure 2 threading Model

In this model, each task is completed in a separate thread. These threads are managed by the operating system and, if they are on a multiprocessor, multicore processor system, may run independently of each other, and if they are on a single processor, they will be interleaved. The key point is that in thread mode, the specific task execution is handled by the operating system. Programmers, however, simply assume that their instruction flows are independent and can be executed in parallel. Although, from the diagram looks very simple, in fact multithreaded programming is very troublesome, you want to ah, the communication between the tasks will be between the threads of communication. Communication between threads that is not a general complication. What mailbox, channel, shared memory 、、、 Alas: (

Some programs use multiprocessor rather than multithreading to implement parallel operations. Although the specifics of the programming are different, they are the same for the model we are going to study.

Let's take a look at the asynchronous programming model,3 shows

Figure 3 Async Model

In this model, the task is staggered, and it is worth noting that this is under single-threaded control. This is much simpler than multithreaded models because programmers can always assume that only one task is executing, while others are in a stopped state. While in a single processor system, threads are alternately performed as shown in Figure 3 . However, as programmers use multi-threading, it is still necessary to use Figure 2 instead of Figure 3 to think about the problem, to prevent the program from moving to a multiprocessor system that does not work properly (considering compatibility). Single-threaded asynchronous programs operate well both on a single processor and on a multi-processor machine.

There is a difference between the asynchronous programming model and the multithreaded model: in multithreaded programs, the decision to stop a thread from starting another thread is not in the programmer's hands but on the operating system, so Programmers must be programmed to assume that at any time a thread is likely to be stopped and start another thread. Conversely, in an asynchronous model, a task must explicitly discard control of the currently running task in order to run it. This is also the most concise place compared to the multithreaded model.

It is important to note that it is possible to mix the asynchronous programming model with the synchronization model in the same system. But most of the time in the introduction, we only study the asynchronous programming model in a single thread.

Motivation

We have seen that the asynchronous programming model is simpler than a multithreaded model in that its single-flow and explicit abandonment of control over tasks rather than being randomly stopped by the operating system. But asynchronous models are much more complex than synchronous models. Programmers must organize tasks into sequences to alternate small steps to complete. Therefore, if one of the tasks uses the output of another task, the dependent task (that is, the task that receives the output) needs to be designed to receive a series of bits or shards instead of receiving them all. Since there is no substantial parallelism, it can be seen from our diagram that an asynchronous program spends the time it takes to synchronize a program, which can take longer due to the performance problems of the asynchronous program.

So, ask, why do you use asynchronous models? Here, we have at least two reasons. First, if you have one or two tasks that need to complete a human-facing interface, if you perform these tasks alternately, the system performs other tasks in the background while maintaining the response to the user. Therefore, although the background tasks may not run faster, such systems may be more welcome.

However, there is a case where the performance of the asynchronous model is higher than the synchronization model, and sometimes even very prominent, that is, in a relatively short period of time to complete all tasks. This is the case where the task is forced to wait or block, as shown in4 :

Figure 4 blocking in the synchronization model

In Figure 4 , the gray section represents a task that is blocked during this time. Why do you want to block a task? The most immediate reason is to wait for the I/O to complete: transfer data or from an external device. The ability of a typical CPU to process data is a multiple of several orders of magnitude for a hard disk or network. Therefore, a synchronization program that requires large I/O operations can take a lot of time to wait for the hard disk or network to prepare the data. It is for this reason that the synchronization program is also known as a blocking program.

As you can see from Figure 4 , a blocking program looks a bit like the asynchronous program depicted in Figure 3 . It's not a coincidence. The main feature behind asynchronous programs is that when a task appears to be blocked like a Synchronizer, other tasks that can be performed continue to execute, rather than blocking them all as they do in the synchronization program. Therefore, an asynchronous program will only "block" if no task is executable, which is why the asynchronous program is called a non-blocking program.

The switch between tasks is either done by this task or it is blocked. Because a large number of tasks can be blocked, the asynchronous program waits less than the synchronization program and uses those times for other real-time work (such as interfaces with people), so the performance of the former is necessarily much higher.

The advantages of an asynchronous model are shown in the following scenarios, compared to the synchronization model:

1. There are a lot of tasks, so at least one task to run at a time

2. The task performs a large number of I/O operations so that the synchronization model wastes a lot of time because the task is blocked

3. tasks are independent of each other, so that there is little interaction within the task.

These conditions are mostly in CS mode where the network is more busy on the server side (such as a WEB server). Each task represents a client that receives a request and replies to an I/O operation. Customer requests (equivalent to read operations) are independent of each other. So a network service is a typical representation of the asynchronous model, which is why twisted is the first and best network library.

Python Twisted Learning Series 1 (reprinted stulife The best Twisted Introductory tutorial)