Deep Learning of twisted and deep learning of twisted

Deep Learning of twisted and deep learning of twisted

I plan to clear my previous blog recently. Write some articles about the twisted framework. twisted is an excellent, mature, and widely used Asynchronous Network Framework of python. It is intended to start from the basics of the Network, to examples, to source code analysis, strive to have a clear understanding of twisted, and I can gain some benefits from it again. It is busy recently and will be updated when it is available. This is the first article, where to write, and finally sort it out.

 

Official Website: https://twistedmatrix.com/trac/

What is Twisted?

Twisted is an event-driven networking engine written in Python and licensed under the open source MIT license. Twisted runs on Python 2 and an ever growing subset also works with Python 3.

Twisted is an event-driven Asynchronous Network Communication architecture implemented by python. The source code of twisted can be viewed online on the official website, and svn can be downloaded on its own. The source code is clear and easy to understand. The following code or examples are provided on the official website.

 

The first question is, why is the project called twisted? My understanding is that the deferred list, a main component of twisted, has two callback chains. Each callback chain contains one (callback, errback) pair. Twisted starts to execute from the first callback according to the callback chain. If an error occurs, it executes the errback corresponding to the next callback Function. If the execution succeeds, it returns to the next callback to continue the execution. Is it twisted?

 

 

The second question is what is an event-driven Asynchronous Network framework?

All of the network communication programs we have written are familiar with sockets. All of them know that select, poll, and epoll network multiplexing programs correspond to read or write operations, the composition of twisted contains a reactor, which is the concept reactor mode of many asynchronous event frameworks. This is a singleton mode, and only one reactor is running at a time. Reactor. run (), we are talking about an event Loop Based on events. twisted waits for the event to be ready, then calls the callback function, and returns the result. Of course, it is an input/output (read and write) Event Based on select/poll/epoll.

 

About the reactor mode: please move here, this ox Translation: http://www.cnblogs.com/pugang/p/4621373.html

 

So what is asynchronous? Similarly, we know that there are several network communication modes. In various interviews, we may also ask about the differences between synchronous, asynchronous, blocking, and non-blocking.

There are many articles about this explanation. For more information, see the unix network I/o mode description in unix network programming. Let me talk about my understanding. In Linux, everything is a file, and socket is, of course, a network I/o is an I/o type. Our application should also contain two parts: User-mode and kernel-mode kernel call. Network I/o contains two parts:

1. The application enters the kernel state and waits for data to be ready.

2. Copy data from the kernel space to the user space.

 

Blocking is the process in which both stages are waiting and all of them are blocked. It is always waiting for the kernel to return.

Non-blocking is the socket we are familiar. setblocking (0), or non-blocking flag set by fnctl; it is only blocked in the second stage. After the application knows that the data is not ready, the error code EAGAIN/EWOULDBLOCK is returned immediately. After the data preparation is complete, the system returns the error code EAGAIN/EWOULDBLOCK.

Synchronization means waiting for the completion of I/O operations.

Asynchronous means that you do not have to wait for the I/O operation to complete and return directly. After the second stage is complete, the kernel actively notifies the process to end.

The difference is that blocking/non-blocking corresponds to a current state. synchronous/asynchronous focuses on the overall situation or operations.

 

Today we want to talk about event-driven. We have mentioned that we rely on select/poll/epoll. They all use network multiplexing I/o, and they do not need to be created with multithreading, the thread pool can monitor multiple fd at the same time. Some people call them asynchronous (synchronous) non-blocking I/0, but in fact they are also synchronous blocking, select/poll/epoll blocking in select (), epoll. in the wait () functions, wait until the data is ready, and then start the first stage. Then, the second stage is also blocked by I/O in the second stage, so they are blocked in two places and need to wait for the I/O operation to complete. Is there any asynchronous I/O? asynchronous I/O requires the underlying support of the operating system. At present, it seems that there is no satisfactory asynchronous I/O, some implementations are also based on Thread/thread pool simulation.

 

Now that twisted uses select/poll/epoll, why is it an event-based Asynchronous framework. Because it adopts asynchronous I/O mode, it uses reactor mode to process event loops and deferred to process callback functions. You can write a callback function and put it in the deferred list, twsited calls these callback functions after the event notification is ready. After the callback function is executed, the result is returned. Check whether it is asynchronous I/O. Other event-based Asynchronous frameworks, such as java's netty, are also simulated.

 

The next article is about to analyze the multiplexing I/O of select/epoll, and start with the socket/epoll example to explain twisted. If there are any errors, please correct them and learn together.