Python coroutine and IO multiplexing

This article describes how to implement Python coroutine and IO multiplexing.

Is a user-mode, please link the thread micro-thread

Benefits

No overhead for thread context switching

No need for atomic operation locking and synchronization overhead

Easy switch control simplifies programming model

High concurrency + high scalability + low cost

Disadvantages

The multi-core resource coroutine cannot be used. in essence, a single thread does not need to use multi-core resources.

Blocking will block the entire program.

Implement a coroutine operation with yield

def consumer(name):    print("--->starting eating baozi...")    while True:        new_baozi = yield        print("[%s] is eating baozi %s" % (name, new_baozi))        # time.sleep(1)  def producer():    r = con.__next__()    r = con2.__next__()    n = 0    while n < 5:        n += 1        con.send(n)        con2.send(n)        print("\033[32;1m[producer]\033[0m is making baozi %s" % n)  if __name__ == '__main__':    con = consumer("c1")    con2 = consumer("c2")    p = producer()

Define a coroutine

1. concurrency must be achieved in only one single thread

2. no locks are required to modify shared data.

3. the user program saves the context stacks of multiple control flows.

4. one coroutine automatically switches to another coroutine when I/O operations are performed.

A small example of a coroutine to manually switch IO operations

Greenlet is a coroutine module implemented in C. compared with yield that comes with python, it allows you to switch between any functions without the need to declare this function as a generator first.

import greenlet def test1():    print(12)    g2.switch()    print(34)    g2.switch() def test2():    print(56)    g1.switch()    print(78) g1 = greenlet.greenlet(test1)g2 = greenlet.greenlet(test2)g1.switch()

Gevent is a third-party library that can easily implement concurrent synchronization or asynchronous programming through gevent. Greenlet is the main mode used in gevent. it is a lightweight coroutine that connects Python in the form of a C extension module. Greenlet runs all inside the operating system processes of the main program, but they are collaboratively scheduled.

Automatic switch of the coroutine when I/O operations are encountered

Import gevent def foo (): print ("Running in foo") gevent. sleep (2) print ("Explicit context switch to foo again") def bar (): print ("Explicit context to bar") gevent. sleep (1) print ("Implicit context switch back to bar") def func3 (): print ("running func3") gevent. sleep (0) print ("running func3 again") gevent. joinall ([gevent. spawn (foo), # generate gevent. spawn (bar), gevent. spawn (func3),])

Performance differences between synchronization and asynchronization

import gevent  def task(pid):    gevent.sleep(0.5)    print("Task %s done" % pid)  def synchronous():    for i in range(1,10):        task(i)  def asynchronous():    threads = [gevent.spawn(task, i) for i in range(10)]    gevent.joinall(threads) print("Synchronous:")synchronous() print("Asynchronous:")asynchronous()

Gevent concurrent webpage crawling

From urllib import requestimport geventfrom gevent import monkey def f (url): print ("GET: % s" % url) resp = request. urlopen (url) data = resp. read () # file = open ("url.html", "wb") # file. write (data) # file. close () print ("% d bytes encoded Ed from % s. "% (len (data), url) monkey. patch_all () # Mark gevent for all IO operations of the current program. joinall ([gevent. spawn (f, "http://www.yitongjia.com"), gevent. spawn (f, "http://www.jinyuncai.cn"), gevent. spawn (f, "https://www.guoshipm.com"),])

Implement multi-socket concurrency in a single thread through gevent

Server

import sysimport socketimport timeimport gevent  from gevent import socket,monkeymonkey.patch_all()    def server(port):    s = socket.socket()    s.bind(('0.0.0.0', port))    s.listen(500)    while True:        cli, addr = s.accept()        gevent.spawn(handle_request, cli)      def handle_request(conn):    try:        while True:            data = conn.recv(1024)            print("recv:", data)            conn.send(data)            if not data:                conn.shutdown(socket.SHUT_WR)      except Exception as  ex:        print(ex)    finally:        conn.close()if __name__ == '__main__':    server(8001)

Client

Import socketimport threading def sock_conn (): client = socket. socket () client. connect ("localhost", 8001) count = 0 while True: # msg = input (">> :"). strip () # if len (msg) = 0: continue client. send ("hello % s" % count ). encode ("UTF-8") data = client. recv (1024) print ("[% s] recv from server:" % threading. get_ident (), data. decode () # result count + = 1 client. close () for I in range (100): t = threading. thread (target = sock_conn) t. start () concurrent 100 sock connections

Event-driven

We usually have several models when writing server processing model programs.

Each time a request is received, a process is generated to process the request.

Every time a request is received, a thread is generated to process the request.

Each time a request is received, an event list is placed for the main process to process the request in non-blocking IO mode.

1st method because the overhead of creating a new process is large, the server performance is poor, but the implementation is relatively simple.

2nd methods may encounter deadlocks and other issues due to thread synchronization.

The logic is more complex than the previous two when writing application code in the 3rd method.

Considering various factors, it is generally believed that 3rd methods are adopted by most network servers.

IO multiplexing

I. INTRODUCTION

Kernel space and user space

Currently, the operating system uses virtual memory. for 32-bit operating systems, the virtual storage space of the addressing space is 4G2 to the power of 32. The core of the operating system is that the kernel is independent from ordinary applications that can access the protected memory space and all permissions to access the underlying hardware devices. To ensure that user processes cannot directly operate on the kernel, ensure kernel security. The system divides virtual space into two parts: kernel space and user space. For linux operating systems, the highest 1G bytes are referred to as the kernel space from the virtual address 0xC0000000 to 0xFFFFFFFF, and the lower 3G bytes are referred to as the user from the virtual address 0x00000000 to 0xbfffff. space.

Process switching

To control the execution of processes, the kernel must be able to suspend processes running on the CPU and resume the execution of a previously suspended process. This behavior is called process switching. Therefore, any process runs with the support of the operating system kernel, which is closely related to the kernel.

Process blocking

The system automatically executes the blocking primitive (block) change yourself from running to blocking. It can be seen that process congestion is an active action of the process itself. Therefore, only a running process that obtains the CPU can turn it into a blocking state. When a process is blocked, it does not occupy CPU resources.

File descriptor fd

File descriptor is an abstract concept used to express references to files in computer science.

The file descriptor is a non-negative integer in form. In fact, it is an index that points to the record table for opening files for each process maintained by the kernel. When the program opens an existing file or creates a new file, the kernel returns a file descriptor to the process. In program design, some underlying programming programs usually focus on file descriptors. However, the file descriptor concept is often only applicable to operating systems such as UNIX and Linux.

Cache IO

Cache I/O is also called standard I/O. The default I/O operations of most file systems are cache I/O. In Linux's cache I/O mechanism, the operating system will cache the I/O data in the file system's page cache, that is, the data will be copied to the buffer zone of the operating system kernel first. then, it will be copied from the buffer zone of the operating system kernel to the address space of the application.

Disadvantages of cache I/O

During data transmission, you need to perform multiple data copy operations in the application address space and kernel. the CPU and memory overhead caused by these data copy operations are very large.

II. IO mode

I just mentioned that for an I/O access, the data with read as an example will be first copied to the buffer zone of the operating system kernel before being copied from the buffer zone of the operating system kernel to the address space of the application. Therefore, when a read operation occurs, it goes through two stages.

1. Waiting for data preparation (Waiting for the data to be ready)

2. copy data from the kernel to the process (Copying the data from the kernel to the process)

In these two phases, the linux system has the following five network modes.

-Block I/Oblocking IO

-Non-blocking I/onblocking IO

-I/O multiplexing IO multiplexing

-Signal-driven I/O signal driven IO

-Asynchronous I/Oasynchronous IO

Blocking IO

In linux, by default, all sockets are blocking. a typical read operation process is like this.

When the user process calls recvfrom, the system calls the kernel and starts the first stage of IO preparation. for network IO, the data is not yet reached at the beginning. For example, you have not received a complete UDP packet. At this time, the kernel will wait for enough data to arrive. This process requires waiting that the data is copied to the buffer zone of the operating system kernel. On the user process side, the whole process will be blocked. of course, the process is blocked by its own choice. When the kernel waits until the data is ready, it will copy the data from the kernel to the user memory, and then the kernel returns the result to the user process to unblock the data and run it again.

So the feature of blocking IO is that it is blocked in both stages of IO execution.

Non-blocking IO

In linux, you can set socket to non-blocking. The process is like this when a non-blocking socket is read.

When a user process sends a read operation, if the data in the kernel is not ready, it does not block the user process, but immediately returns an error. From the perspective of the user process, it does not need to wait but get a result immediately after initiating a read operation. When the user process determines that the result is an error, it will know that the data is not ready, so it can send the read operation again. Once the data in the kernel is ready and the system call of the user process is received again, it immediately copies the data to the user memory and returns it.

Therefore, nonblocking IO is characterized by the fact that the user process needs to actively ask about the kernel data.

IO multiplexing

IO multiplexing is what we call selectpollepoll, which is also called event driven IO in some places. The benefit of select/epoll is that a single process can simultaneously process the IO of multiple network connections. The basic principle of this function is that the selectpollepoll function will continuously poll all the sockets in charge. when a socket has data, it will notify the user process.

When the user process calls the select statement, the entire process will be blocked, and the kernel will "monitor" all sockets in the select statement. when the data in any socket is ready, the select statement will be returned. At this time, the user process then calls the read operation to copy data from the kernel to the user process.

Therefore, I/O multiplexing is characterized by a mechanism in which a process can wait for multiple file descriptors at the same time, and any of these file descriptor socket descriptors enters the read-ready state select () function.

This graph is not much different from the blocking IO graph, but it is actually worse. Here we need to use two system calls (select and recvfrom), while blocking IO only calls one system call (recvfrom ). However, the advantage of using select is that it can process multiple connections at the same time.

Therefore, if the number of connections to be processed is not high, the web server using select/epoll may not have better performance than the web server using multi-threading + blocking IO, but may have a greater latency. The advantage of select/epoll is not that it can process a single connection faster, but that it can process more connections.

In the I/O multiplexing Model, each socket is generally set to non-blocking, but as shown in, the entire user's process is always blocked. However, process is block by the select function rather than block by socket IO.

Asynchronous IO

The user process can start other tasks immediately after initiating the read operation. On the other hand, from the perspective of kernel, when it receives an asynchronous read, it will first return immediately, so it will not generate any block to the user process. Then, the kernel will wait for the data preparation to complete and then copy the data to the user memory. after all this is done, the kernel will send a signal to the user process to tell it that the read operation is complete.

IO multiplexing select, poll, epoll

Select example

Server

Import selectimport socketimport sysimport queue server = socket. socket () server. bind ("0.0.0.0", 6666) server. listen (1000) server. setblocking (False) # Do not block msg_dic ={} inputs = [server,] outputs = [] while True: readable, writeable, predictional = select. select (inputs, outputs, inputs) print (readable, writeable, exceptional) for r in readable: if r is server: # indicates a new connection conn, addr = server. accept () print ("New Connection", addr) inputs. append (conn) # because the newly established connection has not sent data, the program will report an error if it receives it now # so if you want to implement this client to send data, the server can know, select needs to monitor the conn msg_dic [conn] = queue. queue () # initialize a Queue, which is followed by else: data = r. recv (1024) print ("received data", data) msg_dic [r]. put (data) outputs. append (r) # put in the returned connection queue # r. send (data) # print ("send end... ") for w in writeable: # connection list to be returned to the client data_to_client = msg_dic [w]. get () w. send (data_to_client) # The metadata outputs returned to the client. remove (w) # make sure that the writeable does not return the processed connection for e in predictional: if e in outputs: outputs. remove (e) inputs. remove (e) del msg_dic [e]

Client

import socket HOST = "127.0.0.1"  # The remote hostPORT = 6666  # The same port as used by the servers = socket.socket(socket.AF_INET, socket.SOCK_STREAM)s.connect((HOST, PORT))while True:    msg = bytes(input(">>:"), encoding="utf8")    s.sendall(msg)    data = s.recv(1024)    # print(data)    print('Received', repr(data))s.close()

Selectors example

Selectors encapsulates the underlying select or epoll, which can be used to determine whether to use select or epoll based on different operating systems.

Selectors server

import socketimport selectors  sel = selectors.DefaultSelector()  def accept(sock, mask):    conn, addr = sock.accept()    print('accepted', conn, 'from', addr)    conn.setblocking(False)    sel.register(conn, selectors.EVENT_READ, read)  def read(conn, mask):    data = conn.recv(1024)    if data:        print('echoing', repr(data), 'to', conn)        conn.send(data)    else:        print('closing', conn)        sel.unregister(conn)        conn.close() sock = socket.socket()sock.bind(("0.0.0.0", 6666))sock.listen(100)sock.setblocking(False)sel.register(sock, selectors.EVENT_READ, accept) while True:    events = sel.select()    for key, mask in events:        callback = key.data        callback(key.fileobj, mask)

Selectors client

import socket HOST = "127.0.0.1"  # The remote hostPORT = 6666  # The same port as used by the servers = socket.socket(socket.AF_INET, socket.SOCK_STREAM)s.connect((HOST, PORT))while True:    msg = bytes(input(">>:"), encoding="utf8")    s.sendall(msg)    data = s.recv(1024)    # print(data)    print('Received', repr(data))s.close()

The above is a detailed description of the Python coroutine and IO multiplexing methods. For more information, see other related articles in the first PHP community!