Python uses Asyncio encapsulation file read-write _python

Objective

Like network IO, file reading and writing is also a cumbersome operation.

By default, Python uses the system's blocking read and write. This means that in Asyncio, if you call the

f = file (' xx ')
f.read ()

will block the event loop.

This article briefly describes how to use Asyncio. Future object to encapsulate asynchronous reading and writing of files.

Code in GitHub. Linux is currently supported only.

Blocking and non-blocking

First you need to change the file's read and write to a non-blocking form. In a non-blocking situation, every call to read returns immediately, and if the return value is empty, it means that the file operation has not yet been completed, whereas the contents of the file are read.

Blocking and non-blocking switching is related to the operating system, so this article only has a Linux version for the time being. If you have experience with Unix system programming, you will find that Python operations are similar.

Flag = Fcntl.fcntl (SELF.FD, Fcntl. F_GETFL) 
If Fcntl.fcntl (SELF.FD, Fcntl. F_SETFL, Flag | Os. O_nonblock)!= 0: 
  raise OSError () 

Future objects

The Future object resembles a Promise object in Javascript. It is a placeholder and its value will be computed in the future. We can use

result = await future

Returns after future gets a value. and use

future.set_result(xxx)

You can set the value of the future, which means that the future can be returned. The await operator automatically invokes the Future.result () to get the value.

Loop.call_soon

You can insert a function into the event loop by using the Loop.call_soon method.

At this point, our asynchronous file read and write ideas have come out. A function that calls non-blocking read-write files through Loop.call_soon. If a file is not read or written, the number of bytes of read and write remaining is computed, and the event loop is inserted again until the read and write is complete.

It can be found that in traditional Unix programming, a while loop that reads and writes non-blocking files is replaced by a Asyncio event loop.

The following is the schematic code for this process.

def read_step (self, future, N, total):
  res = Self.fd.read (n)
  if res is None:
    Self.loop.call_soon (Self.read_ Step, future, N, Total]
    return
  if not res: # EOF
    future.set_result (bytes (self.rbuffer))
  return Self.rbuffer.extend (res)
  Self.loop.call_soon (Self.read_step, Future, self. Block_size, total)

def read (self, n=-1):
  future = Asyncio. Future (Loop=self.loop)

  self.rbuffer.clear ()
  Self.loop.call_soon (Self.read_step, Future, min (self). Block_size, N), N) return

  future