In Python, asyncio is used to encapsulate file read/write,

In Python, asyncio is used to encapsulate file read/write,

Preface

Like network I/O, file read/write is also a troublesome operation.

By default, Python uses system-blocking read/write. This means that if

f = file('xx')f.read()

Will block the event loop.

This article describes how to use asyncio. Future object to encapsulate asynchronous file read/write.

The code is on GitHub. Currently, only Linux is supported.

Blocking and non-blocking

First, you must change the file read/write to a non-blocking format. In the case of non-blocking, each call to read will return immediately. If the returned value is null, it means that the file operation is not completed, and vice versa.

Blocking and non-blocking switching are related to the operating system. Therefore, this article only writes the Linux version. If you have Unix programming experience, you will find that Python operations are similar.

flag = fcntl.fcntl(self.fd, fcntl.F_GETFL) if fcntl.fcntl(self.fd, fcntl.F_SETFL, flag | os.O_NONBLOCK) != 0:   raise OSError() 

Future object

The Future object is similar to the Promise object in Javascript. It is a placeholder and its value will be calculated in the future. We can use

result = await future

Return after the value is obtained in future. And use

future.set_result(xxx)

You can set the value of future, which means that future can be returned. The await operator automatically calls ure. result () to obtain the value.

Loop. call_soon

You can use the loop. call_soon method to insert a function into an event loop.

So far, our asynchronous file read and write ideas have come out. Call the function of non-blocking read/write files through loop. call_soon. If a file read/write operation is not completed, the remaining number of read/write bytes is calculated, and an event loop is inserted again until the read and write operations are completed.

It can be found that in traditional Unix programming, the while loop for non-blocking file read/write is replaced with the event loop of asyncio.

The following is the sample code for this process.

def read_step(self, future, n, total):  res = self.fd.read(n)  if res is None:    self.loop.call_soon(self.read_step, future, n, total)    return  if not res: # EOF    future.set_result(bytes(self.rbuffer))    return  self.rbuffer.extend(res)  self.loop.call_soon(self.read_step, future, self.BLOCK_SIZE, total)def read(self, n=-1):  future = asyncio.Future(loop=self.loop)  self.rbuffer.clear()  self.loop.call_soon(self.read_step, future, min(self.BLOCK_SIZE, n), n)  return future