Python uses Asyncio encapsulation file read-write _python

Source: Internet
Author: User

Objective

Like network IO, file reading and writing is also a cumbersome operation.

By default, Python uses the system's blocking read and write. This means that in Asyncio, if you call the

f = file (' xx ')
f.read ()

will block the event loop.

This article briefly describes how to use Asyncio. Future object to encapsulate asynchronous reading and writing of files.

Code in GitHub. Linux is currently supported only.

Blocking and non-blocking

First you need to change the file's read and write to a non-blocking form. In a non-blocking situation, every call to read returns immediately, and if the return value is empty, it means that the file operation has not yet been completed, whereas the contents of the file are read.

Blocking and non-blocking switching is related to the operating system, so this article only has a Linux version for the time being. If you have experience with Unix system programming, you will find that Python operations are similar.

Flag = Fcntl.fcntl (SELF.FD, Fcntl. F_GETFL) 
If Fcntl.fcntl (SELF.FD, Fcntl. F_SETFL, Flag | Os. O_nonblock)!= 0: 
  raise OSError () 

Future objects

The Future object resembles a Promise object in Javascript. It is a placeholder and its value will be computed in the future. We can use

result = await future

Returns after future gets a value. and use

future.set_result(xxx)

You can set the value of the future, which means that the future can be returned. The await operator automatically invokes the Future.result () to get the value.

Loop.call_soon

You can insert a function into the event loop by using the Loop.call_soon method.

At this point, our asynchronous file read and write ideas have come out. A function that calls non-blocking read-write files through Loop.call_soon. If a file is not read or written, the number of bytes of read and write remaining is computed, and the event loop is inserted again until the read and write is complete.

It can be found that in traditional Unix programming, a while loop that reads and writes non-blocking files is replaced by a Asyncio event loop.

The following is the schematic code for this process.

def read_step (self, future, N, total):
  res = Self.fd.read (n)
  if res is None:
    Self.loop.call_soon (Self.read_ Step, future, N, Total]
    return
  if not res: # EOF
    future.set_result (bytes (self.rbuffer))
  return Self.rbuffer.extend (res)
  Self.loop.call_soon (Self.read_step, Future, self. Block_size, total)

def read (self, n=-1):
  future = Asyncio. Future (Loop=self.loop)

  self.rbuffer.clear ()
  Self.loop.call_soon (Self.read_step, Future, min (self). Block_size, N), N) return

  future

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.