Using asynchronous I/O greatly improves application performance (AIO)

Introduction:The most common input/output (I/O) model in Linux is synchronous I/O. In this model, when a request is sent, the application blocks until the request is met. This is a good solution because no central processing unit (CPU) is required when calling an application waiting for the completion of I/O requests ). However, in some cases, I/O requests may need to overlap with other processes. POSIX asynchronous I/O (AIO) application interface (API) provides this function. In this article, we will introduce the overview of this API and learn how to use it.

AIO Introduction

Linux asynchronous I/O is a new enhancement provided by the Linux kernel. It is a standard feature of the 2.6 kernel, but we can also find it in the patch of the 2.4 kernel. The basic idea behind AIO is to allow a process to initiate many I/O operations without blocking or waiting for any operation to complete. The process can retrieve the results of an I/O operation later or when it receives a notification that the I/O operation is complete.

I/O model

Before going into the aio api, let's first explore the different I/O models that can be used on Linux. This is not a detailed description, but we will try to introduce some of the most common models to explain the differences between them and asynchronous I/O. Figure 1 shows the synchronous and asynchronous models, as well as the blocking and non-blocking Models.

Figure 1. Simple matrix of basic Linux I/O models

Each I/O model has its own usage mode, which has its own advantages for specific applications. This section briefly introduces them one by one.

Synchronous blocking I/O

Comparison between I/O-intensive and CPU-intensive processes

I/O-intensive processes perform more I/O operations than I/O operations. CPU-intensive processes perform more processing operations than I/O operations. In Linux 2.6, schedulers prefer I/O-intensive processes because they usually initiate an I/O operation and then block the process, this means that other work can be effectively staggered between the two.

The most common model is the synchronous blocking I/O model. In this model, the user space application executes a system call, which causes application blocking. This means that the application will be blocked until the system call is completed (data transmission is completed or an error occurs ). Calling an application is in a state that does not consume the CPU, but simply waits for a response. Therefore, this is very effective from the processing point of view.

Figure 2 shows the traditional blocking I/O model, which is also the most commonly used model in applications. Its behavior is very easy to understand, and its usage is very effective for typical applications. Before callingreadWhen the system is called, the application is blocked and context switches are performed on the kernel. Then, the read operation is triggered. When the response is returned (from the device we are reading from), the data is moved to the user space buffer. Then the application unblocks (readCall back ).

Figure 2. Typical process of synchronous blocking I/O model

From the application perspective,readThe call takes a long time. In fact, when the Kernel performs read operations and other work, the application is indeed blocked.

Synchronous non-blocking I/O

A slightly less efficient variant of synchronous blocking I/O is synchronous non-blocking I/O. In this model, devices are opened in a non-blocking manner. This means that the I/O operation will not be completed immediately,readThe operation may return an error code, indicating that this command cannot be met immediately (EAGAINOrEWOULDBLOCK), As shown in 3.

Figure 3. Typical process of synchronizing non-blocking I/O models

The non-blocking implementation is that the I/O command may not be immediately satisfied, and the application needs to call it many times to wait for the Operation to complete. This may be inefficient, because in many cases, when the kernel executes this command, the application must wait until the data is available or try to execute other work. As shown in figure 3, this method can introduce the latency of I/O operations, because the data becomes available in the kernel for user calls.readThere is a certain interval between the returned data, which leads to a reduction in the overall data throughput.

Asynchronous blocking I/O

Another blocking solution is non-blocking I/O with blocking notifications. In this model, non-blocking I/O is configured, and then blockselectA system call is used to determine when an I/O descriptor is operated. EnableselectThe call is very interesting because it can be used to provide notifications for multiple descriptors, not just for one descriptor. For each prompt, we can request that this descriptor can write data, read data is available, and whether an error occurs.

Figure 4. Typical process of asynchronous blocking I/O model (select)

selectThe main problem with calling is that it is not very efficient. Although this is a convenient model for asynchronous notifications, it is not recommended for high-performance I/O operations.

Asynchronous non-blocking I/O (AIO)

Finally, the asynchronous non-blocking I/O model is a model that processes overlapping I/O. Read requests are returned immediately.readThe request has been initiated successfully. When the read operation is completed in the background, the application then performs other processing operations. WhenreadWhen the response arrives, a signal is generated or a thread-based callback function is executed to complete the I/O processing process.

Figure 5. Typical asynchronous non-blocking I/O model process

The overlapping processing capabilities of computing operations and I/O processing to execute multiple I/O requests in a process utilizes the difference between processing speed and I/O speed. When one or more I/O requests are suspended, the CPU can execute other tasks. or, more commonly, when other I/O operations are initiated, the completed I/O operations are performed.

The next section will introduce this model in depth, explore the APIs used by this model, and then display several commands.

Back to Top

Motivation for asynchronous I/O

From the classification of the previous I/O model, we can see the motivation of AIO. This blocking model needs to block the application at the beginning of the I/O operation. This means that it is impossible to overlap processing and I/O operations at the same time. Synchronous non-blocking model allows overlapping processing and I/O operations, but this requires the application to check the status of I/O operations according to the reproducible rules. In this way, asynchronous non-blocking I/O operations are left, which allow overlapping processing and I/O operations, including notifications of completed I/O operations.

In addition to blocking,selectThe functions provided by the function (asynchronously blocking I/O) are similar to those provided by AIO. However, it blocks notification events rather than I/O calls.

Back to Top

Introduction to AIO on Linux

This section explores the asynchronous I/O model of Linux to help us understand how to use this technology in applications.

In the traditional I/O model, there is an I/O channel identified by a unique handle. In UNIX, these handles are file descriptors (equivalent to files, pipelines, sockets, etc ). In blocking I/O, we initiate a transmission operation. When the transmission operation is completed or an error occurs, the system will return the result after calling the operation.

AIO on Linux

AIO first appeared in the kernel of Version 2.5. It is now a standard feature of the kernel of version 2.6.

In asynchronous non-blocking I/O, we can initiate multiple transmission operations at the same time. This requires that each transmission operation has a unique context so that we can determine which transmission operation is completed in their time zone. In AIO, This Isaiocb(Aio I/O Control Block) structure. This structure contains all information about the transmission, including the user buffer prepared for the data. When an I/O (called completion) Notification is generated,aiocbThe structure is used to uniquely identify the completed I/O operations. This API shows how to use it.

Back to Top

AIO API

The API of the AIO interface is very simple, but it provides the necessary functions for data transmission and provides two different notification models. Table 1 provides the AIO interface functions. This section will be detailed later.

Table 1. AIO API

API functions	Description
aio_read	Request asynchronous read Operations
aio_error	Check the status of asynchronous requests
aio_return	Returns the status of the completed asynchronous request.
aio_write	Request asynchronous write operation
aio_suspend	Suspends the calling process until one or more asynchronous requests have been completed (or failed)
aio_cancel	Cancel asynchronous I/O requests
lio_listio	Initiate a series of I/O operations

 

Each API function usesaiocbStructure start or check. This structure has many elements, but Listing 1 only provides the elements that need to be (or can be) used.

Listing 1. fields related to the aiocb Structure

struct aiocb { int aio_fildes; // File Descriptor int aio_lio_opcode; // Valid only for lio_listio (r/w/nop) volatile void *aio_buf; // Data Buffer size_t aio_nbytes; // Number of Bytes in Data Buffer struct sigevent aio_sigevent; // Notification Structure /* Internal fields */ ...};

 

sigeventThe structure tells AIO what operations should be performed when I/O operations are completed. We will explore this structure in the AIO presentation. Now we will show how the various aio api functions work and how we should use them.

Aio_read

aio_readFunction requests perform asynchronous read operations on a valid file descriptor. This file descriptor can represent a file, socket, or even a pipeline.aio_readThe function prototype is as follows:

 

int aio_read( struct aiocb *aiocbp );

 

aio_readThe function returns immediately after the request is queued. If the execution is successful, the returned value is 0. If an error occurs, the returned value is-1 and the value is set.
errno.

To perform a read operation, the application mustaiocbStructure. The following example shows how to fillaiocbRequest structure, and useaio_readTo execute asynchronous read requests (currently ignore notifications. It also shows
aio_errorBut we will explain it later.

Listing 2. Examples of asynchronous read operations using aio_read

#include <aio.h>... int fd, ret; struct aiocb my_aiocb; fd = open( "file.txt", O_RDONLY ); if (fd < 0) perror("open"); /* Zero out the aiocb structure (recommended) */ bzero( (char *)&my_aiocb, sizeof(struct aiocb) ); /* Allocate a data buffer for the aiocb request */ my_aiocb.aio_buf = malloc(BUFSIZE+1); if (!my_aiocb.aio_buf) perror("malloc"); /* Initialize the necessary fields in the aiocb */ my_aiocb.aio_fildes = fd; my_aiocb.aio_nbytes = BUFSIZE; my_aiocb.aio_offset = 0; ret = aio_read( &my_aiocb ); if (ret < 0) perror("aio_read"); while ( aio_error( &my_aiocb ) == EINPROGRESS ) ; if ((ret = aio_return( &my_iocb )) > 0) { /* got ret bytes on the read */ } else { /* read failed, consult errno */ }

 

In Listing 2, after opening the file from which data is to be read, we cleared it.aiocbStructure, and then allocate a data buffer. And put the reference to this data bufferaio_buf. Then, we will
aio_nbytesThe size of the buffer. And setaio_offsetSet to 0 (the first offset in the file ). We willaio_fildesSet to the file descriptor from which the data is read. Call
aio_readRead requests. We can then callaio_errorTo confirmaio_read. As long as the status is
EINPROGRESS, And waits until the status changes. The request may be successful or fail.

Use the AIO interface to compile the program

We canaio.hFind the function prototype and other required symbols in the header file. When compiling a program using this interface, we must use the POSIX real-time extension Library (librt).

Note that using this API is very similar to reading content from files using standard library functions. Besidesaio_readIn addition to some asynchronous features, another difference is the setting of the read operation offset. In the traditionalreadDuring the call, the offset is maintained in the context of the file descriptor. For each read operation, the offset must be updated so that subsequent read operations can address the next data block. This is impossible for asynchronous I/O operations, because we can execute many read requests at the same time, so we must specify an offset for each specific Read Request.

Aio_error

aio_errorThe function is used to determine the Request status. The prototype is as follows:

 

int aio_error( struct aiocb *aiocbp );

 

This function returns the following content:

• EINPROGRESS, Indicating that the request has not been completed
• ECANCELLEDThe request is canceled by the application.
• -1, Indicating an error occurred. For details about the error cause, refererrno

Aio_return

Another difference between asynchronous I/O and standard block I/O is that we cannot immediately access the returned status of this function because we are not blockingreadCall. In the standardreadIn the call, the return status is provided when the function returns. However, in asynchronous I/O, we need to use
aio_returnFunction. The prototype of this function is as follows:

 

ssize_t aio_return( struct aiocb *aiocbp );

 

Onlyaio_errorThis function is called only after the request is completed (successful or incorrect.aio_returnThe return value is equivalent to that in synchronization.readOr
writeReturned value of the system call (the number of transmitted bytes. If an error occurs, the returned value is-1).

Aio_write

aio_writeA function is used to request an asynchronous write operation. The function prototype is as follows:

 

int aio_write( struct aiocb *aiocbp );

 

aio_writeThe function returns immediately, indicating that the request has been queued (the return value is0, The return value is
-1And set it accordingly.errno).

This correspondsreadSystem calls are similar, but there are some differences in behavior that need attention. Recall thatreadFor calling, the offset to be used is very important. However,writeFor example, this offset is only available when no value is set.
O_APPENDOption file context is very important. IfO_APPENDThe offset will be ignored, and the data will be appended to the end of the file. Otherwise,aio_offsetThe offset of the data in the file to be written.

Aio_suspend

We can useaio_suspendFunction to suspend (or block) The Calling process until the asynchronous request is complete. At this time, a signal or other timeout operations are generated. The caller providesaiocbReference List, any completion will cause
aio_suspend.aio_suspendThe function prototype is as follows:

 

int aio_suspend( const struct aiocb *const cblist[], int n, const struct timespec *timeout );

 

aio_suspendIs very simple to use. We want to provideaiocbReference List. If any one is complete, the call will return0. Otherwise
-1, Indicating that an error has occurred. See listing 3.

Listing 3. Using the aio_suspend function to block asynchronous I/O

struct aioct *cblist[MAX_LIST]/* Clear the list. */bzero( (char *)cblist, sizeof(cblist) );/* Load one or more references into the list */cblist[0] = &my_aiocb;ret = aio_read( &my_aiocb );ret = aio_suspend( cblist, MAX_LIST, NULL );

 

Note,aio_suspendThe second parameter of iscblistNumber of elements, ratheraiocbNumber of references.cblistAnyNULLAll elements are
aio_suspendIgnore.

Ifaio_suspendProvides timeout, And the timeout does occur, then it will return-1,errnoWill containEAGAIN.

Aio_cancel

aio_cancelThe function allows us to cancel one or all I/O requests executed on a file descriptor. The prototype is as follows:

 

int aio_cancel( int fd, struct aiocb *aiocbp );

 

To cancel a request, we need to provide the file descriptor andaiocbReference. If the request is successfully canceled, the function returnsAIO_CANCELED. If the request is complete, this function returnsAIO_NOTCANCELED.

To cancel all requests for a given file descriptor, we need to provide the file descriptor and a pairaiocbpOfNULLReference. If all requests are canceled, this function returnsAIO_CANCELED; If at least one request is not canceled, this function returns
AIO_NOT_CANCELEDIf no request can be canceled, this function returnsAIO_ALLDONE. We can then use
aio_errorTo verify each AIO request. If the request has been canceledaio_errorWill return
-1AnderrnoWill be setECANCELED.

Lio_listio

Finally, Aio provides a method to uselio_listioAPI functions initiate multiple transfers at the same time. This function is very important, because it means that we can start a large number of I/O operations in a system call (one kernel context switch. From the performance perspective, this is very important, so it is worth some time to explore.lio_listioThe API function is prototype as follows:

 

int lio_listio( int mode, struct aiocb *list[], int nent, struct sigevent *sig );

 

modeThe parameter can beLIO_WAITOrLIO_NOWAIT.LIO_WAITThis call will be blocked until all I/O operations are completed. After the operation is queued,LIO_NOWAITWill return.listIsaiocbReference List, the maximum number of elements is
nentDefined. Note:listCan beNULL,lio_listioWill ignore it.sigeventReference defines the method that generates signals when all I/O operations are completed.

Forlio_listioAnd the traditionalreadOrwriteRequests differ slightly in the actions that must be specified, as shown in Listing 4.

Listing 4. Using the lio_listio function to initiate a series of requests

struct aiocb aiocb1, aiocb2;struct aiocb *list[MAX_LIST];.../* Prepare the first aiocb */aiocb1.aio_fildes = fd;aiocb1.aio_buf = malloc( BUFSIZE+1 );aiocb1.aio_nbytes = BUFSIZE;aiocb1.aio_offset = next_offset;aiocb1.aio_lio_opcode = LIO_READ;...bzero( (char *)list, sizeof(list) );list[0] = &aiocb1;list[1] = &aiocb2;ret = lio_listio( LIO_WAIT, list, MAX_LIST, NULL );

 

For read operations,aio_lio_opcodeThe field value isLIO_READ. For write operations, we need to use
LIO_WRITE,LIO_NOPIt is also effective for not performing operations.

Back to Top

AIO notification

Now we have read the available AIO functions. This section describes how to use asynchronous notifications in depth. We will explore the asynchronous function notification mechanism through signal and function callback.

Asynchronous notification using signals

Using signals for inter-process communication (IPC) is a traditional mechanism in Unix. AIO can also support this mechanism. In this example, the application needs to define a signal processing program, which will be called when a specified signal is generated. The application then configures an asynchronous request to generate a signal when the request is complete. As part of the signal context, specificaiocbThe request is provided to record multiple possible requests. Listing 5 shows this notification method.

Listing 5. Using signals as AIO request notifications

void setup_io( ... ){ int fd; struct sigaction sig_act; struct aiocb my_aiocb; ... /* Set up the signal handler */ sigemptyset(&sig_act.sa_mask); sig_act.sa_flags = SA_SIGINFO; sig_act.sa_sigaction = aio_completion_handler; /* Set up the AIO request */ bzero( (char *)&my_aiocb, sizeof(struct aiocb) ); my_aiocb.aio_fildes = fd; my_aiocb.aio_buf = malloc(BUF_SIZE+1); my_aiocb.aio_nbytes = BUF_SIZE; my_aiocb.aio_offset = next_offset; /* Link the AIO request with the Signal Handler */ my_aiocb.aio_sigevent.sigev_notify = SIGEV_SIGNAL; my_aiocb.aio_sigevent.sigev_signo = SIGIO; my_aiocb.aio_sigevent.sigev_value.sival_ptr = &my_aiocb; /* Map the Signal to the Signal Handler */ ret = sigaction( SIGIO, &sig_act, NULL ); ... ret = aio_read( &my_aiocb );}void aio_completion_handler( int signo, siginfo_t *info, void *context ){ struct aiocb *req; /* Ensure it's our signal */ if (info->si_signo == SIGIO) { req = (struct aiocb *)info->si_value.sival_ptr; /* Did the request complete? */ if (aio_error( req ) == 0) { /* Request completed successfully, get the return status */ ret = aio_return( req ); } } return;}

 

In listing 5aio_completion_handlerSet the signal handler in the function to captureSIGIOSignal. Then initializeaio_sigeventStructure generation
SIGIOSignal to send a notification (this is throughsigev_notifyInSIGEV_SIGNAL). When the read operation is complete, the signal processing program starts from
si_valueStructure ExtractionaiocbAnd check the error status and return status to determine whether the I/O operation is complete.

For performance, this handler is also ideal for continuing I/O operations by requesting the next asynchronous transmission. In this way, when a data transmission is completed, we can start the next data transmission operation immediately.

Asynchronous notification using callback Functions

Another notification method is the system callback function. This mechanism does not generate a signal for the notification, but calls a function of the user space to implement the notification function. InsigeventTheaiocbTo uniquely identify the specific request being completed. See Listing 6.

Listing 6. Use thread callback notification for AIO requests

void setup_io( ... ){ int fd; struct aiocb my_aiocb; ... /* Set up the AIO request */ bzero( (char *)&my_aiocb, sizeof(struct aiocb) ); my_aiocb.aio_fildes = fd; my_aiocb.aio_buf = malloc(BUF_SIZE+1); my_aiocb.aio_nbytes = BUF_SIZE; my_aiocb.aio_offset = next_offset; /* Link the AIO request with a thread callback */ my_aiocb.aio_sigevent.sigev_notify = SIGEV_THREAD; my_aiocb.aio_sigevent.notify_function = aio_completion_handler; my_aiocb.aio_sigevent.notify_attributes = NULL; my_aiocb.aio_sigevent.sigev_value.sival_ptr = &my_aiocb; ... ret = aio_read( &my_aiocb );}void aio_completion_handler( sigval_t sigval ){ struct aiocb *req; req = (struct aiocb *)sigval.sival_ptr; /* Did the request complete? */ if (aio_error( req ) == 0) { /* Request completed successfully, get the return status */ ret = aio_return( req ); } return;}

 

In Listing 6aiocbAfter the request, we useSIGEV_THREADA thread callback function is requested as a notification method. Then we will specify a specific notification handler and load the context to be transmitted to the handler (in this case, it isaiocbRequest your own reference ). In this processing program, we simply reference
sigvalPointer and use the AIO function to verify that the request has been completed.

Back to Top

System Optimization for AIO

The proc file system contains two virtual files that can be used to optimize the performance of asynchronous I/O:

• The/proc/sys/fs/AIO-Nr file provides the current number of System-range asynchronous I/O requests.
• The/proc/sys/fs/AIO-max-Nr file is the maximum number of concurrent requests allowed. The maximum number is usually 64 KB, which is sufficient for most applications.

Back to Top

Conclusion

Using asynchronous I/O can help us build applications with faster I/O speeds and higher efficiency. If our applications can overlap processing with I/O operations, then AIO can help us build applications that can more efficiently use available CPU resources. Although this I/O model is different from the traditional blocking mode used in most Linux applications, the asynchronous notification model is very simple in concept and can simplify our design.

 

Source: https://www.ibm.com/developerworks/cn/linux/l-async/

Bytes -----------------------------------------------------------------------------------------------------

Undefined reference to 'aio _ read'

Fedora 9 system, kernel version 2.6.25.14.

Xtep Io program AIO. C. If the undefined reference to 'aio _ read' or undefined reference to 'aio _ write' or undefined reference to 'aio _ return' error occurs during compilation, the reason may be that the POSIX real-time extended library Library link during compilation.

Run ls-L/lib/librt. *, You can see lrwxrwxrwx 1 Root/lib/librt. so.1-> librt-2.8.so

Modify gcc-o aio. C to GCC-o aio. C-LRT.

 

Source: http://blog.sina.com.cn/s/blog_a06ad1b101017mtf.html