Boost application performance using asynchronous I/O

Learn when and how to use the POSIX AIO API

Level: Intermediate M. Tim Jones (mtj@mtjones.com), Consultant Engineer, Emulex 29 Aug 2006 The most common input/output (I/O) model used in Linux is synchronous I/O. After a request is made in this model, the application blocks until the request is satisfied. This is a great paradigm because the calling application requires no central processing unit (CPU) while it awaits the completion of the I/O request. But in some cases there's a need to overlap an I/O request with other processing. The Portable Operating System Interface (POSIX) asynchronous I/O (AIO) application program interface (API) provides this capability. In this article, get an overview of the API and see how to use it. Introduction to AIO Linux asynchronous I/O is a relatively recent addition to the Linux kernel. It's a standard feature of the 2.6 kernel, but you can find patches for 2.4. The basic idea behind AIO is to allow a process to initiate a number of I/O operations without having to block or wait for any to complete. At some later time, or after being notified of I/O completion, the process can retrieve the results of the I/O. I/O models Before digging into the AIO API, let's explore the different I/O models that are available under Linux. This isn't intended as an exhaustive review, but rather aims to cover the most common models to illustrate their differences from asynchronous I/O. Figure 1 shows synchronous and asynchronous models, as well as blocking and non-blocking models. Figure 1. Simplified matrix of basic Linux I/O models

Each of these I/O models has usage patterns that are advantageous for particular applications. This section briefly explores each one. Synchronous blocking I/O One of the most common models is the synchronous blocking I/O model. In this model, the user-space application performs a system call that results in the application blocking. This means that the application blocks until the system call is complete (data transferred or error). The calling application is in a state where it consumes no CPU and simply awaits the response, so it is efficient from a processing perspective.

I/O-bound versus CPU-bound processes A process that is I/O bound is one that performs more I/O than processing. A CPU-bound process does more processing than I/O. The Linux 2.6 scheduler actually favors I/O-bound processes because they commonly initiate an I/O and then block, which means other work can be efficiently interlaced between them.

Figure 2 illustrates the traditional blocking I/O model, which is also the most common model used in applications today. Its behaviors are well understood, and its usage is efficient for typical applications. When the read system call is invoked, the application blocks and the context switches to the kernel. The read is then initiated, and when the response returns (from the device from which you're reading), the data is moved to the user-space buffer. Then the application is unblocked (and the read call returns). Figure 2. Typical flow of the synchronous blocking I/O model

From the application's perspective, the read call spans a long duration. But, in fact, the application is actually blocked while the read is multiplexed with other work in the kernel. Synchronous non-blocking I/O A less efficient variant of synchronous blocking is synchronous non-blocking I/O. In this model, a device is opened as non-blocking. This means that instead of completing an I/O immediately, a

read may return an error code indicating that the command (EAGAIN or EWOULDBLOCK), as shown in Figure 3.

could not be immediately satisfied

Figure 3. Typical flow of the synchronous non-blocking I/O model

The implication of non-blocking is that an I/O command may not be satisfied immediately, requiring that the application make numerous calls to await completion. This can be extremely inefficient because in many cases the application must busy-wait until the data is available or attempt to do other work while the command is performed in the kernel. As also shown in Figure 3, this method can introduce latency in the I/O because any gap between the data becoming available in the kernel and the user calling read to return it can reduce the overall data throughput. Asynchronous blocking I/O Another blocking paradigm is non-blocking I/O with blocking notifications. In this model, nonblocking I/O is configured, and then the blocking select system call is used to determine when there's any activity for an I/O descriptor. What makes the select call interesting is that it can be used to provide notification for not just one descriptor, but many. For each descriptor, you can request notification of the descriptor's ability to write data, availability of read data, and also whether an error has occurred.

Figure 4. Typical flow of the asynchronous blocking I/O model (select)

The primary issue with the select call is that it's not very efficient. While it's a convenient model for asynchronous notification, its use for high-performance I/O is not advised. Asynchronous non-blocking I/O (AIO) Finally, the asynchronous non-blocking I/O model is one of overlapping processing with I/O. The read request returns immediately, indicating that the read was successfully initiated. The application can then perform other processing while the background read operation completes. When the read response arrives, a signal or a thread-based callback can be generated to complete the I/O transaction. Figure 5. Typical flow of the asynchronous non-blocking I/O model

The ability to overlap computation and I/O processing in a single process for potentially multiple I/O requests exploits the gap between processing speed and I/O speed. While one or more slow I/O requests are pending, the CPU can perform other tasks or, more commonly, operate on already completed I/Os while other I/Os are initiated. The next section examines this model further, explores the API, and then demonstrates a number of the commands. Motivation for asynchronous I/O From the previous taxonomy of I/O models, you can see the motivation for AIO. The blocking models require the initiating application to block when the I/O has started. This means that it isn't possible to overlap processing and I/O at the same time. The synchronous non-blocking model allows overlap of processing and I/O, but it requires that the application check the status of the I/O on a recurring basis. This leaves asynchronous non-blocking I/O, which permits overlap of processing and I/O, including notification of I/O completion. The functionality provided by the select function (asynchronous blocking I/O) is similar to AIO, except that it still blocks. However, it blocks on notifications instead of the I/O call.

Introduction to AIO for Linux This section explores the asynchronous I/O model for Linux to help you understand how to apply it in your applications. In a traditional I/O model, there is an I/O channel that is identified by a unique handle. In UNIX, these are file descriptors (which are the same for files, pipes, sockets, and so on). In blocking I/O, you initiate a transfer and the system call returns when it's complete or an error has occurred. In asynchronous non-blocking I/O, you have the ability to AIO for Linux initiate multiple transfers at the same time. This requires a unique context for each transfer so you can identify it when AIO first entered the Linux kernel in it completes. In AIO, this is an aiocb (AIO I/O Control 2.5 and is now a standard feature of Block) structure. This structure contains all of the 2.6 production kernels. information about a transfer, including a user buffer for data. When notification for an I/O occurs (called a completion), the aiocb structure is provided to uniquely identify the completed I/O. The API demonstration shows how to do this.

AIO API The AIO interface API is quite simple, but it provides the necessary functions for data transfer with a couple of different notification models. Table 1 shows the AIO interface functions, which are further explained later in this section.

Table 1. AIO interface APIs API function Description aio_read Request an asynchronous read operation aio_error Check the status of an asynchronous request aio_return Get the return status of a completed asynchronous request aio_write Request an asynchronous operation Suspend the calling process until one or more asynchronous requests have completed aio_suspend (or failed) aio_cancel Cancel an asynchronous I/O request lio_listio Initiate a list of I/O operations Each of these API functions uses the aiocb structure for initiating or checking. This structure has a number of elements, but Listing 1 shows only the ones that you'll need to (or can) use. Listing 1. The aiocb structure showing the relevant fields
struct aiocb { int aio_fildes; Descriptor int aio_lio_opcode; for lio_listio (r/w/nop) volatile void *aio_buf; size_t aio_nbytes; Bytes in Data Buffer struct sigevent aio_sigevent; Structure /* Internal fields */ ... }; // File // Valid only // Data Buffer // Number of // Notification

The sigevent structure tells AIO what to do when the I/O completes. You'll explore this structure in the AIO demonstration. Now I'll show you how the individual API functions for AIO work and how you can use them. aio_read The aio_read function requests an asynchronous read operation for a valid file descriptor. The file descriptor can represent a file, a socket, or even a pipe. The aio_read function has the following prototype:
int aio_read( struct aiocb *aiocbp );

The aio_read function returns immediately after the request has been queued. The return value is zero on success or -1 on error, where errno is defined. To perform a read, the application must initialize the aiocb structure. The following short example

illustrates filling in the aiocb request structure and using aio_read to perform an asynchronous read request (ignore notification for now). It also shows use of the aio_error function, but I'll explain that later. Listing 2. Sample code for an asynchronous read with aio_read
#include <aio.h> ... int fd, ret; struct aiocb my_aiocb; fd = open( "file.txt", O_RDONLY ); if (fd < 0) perror("open"); */ ); /* Zero out the aiocb structure (recommended) bzero( (char *)&my_aiocb, sizeof(struct aiocb)

/* Allocate a data buffer for the aiocb request */ my_aiocb.aio_buf = malloc(BUFSIZE+1); if (!my_aiocb.aio_buf) perror("malloc"); /* Initialize the necessary fields in the aiocb */ my_aiocb.aio_fildes = fd; my_aiocb.aio_nbytes = BUFSIZE; my_aiocb.aio_offset = 0; ret = aio_read( &my_aiocb ); if (ret < 0) perror("aio_read"); while ( aio_error( &my_aiocb ) == EINPROGRESS ) ; if ((ret = aio_return( &my_iocb )) > 0) { /* got ret bytes on the read */ } else { /* read failed, consult errno */ }

In Listing 2, after the file from which you're reading data is opened, you zero out your aiocb structure, and then allocate a data buffer. The reference to the data buffer is placed into aio_buf. Subsequently, you initialize the size of the buffer into aio_nbytes. The aio_offset is set to zero (the first offset in the file). You set the file descriptor from which you're reading into aio_fildes. After these fields are set, you call aio_read to request the read. You can then make a call to aio_error to determine the status of the aio_read. As long as the status is EINPROGRESS, you busy-wait until the status changes. At this point, your request has either succeeded or failed. Building with the AIO interface Note the similarities to reading from the file with the standard library functions. In addition to the asynchronous You can find the function prototypes and other necessary symbolics in the aio.h header file. When building an application that uses this interface, you must use the POSIX real-time