Redis Source Code Analysis (27)---Rio I/o packages

I/O per operating system, one component of it. And I/O business quality, to some extent also affect the efficiency of the system.

Today, I learned about Redis intermediate I/O, the same, Redis in his own system. Also encapsulates an I/O layer. Referred to as Rio. First look at what's in Rio:

struct _rio {/* backend functions. * Since This functions don't tolerate short writes or reads the return * value was simplified To:zero on error, non Z Ero on complete success.    */* Data stream Reading Method */size_t (*read) (struct _rio *, void *buf, size_t len);    /* Data Stream Write Method */size_t (*write) (struct _rio *, const void *BUF, size_t len);    /* Get current Read and write offset */off_t (*tell) (struct _rio *); /* The Update_cksum method if not NULL are used to compute the checksum of * all the data that were read or written so F Ar. The method should is * designed so, can is called with the current checksum, and the BUF * and Len Fields Poin Ting to the new block of data to add to the checksum * computation.    */* When a new data block is read, the current checksum/Void (*update_cksum) (struct _rio *, const void *BUF, size_t len) is updated;    /* The current CHECKSUM *////////uint64_t cksum;    /* Number of bytes read or written */////////////size_t processed_bytes; /* Maximum sinGLE read or write chunk size */* Maximum single read/write sizes */size_t max_processing_chunk; /* backend-specific VARs/*/* Rio in I/O variable */union {//buffer struct struct {//buffer detail s            DS ptr;        Offset off_t POS;        } buffer;            File structure body struct {file *fp; off_t buffered; /* Bytes written since last Fsync. *///sync minimum size off_t autosync; /* Fsync after ' AutoSync ' bytes written.    */} file; } io;};

In addition to the 3 required methods, the Read,write method, and the Tell method to get the offset. There are also 2 struct variables. A buffer structure body, a file structure. The author addresses different I/O situations. Did a different deal. When I am running a temporary I/O operation, I am dealing with rio.buffer when I am I/O with the file. The operation is run with the rio.file. Here's a look at the unified definition of the Rio read and write method:

/* The following functions is our interface with the stream. They ' ll call the * actual implementation of Read/write/tell, and would update the checksum * if needed. *//* Rio's writing method */static inline size_t riowrite (Rio *r, const void *buf, size_t len) {while (len) {//infer if the current operation byte length exceeds the maximum Length size_t bytes_to_write = (r->max_processing_chunk && r->max_processing_chunk < len)?        r->max_processing_chunk:len;        When writing new data, update checksum if (r->update_cksum) r->update_cksum (r,buf,bytes_to_write);        Run the Write method if (r->write (r,buf,bytes_to_write) = = 0) return 0;        BUF = (char*) buf + bytes_to_write;        Len-= Bytes_to_write;    The number of Operations bytes added r->processed_bytes + = Bytes_to_write; } return 1;}        /* Rio's Read method */static inline size_t rioread (Rio *r, void *buf, size_t len) {while (len) {//infer if the current operation byte length exceeds the maximum length size_t Bytes_to_read = (r->max_processing_chunk && r->max_processing_chunk < len)?
r->max_processing_chunk:len;        Read Data method if (R->read (r,buf,bytes_to_read) = = 0) return 0;        When reading the data, update the checksum if (r->update_cksum) r->update_cksum (R,buf,bytes_to_read);        BUF = (char*) buf + bytes_to_read;        Len-= Bytes_to_read;    R->processed_bytes + = Bytes_to_read; } return 1;}

There is a nice place here. Every time there is a change in data, Redis will do a calculation of the checksum processing algorithm, indicating that the data operation of the change action, using the algorithm is introduced before the CRC64 algorithm, for Rio's buffer IO and file Io,redis defined 2 Rio structure:

/* defines Bufferrio */static const Rio Riobufferio = {    Riobufferread,    Riobufferwrite,    based on the method described above. Riobuffertell,    NULL,/           * update_checksum */    0,/              * Current checksum */    0,/              * bytes Read or Written */    0,/              * read/write Chunk Size */    {{NULL, 0}}/* Union for Io-specific VARs */};/* According to the method described above, the Filerio */static const Rio Riofileio = {    Riofileread,    riofilewrite,    Riofiletell,    NULL,/           * Update_checksum */    0,/              * Current checksum */    0,/              * bytes read or written */    0,/              * read/ Write chunk Size */    {{NULL, 0}}/* Union for Io-specific VARs */};

The corresponding read and write methods are defined separately, such as the Read method of buffer and the Read method of file:

/* Returns 1 or 0 for success/failure. *//* reads the buffer contents of Rio into the incoming parameters */static size_t Riobufferread (Rio *r, void *buf, size_t len) {    if (Sdslen. PTR)-r->io.buffer.pos < len)        return 0;/* enough buffer to return len bytes. *    /memcpy (Buf,r->io.buff Er.ptr+r->io.buffer.pos,len);    R->io.buffer.pos + = len;    return 1;}

/* Returns 1 or 0 for success/failure. *//* reads the contents of the FP file in Rio */static size_t Riofileread (Rio *r, void *buf, size_t len) {    return fread (buf,len,1,r->io.file. FP);}

The object variables of the function Rio are different, and finally, in the Redis declaration, 4 types of data are written:

/* Rio writes different types of data methods. Finally called is the Riowrite method */size_t Riowritebulkcount (Rio *r, char prefix, int count); size_t riowritebulkstring (Rio *r, const char * BUF, size_t len); size_t Riowritebulklonglong (Rio *r, Long Long L); size_t Riowritebulkdouble (Rio *r, double D);

One way to achieve this:

/* Write Multi Bulk Count in the format: "*<count>\r\n". *//* Rio writes different types of data methods, called the Riowrite method */size_t Riowritebulkcount (Rio *r, char prefix, int count) {    char cbuf[128];    int Clen;    Cbuf[0] = prefix;    Clen = 1+ll2string (cbuf+1,sizeof (CBUF) -1,count);    cbuf[clen++] = ' \ r ';    cbuf[clen++] = ' \ n ';    if (Riowrite (r,cbuf,clen) = = 0) return 0;    return Clen;}

Call or the Riowrite method inside, depending on whether you define buffer io or file IO, each has its own different implementations.

In the file's write method, there is a detail when you read the content into Rio.file.buffer. Buffer exceeds the given synchronization minimum byte, you have to flush the buffer contents into the file.

/* Returns 1 or 0 for success/failure. *//* writes buf to file files in Rio */static size_t Riofilewrite (Rio *r, const void *buf, size_t len) {    size_t retval;    retval = fwrite (BUF,LEN,1,R->IO.FILE.FP);    R->io.file.buffered + = len;    if (R->io.file.autosync &&        r->io.file.buffered >= r->io.file.autosync)    {    // Interpretation of the need to synchronize        fflush (R->IO.FILE.FP);        Aof_fsync (Fileno (R->IO.FILE.FP));        r->io.file.buffered = 0;    }    return retval;}

Copyright notice: This article Bo Master original articles, blogs, without consent may not be reproduced.

Redis Source Code Analysis (27)---Rio I/o packages