Redis Source Analysis (27)---The Rio system I/O encapsulation

I/O operations are an essential part of each system. And I/O operation is good or bad, to a certain extent, will also affect the efficiency of the system. Today I learned what I am doing with I/O in Redis, and, similarly, Redis has encapsulated an I/O layer in his own system. Referred to as Rio. Let's see what's in Rio.

struct _rio {/* backend functions. * Since This functions don't tolerate short writes or reads the return * value is simplified To:zero on error, non Zero on complete success.
    */////////////////* size_t (*read) (struct _rio *, void *buf, size_t len);
    /* Data Flow Write Method * * size_t (*write) (struct _rio *, const void *BUF, size_t len);
    /* Get current read/write offset/off_t (*tell) (struct _rio *); /* The Update_cksum method if not NULL are used to compute the checksum of Far. The method should is * designed so can is called with the current checksum, and the BUF * and Len Fields PO Inting to the new blocks of data to add to the checksum * computation.

    * * * When the new data block is read, updates the current checksum/Void (*update_cksum) (struct _rio *, const void *BUF, size_t len);

    /* The current checksum////* uint64_t cksum; /* Number of bytes read or written////////////* The byte size currently read or written * * * size_t processed_bytes

    /* Maximum single read or write chunk size/////* max. * * size_t Max_processing_chunk;
            /* backend-specific vars. */* Rio I/O variable/union {//buffer structure struct {//buffer specific content
            SDS PTR;
        Offset off_t POS;
        } buffer;
            The file structure body struct {file *fp; off_t buffered; /* Bytes written since last Fsync. *///sync minimum size off_t autosync; /* Fsync after ' AutoSync ' bytes written.
    */} file;
} io; };

In addition to the 3 necessary methods, Read,write method, and get the offset of the tell method, there are 2 structural variables, a buffer structure, a file structure, the author for different I/O situation, do a different processing, when performing temporary I/O operations, are dealing with rio.buffer, and when I/O is done with the file, the operation between Rio.file is performed. Here's a look at the Rio unified definition of Read and write methods:

/* The following functions are our interface with the stream. They ' ll call the * actual implementation of Read/write/tell, and'll update the checksum * if needed. /* Rio Write method/static inline size_t Riowrite (Rio *r, const void *buf, size_t len) {while (len) {//To determine the current operation byte length is No more than maximum length size_t bytes_to_write = (r->max_processing_chunk && r->max_processing_chunk < len)?
        r->max_processing_chunk:len;
        Update checksum if (r->update_cksum) r->update_cksum (r,buf,bytes_to_write) when new data is written;
        Execute Write method if (r->write (r,buf,bytes_to_write) = = 0) return 0;
        BUF = (char*) buf + bytes_to_write;
        Len-= Bytes_to_write;
    Operation bytes increased r->processed_bytes + = Bytes_to_write;
return 1; 
        /* Rio Read method/static inline size_t Rioread (Rio *r, void *buf, size_t len) {while (len) {//determining whether the current operation byte length exceeds the maximum length size_t Bytes_to_read = (r->max_processing_chunk && r->max_proCessing_chunk < Len)?
        r->max_processing_chunk:len;
        Read Data method if (R->read (r,buf,bytes_to_read) = = 0) return 0;
        Update checksum if (r->update_cksum) r->update_cksum (R,buf,bytes_to_read) when reading data;
        BUF = (char*) buf + bytes_to_read;
        Len-= Bytes_to_read;
    R->processed_bytes + = Bytes_to_read;
return 1;
 }

Here is a relatively good place, every time when there is a change in data, Redis will do a calculation checksum processing algorithm, indicating the data operation of the change action, using the algorithm is previously introduced CRC64 algorithm, for Rio's buffer io and file IO, The Redis defines 2 Rio structures:

/* According to the method described above, Bufferrio/
static const Rio Riobufferio = {
    Riobufferread,
    riobufferwrite,
    Riobuffertell,
    NULL,/           * Update_checksum/
    0,/* Current checksum/*
    0,/              * bytes Read or Written * *
    0,              /* read/write Chunk Size * * {
    NULL, 0}}/* Union for Io-specific VARs
/};

/* According to the method described above, Filerio/
static const Rio Riofileio = {
    Riofileread,
    riofilewrite,
    Riofiletell,
    NULL,/           * update_checksum/0,/* Current              checksum * *
    0,/              * bytes read or written * *
    0,              /* read/write Chunk Size * * {
    NULL, 0}}/* Union for io-specific VARs * *
;

The corresponding read and write methods are defined, such as the Read method of buffer and the Read method of file:

/* Returns 1 or 0 for success/failure. *
///* Read the buffer contents in Rio to the incoming parameters/
static size_t Riobufferread (Rio *r, void *buf, size_t len) {
    if Sdslen (r-> IO.BUFFER.PTR)-r->io.buffer.pos < len) return
        0;/not enough buffer to return len bytes. * *
    memcpy (buf,r-& Gt;io.buffer.ptr+r->io.buffer.pos,len);
    R->io.buffer.pos = len;
    return 1;
}

/* Returns 1 or 0 for success/failure. * * *
Read the contents of the FP file in Rio * *
static size_t Riofileread (Rio *r, void *buf, size_t len) {return
    fread (buf,len,1,r- >IO.FILE.FP);
}

The role of the Rio object variable, and finally in the Redis declaration of 4 different types of data writing methods:

/* Rio writes different types of data methods, the final call is Riowrite method * *
size_t Riowritebulkcount (Rio *r, char prefix, int count);
size_t Riowritebulkstring (Rio *r, const char *BUF, size_t len);
size_t Riowritebulklonglong (Rio *r, Long Long L);
size_t Riowritebulkdouble (Rio *r, double D);

One of the ways to achieve this:

/* Write Multi Bulk Count in the format: "*<count>\r\n". * * *
Rio writes different types of data methods, calling the Riowrite method * *
size_t Riowritebulkcount (Rio *r, char prefix, int count) {
    char cbuf[ 128];
    int Clen;

    Cbuf[0] = prefix;
    Clen = 1+ll2string (cbuf+1,sizeof (CBUF) -1,count);
    cbuf[clen++] = ' \ r ';
    cbuf[clen++] = ' \ n ';
    if (Riowrite (r,cbuf,clen) = = 0) return 0;
    return clen;
}

Call or the inside of the Riowrite method, depending on whether you define buffer io or file IO,. Each has its own different implementations. In the file write method, there is a detail, when you read the content into the Rio.file.buffer, the buffer over the given synchronization minimum byte, you have to flush the contents of the buffer to the file.

/* Returns 1 or 0 for success/failure. *
///* Write buf to the Rio file
/static size_t Riofilewrite (Rio *r, const void *buf, size_t len) {
    size_t retval;
  retval = fwrite (BUF,LEN,1,R->IO.FILE.FP);
    r->io.file.buffered = len;

    if (R->io.file.autosync &&
        r->io.file.buffered >= r->io.file.autosync)
    {
    	// Whether synchronous
        Fflush (R->IO.FILE.FP) is required for interpretation;
        Aof_fsync (Fileno (R->IO.FILE.FP));
        r->io.file.buffered = 0;
    }
    return retval;
}