Redis Source Code Analysis (22)---Networking Network protocol transmission

The last time I only analyzed the parts of the Redis network part of the code, today I took networking's code implementation part also learned, networking's code more emphasis is the operation of client clients. The method operation inside the Addreply () series is the main part. This series of light-and-light methods should occupy half the number of APIs. I have divided the API into 3 parts:

/*------------API----------------------*/void *dupclientreplyvalue (void *o)/* Copy value one copy of */int listmatchobjects (void *a, void *b)/* Compare 2 obj equals */robj *duplastobjectifneeded (list *reply)/* Returns the last element object in the reply list */void Copyclientoutputbuffer (r  Edisclient *dst, Redisclient *src)/* Copy the output buffer of the source client to the target client */static void Acceptcommonhandler (int fd, int flags)/* Call method After network connection */void Accepttcphandler (aeeventloop *el, int fd, void *privdata, int mask) void Acceptunixhandler (Aeeventloop * El, int fd, void *privdata, int mask) void disconnectslaves (void)/* Causes the server's slave to lose connectivity */void Replicationhandlemasterdisco Nnection (void) void flushslavesoutputbuffers (void)/* from the method will be released in Freememoryifneeded (), freeing the memory space function to flush the results of in-memory data operations to disk */int processeventswhileblocked (void)/*-------------addreply API-----------------*/int _addreplytobuffer (redisclient *c , Char *s, size_t len)/* Add content to the client buffer */void _addreplyobjecttolist (redisclient *c, RobJ *o)/* robj add to reply list */void _a Ddreplysdstolist (Redisclient *c, SDS s) /* Add the SDS string object to the reply list */void _addreplystringtolist (redisclient *c, Char *s, size_t len)/* Add a String object in the reply list, the length of the given character in the parameter */voi D addreply (redisclient *c, RobJ *obj)/* Write data in buffer of redisclient, data exists obj->ptr pointer */void Addreplysds (redisclient *c , SDS s)/* Add the SDS string to the reply, the following amount Addreply () series method principle is basically similar to */void addreplystring (redisclient *c, Char *s, size_t len) void Addreplye Rrorlength (redisclient *c, Char *s, size_t len) void Addreplyerror (Redisclient *c, char *err)/* Add information about the error class to reply */void Addreplyerrorformat (redisclient *c, const char *FMT, ...) void Addreplystatuslength (Redisclient *c, Char *s, size_t len) void Addreplystatus (Redisclient *c, char *status) void addRe Plystatusformat (redisclient *c, const char *FMT, ...) void *adddeferredmultibulklength (Redisclient *c)/* Add an empty obj object to the reply list */void setdeferredmultibulklength ( Redisclient *c, void *node, long length) void addreplydouble (Redisclient *c, double D)/* Adds a double type value in bulk reply, meaning of bulk For large chunks, the meaning of bulk reply is the reply of the Big data volume */void AddreplylonglongwIthprefix (Redisclient *c, Long long ll, char prefix) void Addreplylonglong (Redisclient *c, long long ll) void Addreplymultib Ulklen (redisclient *c, long length) void Addreplybulklen (Redisclient *c, RobJ *obj)/* Add bulk chunk of data length */void addreplybulk ( Redisclient *c, RobJ *obj)/* To split the data of an obj into chunks of data added */void Addreplybulkcbuffer (redisclient *c, void *p, size_t len) void add Replybulkcstring (redisclient *c, char *s) void Addreplybulklonglong (Redisclient *c, Long long ll)/*-------------Client A PI-----------------*/redisclient *createclient (int fd)/* Create Redisclient Client, 1. Establish a connection, 2. Set the database, 3. property Settings */int Prepareclienttowrite (redisclient *c)/* This method will be called before the client prepares to accept the new data, and in the fileevent to set writer's handler processing event */static void FREECLIENTARGV (redisclient *c) void Freeclient (Redisclient *c)/* Release freeclient, to be divided into master and Slave2 kinds of situations for different processing */void Freeclientasync (redisclient *c) void Freeclientsinasyncfreequeue (void)/* Asynchronous free client */void sendreplytoclient ( Aeeventloop *el, int fd, void *privdata, int mask)/* The reply data in the clientDeposit file */void resetclient (redisclient *c) int processinlinebuffer (redisclient *c)/* handles the internal chain of the Redis client buffer, which is c-> Querybuf */static void Setprotocolerror (redisclient *c, int pos) int processmultibulkbuffer (redisclient *c)/* Handles large chunks of buffer */void ProcessInputBuffer (redisclient *c)/* processing redisclient query buffer */void readqueryfromclient ( Aeeventloop *el, int fd, void *privdata, int mask)/* Get query from client queries statement */void getclientsmaxbuffers (unsigned long *longes T_output_list, unsigned long *biggest_input_buffer)/* Gets the maximum length value of input buffer and output buffer in the client */void fo Rmatpeerid (char *peerid, size_t Peerid_len, char *ip, int port)/* Format the output of the Ip,port port number, Ip:port */int Genclientpeerid (REDISCL Ient *client, Char *peerid, size_t peerid_len)/* Get Ip,port address information for client clients */char *getclientpeerid (redisclient *c)/* Get c-& Gt;peerid Client Address information */sds catclientinfostring (SDS s, redisclient *client)/* Formatted output client property information, directly returning a stitched string */sds Getallclientsinfostring (void)/* Gets the property information for all client clients and connects to a total string and outputs the */vOID Clientcommand (redisclient *c)/* Execute the client's command */void rewriteclientcommandvector (redisclient *c, int argc, ...)/* Override Client The command set, the old command set of the application count minus 1, the new command vector collection of commands increased 1 */void rewriteclientcommandargument (redisclient *c, int i, robj *newval)/* Override the I parameter in the client */unsigned long getclientoutputbuffermemoryusage (redisclient *c)/* To get the size of the output buffer that has been used in the client */int Getclienttype (redisclient *c) int getclienttypebyname (char *name)/* 3 types of names in the client, normal,slave,pubsub */char * Getclienttypename (int class) int checkclientoutputbufferlimits (redisclient *c)/* Determines if the size of the output buffer for Clint exceeds the soft limit or the hard limit */ void asynccloseclientonoutputbufferlimitreached (Redisclient *c)/* Asynchronously closes the client, if the soft limit in the buffer or the hard limit has arrived, The result of a buffer exceeding the limit causes the release to be unsafe, */

We start by adding the reply data from the simplest _addreplytobuffer in the buffer, because the various addreply methods in the back are more or less called and this song method.

/*-----------------------------------------------------------------------------* Low level functions to add more data To output buffers. *--------------------------------------------------------------------------*//* Add content to the client buffer */int _ Addreplytobuffer (redisclient *c, Char *s, size_t len) {    size_t available = sizeof (C->BUF)-c->bufpos;    if (C->flags & redis_close_after_reply) return REDIS_OK;    /* If there already is entries in the Reply list, we cannot     * add anything + to the static buffer. *     //If the current R Eply the content already exists, the operation error occurs    if (Listlength (c->reply) > 0) return redis_err;    /* Check that the buffer have enough space available for this string. *    /if (len > available) return redis_err;    memcpy (C->buf+c->bufpos,s,len);    c->bufpos+=len;    return REDIS_OK;}

The most direct impact of the sentence is memcpy (C->buf+c->bufpos,s,len), so the content is added to the C->buf, which is the client output buffer, adding an operation there is another form of adding an object type:

/* RobJ added to reply's list */void _addreplyobjecttolist (redisclient *c, RobJ *o) {robj *tail;    if (C->flags & redis_close_after_reply) return;        if (listlength (c->reply) = = 0) {incrrefcount (o);        Add RobJ content Listaddnodetail (c->reply,o) in the Reply list summary;    C->reply_bytes + = Zmalloc_size_sds (o->ptr);        } else {tail = Listnodevalue (Listlast (c->reply)); /* Append to this object when possible. */if (tail->ptr! = NULL && Sdslen (tail->ptr) +sdslen (o->ptr) <= redis_reply_chunk_by            TES) {c->reply_bytes-= Zmalloc_size_sds (TAIL-&GT;PTR);            Tail = duplastobjectifneeded (c->reply);            Tail->ptr = Sdscatlen (Tail->ptr,o->ptr,sdslen (o->ptr));        C->reply_bytes + = Zmalloc_size_sds (tail->ptr);            } else {incrrefcount (o);            Listaddnodetail (C->reply,o);        C->reply_bytes + = Zmalloc_size_sds (o->ptr);   } } asynccloseclientonoutputbufferlimitreached (c);} 

The RobJ object is loaded into the reply list, and the byte size of the reply is changed, and finally a asynccloseclientonoutputbufferlimitreached (c) is called; method, which I found at the bottom of this file. , I really don't know what that means at first. When the data is added, when the client's output buffer size exceeds the limit, it is closed asynchronously:

/* asynchronously close a client if soft or hard limit was reached on the * output buffer size . The caller can check if the client would be a closed * checking if the client REDIS_CLOSE_ASAP flag is set. * * Note:we need to close the client asynchronously because this function was * called from contexts where the client can ' T be freed safely, i.e. from the * lower level functions pushing data inside the client output buffers. *//* the client is closed asynchronously, if the soft limit in the buffer or the hard limit has arrived, the result of the buffer exceeding the limit will cause the release unsafe, */void asynccloseclientonoutputbufferlimitreached (    Redisclient *c) {Redisassert (C->reply_bytes < ulong_max-(1024*64));    if (c->reply_bytes = = 0 | | C->flags & REDIS_CLOSE_ASAP) return;        if (Checkclientoutputbufferlimits (c)) {SDS client = catclientinfostring (Sdsempty (), c);        Freeclientasync (c);        Redislog (redis_warning, "client%s scheduled to is closed ASAP for overcoming of output buffer limits.", client);    Sdsfree (client); }}
 

At the time of the Addreply method invocation, it is sometimes necessary to have a precondition, and I say that when writing data events, you have to create a listener event for the written file:

/* Add SDS string in reply */void Addreplysds (redisclient *c, SDS s) {//Before invoking the add operation, perform Prepareclienttowrite (c) and set the write event of the file event    if (Prepareclienttowrite (c)! = REDIS_OK) {        /* The caller expects the SDS to being free ' d. *        /Sdsfree (s);        return;    }    if (_addreplytobuffer (C,s,sdslen (s)) = = Redis_ok) {        sdsfree (s);    } else {/* This method Free's the        SDS when it is no longer needed. *        /_addreplysdstolist (c,s);    }}

What do you do in this prepareclienttowrite ()?

/* This function is called every time we be going to transmit new data * to the client. The behavior is the following: * * If the client should receive new data (normal clients would) the function * returns Redi S_OK, and make sure to install the write handler in we event * loop so that's when the socket is writable new data gets WRI Tten.  * If the client should not receive new data, because it was a fake client, * a master, a slave not yet online, or because The setup of the write handler * failed, the function returns REDIS_ERR. * * Typically gets called every time a reply is built, before adding + * data to the clients output buffers. If the function returns REDIS_ERR no * data should is appended to the output buffers. *//* This method will be called before the client prepares to accept the new data, and in Fileevent sets the writer's handler processing event */int Prepareclienttowrite (redisclient *c) {if (    C->flags & Redis_lua_client) return REDIS_OK; if ((C->flags & Redis_master) &&! ( C->flags & REDIS_MASTER_FORCe_reply)) return redis_err; if (c->fd <= 0) return redis_err; /* Fake Client */if (C->bufpos = = 0 && listlength (c->reply) = = 0 && (c->replstate = = R         Edis_repl_none | | C->replstate = = redis_repl_online) &&//Create a write file here event aecreatefileevent (Server.el, C-&GT;FD, Ae_wri    TABLE, sendreplytoclient, c) = = Ae_err) return redis_err; return REDIS_OK;}

In the Addreply method mentioned a Addreplybulk type method, bulk Chinese meaning for large chunks, the addreplybulk added are some of the larger data, find a way to see:

/* Add a Redis Object as a bulk reply *//* to split the data of one obj into chunks of data added */void Addreplybulk (redisclient *c, RobJ *obj) {//reply Add long Degree    Addreplybulklen (c,obj);    Reply Add Object    addreply (c,obj);    Addreply (C,SHARED.CRLF);}

Split the original RobJ data into 3 common addreply method calls. It becomes the data that is large. A bad place to reply to big data is when it is time to parse or copy data. The Freeclient () operation is also provided in the networking method:

/* Release freeclient, to be divided into master and Slave2 case for different processing */void freeclient (redisclient *c) {    ListNode *ln;    /* If the marked as current client unset it *    /if (server.current_client = = c) server.current_client = NULL;    /* If It is your master that's beging disconnected we should make sure * to cache the state to     try a partial Resynchron ization later.     *     * Note that before doing this we do sure that the client are not     in * some unexpected state, by checking its FL Ags. */    if (server.master && c->flags & redis_master) {        redislog (redis_warning, "Connection with Master lost. ");        if (! ( C->flags & (redis_close_after_reply|                          redis_close_asap|                          redis_blocked|                          redis_unblocked))        {        //If the master client requires processing of the cache client, the            Replicationcachemaster (c) can be quickly re-enabled;            return;        }    }

... The following code omits the

When the output buffer data in the client is getting more and more ready to be persisted to the disk file, call the following method,

/* Helper function used by freememoryifneeded () on order to flush slave * output buffers without returning control to the Event loop. *//* from the method will be in freememoryifneeded (), freeing the memory space function to flush the results of in-memory data operations to disk */void flushslavesoutputbuffers (void) {    listiter li ;    ListNode *ln;    Listrewind (Server.slaves,&li);    while ((ln = listnext (&li))) {        Redisclient *slave = listnodevalue (LN);        int events;        Events = Aegetfileevents (SERVER.EL,SLAVE->FD);        if (Events & ae_writable &&            slave->replstate = = Redis_repl_online &&            listlength ( slave->reply)        {        //here called the Write method            sendreplytoclient (server.el,slave->fd,slave,0);    }}}

The core call of this method is in the Sendreplytoclient () method, which is to save the client's reply content and buf content to the file. The above is my understanding, the code is a bit large, I really look at the head a bit big.

Redis Source Code Analysis (22)---Networking Network protocol transmission