connection establishment and reconnection mechanism in ZeroMQ (Java)

The previous article analyzes the simplest type of socket in Zeromq, Dealer. However, I think this specific type of socket analysis can be left to later, or later when the use of the analysis is no later ....

However, as a framework for message communication, the most important thing is the reliability of communication, and the most important thing is the reconnection mechanism after disconnect ...

Before looking at the specific reconnection mechanism, let's look at how the ZEROMQ is active in the remote connection, first look at the Connect method defined in Socketbase:

 Connection to remote address public boolean connect (String addr_) {if (ctx_terminated) {throw new Zerror.ctxter
        Minatedexception ();
        }//Process pending commands, if any.
        Boolean BRC = Process_commands (0, false);

        if (!BRC) return false;
        Parse addr_ String.
        URI Uri;   try {uri = new Uri (ADDR_);
        Build Uri Object} catch (URISyntaxException e) {throw new IllegalArgumentException (e);   String protocol = Uri.getscheme ();
        Gets the protocol type String address = uri.getauthority ();
        String path = Uri.getpath ();

        if (address = = NULL) address = path;  Check_protocol (protocol);  Check if the protocol type is qualified if (Protocol.equals ("InProc")) {//If the communication within the process//Todo:inproc Connect is specific With respect to creating pipes//As there ' s no ' reconnect ' functionality implemented. Once that//are inPlace we should follow generic pipe creation algorithm.
            Find the peer endpoint.
            Ctx.endpoint peer = Find_endpoint (ADDR_);
            if (Peer.socket = null) return false;
            The total HWM for a inproc connection should be the sum of//The binder ' s HWM and the connector ' s HWM.
            int SNDHWM = 0; if (options.sndhwm!= 0 && peer.options.rcvhwm!= 0) sndhwm = options.sndhwm + PEER.OPTIONS.RCVHWM
            ;
            int RCVHWM = 0; if (options.rcvhwm!= 0 && peer.options.sndhwm!= 0) rcvhwm = options.rcvhwm + PEER.OPTIONS.SNDHWM

            ;
            Create a bi-directional pipe to connect the peers.
            Zobject[] Parents = {this, peer.socket};
            Pipe[] pipes = {null, NULL};
            Int[] Hwms = {SNDHWM, RCVHWM};
            Boolean[] delays = {options.delay_on_disconnect, options.delay_on_close}; Pipe.pipepair (PArents, pipes, hwms, delays);
            Attach the pipe to this socket object.

            Attach_pipe (pipes [0]);
            If required, send the identity of the peer to the local socket.
                if (peer.options.recv_identity) {msg id = new MSG (options.identity_size);
                Id.put (options.identity, 0, options.identity_size);
                Id.set_flags (msg.identity);
                Boolean written = pipes [0].write (ID);
                ASSERT (written);
            Pipes [0].flush ();
            }//If required, send the identity of the local socket to the peer.
                if (options.recv_identity) {msg id = new MSG (peer.options.identity_size);
                Id.put (peer.options.identity, 0, peer.options.identity_size);
                Id.set_flags (msg.identity);
                Boolean written = pipes [1].write (ID);
            ASSERT (written);    Pipes [1].flush (); }//Attach remote end of the pipe to the peer socket. Note that peer ' s//Seqnum is incremented in find_endpoint function.
            We don ' t need it//increased here.

            Send_bind (Peer.socket, pipes [1], false);

            Save last endpoint URI options.last_endpoint = addr_;

            Remember InProc connections for disconnect inprocs.put (ADDR_, pipes[0]);
        return true;
        //Select a compare IO thread to deploy the session will create love session Iothread Io_thread = Choose_io_thread (options.affinity);
        if (Io_thread = = null) {throw new IllegalStateException ("Empty io thread");

        //Create Address object address PADDR = new Address (protocol, address);
            if (Protocol.equals ("TCP")) {//If TCP paddr.resolved (new Tcpaddress ()); Paddr.resolved (). Resolve (address, options.ipv4only!= 0? true: FALSE);
            else if (protocol.equals ("IPC")) {//interprocess communication paddr.resolved (new ipcaddress ());
        Paddr.resolved (). Resolve (address, true);
        }//Create session. The first parameter is the IO thread that the current session will attach to, and the second parameter indicates the need to actively establish the connection sessionbase session = Sessionbase.create (Io_thread, True, thi
        s, options, PADDR);

        ASSERT (session!= NULL); PGM does not support subscription forwarding;
        Ask for the all data to is//sent to this pipe.
        Boolean icanhasall = false;

        if (Protocol.equals ("PGM") | | | protocol.equals ("EPGM")) Icanhasall = true;  Create a pipe association that connects the session to the current socket if (options.delay_attach_on_connect!= 1 | | icanhasall) {//Create a
            Bi-directional Pipe.
            Zobject[] Parents = {this, session};
            Pipe[] pipes = {null, NULL};
            Int[] Hwms = {OPTIONS.SNDHWM, OPTIONS.RCVHWM}; Boolean[] delays = {Options.delay_on_disconnECT, options.delay_on_close};

            Pipe.pipepair (parents, pipes, hwms, delays);
            Attach the pipe to the socket object.

            Associates the first pipe with the current socket Attach_pipe (pipes [0], Icanhasall);
            Attach remote end of the "pipe to" session object later.
        Associate another pipe with the session so that the session and socket can communicate through pipe session.attach_pipe (pipes [1]);

        }//Save last endpoint URI options.last_endpoint = paddr.tostring ();  Add_endpoint (Addr_, session);
    Associate this session with this address return true; }

Here is mainly concerned about the establishment of TCP connections, after all, in a distributed environment or again with TCP, through the previous article, we know that a socket below may correspond to multiple connections, and each connection in fact corresponds to a Streamengine object, And each of the Streamengine objects are associated with a session object for interaction with the upper socket, so here you can see the most important thing to do is to create the session object, as well as the pipe object .... Then call the Add_endpoint method to deploy the session, and then take a look at this approach:

    This manages the address and session, and actually records all the current connection-setting addresses, as well as the relative session
    private void Add_endpoint (String addr_, Own endpoint_) {
        //  Activate the session. Make it a child of this socket.
        Launch_child (ENDPOINT_);   To deploy this endpoint, the main point here is to add this endpoint to IO thread
        endpoints.put (addr_, endpoint_);
    
    

This is actually used instead of the session object, then the Process_plug method will be executed for the sessions object, so let's look at the definition of this method:

    Execute the plug command, and if you need to connect, start the connection
    protected void Process_plug () {
        io_object.set_handler (this);  Sets the Io object's handler, which is used to respond to IO event if
        (connect) {  //If there is a need to initiate a connection with the remote remotely, then start the connection
            start_connecting (false);   Start the connection, false indicates no wait
        }
    }

This will first set the current IO object's event callback, connect property, set when the session is created, if it is the active creation of the connection will be true, if the listener received the connection, then it will be false, here to see the definition of this method:

    If connect, then you need to call this method to establish a connection to the private void Start_connecting (Boolean wait_) {assert (connect); Choose I/O thread to run connecter in.
        Given that we are already//running in a I/O thread, there must is at least one available.  Iothread Io_thread = Choose_io_thread (options.affinity);

        Select an IO thread to deploy the Tcpconnector assert (Io_thread!= null) for the pending;

        Create the Connecter object. if (Addr.protocol (). Equals ("TCP")) {Tcpconnecter connecter = new Tcpconnecter (io_thread, th
            is, options, addr, wait_);
            Alloc_assert (connecter);  Launch_child (connecter);
        Deploy this tcpconnector return;
                } if (Addr.protocol (). Equals ("IPC")) {Ipcconnecter connecter = new Ipcconnecter (
            Io_thread, this, options, addr, wait_);
            Alloc_assert (connecter);
            Launch_child (connecter);
       Return ASSERT (FALSE); }

Here comes a parameter that will be used when building the Tcpconnector to indicate whether the connection was built to be deferred. When the connection was first established, it was false, which means no delay, and when the connection is attached it will be found that the deferred connection will be used in the reconnection ...

Here can also see the establishment of a specific connection, in fact, is entrusted to the Tcpconnector object to do, it is actually a tool class ...

Concrete it is how to establish the connection is not detailed list come, probably say the process:

(1) Create a Socketchannel object and set it to non-blocking, and then call the Connect method to establish a connection to the remote address

(2) Register the Socketchannel to the IO thread's poller and set the Connect event

(3) for the Connect event callback to do, in fact, on the Poller object to remove this Socketchannel registration, and then create a new Streamengine object to wrap this socketchannel, The Streamengine object is then associated with the just session object ...

Here we can take a look at this connect event callback method to do something about it:

    Connection-established event callback, it is also possible that the connection timeout public void Connect_event () {Boolean err = false;
        Socketchannel FD = null;   try {fd = connect ();
        Gets the channel} catch (Connectexception e) {err = true that already has a connection established;
        catch (SocketException e) {err = true;
        catch (Sockettimeoutexception e) {err = true;
        catch (IOException e) {throw new Zerror.ioexception (e);  } io_object.rm_fd (handle);
        
        The current ioobject can be removed from the Poller and the Tcpconnector is invalidated, handle_valid = false;
            if (err) {//Handle the error condition by attempt to reconnect.   
            Close ();  Add_reconnect_timer ();
        Try to re-establish the connection return;
        
        } handle = null;
            try {utils.tune_tcp_socket (FD); Utils.tune_tcp_keepalives (FD, Options.tcp_keepalive, options.tcp_keepalive_cnt, OPTIONS.TCp_keepalive_idle, OPTIONS.TCP_KEEPALIVE_INTVL);
        catch (SocketException e) {throw new RuntimeException (e);
        
        //Create The engine object for this connection.
        Creates a Streamengine object that encapsulates the channel streamengine engine = NULL, which is a good connection;
        try {engine = new Streamengine (FD, options, endpoint);
            catch (Zerror.instantiationexception e) {socket.event_connect_delayed (endpoint,-1);
        Return
        }//Attach the engine to the corresponding session object.  Send_attach (session, engine);
        Bind this engine to the session, and then bind the current streamengine to the IO thread, which is registered on the Poller//shut the connecter down.  Terminate ();  Closes the current connector socket.event_connected (endpoint, FD); Message to the upper socket notification connection established

What the specific code is very straightforward to see it, here you can also see the connection to establish a timeout will also try to reconnect ...

Well, how to establish a connection here is quite clear. Then look at how the connection will be reconnected after the disconnect, first to see what happens after the connection is disconnected,

The first thing to know is how to tell if the channel connection is disconnected. How to judge, well, this somewhat basic should know, if the connection has been disconnected, then read on channel will return-1, OK then we know where the code should begin to look, Well, look at Streamengine's In_event method and see what it does after read returns-1:

    callback method when the underlying Chanel has data to read the public void In_event () {if (handshaking) if (!handshake ())
        
        Return
        ASSERT (decoder!= null);

        Boolean disconnection = false; If there ' s no data to process in the buffer ... if (insize = 0) {//If there is no information in inbuf it needs to be processed//Retrie
            ve the buffer and read as much data as possible. Note So buffer can be arbitrarily large. However, we assume//the underlying TCP layer has fixed buffer size and thus the/number of B
            Ytes read would be always limited.  Inbuf = Decoder.get_buffer ();  Gets buf from the decoder for writing read data, because the size of the TCP receive buffer that has the underlying socket set insize = Read (INBUF);  Used to write the data sent over to the BUF and record the size inbuf.flip ();
            Here is ready to read the data from the BUF/Check whether the peer has closed the connection.
                if (insize = = 1) {//if is-1, indicates that the underlying socket connection has been problematic insize = 0; DIsconnection = true;
        Set flag bit}//Push the data to the decoder.  int processed = Decoder.process_buffer (Inbuf, insize);
        Resolves these read data if (processed = = 1) {disconnection = true;
            else {//Stop polling for input if we got stuck.

            if (processed < insize)//If the data processed is not yet read much, then the registration Io_object.reset_pollin (handle) of the Read event is canceled;
            Adjust the buffer.  Insize-= processed;
        There is still the size of the data not processed}//Flush All messages The decoder may have produced.  Session.flush (); Decoder resolution of the data to the session//A input error has occurred.
        If the last decoded message//has already been accepted, we terminate the engine immediately. Otherwise, we stop waiting for sockets events and postpone//the termination until after the ' message ' is ACCEP
        Ted. if (disconnection) {//indicates that the connection has been disconnected, you need to process the if (dEcoder.stalled ()) {IO_OBJECT.RM_FD (handle);
            io_enabled = false;
        
            else {error (); }
        }
    }

Well, here you can see that if you return-1, you set the disconnection flag bit and then call the error method to make an error, and then look at what the error method does:

    Error, then let the upper ZMQ socket close the current connection
    private void error ()  {
        assert (session!= null);
        socket.event_disconnected (endpoint, handle);  Here can be understood as notification of upper sockets,
        Session.detach ();   This is mainly used for session cleaning and socket pipe, and then try to reconnect
        unplug ();  Cancellation of the registered
        destroy () above the poller;  Close the underlying channel and close the current
    }

In fact, if the underlying link is disconnected, then the current channel also invalid, then the current Streamengine object is not valid, then the thing to do is to destroy the current object, and then also to remove the registration in the poller above, and then notice the upper socket, The current connection to this link address has been disconnected ... Of course also tell the session object, let it do some processing, session processing includes reconnection, then to see what he did:

    The equivalent is the associated public
    Void detach ()  {
        //Engine is dead to remove the underlying Engine  . Let ' s forget about it.
        engine = NULL;  This is equivalent to releasing the current engine object

        //  Remove any half-done messages from the pipes.
        Clean_pipes ();  Clear those that have not been accepted for MSG
 
        //Send the event to the  derived class.
        Detached ();   Cancel pipe, then reconnect

        //Just in case there's only  a delimiter in the pipe.
        if (pipe!= null)
            pipe.check_read ();
    }

The code for reconnection is not seen here, and then continue to look at the detached method:

    private void detached () {//Transient session self-destructs after peer disconnects.
            if (!connect) {//If the connection is not actively established, then it is better to terminate directly otherwise the attempt to reconnect is terminate ();
        Return  }//For delayed connect situations, terminate the pipe//and reestablish on if (later pipe Null && options.delay_attach_on_connect = 1 && addr.protocol ()!= "PGM" && Addr.pro
            Tocol ()!= "EPGM") {Pipe.hiccup ();
            Pipe.terminate (FALSE);
            Terminating_pipes.add (pipe);
        pipe = NULL;  Reset ();   Reset flag bit//The attempt to initiate reconnection here if (OPTIONS.RECONNECT_IVL!=-1) {start_connecting (true);
        For reconnection attempts, there is a need for some delay}//For subscriber sockets we hiccup the inbound pipe, which will cause
        The socket object to resend all the subscriptions. if (pipe!= null && (Options.type = = ZMQ.zmq_sub | | Options.type = = Zmq.

    zmq_xsub)) Pipe.hiccup (); }

Here you can see that the Start_conneting method is invoked, but the arguments passed here are true, and the specific execution process is similar to the one above, except that the connection is deferred ...

This means that the timer is set on the IO thread and will not be connected until after the timeout ... This also makes the reconnection in a certain frequency ...

The specific timing is not detailed, quite simple ...

Through the above code you can know that zeromq after the connection is disconnected, if the connection was built on its own initiative, rather than listener acquired, then automatically try to reconnect. Well, it's not bad.