On multithreading implementation of message bus client

The last time I talked about a client that was writing a RABBITMQ-based message bus in the face of concurrency problems and the final implementation of the solution. It's a simple and not prone to concurrency problem scenario, and if you read that article, I've given the pros and cons of the final implementation scenario.

The core issue is whether client-created connection with RABBITMQ server is shared or exclusive. One more popular example of this problem: if you want to rent a house, everyone will have different ideas. For example, some people like simple, quiet life and care about personal privacy, then your best choice is to rent a single room cover: There is everything, and is exclusive, its shortcomings are caused by the waste of resources (such as you need to occupy the washing machine, refrigerators, etc. you do not use them efficiently), So you have to pay an expensive price for these resources, and if you don't want to waste resources, there's something to share with others, and you want to have a higher price/performance ratio, then the best option in this case is actually--co-renting. This is also the subversion of my self-realization since the last article. In this article I will explore new ways to achieve this.

As stated in the last article , it is not just a client for this message bus, it is a solution that can be referenced by other generic technology components.

Problem analysis

First of all, what was the last tangled question? The life cycle of a connection is limited to the client Master object (its creation and destruction are dependent on the client Master object), and if connection can be shared by multiple client objects within the same JVM process, the ownership of its life cycle control is a problem.

To share a RABBITMQ connection within a JVM process, I can only choose to implement it as a singleton. How is the life cycle of connection managed for this implementation? is done through the open/close of the client Master object. The first call to the client's Open method triggers the instantiation of the connection, after which the connection has been open, and when Close is called, the connection is closed together (not knowing if anyone will ask, Why do you want to close the connection when calling the Close method, because it must be implemented in a non-multithreaded implementation, if it is a single-threaded, the client object to complete the task, it is necessary to call the Close method, and at this point you have to close the connection , otherwise there is no time for the call). But here is the question of the Close method call and timing, because in a multithreaded environment each channel is created in the shared connection, and if a client shuts down the connection, the other in-use channel will all throw an exception. That's why, in the last article, my thinking shifted to the way each client exclusively connection.

But I still want to find a solution for sharing the connection model. Because the RABBITMQ channel is a multiplexed design that has provided the basis for us to reuse a connection on multiple threads, if each client exclusive connection, it is undoubtedly breaking best practices (Redis's client Jedis implements connection pool to monopolize connection because Redis does not have the concept of channel in communication).

Solution Ideas

The fundamental crux of the above problem is the life cycle management of connection objects. The connection object is hosted inside the client. If we let its creation, initialization and destruction depend only on the object pool, does it not have this problem? At the same time, there are two key objects: Pubsubermanager (used to get a configuration center configuration), ConfigManager (for subscribing to the configuration Center for data changes and parsing the configuration), and they can also be raised together to share the same instance in the same pool.

Technology implementation

The object pool uses the infrastructure provided by Apache Common pool. For the lifecycle management of several shared objects above, there are two methods defined in the pool:

-init: Used to initialize several key objects such as connection

-destroy: Used to destroy key objects in the Init method

Init:

protected void init () {This.exchangemanager = new Exchangermanager (This.pubsuberhost, This.pubsuberport);        if (!this.exchangemanager.ispubsuberalive ()) throw new RuntimeException ("Can not connect to PubSub server.");        This.configmanager = new ConfigManager ();        This.configManager.setExchangeManager (This.exchangemanager);        This.poolid = Randomhelper.randomnumberandcharacter (12); This.exchangeManager.registerWithMultiChannels (Poolid, This.configmanager, new string[]{Constants.pubsub_route R_channel, Constants.pubsub_config_channel, Constants.pubsub_event_channel, constants.pub        Sub_sink_channel, Constants.pubsub_channel_channel,});            try {this.configManager.parseRealTimeData ();            String host = This.configManager.getClientConfigMap (). Get ("Messagebus.client.host"). GetValue ();        ConnectionFactory connectionfactory = new ConnectionFactory ();    Connectionfactory.sethost (host);        This.connection = Connectionfactory.newconnection ();        } catch (IOException e) {throw new RuntimeException (e); }    }

Destroy

public void Destroy () {        This.innerPool.destroy ();        Release Resource        if (Exchangemanager! = null)            exchangemanager.removeregister (this.poolid);        if (ConfigManager! = null)            Configmanager.destroy ();        if (this.connection! = null && this.connection.isOpen ()) {            try {                this.connection.close ();            } catch ( IOException e) {                throw new RuntimeException (e);}}    }

As you can see in the Destroy method above, the connection is closed at the end, after destroy the real object pool first. In this way, the life cycle of the connection object is associated with the pool's life cycle, and is not related to the client object in the pool. They only need to get open connection objects to create the channel for communication (now only the channel is a one-to-the-client object).

Although we have placed several key objects that the client relies on in the pool to build, we also need to pass them on to the client object. Two common ways to inject dependent objects: constructor injection, setter method injection are not feasible here, because we do not expect external objects to understand the details of the client. Therefore, instead of providing public injection points, they are defined as private instance fields within the client, then open access through reflection, and then turn off access after injection.

Private instance fields:

Inject by reflector    private Exchangermanager Exchangemanager;    Private ConfigManager    ConfigManager;    Private Connection       Connection;

References to reflection injection instances:

Public pooledobject<messagebus> Makeobject () throws Exception {constructor<messagebus> Privatector = M        Essagebus.class.getDeclaredConstructor ();        Privatector.setaccessible (TRUE);        Messagebus client = Privatector.newinstance ();        Privatector.setaccessible (FALSE);        class<?> superclient = Messagebus.class.getSuperclass ();        Set private Field field Exchangemanagerfield = Superclient.getdeclaredfield ("Exchangemanager");        Exchangemanagerfield.setaccessible (TRUE);        Exchangemanagerfield.set (client, This.exchangemanager);        Exchangemanagerfield.setaccessible (FALSE);        Field Configmanagerfield = Superclient.getdeclaredfield ("ConfigManager");        Configmanagerfield.setaccessible (TRUE);        Configmanagerfield.set (client, This.configmanager);        Configmanagerfield.setaccessible (FALSE);        Field Connectionfield = Superclient.getdeclaredfield ("Connection");    Connectionfield.setaccessible (TRUE);    Connectionfield.set (client, this.connection); Connectionfield.setaccessible (FALSE);

Multi-Threading and single-threaded consistent model

The pool's build is usually located on the main thread, and it is built on all the clients before it is destroyed by all client tasks, so the connection life cycle is the life cycle of any client, The channel created by the client using the connection can also be turned off when the client is closed. Regardless of the multi-threaded environment, or single-threaded environment, the client main object implementation mechanism is best only one set, if the mechanism is different, the internal implementation will be quite different, so whether from the design of elegance and workload is not a good design solution. Therefore, since the instantiation of connection refers to the outside of the client main object, the corresponding changes must be made under the single-threaded model. The final scenario is that the object pool is also used in a single-threaded environment, but it is a pool of objects with a capacity of 1, which is implemented by inheriting the generic object pool, then configuring the pool's maxtotal (the maximum number of objects) to 1, and allowing the caller to modify the pool's parameter configuration externally.

The built-in Maxtotal is a 1 object pool configuration:

Public Messagebussinglepool (String pubsuberhost, int pubsuberport) {        super (Pubsuberhost,              Pubsuberport,              New Defaultmessagebuspoolconfig ());    }    /**     * Inner class:single messagebus instance Config     * *    private static Class Defaultmessagebuspoolconfig ext Ends Genericobjectpoolconfig {public        defaultmessagebuspoolconfig () {            settestwhileidle (false);            Setmaxtotal (1);            Setminevictableidletimemillis (60000);            Settimebetweenevictionrunsmillis (30000);            Setnumtestsperevictionrun ( -1);        }    }

Privatization constructor:

Private Messagebus () {        super ();        Producer = new Genericproducer ();        Consumer = new Genericconsumer ();        Publisher = new Genericpublisher ();        Subscriber = new Genericsubscriber ();        Requester = new Genericrequester ();        Responser = new Genericresponser ();        Broadcaster = new Genericbroadcaster ();    }

It's not going to work, we have to stop users from being able to create all the ways of the client so they can't get objects from the object pool, so we also need to set the constructor of the client master object to private, and then create the object through reflection in the method of creating the pooled object in the object pool:

Public pooledobject<messagebus> Makeobject () throws Exception {        constructor<messagebus> privatector = Messagebus.class.getDeclaredConstructor ();        Privatector.setaccessible (true);        Messagebus client = Privatector.newinstance ();        Privatector.setaccessible (FALSE);

The problem is not over here, in the original program connection, such as the creation of a few key objects, destruction is actually dependent on the client's two key Api:open, close. Now in this mode, the life cycle of these key objects can no longer be controlled by these two APIs (because they are shared by connection), and these two APIs only control the channel objects associated with each client Master object. However, objects in the object pool should be stateless or state-consistent, and the Apache pool itself provides a way to control the state of the object: Make/destroy/active, and so on. So in order for the client Master object state to be consistent (without the channel of some client objects open and some closed), it is necessary to reclaim the external visibility of these two APIs (the access identifier is set to: private). It is then called uniformly by the pool. For the invocation of these two methods, we still invoke them through reflection:

To obtain a reflection instance of a Method object (an instance of the methods objects):

try {            OpenMethod = Messagebus.class.getSuperclass (). Getdeclaredmethod ("open");            Openmethod.setaccessible (true);            CloseMethod = Messagebus.class.getSuperclass (). Getdeclaredmethod ("close");            Closemethod.setaccessible (True);        } catch (Nosuchmethodexception e) {            throw new RuntimeException (e);        }

Unified change of State in thread security and consistent environment (the following methods are triggered within the pool):

public void Destroyobject (Pooledobject<messagebus> pooledobject) throws Exception {        Messagebus client = Pooledobject.getobject ();        if (client! = NULL) {            if (Client.isopen ()) {                closemethod.invoke (client);}}    }

public void Activateobject (Pooledobject<messagebus> pooledobject) throws Exception {        Messagebus client = Pooledobject.getobject ();        if (client! = NULL) {            if (!client.isopen ()) {                openmethod.invoke (client);}}    }

public void Passivateobject (Pooledobject<messagebus> pooledobject) throws Exception {        Messagebus client = Pooledobject.getobject ();        if (client! = NULL) {            if (Client.isopen ()) {                closemethod.invoke (client);}}    }

Now, whether we use the client in single-threaded or multi-threaded, we get it from the object pool, and the client is now only responsible for communicating with the RABBITMQ server through the channel, and it doesn't have to worry about anything else.

Above, three motives for using reflection are explained:

• Dynamic injection of client main object depends on the connection object, the external access interface is not exposed;
  
• Dynamic creation of client main object, no external exposure to the constructor;
  
• Dynamic call to client's open/close, external exposure method;
  

The biggest advantage of model advantage sharing connection is that it conforms to the HttpbridgeThe implementation. In addition to providing a way for the Java client to call, the message bus also provides a cross-platform communication scheme--httpbridge. The principle of this mode is also very simple, is to use the platform independence of HTTP to pass the request parameters to the background, the background is the servlet container, you can again convert the environment to Java, and then again rely on the Java client call, you can consider it as a client agent. Now that the client Master object is pooled, you can control the concurrency of the Httpbridge deployment node by configuring the size of the objects in the pool. And the use of RABBITMQ server resources (TCP connections consumed by connection) is much less than before, in which case a deployment instance requires a connection object, whereas the previous model was in Httpbridge. How many connection are required for concurrent requests that are being processed. Now this model can be very convenient to scale the horizontal expansion.

Complete implementation code, please visit: Banyan

On multithreading implementation of message bus client