Openfire cluster source code analysis

Openfire cluster source code analysis

To solve the throughput problem, if the number of users increases, you need to introduce the cluster, provide cluster support in openfire, and implement two cluster plug-ins: hazelcast and clustering. To understand the working principle of the cluster, I analyzed the source code of openfire, which is also a learning process.

First, understand some simple concepts about clusters. The purpose of a cluster is to make multiple instances run like one. In this way, you can increase the computing power by increasing instances. That is, the so-called distributed computing problem. One of the most important features is the CAP theory, which is also called consistency, availability, and partition fault tolerance. CAP is the core solution in the cluster. The overall understanding of CAP is what I wrote above. Multiple instances run as one instance. Therefore, the so-called cluster is to share some data or synchronize it to different instances. In this way, the system uses the same algorithm and the results should be of course the same. Therefore, master-slave replication and cache data clusters of some databases are similar to this solution. It's just about code implementation quality and processing scale. With this foundation, let's take a look at how openfire solves this problem. Openfire cluster Design 1. What needs to be synchronized between clusters? For openfire, data in these aspects must be synchronized between clusters: data inventory data, cache data, and session. This seems like it's all about it, right?

Because the database is basically transparent to openfire, this is done by the database itself. The cached data cache exists in the memory. Therefore, sessionsession to be synchronized does not need to be synchronized across all instances in openfire, but user routing cache is required, otherwise, the corresponding session cannot be found during message sending. Therefore, the user route must be synchronized.

2. Cache Design

• Cache Interface

Openfire provides a packaging interface for the cached data container, which provides the basic method for caching data for unified data operations.

public interface Cache<K,V> extends java.util.Map<K,V>

If the default cache container class is public class DefaultCache <K, V> when the cluster is not enabled, DefaultCache uses a Hashmap to store data.

• Cache factory type

To ensure that the cache is scalable, a factory class is provided:

public class CacheFactory

The CacheFactory class manages all the cache containers. The Code is as follows:

    /**     * Returns the named cache, creating it as necessary.     *     * @param name         the name of the cache to create.     * @return the named cache, creating it as necessary.     */    @SuppressWarnings("unchecked")    public static synchronized <T extends Cache> T createCache(String name) {        T cache = (T) caches.get(name);        if (cache != null) {            return cache;        }        cache = (T) cacheFactoryStrategy.createCache(name);         log.info("Created cache [" + cacheFactoryStrategy.getClass().getName() + "] for " + name);         return wrapCache(cache, name);    }

In the code above, a cache container is created through the cache Factory policy object, and the warpCache method puts the container into caches.

• Cache factory policies

In CacheFactory, A DefaultLocalCacheStrategy is used by default to create a cache. In addition, Cache Policy access under cluster conditions is also provided. That is, the cache management solution is switched by instantiating different policies. For example, hazelcast mentioned later replaces the local cache policy. From the interface design point of view, openfire's cache policy is to implement clusters and non-clusters. 3. clusters in openfire mainly include cluster management, data synchronization management, and cluster computing tasks. The Cluster Manager is mainly implemented by a class in openfire: ClusterManager, which adds and exits the cluster instance in ClusterManager because the master-slave structure is not used, so ClusterManager implements a non-central management. I don't know if I understand it correctly. As long as the cluster is enabled for the current instance, ClusterManager automatically loads the cluster management and synchronizes it with other clusters.

• Startup

Startup is the method for starting a cluster. The Code is as follows:

    public static synchronized void startup() {        if (isClusteringEnabled() && !isClusteringStarted()) {            initEventDispatcher();            CacheFactory.startClustering();        }    }

First, you must determine whether the cluster is enabled and the current cluster instance is not running. The event distributor is initialized to process cluster synchronization tasks. Then, call the startClustering of CacheFactory to run the cluster. The startClustering method mainly involves the following:

• It will start with the cluster's cache Factory policy and add itself to the cluster.
• Enable a thread to synchronize the cache status

In the initEventDispatcher method in the preceding startup, a distribution thread is registered here to listen to cluster events. After the event is received, the joinedCluster or leftCluster operation is executed. joinedCluster is added to the cluster. During joinedCluster, local cache containers are converted to cluster cache. The cluster is initialized and added to the cluster.

• Shutdown

Shutdown is relatively simple, that is, to exit the cluster and restore the cache factory to a local cache. Synchronization management is mainly about how to manage clusters. Next, it is important to synchronize data between clusters? This part mainly depends on the implementation of the specific distributed computing system. In openfire, the data is stored in the cluster cache and then completed through cluster components, such as hazelcast. Because the cache is used to solve the problem, there will be many codes about the cluster in CacheFactory, especially the Cache Policy Switching and cluster task processing are all made public in CacheFactory as an interface method. In this way, the implementation of the cluster is transparent. The cluster computing task has not mentioned the computing problem in the cluster before, because can we use the cluster advantages for parallel computing? I am not too sure about this part, but I only see the relevant code, so I will simply list it. There are several methods in the CacheFactory class: doClusterTask and doSynchronousClusterTask. Both of them are the overload method and the parameters are different. These methods are used to execute some computing tasks. Let's take a look at doClusterTask:

    public static void doClusterTask(final ClusterTask<?> task) {        cacheFactoryStrategy.doClusterTask(task);    }

Here, there is a limit that it must be a class derived from ClusterTask. Let's take a look at its definition:

public interface ClusterTask<V> extends Runnable, Externalizable {     V getResult(); }

It is mainly for asynchronous execution and serialization, because it cannot be blocked, and serialization is of course to be transmitted in the cluster.

Looking at the CacheFactory doClusterTask method, we can find that it only acts as a proxy for the doClusterTask of the cache policy factory. The specific implementation depends on the implementation of the cluster. Let's take a look at the implementation of hazelcast. The openfire cluster has a cluster plug-in implementation in openfire. Here we will take hazelcast as an example to briefly analyze and learn.

• Cache Policy factory class (ClusteredCacheFactory)

ClusteredCacheFactory implements CacheFactoryStrategy. The Code is as follows:

public class ClusteredCacheFactory implements CacheFactoryStrategy {

First, the startCluster method is used to start the cluster, mainly to do the following:

• • Set the cache serialization tool class ClusterExternalizableUtil. This is a serialization tool used for data replication between clusters.
  • Set the remote session locator, RemoteSessionLocator. Because the session is not synchronized, it is mainly used for session reading between multiple instances.
  • Set the remote packet router ClusterPacketRouter to send messages in the cluster.
  • Set NodeID for the instance that loads Hazelcast and ClusterListener

As mentioned above, cache switching is mentioned during cluster startup. What is the specific implementation? The cluster is added to the cluster by using the CacheFactory. joinedCluster method after it is started. Let's take a look at the added code:

   /**     * Notification message indicating that this JVM has joined a cluster.     */    @SuppressWarnings("unchecked")    public static synchronized void joinedCluster() {        cacheFactoryStrategy = clusteredCacheFactoryStrategy;        // Loop through local caches and switch them to clustered cache (copy content)        for (Cache cache : getAllCaches()) {            // skip local-only caches            if (localOnly.contains(cache.getName())) continue;            CacheWrapper cacheWrapper = ((CacheWrapper) cache);            Cache clusteredCache = cacheFactoryStrategy.createCache(cacheWrapper.getName());            clusteredCache.putAll(cache);            cacheWrapper.setWrappedCache(clusteredCache);        }        clusteringStarting = false;        clusteringStarted = true;        log.info("Clustering started; cache migration complete");    }

Here we can see that all the Cache containers are read and packaged one by one using Wrapper, and then a new Cache is created using the same Cache name, this step uses the switched cluster Cache Policy factory, that is, ClusteredCacheFactory is used to create a new cache container. Finally, write the cache to the new clusteredCache to complete the cache switching.

Of course, here we should look at the createCache Implementation of ClusteredCacheFactory:

    public Cache createCache(String name) {        // Check if cluster is being started up        while (state == State.starting) {            // Wait until cluster is fully started (or failed)            try {                Thread.sleep(250);            }            catch (InterruptedException e) {                // Ignore            }        }        if (state == State.stopped) {            throw new IllegalStateException("Cannot create clustered cache when not in a cluster");        }        return new ClusteredCache(name, hazelcast.getMap(name));    } 

ClusteredCache is used here, and the most important thing is that the second map parameter passed in is replaced with hazelcast. Then, when accessing this Cache container, it is no longer the original local Cache, it is a map object of hazelcast. Hazelcast automatically synchronizes and manages map data, which completes cache synchronization.

• Cluster Computing

Let's take a look at the implementation of hazelcast. Here is an example of doClusterTask in ClusteredCacheFactory:

    public void doClusterTask(final ClusterTask task) {        if (cluster == null) { return; }        Set<Member> members = new HashSet<Member>();        Member current = cluster.getLocalMember();        for(Member member : cluster.getMembers()) {            if (!member.getUuid().equals(current.getUuid())) {                members.add(member);            }        }        if (members.size() > 0) {            // Asynchronously execute the task on the other cluster members            logger.debug("Executing asynchronous MultiTask: " + task.getClass().getName());            hazelcast.getExecutorService(HAZELCAST_EXECUTOR_SERVICE_NAME).submitToMembers(                new CallableTask<Object>(task), members);        } else {               logger.warn("No cluster members selected for cluster task " + task.getClass().getName());        }    }

The process is to first obtain the instance members in the cluster, of course, to exclude yourself. Then hazelcast provides ExecutorService to execute this task. The method is submiteToMembers. In this way, an operation task is submitted. However, it is not clear how to allocate calculations and collect results.

Summary

It took a day to read about the openfire cluster. As a result, I wrote an article, and I did get something. It seems that you are more willing to use redies to complete cache sharing and implement clusters through proxy, rather than using the openfire cluster solution. I don't know how to deal with high concurrency requirements. I will try to write a redies plug-in later.

Detailed installation process of Openfire in CentOS

Openfire server configuration notes based on Jabber/XMPP protocol in CentOS 5.4

Install Openfire on Ubuntu 12.04

Openfire solves Chinese garbled characters After MySQL database is used

Load Balancing for Openfire clusters using Nginx

Openfire details: click here
Openfire: click here

This article permanently updates the link address: