Openfire Hazelcast cluster details

Openfire Hazelcast cluster details

I. Overview
The Openfire Hazelcast plug-in supports running multiple redundant Openfire servers on a cluster. By running Openfire as a cluster, terminal connections can be allocated to multiple Openfire servers, and server failover is also provided. The Hazelcast plug-in is an alternative to the original Openfire cluster plug-in. It uses the open-source Hazelcast data distribution framework to replace expensive proprietary third-party products. Currently, Hazelcast is 3.5.1.

Hazelcast home page: http://www.hazelcast.org/

2. Comparison between clusters and Federation
The XMPP protocol is designed similar to the mailbox. Each Openfire installation supports a single XMPP domain, and the Server to Server (S2S) the XMPP protocol is described in the XMPP specification to connect multiple XMPP domains, namely Federation ). Federation represents an effective method of XMPP horizontal scaling because it allows an XMPP user to securely communicate with any user in other federated domains. These Federation can be public or private. The federated domain can exchange XMPP Festival (Stanzas) on the Internet, or even use DNS-based service lookup or address resolution for discovery.

In contrast, a cluster is a vertical extension technology for a single XMPP domain, and the same configuration is shared in good faith within the cluster. Each member of the cluster allows any user to connect, authorize, and authenticate and exchange segments within the domain. Servers in all clusters share a single database. Therefore, they must be deployed in the same LAN to maintain low network latency. Cluster deployment is suitable for providing runtime redundancy, and supports a larger number of users and more connections.

For a relatively large Openfire deployment, the combination of federated and cluster technologies can provide better results. A single XMPP domain can support tens of thousands or even hundreds of thousands of users. Combined with federated deployment, it can support more than one million users.

Iii. Installation
To create an Openfire cluster, you must have at least two Openfire servers. The Hazelcast plug-in must be installed for each server. To follow Hazelcast, simply put the hazelcast. jar package into the plugins subdirectory of Openfire. You can also directly install the plug-in on the plug-in page of the Management Console.

Note that each Openfire server in the cluster must be configured to connect to the same external database.

By default, when Openfire starts and initializes a process, server nodes discover each other (by exchanging UDP broadcast packets ). Note that if the LAN does not support broadcast communication, you can use some Initialization Configuration options.

After the Hazelcast plug-in is deployed on each Openfire server, use the single-choice button on the cluster page of the Management Console to activate or enable the cluster. You only need to enable the cluster once. This change will automatically notify other nodes in the cluster. After refreshing the cluster page, you can see that all servers have successfully joined the cluster.

Note that Hazelcast and earlier cluster plug-ins (such as clustering. jar or enterprise. jar) are mutually exclusive. That is to say, when the Hazelcast plug-in is used, the early cluster plug-in cannot be used.

After the cluster runs, you need some form of Server Load balancer to allocate user access to each Openfire node. There are multiple commercial and open-source solutions. If you use the HTTP/BOSH Openfire connector to connect to Openfire, you can use Apache HTTP Server for Load balancer (using the mod_proxy_balancer module ). Some popular options such as F5 LTM (commercial solutions) and HAProxy (open-source solutions) are acceptable.

F5 LTM: http://www.f5.com/products/big-ip/big-ip-local-traffic-manager/overview/
HAProxy: http://haproxy.1wt.eu/

There is also a simple Round-robin DNS configuration that can help assign xmpp to the uogeOpenfire server node in the cluster. Although popular lightweight and low-cost methods provide basic scalability, note that these methods do not meet the actual load balancing requirements and do not provide high availability.

Iv. Upgrade the Hazelcast plug-in
Upgrading the Hazelcast plug-in requires additional steps, and each node in the cluster needs to install the same plug-in version to avoid various errors and data synchronization problems.

Method 1: offline upgrade
This upgrade method is simple, but may cause a short service interruption.
1) Disable all Openfire services in the cluster.
2) perform the following steps on the first Openfire server in the Cluster:
A. Remove the existing Hazelcast plug-in hazelcast. jar in the plugins directory;
B. Remove the existing hazelcast subdirectory under the plugins directory;
C. Copy the latest hazelcast. jar file to the plugins directory;
D. Restart the Openfire server to decompress and install the plug-in.
3) Repeat the above work on each remaining Openfire server.

Method 2: Online upgrade
This method does not affect Openfire services during the upgrade.
1. Shut down the Openfire server in the cluster and keep only one of them for running.
2. log on to the Management Console of this Openfire server and remove the existing Hazelcast plug-in from the plug-in page.
3. Upload the latest Hazelcast plug-in and complete the installation.
4. Use the "offline" step to upgrade and start the remaining servers.

Method 3: Split and upgrade
Note: If you can only access the Openfire console, you can use this method. However, during the upgrade, users may not be able to communicate with each other.
1) Disable the cluster (Disable) on the cluster page of The Openfire console. This will Disable the cluster for all cluster members.
2) Update the Hazelcast plug-in for each Openfire.
3) when the Hazelcast plug-in of all Openfire servers is updated, Enable on the cluster page of the Management Console activates all Openfire members in the cluster.

V. Configuration
Hazelcast has several configuration options that can be used as the system attribute of Openfire.

1. hazelcast. startup. delay. seconds (5)
The waiting time (in seconds) for running the Hazelcast plug-in ). This allows Openfire to deploy other plug-ins when initializing the cluster cache.

2. hazelcast. startup. retry. count (1)
The number of times the cluster is restarted after the cluster fails to start for the first time.

3. hazelcast. startup. retry. seconds (10)
The interval between two adjacent start operations. (Seconds)

4、hazelcast.max.exe cution. seconds (30)
The maximum waiting time for cluster members to execute synchronization tasks.

5. hazelcast. config. xml. filename (hazelcast-cache-config.xml)
The name of the Hazelcast plug-in configuration file to facilitate the installation of custom cache configuration files.

6. hazelcast. config. xml. directory ({OPENFIRE_HOME}/conf)
You can add the directory to the plug-in class path to facilitate loading the custom Hazelcast configuration file.

7. hazelcast. config. jmx. enabled (false)
Enable jmx for the Hazelcast cluster.

The Hazelcast plug-in uses the XML configuration builder to initialize the cluster. By default, cluster members attempt to discover each other through the positions under the broadcast.
IP Address: 224.2.2.3
Port 54327

6. Use Hazelcast
Hazelcast is very sensitive to latency, which may lead to a long-term GC cycle, especially when using the default JVM configuration. In most cases, the parallel garbage collector (CMS) is preferentially activated or the new G1 garbage collector is used to minimize blocking.
In addition, the running Features of Openfire clusters depend largely on the number of clients, connection types, and XMPP service types deployed. However, it should be noted that many objects allocated on the Heap are in short active periods. Increasing the space for young generations may have a positive impact on performance.

For example, the following provides a server cluster (4-core CPU) suitable for three nodes and supports about 50 thousand active users:

OPENFIRE_OPTS = "-Xmx4G-Xms4G-XX: NewRatio = 1-XX: Export vorratio = 4
-XX: + UseConcMarkSweepGC-XX: + CMSIncrementalMode-XX: + UseParNewGC
-XX: + CMSParallelRemarkEnabled-XX: CMSFullGCsBeforeCompaction = 1
-XX: CMSInitiatingOccupancyFraction = 80-XX: + UseCMSInitiatingOccupancyOnly
-XX: + PrintGCDetails-XX: + PrintPromotionFailure"

-------------------------------------- Split line --------------------------------------

Detailed installation process of Openfire in CentOS

Openfire server configuration notes based on Jabber/XMPP protocol in CentOS 5.4

Install Openfire on Ubuntu 12.04

Openfire solves Chinese garbled characters After MySQL database is used

Load Balancing for Openfire clusters using Nginx

-------------------------------------- Split line --------------------------------------

This article permanently updates the link address: