Tomcat cluster and Server Load balancer-Part 2

Original English version, Chinese versionTony TangTranslation Arrangement

In the first part, I briefly described the various factors that need to be taken into account during the design of a large J2EE system with upgrading and high reliability.

This article discusses Tomcat's support for cluster, Server Load balancer, fault tolerance, session replication, and other capabilities.

In this section, we will see the complete cluster architecture and installation and configuration details during cluster deployment (by running multiple Tomcat server instances ).

+ Cluster settings

The following lists the implementation goals of the Tomcat cluster example:

* Scalability

* Fault Tolerance

* Dynamic configuration, easy to manage

* Automatic discovery of new members

* Failed restart and load balancing, session data memory Replication

* Pluggable/configured Load Balancing policies

* When a member joins or leaves, the group member can be notified.

* No packet loss is transmitted through multicast.

* Cluster-to-Web Applications and Service Is seamless.

In this cluster environment, four Tomcat server instances are installed. One Server Load balancer instance and three clusters.

The cluster is vertically scaled (multiple Tomcat server instances run on one machine ).

The following is the configuration of the main components of the cluster:

* Server Load balancer: a tomcat instance that distributes transactions to nodes in the cluster. Code TC-lb.

* Cluster: the cluster contains three Tomcat server instances, codenamed tc01, tc02, and tc03.

* Session Persistence: Select Memory Replication method. When When the session object changes, the session data is copied to all three cluster members.

* Restart upon failure: the Server Load balancer application that comes with Tomcat installation cannot be restarted upon failure.

I wrote a tool class serverutil to check the server status before forwarding requests to the server.

There are two ways to check the status of cluster nodes. In the first method, use mcastservice to check whether a specified server instance is running.

The second method creates a URL object with the URL of the web page as the parameter to verify the validity of the cluster node.

To use this class, make sure that the catalina-cluster.jar (located in the % tomcat_home %/Server/library directory)

And the commons-logging-api.jar (located in the % tomcat_home %/bin directory) file is specified in classpath.

The architecture of the main components of the following cluster.

Figure 1 Tomcat cluster architecture

+ Install and configure Tomcat instances

Table 1. In this example, set the hardware and software used in the Tomcat cluster environment.

Processor HP pavilion Pentium III with 800 MHz

Apsaradb for memory 512 MB RAM

Hard Disk 40 GB

Operating System Windows 2000 Server with Service Pack 4

JDK version 1.4.0 _ 02 (Note: JDK version 1.4 or a later version is required to enable Tomcat clustering)

Tomcat version 5.0.19

Tools used ant 1.6.1, log4j, jmeter, JBuilder

+ Main elements of the cluster framework

++ Java class

* Baseloadbalancingrule

Abstract class, which encapsulates general rule logic. In this example, the custom Server Load balancer rules are extended from this base class.

* Randomredirectrule

Use random rules to define the logic for redirecting Web requests to a valid server. Use the current system time as the seed to generate a random number.

* Roundrobinrule

This class defines the logic of a server Load balancer based on the "Round Robin" rule. When a request enters, It redirects it to the next member in the cluster member list.

Use a static variable to track the next valid cluster member. Each time a request is processed, the value is added to 1.

* Serverutil

A tool class used to check whether a specified cluster node is valid.

This class uses mcastservice (Org. Apache. Catalina. Cluster. MCAST package) to check whether a cluster member has left the group.

The following class diagram shows the relationships between these Java classes.

Figure 2 cluster Application diagram

++ Configuration file

* Server. xml

It is used to configure a cluster for Tomcat server instances. After Tomcat is installed in this version, the server. xml file contains the cluster configuration details that have been commented out.

* Web. xml

In this file, the session data of the Web application needs to be copied.

* Rules. xml

This file is used to define the load balancing rules.

++ Script

* Test. jsp

A simple test JSP script is used to check the server status. Displays the name and system time of the running Tomcat instance.

* Testlb. jsp

In this application, this is the starting page. It uses HTML redirection to forward Web requests to the server Load balancer filter.

* Sessiondata. jsp

This script is used to verify that session data is not lost when a cluster node is suspended. Display the session content and use the HTML field to operate the http session object.

* Build. xml

Ant build script to automate the tasks of starting and stopping Tomcat instances (this script is executed by ANT 1.6.1 ). Once a tomcat instance is successfully started, you can call test. jsp by specifying the IP address and port number to verify whether the Tomcat instance is running. This JSP page displays the current system time and tomcat Instance name. You need to change the home directory in the build. properties file and run this script in your own environment.

In the build script, the following targets are used to start or stop Tomcat instances:

* Call target "start. tomcat5x" to start a specific Tomcat instance (for example, tomcat50 ).

* Call stop. tomcat5x to stop a specific Tomcat instance

* Call stop. allpolicats to stop all running Tomcat instances.

+ Sample code

Code in this exampleTomcatclustering.zip. After the Tomcat server instance is installed (4), decompress the files in the ZIP file to the Tomcat directory.

The example code uses roundrobinrule as the load balancing rule. If you want to use random redirection rules, modify the rules. xml file (in the tomcat50/webapps/balancer/WEB-INF/conf directory ).

Comment out the roundrobinrule element and cancel the comment on the randomredirectrule element. Similarly, if you want to use two instances instead of three, comment out the third one and change the value of the maxserverinstances attribute to 2 (replace the original 3 ).

Note: by default, Tomcat will contain several other applications after installation. I have deleted all other Web applications (JSP-examples, and so on ), only the balancer and the Web applications in this example are retained.

+ HTTP Request Process

In this example, the Web request process in the cluster environment is as follows:

1. Start page (http: // localhost: 8080/balancer/testlb. jsp );

2. jsp redirects requests to the server Load balancer filter (URL: http: // localhost: 8080/balancer/loadbalancer)

3. The Load balancer (TC-LB) Intercepts Web requests and redirects to the next valid cluster member (tc01, tc02 or tc03) according to the load balancing rules specified in the configuration file );

4. The sessiondata. jsp (located in the clusterapp Web Application) of the selected cluster member is called;

5. If the session is modified, the session terappsessionlistener session listener method will be called to record the session modification event;

6. sessiondata. jsp displays session details in a web browser (for example, session ID, last access event, and so on );

7. Stop one or two cluster nodes randomly (call the "Stop. tomcat5x" target of the ant script );

8. Repeat the above seven steps to check whether the request failed for a valid cluster member. At the same time, check whether the session information is copied within the cluster members without data loss.

Figure 3 shows a Web Request Process

Cluster application sequence diagram

+ Cluster configuration

In this cluster, run a clusterapp web application. To optimize session replication, all instances have the same directory structure and content.

Because the Tomcat server instance uses IP multicast to transmit sessions, we must ensure that the IP multicast function on the Cluster machine is available. For verification, you can run "if you write multicast services and customer programs. Tomcat: the definitive guide contains examples of the Java program multicastnode. For more information, see http://java.sun.com/docs/books/tutorial/networking/?rams/broadcasting.html.

When a cluster node is started, other members of the cluster will display a record on the server console, indicating that a member has been added to the cluster. Similarly, when a cluster node is deprecated, other nodes will display the records of a cluster member leaving on the console.

Figure 4 records generated when a Member is added or deleted in the Cluster

Follow these steps to enable the Tomcat server cluster and session replication function:

1. All session attributes must implement the java. Io. serailizable interface.

2. uncomment the cluster element in the server. xml file. The userdirtyflag and replicationmode attributes are used to optimize the frequency and session replication mechanism.

3. uncomment the value element in server. xml. Replicationvalue is used to intercept HTTP requests and copy session data within cluster members. The value element has a "filter" attribute that can be used to filter requests that do not modify Sessions (such as HTML pages and image files ).

4. Because all Tomcat instances are running on the same machine, the tcplistenport attribute of each Tomcat instance must be set to unique. The attributes of mcastxxx (mcastaddr, mcastport, mcastfrequency, and mcastdroptime) are used for multicast Ping of the cluster relationship, while the names are in the format of tcpxxx (tcpthreadcount, tcplistenaddress, tcplistenport and latency) is used for session replication (the following cluster configuration parameter table shows different configurations of Tomcat server instances)

5. The web. xml Meta File (located in the clusterapp/WEB-INF directory) should have elements. Copies the session Status of a specified web application. The distributable element must be defined. This indicates that if more than one web application requires session replication, you need to add distributable to the Web. xml files of all web applications. 《Tomcat: the definitive guide"Tomcat cluster" in this book has a good explanation of this problem.

Table 2 cluster configuration parameters

Configuration parameters	Instance 1	Instance 2	Instance 3	Instance 4
Instance type	Server Load balancer	Cluster node 1	Cluster node 2	Cluster node 3
Code name	TC-LB	Tc01	Tc02	Tc03
Home Directory	C:/web/tomcat50	C:/web/tomcat51	C:/web/tomcat52	C:/web/tomcat53
Server Port	8005	9005	10005	11005
Connector	8080	9080	10080	11080
Coyote/JK2 AJP Connector	8009	9009	10009	11009
Cluster mcastaddr	228.0.0.4	228.0.0.4	228.0.0.4	228.0.0.4
Cluster mcastport	45564	45564	45564	45564
Tcplistenaddress	127.0.0.1	127.0.0.1	127.0.0.1	127.0.0.1
Cluster tcplistenport	4000	4001	4002	4003

Note: because all cluster members are running on the same machine, they use the same IP address (127.0.0.1 ).

If you do not use the ant script to start or stop Tomcat instances, do not set the catalina_home environment variable on your machine. If this variable is set, all instances attempt to start the Tomcat instance using the same directory (specified by the catalina_home variable. As a result, only the first instance can be started successfully, and other instances will crash. The notification port "java.net. bindexception: address already in use: jvm_bind: 8080 ".

+ Server Load balancer settings

I have written two simple, custom load balancing rules (roundrobinrule and randomredirect) for redirecting incoming Web requests. These rules are based on Server Load balancer algorithms (such as round robin and random redirection ). You can write custom Server Load balancer rules similar to other factors (such as weighting and last access time. Tomcat server Load balancer provides a sample (parameter-Based Server Load balancer rules) that redirects Web requests to different URLs Based on HTTP request parameters.

Maintain the state of comment on the cluster and Value Elements in server. XML (TC-LB instance) because the instance is not a cluster member.

+ Test settings

++ Session persistence test

In the session persistence test, the main goal is to verify that session data is not lost when a cluster member crashes during a Web request. JSP sessiondata. jsp is used to display the session content. This script also provides the HTML text field for adding, modifying, and deleting session attributes. After adding the attribute to the HTTP session, I randomly stop the cluster node and check the session on valid cluster members.

++ Load Testing

The purpose of the Server Load balancer is to study how a custom Server Load balancer algorithm effectively distributes Web requests to a specified cluster node when one or more nodes stop the service. The jmeter load testing tool is used to simulate multiple concurrent web users.

To test Server Load balancer, follow these steps:

1. Start the Server Load balancer and cluster instances.

2. Run the starting JSP script (testlb. jsp ).

3. manually stop one or more containers to simulate server crashes.

4. Check the load distribution mode.

5. Repeat steps 1 to 4 for the first 100 times.

All record information is redirected to a text file called tomcat_cluster.log (in the tomcat50/webapps/balancer directory ). In a sequence diagram (Figure 2), the response time of all web objects is recorded using log4j information. Table 3 is a time-consuming (millisecond) table.

The following table shows the time consumed by the load test (using the roundrobinrule algorithm) and the load distribution percentage (using the randomredirectrule algorithm ).

Table 3. Load Testing time consumption

#	Scenario	Testlb. jsp MS)	Roundrobinrule MS)	Sessiondata. jsp MS)	Total MS)
1	All three servers are running.	54	76	12	142
2	Two server instances are running (tc02 was stopped)	55	531	14	600
3	A server is running (Tc01 and tc02 were stopped)	56	1900	11	1967

Note: all consumed time is the average value of 100 concurrent users.

Table 4. When a Random Load Balancing rule is used, load distribution is performed.

#	Scenario	Tc01 (%)	Tc02 (%)	Tc03 (%)
1	All service instances are running	30	46	24
2	Two service instances are running (tc02 was stopped)	56	0	44

Note: The load distribution percentage is also based on the load of 100 concurrent users.

+ Summary

In the session persistence test, after the session attribute is added, one of the cluster nodes hangs and passes verification, it is confirmed that the session attribute is not lost during server downtime. The specific content of the session attribute is recorded in a text file.

In the load test, when one or two server instances stop and only one Tomcat instance runs, the response time is longer than that of all three instances. When the stopped instance is restarted, the Server Load balancer automatically finds that these servers are valid again, and then reschedule the subsequent requests to these server instances, which can immediately increase the response time.

The serverutil mechanism is not the fastest way to find out whether the cluster members are valid.

One drawback of this cluster setting is that it only provides a Load balancer. What happens when the Tomcat instance used as the Load balancer instance is suspended? There is no way to forward requests to the cluster. This result is called spof. One solution is that another tomcat instance runs as a backup Load balancer instance. If the primary Server Load balancer crashes, the secondary server Load balancer takes over. A typical high-reliability cluster (HA) includes two load balancers to prevent spof.

In the preceding example, all Tomcat instances (including Server Load balancer) are configured to run on the same machine. A better setting is to run the Server Load balancer on an independent machine. Similarly, each machine is limited to two cluster nodes, making full use of the horizontal scaling method to ensure cluster efficiency.

For J2EE Web application servers, HTTP session replication is an expensive operation. In the J2EE cluster environment, the implementation of session management should be considered in the project analysis and design phase. The cluster environment must be considered during encoding. If the implementation of the cluster is not considered in the design phase, the Code may need to be completely rewritten to enable the application to work in the cluster environment. This will have a very large impact.

If Web applications support various object caching mechanisms, cache objects in the cluster environment should be considered in the initial stage of application development. This is very important, because it is very dangerous to keep the cached data synchronized among all cluster nodes for providing precise and real-time transaction data to Web users.

Once the J2EE cluster is successfully set and run, its management and maintenance become very important. Keep the cluster running and push application changes to all cluster nodes. You need to provide these services in a way that implements a Monitoring Service and periodically checks the server's validity. If any node in the cluster is invalid, it will send a notification. This service detects invalid nodes at regular intervals and deletes them from the list of active cluster nodes. It should have the ability to synchronize and update all servers in the cluster when changes and updates occur. Because all requests to Web applications must pass the Server Load balancer system, the system can detect the number of active sessions, the number of active sessions, the number of responses, and the number of peak loads, the number of active sessions during the peak hours, the number of active loads during the trough, and so on. These audit information can be used as a reference to improve performance and optimize the entire system.

Here, you can manually adjust the configuration file (server. xml and rules. XML) to meet all the Configuration Requirements for setting the cluster and Server Load balancer. If the Jakarta project team provides a web-based cluster management tool, we can use the management tool to modify the configuration to manage clusters and Server Load balancer.

+ Resources

*Tomcat 5Home Page

*TomcatCluster homepage of the site

*TomcatServer Load balancer Homepage

*Tomcat: the definitive guideBy Jason Brittain and ian F. Darwin

*Java performance tuning, 2nd EditionBy Jack shiraji

* Creating highly available and scalable applications using J2EE, the middleware company, EJB essentials training class material