Tomcat cluster and Server Load balancer

There are some major problems in executing Web applications on a single server. When a website is successfully built and receives a large number of requests, a single server cannot satisfy the load to be processed, so it seems a little powerless. Another common problem is that a single point of failure (spof) occurs. If the server breaks down, the website will become unavailable immediately. Whether it is better scalability or fault tolerance, we want to execute web applications on more than one server computer. Therefore, we need to use the cluster technology at this time.

Before discussing the cluster system architecture, define some special terms:
1. Cluster: A group of independent computer systems constitute a loosely coupled multi-processor system. They communicate with each other through a network. Applications can transmit messages through the network shared memory to implement distributed computers.
2. load Balancing: the cluster is a group of computers that are connected together. From the outside, it is a system, each node can be a computer composed of different operating systems or different hardware. For example, a cluster that provides Web Services is a large Web server. However, cluster nodes can also provide services separately.
3. Features: based on the existing network structure, Server Load balancer provides a cheap and effective method to expand server bandwidth and increase throughput, enhance network data processing capabilities, and improve network flexibility and availability. Cluster mainly solves the following problems:
High reliability (HA): With cluster management software, when the master server fails, the backup server can automatically take over the work of the master server and switch over in time, to provide uninterrupted services to users.
High-performance computing (HP): makes full use of the resources of every computer in the cluster to implement parallel processing of complex operations. It is usually used in scientific computing, such as genetic analysis and chemical analysis.
Load Balancing: distributes the load pressure to each computer in the cluster according to a certain algorithm to reduce the pressure on the master server and reduce the hardware and software requirements on the master server.

Currently, Server Load balancer is commonly used in the following scenarios:
1. DNS-Based Load Balancing
Server Load balancer is achieved through random name resolution in the DNS service. In the DNS server, you can configure the same name for multiple different addresses, the client that finally queries the name will get an address when parsing the name. Therefore, for the same name, different customers get different addresses and access the Web servers at different addresses to achieve load balancing.

2. Reverse Proxy Server Load balancer (such as Apache + JK2 + Tomcat)
The proxy server can forward requests to the internal web server so that the proxy server can evenly distribute requests to one of multiple internal Web servers to achieve load balancing. This proxy method is different from the common proxy method. The standard proxy method is that the customer uses the proxy to access multiple external web servers. This proxy method is used by multiple customers to access internal web servers, it is also called reverse proxy mode.

3. Server Load balancer Technology Based on NAT (Network Address Translation) (such as Linux virtual server (LVS)
The network address is converted between the internal address and the external address so that computers with the internal address can access the external network, when a computer in an external network accesses an external address owned by the address translation gateway, the address translation gateway can forward it to a mapped internal address. Therefore, if the address translation gateway can evenly convert each connection to a different internal server address, then the computers in the external network will communicate with the server on the address they have obtained, this achieves load balancing.

After introducing the above cluster technology, the following describes the tomcat-based cluster architecture solution:

The above uses Apache httpd as the Web server, that is, as the front-end Processor of Tomcat. It depends on the specific situation. In some cases, Apache httpd is not required as the web server, if the system does not display static pages, Apache httpd is not required. At that time, Tomcat can be directly used as a web server. Apache httpd is mainly used to process more static pages than tomcat.
1. After the user's web browser completes the request/response from the local DNS and enterprise-authorized DNS, the enterprise-authorized DNS (21cn boss DNS) the local DNS server will be provided with a NAT request distributor (that is, the gateway) ip address.

2. The Nat distributor determines the internal Apache httpd to which the connection is to be handed over based on a specific allocation algorithm to process the request. Most Nat request splitters provide Fault Tolerance capabilities: To detect the failure status of various web servers, stop allocating requests to the down servers. In addition, some splitters can monitor the load of Web server machines and allocate requests to servers with the least load. Linux Virtual Server is an open source VS-NAT software suite Based on Linux operating systems, and it has a wealth of features and good documentation. Serveriron of Foundry Networks, a commercial hardware solution, is one of the most widely recognized request distributors in the industry.

3. Apache httpd + mod_jk2 is used as a server Load balancer. Why do we need a cluster? If the cluster system must have fault tolerance capabilities to make it 100% available when any single hardware or software component fails, the cluster system must be free of spof concerns. Therefore, you cannot set up only one Apache httpd with mod_jk2, because if httpd or mod_jk2 becomes invalid, no more requests will be sent to any Tomcat instance. In this case, Apache httpd is a bottle of strength, especially for websites with high traffic volumes.

4. mod_jk2 Server Load balancer and fault recovery decide to use Apache httpd as a web server and use mod_jk2 to send requests to Tomcat. The load balancing and fault tolerance functions of mod_jk2 can be used. In the cluster system, Apache httpd with mod_jk2 can do the following:
A. Allocate requests to one or more Tomcat instances.
You can set many Tomcat instances in the workers. properties file of mod_jk2 and assign each instance an lb_factor value as the weighted factor for request allocation.

B. Check whether the Tomcat instance fails.
When the connector service of the Tomcat instance no longer responds, mod_jk2 will detect it in time and stop sending the request to it. Other Tomcat instances will accept the load of the invalid instance.

C. Detect when the Tomcat instance is restored after it expires
After a request is stopped and assigned to the Tomcat instance due to a connector service failure, mod_jk2 periodically checks whether the application has been resumed and automatically adds the request to the current Tomcat instance pool.

5. In tomcat, the cluster principle is to find nodes through multicast and use TCP connections to copy sessions. The prompt here is that Tomcat will perform session replication for each request processing, and the copied session will gradually become large.

6. mod_jk2 supports both session affinity and session replication. In Tomcat 5, how does one implement session affinity and session replication? Remove the <cluster/> label in server. XML to achieve session affinity, and add the <cluster/> label to implement session replication.

7. Session affinity: indicates that all requests from the same session are processed by the same Tomcat instance. In this case, if the Tomcat instance or the server machine executed fails, the servlet session data will also be lost. Even if more Tomcat instances are executed in the cluster system, session data will never be copied. This is a solution to improve cluster performance, but it is not capable of fault tolerance.

8. When session replication is used, when a tomcat instance goes down, the data will not be lost because at least one Tomcat instance has a copy of session status data. However, the performance will be reduced.