Nginx tutorial 5: Load Balancing of clusters using nginx + Tomcat (session replication)

Architecture Description
　　
A front-end nginx server is used as a Load balancer, and N Tomcat servers are deployed at the backend to form a cluster processing service, which is forwarded to the backend through nginx (Note: No static and dynamic separation is performed, and all static dynamics are forwarded to Tomcat)
　　
Advantage: a flexible architecture is implemented. When the pressure increases, you can temporarily add Tomcat servers to this architecture.
　　
1. Configure nginx
　　
1. Download the package
　　
Wget http://sysoev.ru/nginx/nginx-0.6.32.tar.gz
　　
Ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/
　　
2. Install the nginx package
　　
A. Install PCRE
　　
Tar zxvf pcre-7.2.tar.gz
　　
CD PCRE
　　
./Configure-Prefix =/PCRE
　　
Make; make install
　　
B. Install nginx
　　
Tar zxvf nginx-0.6.32.tar.gz
　　
CD nginx-0.6.32
　　
./Configure-Prefix =/nginx-with-PCRE =/PCRE-with-http_rewrite_module
　　
Make; make install
　　
3. modify the configuration file
　　
VI/nginx/CONF/nginx. conf
　　
# User Group
　　
User nobody Nobody;
　　
# Number of CPUs, which can be calculated based on the actual Server
　　
Worker_processes 8;
　　
Worker_rlimit_nofile 51200;
　　
Events {
　　
Use epoll;
　　
# Connections
　　
Worker_connections 8192;
　　
}
　　
HTTP {
　　
Include mime. types;
　　
Default_type application/octet-stream;
　　
Server_names_hash_bucket_size 128;
　　
# Access_log off;
　　
# Access_log logs/access. log;
　　
# Cache time (you can set different times based on different files)
　　
# Expires 2 h;
　　
Tcp_nodelay on;
　　
Keepalive_timeout 30;
　　
Gzip on;
　　
Gzip_min_length 10;
　　
Gzip_buffers 4 8 K;
　　
Gzip_http_version 1.1;
　　
Gzip_types text/plain application/X-JavaScript text/CSS text/HTML Application/XML;
　　
Sendfile on;
　　
Tcp_nopush on;
　　
Reset_timedout_connection on;
　　
Client_max_body_size 30 m;
　　
# Set the Server Load balancer list
　　
Upstream backend
　　
{
　　
Server 172.23.254.2: 8080;
　　
Server 172.23.254.3: 8080;
　　
}
　　
# Set Virtual Hosts
　　
Server {
　　
Listen 80;
　　
SERVER_NAME www.abc.com;
　　
# Load Balancing for/all (the local nginx Adopts full forwarding, and all requests are forwarded to the backend Tomcat cluster)
　　
Location /{
　　
Root/web/WWW;
　　
Index index. jsp index.htm index.html;
　　
Proxy_redirect off;
　　
# Retaining real user information
　　
Proxy_set_header host $ host;
　　
Proxy_set_header X-real-IP $ remote_addr;
　　
Proxy_set_header X-forwarded-for $ proxy_add_x_forwarded_for;
　　
Proxy_pass http: // backend;
　　
}
　　
}
　　
}
　　
Mainly in the configuration of proxy and upstream
　　
Upstream is capable of load balancing. It can automatically identify the following machines and automatically kick out the machines that cannot provide services normally.
　　
4. Start the program
　　
/Nginx/sbin/nginx
　　
5. Write the Startup Script
　　
VI nginx. Sh
　　
#! /Bin/sh
　　
CWD = 'pwd'
　　
Case $1 in
　　
Start)
　　
/Nginx/sbin/nginx;
　　
;;
　　
Stop)
　　
Kill-2 'ps-Ef | grep "/nginx/sbin/nginx" | grep-V "grep" | awk '{print $2 }''
　　
;;
　　
Restart)
　　
CD "$ cmd"
　　
$0 stop
　　
$0 start
　　
;;
　　
*)
　　
Echo $ "usage: $0 {START | stop | restart }"
　　
Exit 1
　　
Esac
　　
Exit 0
　　
2. Configure Tomcat
　　
1. Download tomcat5.59
　　
Tar zxvf tomcat5.59
　　
2. modify the configuration file
　　
A. Configure the data source.
　　
B. Optimize Max Tomcat concurrency
　　
Maxthreads = "2048" minsparethreads = "100" maxsparethreads = "200 ″
　　
Enablelookups = "false" redirectport = "8443" acceptcount = "500 ″
　　
Connectiontimeout = "20000" disableuploadtimeout = "true"/>
　　
C. Add a VM
　　
(Note: The Virtual Host forwarded by the master must use localhost. Otherwise, nginx cannot forward data through the Intranet IP address, but only through domain name forwarding.
　　
D. Test
　　
Open http: // ip: 8080
　　
Page access is normal
　　
2. Other Tomcat servers use the same configuration.
　　
3. Create a tomcat Cluster
　　
Two machines: 172.23.254.2 172.23.254.3
　　
The file configuration to be modified for the cluster is in three places.
　　
1. Modify the conf/server. xml configuration file.
　　
A. Locate the engine tag and add the attribute jvmroute = "worker1 ″
　　
B. Locate the cluster label, remove the annotation, and change tcplistenaddress to the local IP address 172.23.254.2 (note: this segment of cluster must be placed in hosts)
　　
2. Modify the Web. xml of the application
　　
Modify the Web. xml file under the WEB-INF directory in the Web application, add tags
　　
Just add it to the previous step.
　　
This step is required for Tomcat cluster creation. Otherwise, your session will not work properly.
　　
3. Enable Firewall
　　
Firewall trust must be enabled between the two Tomcat servers.
　　
Start two Tomcat servers, and check whether port 8080 and port 4001 are enabled for each Tomcat server.
　　
Use netstat-An to view the link information.
　　
TCP 0 0 172.23.254.2: 43320 172.23.254.3: 4001 established
　　
TCP 0 0 172.23.254.2: 46544 172.23.254.3: 4001 time_wait
　　
TCP 0 0 172.23.254.2: 40118 172.23.254.3: 4001 established
　　
TCP 0 0 172.23.254.2: 4001 172.23.254.3: 48804 established
　　
TCP 0 0 172.23.254.2: 4001 172.23.254.3: 34254 established
　　
If the connection is established on port 4001 of both machines, the cluster configuration is successful and session replication can be performed.
　　
Possible problems
　　
1. session replication Problems
　　
When I used Apache for load balancing, I chose the session sticky mode. In this way, every time a user comes in, the session is in the same server, it will not be forwarded to other servers. In this case, Tomcat does not affect user access even if session replication is not performed. However, nginx does not support sticky. Therefore, session replication is required. Otherwise, it will be useless in many places. For example, during the login process, we waited for the First tomcat to generate a session, refreshed the page, and flushed it to another tomcat machine. Without this session, the problem may occur. So programmers should pay attention to this when writing JSP files.
　　
For example, if we modify a user's data in a session in a standalone application, it is usually:
　　
User user = (User) request. getsession (). getattribute ("user ");
　　
User. setname ("My name ");
　　
In this way, we can directly access it and then modify it. Although there is no problem in a single machine, but under the cluster conditions, this will lead to the problem of non-synchronization of sessions on multiple web servers, because the session has not changed, Tomcat cannot monitor whether the value of a data in the session has changed. Therefore, we also need to execute the following operations to ensure synchronization of sessions:
　　
Request. getsession (). setattribute ("user", user );
　　
Therefore, we should pay special attention when operating the session! We recommend that you do not modify the data in the session as much as possible.
　　
Session replication may be abnormal frequently. In addition to finding the cause on the server, you can find the cause on the program. All of them may cause abnormal session replication.
　　
2. Page Synchronization
　　
To ensure that the page programs on the Tomcat server are consistent, you can use the following method:
　　
A, rsync synchronization, or a page button, provided for editing, modified the program even if you click Synchronization
　　
B. Shared zone storage, or drbd network raid Mode
　　
3. Confirm that nginx can be forwarded successfully,
　　
On nginx, wget refers to the URL forwarded after (Packet over port). If it can be opened, it can be forwarded in the past. If it cannot be opened, it cannot be forwarded.