Nginx+tomcat Cluster load Balancing (for session replication)

Transferred from: http://talangniao.iteye.com/blog/341512

Schema description

The front-end of an nginx server to do load balancer, back-end put n Tomcat to form a cluster processing service, through Nginx forward to the back (note: No movement separation, static dynamics are all transferred to Tomcat)

Pros: An elastic architecture is implemented to temporarily add a Tomcat server to this architecture when the pressure increases

One, configure Nginx

1, download package

Wget http://sysoev.ru/nginx/nginx-0.6.32.tar.gz

Ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/

2, install the Nginx package

A. Installing the Pcre

Tar zxvf pcre-7.2.tar.gz

CD Pcre

./configure--prefix =/pcre

Make;make Install

B, Installing Nginx

Tar zxvf nginx-0.6.32.tar.gz

CD nginx-0.6.32

./configure--prefix=/nginx–with-pcre=/pcre--with-http_rewrite_module

Make;make Install

3, Modifying a configuration file

Vi/nginx/conf/nginx.conf

#用户组

User nobody nobody;

#cpu个数, you can calculate it according to the actual server

Worker_processes 8;

Worker_rlimit_nofile 51200;

Events {

Use Epoll;

#连接数

Worker_connections 8192;

}

HTTP {

Include Mime.types;

Default_type Application/octet-stream;

Server_names_hash_bucket_size 128;

# Access_log off;

# Access_log Logs/access.log;

#缓存的时间, (different time can be set according to different files)

# Expires 2h;

Tcp_nodelay on;

Keepalive_timeout 30;

gzip on;

Gzip_min_length 10;

Gzip_buffers 4 8k;

Gzip_http_version 1.1;

Gzip_types text/plain application/x-javascript text/css text/html application/xml;

Sendfile on;

Tcp_nopush on;

Reset_timedout_connection on;

Client_max_body_size 30m;

#设定负载均衡列表

Upstream Backend

{

Server 172.23.254.2:8080;

Server 172.23.254.3:8080;

}

#设定虚拟主机

server {

Listen 80;

server_name www.abc.com;

#对/All load balancing (native Nginx with full forwarding, all requests forwarded to the back-end tomcat cluster)

Location/{

root/web/www;

Index index.jsp index.htm index.html;

Proxy_redirect off;

#保留用户真实信息

Proxy_set_header Host $host;

Proxy_set_header X-real-ip $remote _addr;

Proxy_set_header x-forwarded-for $proxy _add_x_forwarded_for;

Proxy_pass Http://backend;

}

}

}

Mainly in the configuration of proxy and upstream

The upstream has load balancing capability, can automatically determine the following machine, and automatically kicked out of the machine can not provide services normally.

4, start the program

/nginx/sbin/nginx

5, write the startup script

Vi nginx.sh

#!/bin/sh

cwd= ' pwd '

Case $ in
Start
/nginx/sbin/nginx;
;;
Stop
Kill-2 ' Ps-ef|grep "/nginx/sbin/nginx" |grep-v "grep" |awk ' {print $} '
;;
Restart
CD "$CMD"
$ stop
$ start
;;
*)
echo $ "Usage: $ {Start|stop|restart}"
Exit 1
Esac
Exit 0

Two, configure Tomcat

1, download tomcat5.59

Tar zxvf tomcat5.59

2. Modify the configuration file

A, configure the data source

B, optimize tomcat max concurrency

<connector port= "8080" maxhttpheadersize= "8192"

maxthreads= "2048" minsparethreads= "maxsparethreads=" 200 "

Enablelookups= "false" redirectport= "8443" acceptcount= "500"

connectiontimeout= "20000" disableuploadtimeout= "true"/>

C, add a virtual host

(Note that the main forwarding of the virtual host must be localhost, otherwise nginx cannot be forwarded through the intranet IP, but only through the domain name forwarding

D, test

Open http://ip:8080

The page can be accessed normally

2, the other Tomcat servers are also configured with the same

Three, do tomcat cluster

Two machines 172.23.254.2 172.23.254.3

There are three locations for the file configuration to be modified by the cluster

1. Modify the Conf/server.xml configuration file
A. Locate the engine tag and add the attribute jvmroute= "Worker1"

B. Locate the cluster tag, remove the comment, and modify the tcplistenaddress to native IP 172.23.254.2 (Note: This section of cluster must be placed inside the hosts)

2. Modify the application's Web. xml

Modify the Web. xml file in the Web-inf directory, add tags

<distributable/>

Just add it to </web-app> before you can.

This is to join the session of Tomcat replication, do tomcat cluster must need this step, otherwise the user's session will not be able to use normal.

3, turn on the firewall

The firewall trust must be turned on between these two tomcat.

Launch two Tomcat respectively to see if each tomcat has 8080 ports and 4001 ports enabled

Re-use Netstat–an to view the link situation

TCP 0 0 172.23.254.2:43320 172.23.254.3:4001 established

TCP 0 0 172.23.254.2:46544 172.23.254.3:4001 time_wait

TCP 0 0 172.23.254.2:40118 172.23.254.3:4001 established

TCP 0 0 172.23.254.2:4001 172.23.254.3:48804 established

TCP 0 0 172.23.254.2:4001 172.23.254.3:34254 established

If the 4001 ports of the two machines are connected separately, the cluster configuration is successful and session replication is possible.

Problems that may exist

1, Session replication problem

Before using Apache for load balancing, is the choice of the session sticky mode, so that the user every time in the same server will be the session, will not be forwarded to other servers. In such cases, Tomcat does not affect user access even if the session is not replicated. However, Nginx does not support the sticky feature. So you have to do a session copy. Otherwise, a lot of places can't be used at all. For example, the login process, first to wait until the first tomcat, generated a session, in the Refresh page, brush to another Tomcat machine, without this session, there will be problems. So the programmer should also be aware of this when writing JSP

For example, if we modify a user's data in a session in a single application, it is usually:
User user = (user) request.getsession (). getattribute ("user");
User.setname ("My Name");
This allows us to directly access and then make changes, although there is no problem in the case of a single machine, but under the cluster conditions, this causes the session on multiple Web servers to be out of sync, because the session has not changed, Tomcat cannot monitor whether the value of one of the data in the session has changed. Therefore, we also need to do the following to ensure session synchronization:
Request.getsession (). SetAttribute ("user", user);
So, we should pay special attention when we operate the session! Another suggestion is that we should not modify the data in the session as much as possible.

You may often encounter a situation where the session is not replicating properly. In addition to the service side to find the reasons for the procedure to find the reason. May cause the session to replicate abnormally.

2. Page synchronization

To ensure that the page programs on the server behind Tomcat are consistent, you can use the following methods

A,rsync sync, or make page button, provide to edit, modify program even click Sync

B, shared area storage, or take DRBD network RAID mode

3, confirm that Nginx can be forwarded successfully,

On the Nginx wget the back of the URL (packet over the port), if you can open, then you can forward the past. cannot be forwarded if it cannot be opened

Nginx+tomcat Cluster load Balancing (for session replication)