Nginx + Tomcat + Memcached server load balancer cluster service construction

Source: Internet
Author: User
Tags couchbase epoll
: This article mainly introduces how to build the Nginx + Tomcat + Memcached server load balancer cluster service. For more information about PHP tutorials, see.
Reprinted please indicate the source:

Operating system: CentOS6.5

This document describes how to set up an Nginx + Tomcat + Memcached server load balancer cluster under CentOS6.5, Nginx is responsible for load balancing, Tomcat is responsible for actual services, and Memcached is responsible for synchronizing Tomcat sessions, to achieve Session sharing.

1. install Nginx

Official Nginx website:

Download the latest stable version. Install the gcc, openssl, pcre, and zlib software libraries before installing Nginx.

1.1 install gcc and gcc-c ++

Installation Command:

#sudo yum install gcc# sudo yum install gcc-c++
1.2 install openssl

Openssl official website:

Installation version: openssl-1.0.1i.tar.gz

Installation Command:

# Tar-zxvf openssl-1.0.1i.tar.gz # cd openssl-1.0.1i # sudo./config -- prefix =/usr/local/openssl-1.0.1i # prefix specifies the installation directory # sudo make install

[Note]: the config command is used here, instead of the common configure command.

After the installation is complete, go to/usr/local/to check whether the installation is successful. If an error occurs during installation, you need to re-compress and reinstall it.

1.3 install pcre

Pcre official website:

Installation version: pcre-8.35.tar.gz

Installation Command:

# Tar-zxvf pcre-8.35.tar.gz # cd pcre-8.35 # sudo./configure -- prefix =/usr/local/pcre-8.35 # prefix specifies the installation directory # sudo make install

After the installation is complete, go to/usr/local/to check whether the installation is successful. If an error occurs during installation, you need to re-compress and reinstall it.

[Note]: if the c ++ compiler is not installed, an error will be reported during installation of this software!

1.4 install zlib

Zlib official website:

Installation version: zlib-1.2.8.tar.gz

Installation Command:

# Tar-zxvf zlib-1.2.8.tar.gz # cd zlib-1.2.8 # sudo./configure -- prefix =/usr/local/zlib-1.2.8 # prefix specifies the installation directory # sudo make install

After the installation is complete, go to/usr/local/to check whether the installation is successful. If an error occurs during installation, you need to re-compress and reinstall it.

1.5 install Nginx

Installation version: nginx-1.6.1.tar.gz

Installation Command:

# Tar-zxvf nginx-1.6.1.tar.gz # cd nginx-1.6.1 # sudo. /configure -- prefix =/usr/local/nginx-1.6.1 # prefix specifies the installation directory -- with-openssl =/home/zht/src/openssl-1.0.1i # refers to the openssl source code path -- with-pcre =/home/zht/src/pcre-8.3.5 # refers to the pcre source code path -- with-zlib =/home/zht/src/zlib-1.2.8 # refers to the source code path of zlib -- with-http_ssl_module # sudo make # make install

After the installation is complete, go to/usr/local/to check whether the installation is successful. If an error occurs during installation, you need to re-compress and reinstall it.

1.5.1 Configure Nginx

Configuration file directory:/usr/local/nginx-1.6.1/conf/nginx. conf

# cd /usr/local/nginx-1.6.1/conf# sudo vi nginx.conf

[The modified configuration file is as follows ]:

# Create a process user and user Group user zht; # Number of service processes, which is generally equal to the number of CPUs worker_processes 1; # global error log definition. we recommend that you enable error-level logs. [debug | info | notice | warn | error | crit] error_log logs/error. log error; # error_log logs/error. log notice; # error_log logs/error. log info; # file that records the process ID # pid logs/nginx. pid; events {# epoll is a method of Multiplexing IO (I/O Multiplexing), but it is only used for Linux and later kernels, which can greatly improve nginx performance. epoll is recommended for Linux, and kqueue is recommended for FreeBSD. useepoll; # A worker_processe Number of recent concurrent connections allowed worker_connections 1024;} http {include mime. types; default_type application/octet-stream; # log_format main '$ remote_addr-$ remote_user [$ time_local] "$ request"' # '$ status $ response "$ http_referer"' # '"$ http_user_agent" "$ http_x_forwarded_for "'; # access_log logs/access. log main; sendfile on; # tcp_nopush on; # keepalive_timeout 65; # gzip compression setting gzip on; # enable gzip gzip_mi N_length 1 k; # minimum compressed file size gzip_buffers 4 16 k; # compressed buffer # http protocol version (1.0/1.1), default 1.1, if the frontend is squid2.5, use 1.0 gzip_http_version 1.1; # gzip compression ratio. the minimum compression ratio is the fastest, and the maximum compression ratio is 9, but the processing speed is the slowest (fast transmission but cpu consumption) gzip_comp_level 2; # It has something to do with the http header. add a vary header for the proxy server. some browsers support compression and some do not support compression, so avoid wasting and compressing unsupported ones, therefore, based on the HTTP header of the client, determine whether to compress gzip_varyon; # gzip compression type, without adding text/html, otherwise, there will be a warning message gzip_types text/plain text/javascript text/css application/xmlapplication/x-plain Crip. T application/json; # set the server list of server load balancer. multiple upstreams can be set. However, the name of mysvr must be distinguished by upstreammyClusterServer1 {# The weigth parameter indicates the weight, the higher the weight, the higher the chance of being allocated to it # enable port 3128 server 8081 weight = 5 on the Squid on the local machine; server 8082 weight = 5; server 8083 weight = 5;} server {# nginx listening port listen 80; # multiple domain names can be added, separated by spaces server_name; # Character encoding charset UTF-8; # set the access logs for this VM. Disabling logs can reduce IO and improve performance. # Access_log logs/host. access. log main; # default request location/{# define the server's default website root directory location root html; # define the name of the home index file index index.html index.htm index. jsp; # Redirect requests to the server list defined by mysvr proxy_pass http://myClusterServer1 ; Proxy_redirect default; # timeout time for connecting to the proxy server, you must note that this time out time cannot exceed 75 seconds. when one server crashes, it will be forwarded to another server after 10 seconds. Proxy_connect_timeout 10 ;}# error_page 404/404 .html; # redirect server error pages to the static page/50x.html # error_page 500 502 503 x.html; location =/50x.html {root html ;} # proxy the PHP scripts to Apache listening on 80 # location ~ \. Php $ {# proxy_pass #}# Pass the PHP scripts to FastCGI server listening on 9000 ## location ~ \. Php $ {# root html; # fastcgi_pass 9000; # fastcgi_index index. php; # fastcgi_param SCRIPT_FILENAME/scripts $ fastcgi_script_name; # include fastcgi_params; #}# deny access. htaccess files, if Apache's document root # concurs with nginx's one # location ~ /\. Ht {# deny all ;#}# anothervirtual host using mix of IP-, name-, and port-based configuration # server {# listen 8000; # listen somename: 8080; # server_name somename alias another. alias; # location/{# root html; # index index.html index.htm; #}#}# HTTPSserver # server {# listen 443 ssl; # server_name localhost; # ssl_certificate cert. pem; # ssl_certificate_key cert. key; # ssl_sess Ion_cache shared: SSL: 1 m; # ssl_session_timeout 5 m; # ssl_ciphers HIGH :! ANULL :! MD5; # ssl_prefer_server_ciphers on; # location/{# root html; # index index.html index.htm ;#}#}}
1.5.2 start and stop Nginx

(1) start

# sudo  /usr/local/nginx-1.6.1/sbin/nginx

Ensure that Port 80 of the system is not occupied by other programs


# sudo  /usr/local/nginx-1.6.1/sbin/nginx  -s  reload

(2) disable:

ps -ef | grep nginx

Find the master process ID and kill it, for example:

# Kill-9 [process number]

(3) Check whether startup is successful:

netstat -ano | grep80

If a result is entered, the startup is successful.

Open the browser to access the IP address of this machine. if the browser displays Welcome to nginx! Nginx has been installed and runs successfully. If you have configured the server load balancer server, the website page in Tomcat is displayed, for example:

2. install Memcache

Memcached official website:

To install memcached, you must first install libevent, libevent official website:

This installation version:



2.1 install livevent

Check whether it is installed: # rpm qa | grep libevent

If you have installed and the version is earlier than 1.3, you must first use:

# rpm -e libevent --nodeps


# Tar zxvf libevent-2.0.21-stable.tar.gz # cd libevent-2.0.21-stable # sudo./configure -- prefix =/usr/local/libevent-2.0.21-stable # prefix specifies the installation path # sudo make install

After the installation is complete, go to the directory specified by prefix to see if the libevent-2.0.21-stable directory exists, as shown in.

2.2 install Memcached

# tar zxvf memcached-1.4.20.tar.gz# cd memcached-1.4.20# sudo ./configure--prefix=/usr/local/memcached-1.4.20--with-libevent=/usr/local/libevent-2.0.21-stable# sudo make# sudo make install

2.2.1 Check installation status

After the installation is complete, go to the directory specified by prefix to check if there is a memcached-1.4.20 directory, as shown in.

2.2.2 view memcached and libevent versions

First, go to the bin directory of Memcached:

# cd /usr/local/memcached-1.4.20/bin

Run the following command:

# sudo ./memcached -i

2.2.3 start memcached

# sudo ./memcached -d -v -p 12000 -m 512 -u zht

Explanation:-d indicates running memcached as a daemon;-v indicates the output extract and error message;-p indicates the listening port number;-m indicates the maximum memory available, in MB; -u indicates the account that runs memcached, not the root user.

Use # ps-ef | grep memcached to view the process.

Follow the basic options:



TCP port of The Listener (default: 11211)


Run memcached as a daemon


The account that runs memcached. it is not a root user.


Maximum memory usage. the unit is MB. the default value is 64 MB.


Number of soft connections. the default value is 1024 (maximum number of concurrent connections)


Output warning and error messages


Print client requests and returned information


Print Help information


Print the copyright information of memcached and libevent

2.2.4 use telnet to verify whether the service is available

Install telnet in Win7: Control Panel> programs and functions> Enable or install Windows functions> select Telnet server and Telnet client.

Install the telnet client on CentOS:

# sudo yum install telnet

Windows command:

CentOS (Linux) command:

# telnet 12000


Connected to localhost ( ).

Escape character is '^]'.

After the connection is successful, manually enter the command: stats

You can see the following information:

Manually enter: quit // exit

Connection closed by foreign host

2.2.5 stop the memcached service

# ps -ef | grep memcached

Find the memcached process ID, and then force kill:

kill - 9 2439

3. install Tomcat + configure memcached

Tomcat official website:

3.1 install Tomcat

Use Tomcat version: apache-tomcat-7.0.55.tar.gz

Decompress Tomcat to any directory.

3.2 configure memcached3.2.1 for Tomcat to add a library file for Tomcat

To support memcached Session management in Tomcat, you need to call some jar files:

Add the dependent jar packages of mem and msm:











Note ]:

Msm1.6.5 depends on Couchbase. you need to add the jar package of couchbase-client. Otherwise, the following error will be reported during startup: java. lang. NoClassDefFoundError: com/couchbase/client/CouchbaseClient.

Tomcat6 and Tomcat7 use different msm support packages: memcached-session-manager-tc6-1.6.5.jar and memcached-session-manager-tc7-1.6.5.jar, only one option, otherwise startup error is reported.

The lib package version in the msm source code is too low: spymemcached needs to use 2.10.2; otherwise, an error is returned when tomcat is started:

Java. lang. NoSuchMethodError: net. spy. memcached. MemcachedClient. set (Ljava/lang/String; ILjava/lang/Object;) Lnet/spy/memcached/internal/OperationFuture;

Atde. javakaffee. web. msm. BackupSessionTask. storeSessionInMemcached (BackupSessionTask. java: 227)

Kryo-serializers must use version 0.10; otherwise, an error is returned:

Caused by: java. lang. ClassNotFoundException: de. javakaffee. kryoserializers. DateSerializer

Partial files:

Find other files by yourself.

After the download, put these library files under the tomcat \ lib directory.

3.2.2 configure memcached for Tomcat

Configuration file directory: tomcat \ conf \ context. xml

Open the configuration file and go ... Add the following content to the node:


[Parameter description ]:

DocBase: The appBase in is consistent with the website deployment directory.

MemcachedNodes: memcached server information. Separate multiple servers with spaces, for example:

N1: 12001 n2: 12002 n3: 12003

Shows the configuration file:

Also configured in server. xml Node appBase = "deployment Directory"

3.2.3 test Session sharing

The test JSP code is as follows: index. jsp

Start multiple Tomcat servers at the same time. I deployed three Tomcat servers. open the browser to access the first Tomcat server and then access the second and third Tomcat servers. the Session information on the page is as follows:

As shown in the figure, the sessionids of the three Tomcat servers are the same: 5fbf6d6b6f37be4158ed965536427005-n1. as long as the browser is not closed, the SessionID remains unchanged no matter how the server is refreshed. Thus, the three Tomcat servers share Session information through memcached.

4. install the Samba shared file service

Check whether samba has been installed:

# rpm -qa | grep samba

4.1 install samba

The network installation is fast and convenient. the command is as follows:

# sudo yum install samba samba-client

4.2 configure shared folders

(1) modify the configuration file:

# sudo vi /etc/samba/smb.conf


Path =/home/zht/SharedFolder # Share directory path writeable = yes # whether to allow write browsable = yes # whether to allow access to directory contents valid user = zht # Name of the account accessing the Directory

(2) add an account

Add zht account:

# sudosmbpasswd -a zht

Output the access password as prompted.

4.3 enable port number
Port 137 (UDP)  --NetBIOS name server and nmbdPort 138 (UDP) --NetBIOS datagram servicePort 139 (TCP) --File and printer sharing and smbdPort 389 (TCP) --for LDAP(Active Directory Mode)Port 445 (TCP) --NetBIOS was moved to 445 after 2000 and beyond(CIFS)Port 901 (TCP) --for SWAT

(1) Add the port: sudo setup using a graphical CentOS;

(2) use the following command to enable:

# Iptables-I input-p udp -- dport [port number]-j ACCEPT # iptables-I INPUT-p tcp -- dport [port number]-j ACCEPT


# sudo service iptables save

4.4 disable SELinux

View Status:

# getenforce


# setenforce 0

Several SELinux statuses:

Enforcing: Forced mode, which indicates that SELinux is running and has correct restrictions;

Permissive: tolerant mode, which indicates that SELinux is running, but only warning information is provided, which is not actually limited;

Disable: Disabled. SELinux is not running.

Permanent close method:

# Sudo vi/etc/selinux/config # SELINUX = enforcing comment out # SELINUXTYPE = targeted comment out

Add a row at the end:

SELINUX = disable

Save closed:


Restart the system

4.5 start and stop services

(1) view the running Status:

# sudo service smb status

(2) start and stop the service:

# sudo /etc/init.d/smb   start/stop/restart


# sudo service smb    start/stop/restart

(3) enable startup:

# sudo chkconfig --level 35 smb on

Automatically run the smb service at level 3 and 5

(4) verification service

# smbclient -L // -U rxyy


# smbclient // -U rxyy

5. install and configure JDK

The JDK Version used this time is: jdk-7u60-linux-x64.tar.gz

5.1 uninstall installed JDK

View installed JDK

# rpm -qa | grep jdk*

Uninstall JDK:

# Sudo yum-y remove [package name]

As shown in:

Install the new JDK: extract jdk-7u60-linux-x64.tar.gz to the specified directory.

5.2 configure Java environment variables

# sudo vi /etc/profile

Add at the end:


exportJAVA_HOME=/home/zht/BalanceServer/Java/jdk1.7.0_60export JRE_HOME=$JAVA_HOME/jreexport CLASSPATH=$CLASSPATH:.:$JRE_HOME/lib:$JAVA_HOME/libexport PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin

As shown in:

6. install the MySQL database

6.1 uninstall installed mysql

View the installed mysql database:

# rpm -qa | grep mysql

Uninstall the installed mysql database:

# Rpm-e [package name] // Normal uninstallation, may be unable to uninstall because of dependency # rpm-e -- nodeps [package name] // force uninstallation, can also be uninstalled if dependency exists


# Yum-y remove [package name] // automatically handles dependencies when uninstalling

6.2 install mysql

To install a MySQL database, you must install four components: mysql -- shared-compat, mysql-server, mysql-client, and mysql-workbench. the installation method is as follows.

(1) install the mysql shared library:

# rpm -ivh MySQL-shared-compat-5.6.20-1.el6.x86_64.rpm

(2) install the mysql server ::

# rpm -ivh MySQL-server-5.6.20-1.el6.x86_64.rpm

(3) install the mysql client:

# rpm -ivh MySQL-client-5.6.20-1.el6.x86_64.rpm

(4) install the mysql-workbench tool:

# rpm -ivh mysql-workbench-community-6.1.7-1.el6.x86_64.rpm

[Description]: RPM installation package. you can double-click to run the installation package under CentOS. The system automatically processes the dependency.

Attachment: yum installation method

View the installed version on yum:

# yumlist | grep mysql


# yum install -y mysql-server mysql mysql-devel

6.3 configure mysql

Configuration file directory:

/Usr/my. cnf or/etc/mysql/my. cnf

After opening the file, add the following configuration after [mysqld:

# Character encoding


# Storage Engine


# Buffer pool


# Maximum number of connections


# Maximum allowed package size (for example, a Blob field)


6.4 start mysql

(1) start mysql:

# sudo service mysql  start/stop/restart


# sudo /etc/rc.d/init.d/mysql  start/stop/restart

(Or # sudo/etc/rc. d/init. d/mysqld start/stop/restart)

When an error occurs during mysql startup, go to/var/liv/mysql/xxx. err to view the error message.

(2) log on to MySQL

# Mysql-u root-p # enter the password # mysql> show databases; # mysql> show variables like '% character_set %' # mysql>...

(3) Create a remote connection user

Enable the sxzl user's logon permission on the local machine (localhost) and password hymmldr.

# mysql> grant all privileges on *.* tosxzl@localhost identified by 'hymmldr.' with grant option;

Enable the sxzl user's remote (%) logon permission and password hymmldr.

# mysql> grant all privileges on *.* to sxzl@"%"identified by 'hymmldr.' with grant option;

Refresh permission

# mysql> flush privileges;

You need to reset the mysql password for the first installation. for details, see the following.

6.5 reset the mysql password (official website)

Start mysql and run the following command:

# ps -ef | grep -i mysql

View the path of mysqld_safe, for example:/usr/bin/mysqld_safe

Stop mysql and start mysql securely:

# sudo  /usr/bin/mysqld_safe  --skip-grant-tables  >/dev/null 2>&1  &

5 seconds later:

# sudo /usr/bin/mysql -u root mysql

Reset the password of the root user:

# mysql> update user SET PASSWORD=PASSWORD('root')where user='root'

Refresh permission

# mysql> flush privileges;# mysql> exit;

Log on to mysql again:

# mysql -u root -p

# Enter the password

Then execute:

# mysql> show databases;

Error: You must set password before executing thisstatement.

Solution: reset the password once:

# mysql> SET PASSWORD=PASSWORD('root');# mysql> flush privileges;

After setting, you can open mysql-workbench to connect to the database.

7 Command description: yum

Description of yum installation options:

Yum-y install package name (* supported): automatically select y, fully automatic yum install package name (* supported): manually select y or nyum-y remove [package name]: automatic Processing dependent yum remove package name (not supported *) rpm-ivh package name (supported *): install the rpm Package

Copyright Disclaimer: This article is an original article by the blogger and cannot be reproduced without the permission of the blogger.

The above describes how to build the Nginx + Tomcat + Memcached server load balancer cluster service, including some content, and hope to help friends who are interested in PHP tutorials.

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.