Lvs+keepalived+nginx implementation of high performance load Balancing cluster high performance JSP cluster

Source: Internet
Author: User
Tags rar

lvs-master:192.168.254.134
lvs-backup:192.168.254.135
lvs-vip:192.168.254.88
nginx+tomcat:192.168.254.131
nginx+tomcat:192.168.254.132
nginx+tomcat:192.168.254.133
(multiple can be added as required)

Install the base package
Yum install-y gcc-c++ pcre pcre-devel zlib zlib-devel OpenSSL openssl-devel

Installing Nginx
(Download path for your own selection)
Download nginx1.14.0 stable version
wget http://nginx.org/download/nginx-1.14.0.tar.gz
Decompression: TAR-ZXF nginx-1.14.0.tar.gz
Enter the extracted source directory: CD nginx-1.14.0
Use the default configuration:./configure
Compiling and installing Nginx:make && make install
Start Nginx:/usr/local/nginx/sbin/nginx
Quick Stop Nginx:/usr/local/nginx/sbin/nginx-s Stop
Graceful Stop Nginx (will process the request in Process complete):/usr/local/nginx/sbin/nginx-s quit
Detect configuration Error:/usr/local/nginx/sbin/nginx-t
Reload configuration:/usr/local/nginx/sbin/nginx-s Reload
1, write Nginx startup script, and join the system services
Vim/etc/init.d/nginx
and write the following in it:

#!/bin/bash
#chkconfig:-30 21
#description: HTTP service.
#Source Function Library
. /etc/init.d/functions
#Nginx Settings
Nginx_sbin= "/usr/local/nginx/sbin/nginx"
nginx_conf= "/usr/local/nginx/conf/nginx.conf"
Nginx_pid= "/usr/local/nginx/logs/nginx.pid"
Retval=0
Prog= "Nginx"
Start () {
Echo-n $ "Starting $prog:"
Mkdir-p/dev/shm/nginx_temp
Daemon $NGINX _sbin-c $NGINX _conf
Retval=$?
Echo
Return $RETVAL
}
Stop () {
Echo-n $ "Stopping $prog:"
Killproc-p $NGINX _pid $NGINX _sbin-term
Rm-rf/dev/shm/nginx_temp
Retval=$?
Echo
Return $RETVAL
}
Reload () {
Echo-n $ "Reloading $prog:"
Killproc-p $NGINX _pid $NGINX _sbin-hup
Retval=$?
Echo
Return $RETVAL
}
Restart () {
Stop
Start
}
Configtest () {
$NGINX _sbin-c $NGINX _conf-t
return 0
}
Case "$" in
Start
Start
;;
Stop
Stop
;;
Reload
Reload
;;
Restart
Restart
;;
Configtest)
Configtest
;;
*)
echo $ "Usage: $ start|stop|reload|
Restart|configtest} "
Retval=1
Esac
Exit $RETVAL

and change the file execution permissions: chmod 755/etc/init.d/nginx
Join system service Startup list: Chkconfig--add nginx
and enable Boot: Chkconfig nginx on
Open service: Nginx start or/etc/init.d/nginx Start|stop|restart
Check to see if the Nginx page is properly accessible.
http://192.168.254.131/

Installing the base Environment JDK
wget http://download.oracle.com/otn-pub/java/jdk/8u171-b11/512cd62ec5174c3487ac17c61aaa89e8/ Jdk-8u171-linux-x64.tar.gz? Authparam=1530931780_fdbf6ecf3dfcc8e6eb6fcef06a7eaaec

TAR-ZXF jdk-8u171-linux-x64.tar.gz-c/usr/local/
cd/usr/local/
MV jdk1.8.0_171/jdk1.8
Vi/etc/profile
At the end of the Add JDK environment variable
Export java_home=/usr/local/jdk1.8
Export path= $JAVA _home/bin: $PATH
Export classpath=.: $JAVA _home/lib/dt.jar: $JAVA _home/lib/tools.jar
Make environment variables effective
Source/etc/profile
View JDK version
Java-version
Java Version "1.8.0_171"
Java (TM) SE Runtime Environment (build 1.8.0_171-B11)
Java HotSpot (TM) 64-bit Server VM (build 25.171-b11, Mixed mode)
View JDK paths
Echo $JAVA _home
/usr/local/jdk1.8

Installing Tomcat
wget http://mirrors.hust.edu.cn/apache/tomcat/tomcat-9/v9.0.10/bin/apache-tomcat-9.0.10.tar.gz

TAR-ZXF apache-tomcat-9.0.10.tar.gz
MV Apache-tomcat-9.0.10/usr/local/tomcat
Create a project directory
Mkdir-p/home/www/web/(Write a index.jsp inside)
VI index.jsp (save to launch)
Modify the Tomcat access path
cd/usr/local/tomcat/conf/
VI Server.xml
Unpackwars= "true" autodeploy= "true" >
This line is added.
<context path= "" docbase= "/home/www/web" reloadable= "true"/>

See if the Tomcat page will be accessed properly
http://192.168.254.131:8080/

Configuring Nginx and Tomcat load balancing/reverse Proxies
cd/usr/local/nginx/conf/
VI nginx.conf
User www www;
Worker_processes 4;

Error_log/usr/local/nginx/logs/error.log;
Error_log/usr/local/nginx/logs/error.log notice;
Error_log/usr/local/nginx/logs/error.log info;

Pid/usr/local/nginx/logs/nginx.pid;

#工作模式及连接数上限
Events {
Use Epoll;

worker_connections  65535;

}

#设定http服务器, which provides load balancing support using its reverse proxy function
http {
#设定mime类型
include mime.types;
Default_type Application/octet-stream;
Include/usr/local/nginx/conf/proxy.conf;
Log_format main ' $remote _addr-$remote _user [$time _local] "$request" '
' $status $body _bytes_sent "$http _referer "'
'" $http _user_agent "" $http _x_forwarded_for "';

Access_log Logs/access.log main; #设定请求缓冲server_names_hash_bucket_size 128;client_header_buffer_size 32k;large_ Client_header_buffers 4 32k;# client_max_body_size 8m;sendfile on;tcp_nopush on;tcp_nodelay on; #keepali Ve_timeout 0;keepalive_timeout 65;gzip on;gzip_min_length 1k;gzip_buffers 4 16k;gzip_http_version 1.1;gzip_comp_lev    El 2;gzip_types text/plain application/x-javascript text/css application/xml;gzip_vary on;   #此处为你tomcat的地址, you can write multiple tomcat addresses upstream Tomcat_pool {server 192.168.254.133:8080 weight=4 max_fails=2 fail_timeout=30s;   Server 192.168.254.132:8080 weight=4 max_fails=2 fail_timeout=30s;    Server 192.168.254.131:8080 weight=4 max_fails=2 fail_timeout=30s;}    server {Listen 80; server_name www.web2.com; #此处替换为你自己的网址, if there are multiple intermediate spaces index index.jsp index.htm index.html index.do; #设定访问的默认首页地址 ro Ot/home/www/web;    #设定网站的资源存放路径 #charset Koi8-r;    #access_log Logs/host.access.log Main; Location/{# root HTml    Index index.jsp index.html index.htm; } location ~ \.        (JSP|JSPX|DP)? $ #所有JSP的页面均交由tomcat处理 {proxy_set_header Host $host;         Proxy_set_header X-real-ip $remote _addr; Proxy_pass Http://tomcat_pool, #转向tomcat处理} #设定访问静态文件直接读取不经过tomcat location ~. *\.     (HTM|HTML|GIF|JPG|JPEG|PNG|BMP|SWF|IOC|RAR|ZIP|TXT|FLV|MID|DOC|PPT|PDF|XLS|MP3|WMA) $ {expires 30d; } location ~. *\.     (JS|CSS)? $ {Expires 1h;    } access_log/usr/local/nginx/logs/ubitechtest.log main; #设定访问日志的存放路径 error_page 502 503 504/50x.html;    Location =/50x.html {root html;           # Pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000 # #location ~ \.php$ {# root    html    # Fastcgi_pass 127.0.0.1:9000;    # Fastcgi_index index.php;    # Fastcgi_param Script_filename/scripts$fastcgi_script_name;    # include Fastcgi_params; #} # Deny access to. htaccess filES, if Apache ' s document Root # concurs with Nginx ' s one # #location ~/\.ht {# deny all;    #}}server {Listen 80;    server_name bbs.yourdomain.com; Location/{root/home/www/web/springmvc; #设定网站的资源存放路径 index index.jsp index.htm index.html index.do WELC ome.jsp; #设定访问的默认首页地址} location ~ \.        (JSP|JSPX|DP)? $ #所有JSP的页面均交由tomcat处理 {proxy_set_header Host $host;         Proxy_set_header X-real-ip $remote _addr; Proxy_pass Http://tomcat_pool, #转向tomcat处理} #设定访问静态文件直接读取不经过tomcat location ~. *\.     (HTM|HTML|GIF|JPG|JPEG|PNG|BMP|SWF|IOC|RAR|ZIP|TXT|FLV|MID|DOC|PPT|PDF|XLS|MP3|WMA) $ {expires 30d; } location ~. *\.     (JS|CSS)? $ {Expires 1h;    } access_log/usr/local/nginx/logs/ubitechztt.log main; #设定访问日志的存放路径 error_page 502 503 504/50x.html;    Location =/50x.html {root html; }}# Another virtual host using mix of ip-, name-, and port-based Configuration# #server {# Listen 8000;# listen somename:8080;# server_name somename alias another.alias;# Location/{# root html;# index index.html index.htm;#}#}

}

Create user www and user group www to use the file directory
/usr/sbin/groupadd www
/usr/sbin/useradd-g www www-s/sbin/nologin
Mkdir-p/home/www
chmod +w/home/www
Chown-r www:www/home/www

Vi/usr/local/nginx/conf/proxy.conf
Proxy_redirect off;
Proxy_set_header Host $host;
Proxy_set_header X-real-ip $remote _addr;
Client_max_body_size 10m;
Client_body_buffer_size 128k;
Proxy_connect_timeout 90;
Proxy_send_timeout 90;
Proxy_read_timeout 90;
Proxy_buffer_size 4k;
Proxy_buffers 4 32k;
Proxy_busy_buffers_size 64k;
Proxy_temp_file_write_size 64k;

/usr/local/nginx/sbin/nginx-t
Nginx:the configuration file/usr/local/nginx/conf/nginx.conf syntax is OK
Nginx:configuration file/usr/local/nginx/conf/nginx.conf Test is successful

Simple test
3 Nginx can access the agent's three Tomcat separately.
[Email protected] conf]# Curl 192.168.254.131
2222222222222222222222222222222222222222
[Email protected] conf]# Curl 192.168.254.131
33333333333333333333333333333
[Email protected] conf]# Curl 192.168.254.131
111111111111111111111111111111111111111
OK, here, Nginx and Tomcat are done.
(Note: Each of these configurations is the same)

Install lvs+keepalived.

Yum-y Install Ipvsadm keepalived (yum installation is not much to say)

Turn on route forwarding
Permanently open
echo "Net.ipv4.ip_forward = 1" >>/etc/sysctl.conf
SYSCTL-P----load, making the configuration file effective immediately
Check:
Sysctl-a |grep "Ip_forward"
Net.ipv4.ip_forward = 1
NET.IPV4.IP_FORWARD_USE_PMTU = 0

Configuring Keepalived (Master Configuration)
Vi/etc/keepalived/keepalived.conf
! Configuration File for Keepalived

Global_defs {
Notification_email {br/>[email protected]

Notification_email_from [email protected]
Smtp_server 127.0.0.1
Smtp_connect_timeout 30
router_id Lvs_master #备份服务器上将MASTER改为BACKUP
}

Vrrp_instance Vi_1 {
State BACKUP #备份服务器上将MASTER改为BACKUP
Interface Ens32 #该网卡名字需要查看具体服务器的网口
VIRTUAL_ROUTER_ID 51
Priority 100 # Change 100 to 90 on backup service
Nopreempt
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1111
}
virtual_ipaddress {
192.168.254.88
# (if you have more than one VIP, continue with line wrapping.)
}
}

Virtual_server 192.168.254.88 80 {
Delay_loop 6 # (query Realserver status every 6 seconds)
Lb_algo RR # (RR algorithm)
Lb_kind DR # (Direct Route)
Persistence_timeout 50 # (connection of the same IP is assigned to the same realserver within 60 seconds)
Protocol TCP # (check realserver status with TCP protocol)

real_server 192.168.254.131 80 {    weight 1   #(权重)    TCP_CHECK {        connect_timeout 10    #(10秒无响应超时)        nb_get_retry 3        delay_before_retry 3        connect_port 80    } } real_server 192.168.254.132 80 {    weight 1    TCP_CHECK {        connect_timeout 10        nb_get_retry 3        delay_before_retry 3        connect_port 80    } } real_server 192.168.254.133 80 {    weight 1    TCP_CHECK {        connect_timeout 10        nb_get_retry 3        delay_before_retry 3        connect_port 80    } }

}

Backup configuration
Vi/etc/keepalived/keepalived.conf
! Configuration File for Keepalived

Global_defs {
Notification_email {br/>[email protected]

Notification_email_from [email protected]
Smtp_server 127.0.0.1
Smtp_connect_timeout 30
router_id Lvs_backup #备份服务器上将MASTER改为BACKUP
}
Vrrp_instance Vi_2 {
State BACKUP #备份服务器上将MASTER改为BACKUP
Interface Ens32 #该网卡名字需要查看具体服务器的网口
VIRTUAL_ROUTER_ID 51
Priority 90 # Change 100 to 90 on backup service
Nopreempt
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass 1111
}
virtual_ipaddress {
192.168.254.88
# (if you have more than one VIP, continue with line wrapping.)
}
}

Virtual_server 192.168.254.88 80 {
Delay_loop 6 # (query Realserver status every 6 seconds)
Lb_algo RR # (RR algorithm)
Lb_kind DR # (Direct Route)
Persistence_timeout 50 # (connection of the same IP is assigned to the same realserver within 60 seconds)
Protocol TCP # (check realserver status with TCP protocol)

real_server 192.168.254.131 80 {    weight 1   #(权重)    TCP_CHECK {        connect_timeout 10    #(10秒无响应超时)        nb_get_retry 3        delay_before_retry 3        connect_port 80    } } real_server 192.168.254.132 80 {    weight 1    TCP_CHECK {        connect_timeout 10        nb_get_retry 3        delay_before_retry 3        connect_port 80    } } real_server 192.168.254.133 80 {    weight 1    TCP_CHECK {        connect_timeout 10        nb_get_retry 3        delay_before_retry 3        connect_port 80    } }

}

Service keepalived Start
ENS32: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU Qdisc pfifo_fast State up group default Qlen 1000
Link/ether 00:0C:29:E3:F4:FF BRD FF:FF:FF:FF:FF:FF
inet 192.168.254.135/24 BRD 192.168.254.255 scope Global Noprefixroute ENS32
Valid_lft Forever Preferred_lft Forever
inet 192.168.254.88/32 Scope Global ENS32
Valid_lft Forever Preferred_lft Forever

显示虚拟IP 启动好了 最后就是在ngnx 服务器上配置vi /etc/init.d/realserver #!/bin/bash

sns_vip=192.168.254.88
/etc/rc.d/init.d/functions
Case "$" in
Start
Ifconfig lo:0 $SNS _vip netmask 255.255.255.255 broadcast $SNS _VIP
/sbin/route add-host $SNS _vip Dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
Sysctl-p >/dev/null 2>&1
echo "Realserver Start OK"
;;
Stop
Ifconfig lo:0 Down
Route del $SNS _VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "Realserver stoped"
;;
*)
echo "Usage: $ {start|stop}"
Exit 1
Esac
Exit 0

chmod 755/etc/init.d/realserver
Service Realserver Start
Lo: <LOOPBACK,UP,LOWER_UP> MTU 65536 qdisc noqueue State UNKNOWN Group default Qlen 1000
Link/loopback 00:00:00:00:00:00 BRD 00:00:00:00:00:00
inet 127.0.0.1/8 Scope host Lo
Valid_lft Forever Preferred_lft Forever
inet 192.168.254.88/32 BRD 192.168.254.88 Scope Global lo:0
Valid_lft Forever Preferred_lft Forever

看到虚拟IP 就好了(同理在其他nginx服务器都要操作)测试 [[email protected] ~]# curl 192.168.254.88

444444444444444444444444
[email protected] ~]# Curl 192.168.254.88
55555555555555555555555555
[email protected] ~]# Curl 192.168.254.88
22222222222222222222222222222222222222222222222222222
[email protected] ~]# Curl 192.168.254.88
444444444444444444444444
[email protected] ~]# Curl 192.168.254.88
33333333333333333333333333333
[email protected] ~]# Curl 192.168.254.88
111111111111111111111111111111111111111

Lvs+keepalived+nginx implementation of high performance load Balancing cluster high performance JSP cluster

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.