LVS (DR) + keepalived on centos6 installation and configuration Manual

1. operating system version and LVS software version:

CentOS6 (Linux Kernel 2.6.32-71.el6.i686)ipvsadm 1.2.4keepalived 1.1.17

2. Required IP Configuration:

LVS Master Ip: 10.0.0.5lvs backup IP: 10.0.0.6realserver IP: 10.0.0.7, 10.0.0.5 (and), 10.0.0.6 (and) netgetway IP: 10.0.0.1 virtual IP (VIP): 10.0.0.2

3. Notes:

Centos6 installation select development workstation mode firewall is disabled (because the business machine front-end has a dedicated firewall, if not, then the master can also)

4. network extension diagram:

5. Install ipvsadm and keepalived:

[Root @ rserver2/] # mkdir/soft [root @ rserver2/] # cd/soft [root @ rserver2 soft] # wget ‑soft] # ln-S /usr/src/kernels/2.6.32-71. el6.i686 // usr/src/Linux [root @ rserver2 soft] # tar-zxvf ipvsadm-1.24.tar.gz [root @ rserver2 soft] # cd unzip sad M-1.24 [root @ rserver2 ipvsadm-1.24] # Make; make install [root @ rserver2 ipvsadm-1.24] # CD .. [root @ rserver2 soft] # tar-zxvf keepalived-1.1.17.tar.gz [root @ rserver2 soft] # cd keepalived-1.1.17 [root @ rserver2 soft] #. /configure [root @ rserver2 keepalived-1.1.17] # Make; make install smoothly on these steps, if there is an error prompt, then according to the specific error processing, generally possible errors: 1) OpenSSL, the prompt may be as follows !!! OpenSSL is not properly installed on your system .!!!!!! Can not include OpenSSL headers files. solution: run the Yum-y installopenssl-devel2) and prompt that there is no GCC compiler solution: run Yum installncurses-devel GCC gcc-C ++ make rpm-Build [root @ rserver2 keepalived-1.1.17] # cp/usr/local/etc/rc. d/init. d/keepalived/etc/init. d/keepalived [root @ rserver2 keepalived-1.1.17] # cp/usr/local/sbin/keepalived/usr/sbin/[root @ rserver2 keepalived-1.1.17] # cp/usr/local/etc/sysconfig /keepalived/etc/sysconfig/[root @ rserver2 keepalived-1.1.17] # mkdir-P/etc/keepalived/[root @ rserver2 keepalived-1.1.17] # cp/usr/local/etc/keepalived. conf/etc/keepalived. conf [root @ rserver2 keepalived-1.1.17] # chmod + x/etc/init. d/keepalived

6. edit/etc/keepalived. conf on The LVS master machine:

Open the editor, such as VI or gedit/etc/keepalived. conf, and modify the content of the program as follows. # Configure/etc/keepalived on the master server. confglobal_defs {icationication_email {leekexi@gmail.com # multiple addresses} icationication_email_from leekexi@gmail.com smtp_server smtp.gmail.com smtp_connect_timeout 30 router_id lvs_devel} # monitor ipvsadm Process status, run vrrp_script chk_rjsadm {script "/usr/local/keepalived/chk_1_sadm.sh" interval 3 weight 3} vrrp_instance vi_1 {state master # indicates that the master backup machine is backup interface eth0 every 3 seconds. virtual_router_id 51 priority 100 # Master permission is higher than backup. For example, backup is 99 advert_int 1 # mcast_src_ip 10.0.0.5 # master server IP address, for a backup machine, enter the IP address authentication {auth_type pass # auth_pass 1111} virtual_ipaddress {10.0.0.2 # multiple virtual IP addresses, line feed }}# virtual_server 10.0.0.2 21 {delay_loop 6 # (RealServer status query every 10 seconds) lb_algo RR # (LVS algorithm) lb_kind Dr # (direct route) persistence_timeout 60 # (connections from the same IP address are allocated to the same RealServer within 60 seconds) protocol TCP # (check the RealServer status using the TCP protocol) # actual Server IP address and port real_server 10.0.0.5 21 {weight 5 tcp_check {connect_timeout 10 nb_get_retry 3 delay_before_retry 3 connect_port 21 }}# actual Server IP address and port real_server 10.0.0.6 21 {weight 5 tcp_check {connect_timeout 10 listen 3 listen 3 connect_port 21 }}# configure virtual_server 10.0.0.2 80 {delay_loop 6 lb_algo RR comment Dr protocol TCP real_server 10.0.0.5 80 {weight 5 tcp_check {connect_timeout 10 when 3 then 3 connect_port 80} real_server 10.0.0.6 80 {weight 5 tcp_check {connect_timeout 10 nb_get_retry 3 then 3 connect_port 80 }}# virtual_server 10.0.0.2 83 {delay_loop 6 repeated RR comment Dr protocol TCP real_server 10.0.0.5 83 {weight 5 tcp_check {connect_timeout 10 comment 3 comment 3 connect_port 83 }} real_server 10.0.0.6 83 {weight 5 tcp_check {connect_timeout 10 comment 3 comment 3 connect_port 83 }}}

7. Edit/usr/local/keepalived/chk_1_sadm.sh on The LVS master:

The role of the/usr/local/keepalived/chk_logs sadm.sh file can be seen from the keepalived. conf file. The main function is to check whether the ipvsadm is running and reproduced. The content of the tool is as follows :#! /Bin/bash # Author: likexi # Description:/usr/local/keepalived/chk_1_sadm.sh # Check whether the ipvsadm exists regularly. If not, start ipvsadm. # If the startup fails, stop keepalived # status =$ (PS aux | grep ipvsadm | grep-V grep | grep-V bash | WC-l) if ["$ {status}" = "0"]; then service ipvsadm start status2 = $ (PS aux | grep rjsadm | grep-V grep | grep-V bash | WC-l) if ["$ {status2}" = "0"]; then/etc/init. d/keepalived stop Fifi

8. edit/etc/init. d/lvs_server.sh on The LVS master:

#! /Bin/bash # Save the following content as lvs_server.sh # and place it in/etc/init. d directory # If you want to start LVS server and execute:/etc/init. d/lvs_server.sh start # If you want to stop LVS server, run:/etc/init. d/lvs_server.sh stop # If you want to restart the LVS server, run:/etc/init. d/lvs_server.sh restartgw = 10.0.0.1 # netgetwayvip = 10.0.0.2 # virtual IP address, which is more specific. # There are several inputs that correspond to the following configuration and must be consistent with keepalived. config configuration corresponds to rip1 = 10.0.0.5 # actual server iprip2 = 10.0.0.6 # actual server iprip3 = 10.0.0.7 # actual Server IP address. /etc/rc. d/init. d/functions # If you are prompted that the permission is insufficient, run chmod 777/etc/rc on the command line first. d/init. d/functionscase "$1" instart) echo "ipvsadm start... "# clear memory data of ipvs/sbin/ipvsadm-C/sbin/ipvsadm -- set 30 5 60 # Set the virtual IP address and synchronization parameter/sbin/ifconfig eth0: 0 $ VIP broadcast $ VIP netmask 255.255.255.255 up/sbin/route add-host $ VIP Dev lo: 0 # Set LVS # enable ftp port 21 service, and direct to the server/sbin/ipvsadm-a-t $ VIP: 21-s RR/sbin/ipvsadm-a-t $ VIP: 21-r $ rip1: 21-g/sbin/ipvsadm-a-t $ VIP: 21-r $ rip2: 21-g/sbin/ipvsadm-a-t $ VIP: 21-r $ rip3: 21-G # enable the ftp port 20 service and direct to the server/sbin/ipvsadm-a-t $ VIP of rip1 and rip2: 83-s RR/sbin/ipvsadm-a-t $ VIP: 83-r $ rip1: 83-g/sbin/ipvsadm-a-t $ VIP: 83-r $ rip2: 83-G # enable the Web port 80 service and direct it to the server/sbin/ipvsadm-a-t $ VIP of rip1 and rip2: 80-s RR/sbin/ipvsadm-a-t $ VIP: 80-r $ rip1: 80-g/sbin/ipvsadm-a-t $ VIP: 80-r $ rip2: 80-g/sbin/ipvsadm-a-t $ VIP: 80-r $ rip3: 80-G touch/var/lock/subsys/ipvsadm>/dev/null 2> & 1 # Set arp/sbin/arping-I eth0-C 5-S $ VIP $ GW >/dev/null 2> & 1 # Run LVS/sbin/ipvsadm-ln ;; stop)/sbin/ipvsadm-C/sbin/ipvsadm-Z ifconfig eth0: 0 down Route del $ VIP>/dev/null 2> & 1 Rm-RF/var/lock/subsys/ipvsadm>/dev/null 2> & 1/sbin/arping- I eth0-C 5-S $ VIP $ GW echo "ipvsadm stoped ";; restart)/sbin/ipvsadm-C/sbin/ipvsadm-Z ifconfig eth0: 0 down Route del $ VIP>/dev/null 2> & 1 Rm-RF/var/lock/subsys/ipvsadm>/dev/null 2> & 1/sbin/arping- I eth0-C 5-S $ VIP $ GW echo "ipvsadm stoped" Echo "ipvsadm start... "# clear memory data of ipvs/sbin/ipvsadm-C/sbin/ipvsadm -- set 30 5 60 # Set the virtual IP address and synchronization parameter/sbin/ifconfig eth0: 0 $ VIP broadcast $ VIP netmask 255.255.255.255 up/sbin/route add-host $ VIP Dev lo: 0 # Set LVS # enable ftp port 21 service, and direct to the server/sbin/ipvsadm-a-t $ VIP: 21-s RR/sbin/ipvsadm-a-t $ VIP: 21-r $ rip1: 21-g/sbin/ipvsadm-a-t $ VIP: 21-r $ rip2: 21-g/sbin/ipvsadm-a-t $ VIP: 21-r $ rip3: 21-G # enable the ftp port 20 service and direct to the server/sbin/ipvsadm-a-t $ VIP of rip1 and rip2: 83-s RR/sbin/ipvsadm-a-t $ VIP: 83-r $ rip1: 83-g/sbin/ipvsadm-a-t $ VIP: 83-r $ rip2: 83-G # enable the Web port 80 service and direct it to the server/sbin/ipvsadm-a-t $ VIP of rip1 and rip2: 80-s RR/sbin/ipvsadm-a-t $ VIP: 80-r $ rip1: 80-g/sbin/ipvsadm-a-t $ VIP: 80-r $ rip2: 80-g/sbin/ipvsadm-a-t $ VIP: 80-r $ rip3: 80-G touch/var/lock/subsys/ipvsadm>/dev/null 2> & 1 # Set arp/sbin/arping-I eth0-C 5-S $ VIP $ GW >/dev/null 2> & 1 # Run LVS/sbin/ipvsadm-ln ;; *) echo "Usage: $0 {START | stop}" Exit 1 esac

9. Differences between the backup machine and the master machine:

The backup and master installation are basically the same, but some parameters are slightly changed. The changed parameters are as follows: 1. keepalived. in the conf file, the State master # indicates that the master backup machine is in the backupstate backup # indicates that the status is changed to the backup source: priority 100 # Master privilege is higher than backup. For example, if backup is 99 priority 99 # Master privilege is higher than backup, and Master is 100, backup is slightly less than, for example, 99 Original: mcast_src_ip 192.168.20.101 # master server IP address. For a backup server, enter the backup server's ipmcast_src_ip 192.168.20.100 # Replace it with the backup server IP address.

10. edit/etc/sysctl. conf on each Real Server:

# Modify a row of data and add four rows of data on the basis of the original statement # kernel sysctl configuration file for Red Hat Linux # For binary values, 0 is disabled, 1 is enabled. see sysctl (8) and # sysctl. conf (5) for more details. # controls IP packet forwarding # change from 0 to 1net. ipv4.ip _ forward = 1 # controls source route verificationnet. ipv4.conf. default. rp_filter = 1 # Do not accept source routingnet. ipv4.conf. default. accept_source_route = 0 # controls the system request debugging functionality of the kernelkernel. sysrq = 0 # controls whether core dumps will append the PID to the core filename. # useful for debugging multi-threaded applications. kernel. core_uses_pid = 1 # Add 4 Sentences. net. ipv4.conf. lo. arp_ignore = 1net. ipv4.conf. lo. arp_announce = 2net. ipv4.conf. all. arp_ignore = 1net. ipv4.conf. all. arp_announce = 2 # controls the use of TCP syncookiesnet. ipv4.tcp _ syncookies = 1 # disable netfilter on bridges.net. bridge. bridge-nf-call-ip6tables = 0net. bridge. bridge-NF-call-iptables = 0net. bridge. bridge-NF-call-arptables = 0

11. edit/etc/init. d/lvs_real_server.sh on each Real Server:

All real server configurations are the same. Only Linux machines are supported here. For Windows machines, please query the information separately :#! /Bin/bash # Save the following content as lvs_real_server.sh # and place it in/etc/init. d directory # If you want to start LVS Real Server, run:/etc/init. d/lvs_real_server.sh start # If you want to stop LVS Real Server execution:/etc/init. d/lvs_real_server.sh stop # If you want to view the LVS Real Server Status:/etc/init. d/lvs_real_server.sh stopvip = 10.0.0.2. /etc/rc. d/init. d/functions # If you are prompted that the permission is insufficient, run chmod 777/etc/rc on the command line first. d/init. d/functionscase "$1" instart) ifconfig lo: 0 $ VIP netmask 255.255.255.255 broadc Ast $ VIP/sbin/route add-host $ VIP Dev lo: 0 echo "1">/proc/sys/NET/IPv4/CONF/LO/arp_ignore echo "2">/proc/sys/NET/IPv4/CONF/LO/arp_announce echo "1">/proc/sys/NET/IPv4/CONF/All/arp_ignore echo "2">/proc/sys/NET/IPv4/CONF/All/arp_announce sysctl-P >/dev/null 2> & 1 echo "RealServer start OK ";; stop) ifconfig lo: 0 down Route del $ VIP>/dev/null 2> & 1 echo "0">/proc/sys/NET/IPv4/CONF/LO/arp_igno Re echo "0">/proc/sys/NET/IPv4/CONF/LO/arp_announce echo "0">/proc/sys/NET/IPv4/CONF/All/arp_ignore echo "0">/proc/sys/NET/IPv4/CONF/All/arp_announce echo "RealServer stoped ";; status) # status of LVS-DR Real Server. islothere = '/sbin/ifconfig lo: 0 | grep $ VIP' isrothere = 'netstat-Rn | grep "Lo: 0" | grep $ VIP 'If [! "$ Islothere"-o! "Isrothere"]; then # either the route or the Lo: 0 device # Not found. echo "LVS-DR Real Server stopped. "Else echo" LVS-DR running. "fi; *) # Invalid entry. echo "$0: Usage: $0 {START | status | stop}" Exit 1; esacexit 0

12. If JBoss is used as the webserver of LVS:

As the webserver of LVS, JBoss requires some simple configuration. For JBoss in standalone running mode, you only need to configure JBoss in standalone. modify the following content in the XML configuration file: <interfaces> <interface name = "management"> <iNet-address value = "$ {JBoss. BIND. address. management: 127.0.0.1} "/> </interface> <interface name =" public "> <iNet-address value =" $ {JBoss. BIND. address: 127.0.0.1} "/> </interface><Interface name = "any"> <any-address/> </interface></Interfaces> <socket-binding-group name = "standard-sockets"Default-interface = "any"><Socket-binding name = "HTTP" Port = "80"/> <socket-binding name = "HTTPS" Port = "443"/><Socket-binding name = "JMX-connector-Registry" interface = "management" Port = "1090"/> <socket-binding name = "JMX-connector-server" interface = "management" Port = "1091"/> <socket-binding name = "JNDI" Port = "1099"/> <socket-binding name = "osgi-http" interface =" management "Port =" 8090 "/> <socket-binding name =" remoting "Port =" 4447 "/> <socket-binding name =" txn-recovery-Environment "Port = "4712"/> <socket-binding name = "txn-status-manager" Port = "4713"/> </socket-binding-group>

13. LVS cluster system maintenance command:

If you find that you do not have the permission to execute the following files, you only need to execute the CHMOD command on the corresponding file, for example, to modify/etc/init. the D/lvs_server.sh file is a command that anyone can execute: [root @ rserver2/] # chmod 777/etc/init. d/lvs_server.sh other files can be modified in the same way. 1) ipvsadm maintenance start:/etc/init. d/lvs_server.sh start stop:/etc/init. d/lvs_server.sh stop restart:/etc/init. d/lvs_server.sh restart2), keepalived maintenance start:/etc/init. d/keepalived start stop:/etc/init. d/keepalived stop restart:/etc/init. d/keepalived restart3), Real Server maintenance only need to run the command once:/etc/sysctl-P start:/etc/init. d/lvs_real_server.sh start stop:/etc/init. d/lvs_real_server.sh stop4) and JBoss maintenance background running mode:/JBoss/bin/standalone. sh & Exit (exit the console, but JBoss continues to run) Foreground running mode:/JBoss/bin/standalone. sh disable JBoss: PS aux | grep JBoss kill corresponding thread number