LVS (DR) + keepalived on CentOS6 installation configuration manual

1, operating system version and LVS and other software versions:

CentOS6 (Linux Kernel 2.6.32-71.el6.i686)

ipvsadm 1.2.4 keepalived 1.1.17

2, the required IP configuration:

LVS Master ip:10.0.0.5

lvs Backup ip:10.0.0.6

realserver ip:10.0.0.7, 10.0.0.5 (concurrently), 10.0.0.6 (and)

Netgetway ip:10.0.0.1

Virtual IP (VIP): 10.0.0.2

3, Attention matters:

CENTOS6 Installation Select Development Workstation mode

firewall is closed (because the business machine front-end has a dedicated firewall, if not, then master can also)

4, the network extension complement diagram:

5, Ipvsadm and keepalived installation:

[Root@rserver2/] #mkdir/soft [root@rserver2/] #cd/soft [root@rserver2 soft]# wget oftware/kernel-2.6/ipvsadm-1.24.tar.gz [Root@rserver2 soft]# wget http://www.keepalived.org/software/ keepalived-1.1.17.tar.gz [Root@rserver2 Soft] #ln-S/usr/src/kernels/2.6.32-71.el6.i686//usr/src/linux [ Root@rserver2 soft]# tar-zxvf ipvsadm-1.24.tar.gz [root@rserver2 soft]# cd ipvsadm-1.24 [Root@rserver2 ipvsadm-1.24]# m

Ake;make Install [Root@rserver2 ipvsadm-1.24]# CD. [Root@rserver2 soft]# tar-zxvf keepalived-1.1.17.tar.gz [root@rserver2 soft]# cd keepalived-1.1.17 [Root@rserver2 soft] #./configure [root@rserver2 keepalived-1.1.17]# make;make Install smooth words on these steps, if there are errors, then according to the specific error specific treatment, the general may occur errors: 1, OpenSSL, the hint may be as follows!!! OpenSSL is isn't properly installed on your system.

!!! !!!

Can not include OpenSSL headers files. Solution: Run yum-y installopenssl-devel 2), Tip no gcc compiler solution: Run yum installncurses-devel gcc gcc-c++ make rpm-build [Root@rserv Er2 Keepalived-1.1.17]# cp/usr/local/etc/rc.d/init.d/keepalived/etc/init.d/keepalived [Root@rserver2 keepalived-1.1.17]# CP/ usr/local/sbin/keepalived/usr/sbin/[Root@rserver2 keepalived-1.1.17]# cp/usr/local/etc/sysconfig/keepalived/etc/ sysconfig/[Root@rserver2 keepalived-1.1.17]# mkdir-p/etc/keepalived/[Root@rserver2 keepalived-1.1.17]# Cp/usr/loca l/etc/keepalived/keepalived.conf/etc/keepalived/keepalived.conf [Root@rserver2 keepalived-1.1.17]# chmod +x/etc/
 Init.d/keepalived

6. Edit the/etc/keepalived/keepalived.conf on the LVS Master machine:

Open the editor, such as VI or gedit/etc/keepalived/keepalived.conf content as follows, you modify the content of the program. #Master服务器上的配置/etc/keepalived/keepalived.conf global_defs {   Notification_email {        leekexi@gmail.com #可以多个地址   }    notification_email_from leekexi@gmail.com & nbsp;  smtp_server smtp.gmail.com    smtp_connect_timeout    router_id Lvs_devel} #监测ipvsad M process status, execute once every 3 seconds vrrp_script chk_ipvsadm{    script "/usr/local/keepalived/chk_ipvsadm.sh"      Interval 3     weight 3} vrrp_instance vi_1 {    state MASTER #标示状态为MASTER backup machine for back Up     interface eth0     virtual_router_id Wuyi     Priority 100    #MASTER权重要高于BACKUP such as backup for     advert_int 1     #mcast_src_ip 10.0.0.5 master server IP, if the backup machine please fill in the Backup machine IP     authentication {        Auth_typE Pass #主从服务器验证方式         auth_pass 1111    }     virtual_ipaddress {        10.0.0.2 #可以多个虚拟IP, line wrap    } #虚拟服务器 21 Port configuration virtual_server 10.0.0.2 {    delay_loop 6             # (Query realserver status every 10 seconds)     Lb_algo rr               # (LVS algorithm)     lb_kind dr               # (Direct Route)     persistence_timeout 60  # ( The same IP connection is assigned to the same realserver in 60 seconds     Protocol tcp             # (check realserver status with TCP protocol)     #实际服务器的IP和端口     real_server 10.0.0.5 21 {        weight  5         Tcp_check {            connect_timeout 10         nb_get_retry 3           
  Delay_before_retry 3             connect_port 21        }    }     #实际服务器的IP和端口     Real_server 10.0.0.6 {        weight  5      
   Tcp_check {            connect_timeout 10         nb_get_retry 3           
  Delay_before_retry 3             connect_port 21        }    }     } #虚拟服务Configuration of Port 80 virtual_server 10.0.0.2 {    delay_loop 6     Lb_algo rr     LB _kind DR     Protocol TCP     Real_server 10.0.0.5      &nbs p;  weight  5         tcp_check {             connect_timeout         nb_get_retry 3              Delay_before_retry 3       

      connect_port        }    }     real_server 10.0.0.6 {        weight  5          Tcp_check {            Connect_timeout         nb_get_retry 3             Delay_before_retry 3              Connect_port        }     } #虚拟服务器 83 Port configuration virtual_server 10.0.0.2 {    delay_loop 6     Lb_algo RR     lb_kind DR     protocol TCP     Real_server 10.0.0.5  &nbsp ;      weight  5         tcp_check {             connect_timeout         nb_ Get_retry 3             delay_before_retry 3              Connect_port         }    }     real_server 10.0.0.6         weight  5         tcp_check {             connect_timeout         nb_ Get_retry 3             delay_before_retry 3              Connect_port        

 }    }}

7. Edit the/usr/local/keepalived/chk_ipvsadm.sh on the LVS Master machine:

/usr/local/keepalived/chk_ipvsadm.sh the role of the document can be seen from the keepalived.conf, the main role is to confirm Ipvsadm is in operation reproduced, the contents are as follows:
#!/bin/ Bash
# #
Author:likexi #
Description:/usr/local/keepalived/chk_ipvsadm.sh
# timed to see if ipvsadm exists, Start Ipvsadm If it does not exist,
# If startup fails, stop keepalived
#
status=$ (ps aux|grep ipvsadm | grep-v grep | grep-v Bash | wc-l) c8/>if ["${status}" = "0"]; Then
        service ipvsadm start
        status2=$ (ps aux|grep ipvsadm | grep-v grep | grep-v bash |wc-l)
        if [${statu S2} "=" 0 "  ]; Then
                /etc/init.d/keepalived Stop
        fi
fi

8. Edit the/etc/init.d/lvs_server.sh on the LVS Master machine:

#!/bin/bash #把一下内容保存成: lvs_server.sh #并放置在/etc/init.d directory #如果想启动LVS Server execution:/etc/init.d/lvs_server.sh start #如果想停止LVS
Server execution:/etc/init.d/lvs_server.sh stop #如果想重启LVS Server execution:/etc/init.d/lvs_server.sh Restart gw=10.0.0.1 # Netgetway vip=10.0.0.2 #虚拟IP, more specific circumstances #有几个输入几个, corresponding to the following configuration, and must correspond to Keepalived.config configuration rip1=10.0.0.5 #实际的服务器IP rip2=10.0.0.6 # The actual server IP rip3=10.0.0.7 #实际的服务器IP. /etc/rc.d/init.d/functions # If you are prompted for insufficient permissions, then execute on the command line: chmod 777/etc/rc.d/init.d/functions case "in Start" Echo IPVSA DM start ... "#清空 Ipvs memory Data/sbin/ipvsadm-c/sbin/ipvsadm--set 5 #设置虚拟IP和同步参数/sbin/ifconf  IG eth0:0 $VIP broadcast $VIP netmask 255.255.255.255 up/sbin/route add-host $VIP Dev lo:0 #设置LVS #开启FTP 21 Port services and points to RIP1 and RIP2 servers/sbin/ipvsadm-a-T $VIP: 21-s rr/sbin/ipvsadm-a-T $VIP: 21-r $RIP 1:21-g/sbin/ipvs Adm-a-T $VIP: 21-r $RIP 2:21-g/sbin/ipvsadm-a-t $VIP: 21-r $RIP 3:21-g #开启FTP 20 port service and point to RIP1 and RIP2 server/s Bin/ipvsaDm-a-T $VIP: 83-s rr/sbin/ipvsadm-a-T $VIP: 83-r $RIP 1:83-g/sbin/ipvsadm-a-t $VIP: 83-r $RIP 2:83-g # Open the Web 80 port service and point to RIP1 and RIP2 server/sbin/ipvsadm-a-T $VIP: 80-s rr/sbin/ipvsadm-a-T $VIP: 80-r $RIP 1:80-g/s Bin/ipvsadm-a-T $VIP: 80-r $RIP 2:80-g/sbin/ipvsadm-a-t $VIP: 80-r $RIP 3:80-g Touch/var/lock/subsys/ipvsad
    M >/dev/null 2>&1 # set arp/sbin/arping-i eth0-c 5-s $VIP $GW >/dev/null 2>&1
	
#运行LVS/sbin/ipvsadm-ln;;
    Stop)/sbin/ipvsadm-c/sbin/ipvsadm-z ifconfig eth0:0 down Route del $VIP >/dev/null 2>&1 Rm-rf/var/lock/subsys/ipvsadm >/dev/null 2>&1/sbin/arping-i eth0-c 5-s $VIP $GW echo "Ipvsadm sto

Ped ";;
    Restart)/sbin/ipvsadm-c/sbin/ipvsadm-z ifconfig eth0:0 down Route del $VIP >/dev/null 2>&1 Rm-rf/var/lock/subsys/ipvsadm >/dev/null 2>&1/sbin/arping-i eth0-c 5-s$VIP $GW echo "Ipvsadm stoped" echo "ipvsadm start ..." #清空 Ipvs memory Data/sbin/ipvsadm-c M--set 5 #设置虚拟IP和同步参数/sbin/ifconfig eth0:0 $VIP broadcast $VIP netmask 255.255.255.255 up/sbin/rout e add-host $VIP Dev lo:0 #设置LVS #开启FTP 21 Port service and points to RIP1 and RIP2 servers/sbin/ipvsadm-a-T $VIP: 21-s rr/sbin/i Pvsadm-a-T $VIP: 21-r $RIP 1:21-g/sbin/ipvsadm-a-t $VIP: 21-r $RIP 2:21-g/sbin/ipvsadm-a-t $VIP: 21-r $RIP 3:21-g #开启FTP 20 Port service and points to RIP1 and RIP2 servers/sbin/ipvsadm-a-T $VIP: 83-s rr/sbin/ipvsadm-a-T $VIP: 83-r $RIP 1 : 83-g/sbin/ipvsadm-a-T $VIP: 83-r $RIP 2:83-g #开启WEB 80 Port service and point to RIP1 and RIP2 servers/sbin/ipvsadm-a-T $VIP: 80 -S rr/sbin/ipvsadm-a-t $VIP: 80-r $RIP 1:80-g/sbin/ipvsadm-a-t $VIP: 80-r $RIP 2:80-g/sbin/ipvsadm-a- T $VIP: 80-r $RIP 3:80-g touch/var/lock/subsys/ipvsadm >/dev/null 2>&1 # set Arp/sbin/arp Ing-i eth0-c 5-s $VIP $GW >/dev/null 2>&1 #运行LVS/sbin/ipvsadm-ln;;
 *) echo "Usage: $ {Start|stop}" Exit 1 Esac

9. The difference between the backup machine and master machine:

The installation of backup is basically consistent with Master's installation, except that some parameters change slightly.

the parameters of the change are as follows:

1, keepalived.conf file

Original: State MASTER #标示状态为MASTER backup machine as backup

State backup  # Change the status to backup

 

: Priority   #MASTER权重要高于BACKUP such as backup for

priority   #MASTER权重要高于BACKUP, Master is 100, then backup is slightly smaller, such as

 

: Mcast_src_ip 192.168.20.101 #Master服务器IP, if the backup machine, please fill in the Backup machine IP

mcast_src_ip 192.168.20.100  #换成BACKUP服务器IP

10. Edit the/etc/sysctl.conf on each real server machine:

#在原来的基础上修改一行数据和添加四行数据 # Kernel sysctl configuration file for Red Hat Linux # for binary values, 0 are disabled, 1 is Enab

led.  the Sysctl (8) and # sysctl.conf (5) for more details. # Controls IP packet forwarding #从0 changed to 1 net.ipv4.ip_forward = 1 # Controls source Route Verification-Net.ipv4.conf.defau Lt.rp_filter = 1 # does not accept source routing Net.ipv4.conf.default.accept_source_route = 0 # Controls the System requ EST debugging functionality of the kernel KERNEL.SYSRQ = 0 # Controls Whether core dumps'll append the PID to the core
FileName.
# Useful for debugging multi-threaded applications. Kernel.core_uses_pid = 1 #添加的4句 net.ipv4.conf.lo.arp_ignore = 1 net.ipv4.conf.lo.arp_announce = 2 Net.ipv4.conf.all.arp_  Ignore = 1 net.ipv4.conf.all.arp_announce = 2 # Controls The use of TCP syncookies net.ipv4.tcp_syncookies = 1 # Disable
NetFilter on bridges. Net.bridge.bridge-nf-call-ip6tables = 0 Net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-arptables = 0

11. Edit the/etc/init.d/lvs_real_server.sh on each real server machine:

As with all real server configurations, only the Linux version of the system machine, for the Windows version of the machine, please inquire further information: #!/bin/bash #把一下内容保存成: lvs_real_server.sh #并放置在/etc/ INIT.D directory #如果想启动LVS Real server execution:/etc/init.d/lvs_real_server.sh start #如果想停止LVS real Server execution:/etc/init.d/lvs_real_ server.sh Stop #如果想查看LVS Real server state:/etc/init.d/lvs_real_server.sh stop vip=10.0.0.2. /etc/rc.d/init.d/functions # If the prompt is not sufficient, execute it first on the command line: chmod 777/etc/rc.d/init.d/functions case "on Start" ifconfi G lo:0 $VIP netmask 255.255.255.255 broadcast $VIP/sbin/route add-host $VIP Dev lo:0 echo "1" >/proc/sy S/net/ipv4/conf/lo/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "1" >/proc/sys/ne T/ipv4/conf/all/arp_ignore echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce sysctl-p >/dev/null 2&G
	   
t;&1 echo "Realserver Start OK";; Stop) Ifconfig lo:0 down Route del $VIP >/dev/null 2>&1 echo "0" >/proc/sys/net/ipv4/con F/lo/arp_ignorE echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
	   
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce echo "Realserver stoped";
        Status) #Status of LVS-DR Real server. Islothere= '/sbin/ifconfig lo:0 | grep $VIP ' isrothere= ' netstat-rn | grep "lo:0" | grep $VIP ' if [! "$islothere"-O!
            "Isrothere"];then # either the route or the lo:0 device # not found.
        echo "LVS-DR Real server Stopped."
        else echo "Lvs-dr Running."

fi;;
        *) #Invalid entry.
echo "$0:usage: $ {Start|status|stop}" exit 1;;
 ESAC Exit 0

12, if the use of JBoss as LVs webserver:

JBoss as the webserver of LVs need to do some simple configuration for standalone run mode JBoss, simply modify the following in Standalone.xml configuration file: <interfaces> <

        Interface name= "Management" > <inet-address value= "${jboss.bind.address.management:127.0.0.1}"/> </interface> <interface name= "public" > <inet-address value= "${jboss.bind.address: 127.0.0.1} "/> </interface><interface name= "any" > <any-address/> </interface></interfaces> <socket-binding-group name= "Standard-sockets"default-interface= "any"><socket-binding name= "http" port= "/> <socket-binding name=" https "port=" 443 "/><socket-binding name= "Jmx-connector-registry" interface= "Management" port= "1090"/> <socket-binding

        "Jmx-connector-server" interface= "Management" port= "1091"/> <socket-binding name= "Jndi" port= "1099"/> <socket-binding name= "Osgi-http" interface= "Management" port= "8090"/> <socket-binding "name=" Remot ing "port=" 4447 "/> <socket-binding name=" txn-recovery-environment "port=" 4712 "/> <socket-bin Ding name= "Txn-status-manager" port= "4713"/> </socket-binding-group>

13. LVS Cluster System maintenance command:

If you find that you do not have permission to execute the following files, simply execute them on the corresponding file: chmod commands
such as: To modify the/etc/init.d/lvs_server.sh file for anyone to execute the command that executes:
[Root@rserver2/]# chmod 777/etc/init.d/lvs_server.sh
Other files in the same way as modified.

1), Ipvsadm maintenance

start:/etc/init.d/lvs_server.sh start

stop:/etc/init.d/lvs_server.sh stop restart

:/etc/ init.d/lvs_server.sh Restart

2), keepalived maintenance

boot:/etc/init.d/keepalived start

stop:/etc/init.d/ keepalived Stop restart

:/etc/init.d/keepalived restart

3), Real server maintenance

only one command to run once:/etc/sysctl-p

Start:/etc/init.d/lvs_real_server.sh start

stop:/etc/init.d/lvs_real_server.sh stop

4), JBoss maintenance

background operation mode: /jboss/bin/standalone.sh & 

Exit (Exit console, but JBoss continues to run)

foreground operation mode:/jboss/bin/standalone.sh

shutdown Jbos S:ps aux |

the thread number corresponding to grep JBoss kill