How does LVS/nginx handle session problems?

Business System Architecture:

Extension 1: nginx (master) + keepalived + nginx (Backup) + 3 web clusters + MySQL (master-slave) + EMC clariion cx4 Storage
Extension 2: LVS (master) + keepalived + LVS (Backup) + 3 web clusters + MySQL (master-slave) + EMC clariion cx4 Storage

The operating system uses 64-bit rhel5.4/centos5.4, the server uses hp360g6 + hp580g5, the front-end firewall of the Business System for wasai usg5000 + WAF-T3-500 (Anti-DDoS, phishing and injection attacks, etc)

If nginx Load balancer is used, ip_hash is used to replace the default RR method. That is, requests from a client IP address can be hashedAlgorithmLocate on the same backend Web server to avoid session loss and solve the session problem. However, the ip_hash command cannot ensure the load balancing of backend servers. Some backend servers may receive many requests, but some backend servers may receive fewer requests. In this way, the significance of load balancing is lost. Our solution is to write the user's login session information into the backend MySQL database, which is also achieved in the subsequent CMS system, and the effect is also good. Later, I proposed a compromise, if the number of concurrent nginx connections (that is, the nginx Server Load balancer's nginxstatus's active connections) is greater than 2000, the session is written into the MySQL database. If the number of concurrent connections is small, ip_hash works well.

In addition, if the ip_hash parameter is added to upstream, the test shows that a backend server does not automatically jump after it fails, the following method is recommended:

1. upstream njzq.com {
2. ip_hash;
3. server 172.16.94.216: 9000 max_fails = 0 ;
4. server 172.16.94.217: 9000 max_fails = 0 ;
5. server 172.16.94.218: 9000 max_fails = 0 ;
6. }

The ipvsadm-P scheme adopted by LVS in the second-to-second extension. The session persistence time is measured in seconds. I usually set it to 120 s. This option is very useful for dynamic websites: This session persistence function is available when users log on to the website from a remote account, you can forward user requests to the same application server. When a user accesses the server for the first time, his access request is forwarded to a real server by the Server Load balancer, so that he can see a login page, the first access is complete; then he enters the user name and password in the login box, then submit. At this time, the problem may occur-login fails. Because there is no session persistence, the Server Load balancer may forward 2nd requests to other servers. So is the connection always recommended between the client and the server after the configuration? Is the connection to another real physical server after 120 seconds? I tried the following experiment: LVS uses a single server, 192.168.1.102, VIP is 192.168.1.188, And the backend is two Web servers, 192.168.1.103 and 192.168.1.104.

Run the following script on LVS. Run the script on two real servers and bind the VIP address 192.168.1.188. Use the lvs_dr.sh and real. Sh scripts on the LVS and physical servers respectively.

2. [Root @ ltos LVS] # Cat lvs_dr.sh
4. #! /Bin/bash
8. # Website Director VIP.
10. Sns_vip=192. 168.1.188
12. Sns_rip1=192. 168.1.103
14. Sns_rip2=192. 168.1.104
16.  
18. ./Etc/rc. d/init. d/functions
20.  
22. Logger $0 called with $1
24.  
26. Case "$1" in
28.  
30. Start)
32. # Set squid VIP
34. /Sbin/ipvsadm -- set 30 5 60
36. /Sbin/ifconfig eth0: 0 $ sns_vip broadcast $ sns_vip netmask 255.255.255.255 broadcast $ sns_vip up
38. /Sbin/route add-host $ sns_vip Dev eth0: 0
40. /Sbin/ipvsadm-a-t $ sns_vip: 80-s wlc-P 120
42. /Sbin/ipvsadm-a-t $ sns_vip: 80-r $ sns_rip1: 80-g-W 1
44. /Sbin/ipvsadm-a-t $ sns_vip: 80-r $ sns_rip2: 80-g-W 1
46. Touch/var/lock/subsys/ipvsadm>/Dev/null 2>& 1
48.  
50. ;;
52. Stop)
54. /Sbin/ipvsadm-C
56. /Sbin/ipvsadm-z
58. Ifconfig eth0: 0 down
60. Route del $ sns_vip
62. Rm-RF/var/lock/subsys/ipvsadm>/Dev/null 2>& 1
64. Echo "ipvsadm stoped"
66. ;;
68.  
70. Status)
72.  
74. If [! -E/var/lock/subsys/ipvsadm]; then
76. Echo "ipvsadm stoped"
78. Exit 1
80. Else
82. Echo "ipvsadm OK"
84. Fi
86. ;;
88.  
90. *)
92. Echo "Usage: $0 {START | stop | status }"
94. Exit 1
96. Esac
98. Exit 0

Two real Web physical servers run the real. Sh script

2. #! /Bin/bash
4. Sns_vip=192. 168.1.188
6. ./Etc/rc. d/init. d/functions
8.  
10. Case "$1" in
12. Start)
14. Ifconfig lo: 0 $ sns_vip netmask 255.255.255.255 broadcast $ sns_vip
16. /Sbin/route add-host $ sns_vip Dev lo: 0
18. Echo "1">/Proc/sys/NET/IPv4/CONF/LO/arp_ignore
20. Echo "2">/Proc/sys/NET/IPv4/CONF/LO/arp_announce
22. Echo "1">/Proc/sys/NET/IPv4/CONF/All/arp_ignore
24. Echo "2">/Proc/sys/NET/IPv4/CONF/All/arp_announce
26. Sysctl-P>/Dev/null 2>& 1
28. Echo "RealServer start OK"
30. ;;
32. Stop)
34. Ifconfig lo: 0 down
36. Route del $ sns_vip>/Dev/null 2>& 1
38. Echo "0">/Proc/sys/NET/IPv4/CONF/LO/arp_ignore
40. Echo "0">/Proc/sys/NET/IPv4/CONF/LO/arp_announce
42. Echo "0">/Proc/sys/NET/IPv4/CONF/All/arp_ignore
44. Echo "0">/Proc/sys/NET/IPv4/CONF/All/arp_announce
46. Echo "RealServer stoped"
48. ;;
50. *)
52. Echo "Usage: $0 {START | stop }"
54. Exit 1
56. Esac
58. Exit 0

It is observed that when the client 192.168.1.100 initiates the first connection request, The LVS Server Load balancer assigns it to the following physical server 192.168.1.104. After three handshakes are completed, the connection status is established, after the TCP connection is terminated for a considerable period of time, the actual Web server status is fin_wait, and the new connection initiated at 192.168.1.100 will always be connected to 192.168.1.104.

Note: a dynamic website refers to a PHP login. If the backend is a cache cluster, this session option can be removed. However, f5, currently, there is no chance to test the function.

During project implementation, I used to share with my colleagues the habit of dividing the entire system architecture into three layers: Server Load balancer layer, web layer, and database layer. I found that everyone liked to talk about the concept of cluster, I am confused about this concept. Although I know they are referring to LVS, I prefer to use the term Server Load balancer. The Server Load balancer is the nginx/lvs I mentioned above, they can distribute client requests to backend server clusters based on different algorithms, such as Apache, tomcat, and squid clusters. High Availability is to use the frontend Server Load balancer as a failover, that is, replace the standby machine with the faulty machine in a short time (<1 s, currently, mature load models are available in the LVS + keepalived and nginx + keepalived architectures (heartbeat I am mainly used in the intranet development environment and has not yet invested in the production environment, I suggest say Linux cluster, so you will know it is LVS environment, if the above statement or configuration is incorrect, please notify 51cto editor or the author of fuqin cooking wine yuhongchun027@163.com, we will Make corrections immediately to avoid misleading readers.

[51cto.com exclusive special article. This article will not be reprinted without authorization. Please indicate the author and source of the original article for the reprinted by the partner media !]