LVS Persistence working principle and configuration

Author: isadba| FH. CN Date: ./ on/ -reprint Please specify: isadba.com|FH. Cnblog:http://isadba.comOriginal: http://isadba.com/?p=67LVS Persistent Connection technology The persistent connection of LVS has two aspects: record the request information of the same client to the hash table of LVS, save time using Persistence_timeout control, the unit is seconds.
The persistence_granularity parameter is in conjunction with Persistence_timeout, which is especially useful in some cases where his value is the subnet mask, which represents the granularity of the persistent connection, which is 255 by default.255.255.255, which is a separate client IP
, if you change it,255.255.255. 0 is the client IP a network segment will be assigned to the same real server. Time-out after a connection is created idle, this time is 3 TCP idle timeout time LVS receives the client TCP fin timeout time how to view these values for UDP timeout?
Ipvsadm # To see the time-out period for connection idle (persistentTen). IP Virtual Server Version1.2.1(size=4096) Prot localaddress:port Scheduler Flags-remoteaddress:port Forward Weight activeconn inactconntcp122.225.32.142: HTTP RR PersistentTen-122.225.32.137: http Route1      0          1-122.225.32.136: http Route1      0          0Ipvsadm-L-Timeout # View TCP tcpfin UDP time-out timeout (TCP tcpfin UDP): the  -  -How did they work? Ipvsadm-LCN IPVS Connection Entries Pro expire State sourceVirtualDestination TCPxx: $NONE110.184.96.173:0 122.225.32.142: the 122.225.32.136: theTcp on: $Fin_wait110.184.96.173:54568 122.225.32.142: the 122.225.32.136:80

when a client accesses a VIP, Ipvs or records a message with a status of none, the expire initial value is the value of persistence_timeout, and then is smaller according to the clock primary key, which is connected to the same client IP during the existence of the following record. will be assigned to the same back end. The value of fin_wait is TCP Tcpfin UDP timeout time, when the value of none is 0 o'clock, if fin_wait still exists, then the value of none will be changed from new to 60 seconds, and then reduced, until the fin_wait disappears, none will disappear, as long as none exists, Access to the same client is assigned to unified Real server. How do I set these values? Persistence_timeout can be set in the following two ways Ipvsadm-a-t192.168.20.154: the-S Rr-p -Note: The red labeled 80 port in the above command, which represents the same client Access server's 80 port, will be defined to the same real server, if the 80 port is changed to 0, then any service of the same client Access server will be forwarded to the same real server. Modify the Keepalived configuration file and add persistence_timeout below the virtual server configuration -configuration of TCP Tcpfin UDP ipvsadm–SetTCP Tcpfin UDP # ipvsadm–Set  -  -  - //ExampleRecommendation: The value of tcpfin is preferably less than the value of persistence_timeout, which makes it easier to calculate.



LVS Persistence Reference: http://www.linuxvirtualserver.org/docs/persistence.htmlPersistent connection for LVS this article is from the "Grow Full record" blog, so be sure to keep this source http://lymrg.blog.51cto.com/1551327/684681since HTTP is a stateless protocol that disconnects immediately after each request, when a user browses to a shopping site to pick a product, a product is added to the shopping cart, and the process is redirected to REALSERVER1. When the second item is added to the shopping cart and redirected to the REALSERVER2, the final checkout time on the REALSERVER2 above, there is only one item, which is obviously unacceptable to the user, at this time need a persistent connection mechanism, To redirect the HTTP request of the same user to the same realserver within the timeout period, the timeout time can be defined by itself, say 2 hours, the server will keep track of the user's access request during the timeout period, and forward all requests of a user to the same realserver. If the user is still accessing after the timeout period, the default is unlimited for two minutes each time all operations here take the DR model as an example, the test environment see: http://lymrg.blog.51cto.com/1551327/660925for the persistent connection of LVS, there are three kinds of PCC,PPC and persistent connection based on firewall tag, we will discuss the PCCPCC used to direct all the access of a user to the same realserver within the timeout period. This method is not used in practice Ipvsadm-a-t192.168.0.1:0-S Wlc-p -(Unit is s) ipvsadm-a-T192.168.0.1:0-R192.168.1.2-W4-G ipvsadm-a-T192.168.0.1:0-R192.168.1.3-W2-g at this point test will find that access to the VIP via HTTP and SSH login VIP is directed to the same realserver above the PPCPPC used to send a user access to the same service within the timeout period to the same realserveripvsadm -a-t192.168.0.1: the-S Wlc-p -Ipvsadm-a-T192.168.0.1: the-R192.168.1.2-W4-G ipvsadm-a-T192.168.0.1: the-R192.168.1.3-W2-G ipvsadm-a-T192.168.0.1: A-S Wlc-p -Ipvsadm-a-T192.168.0.1: A-R192.168.1.2-G ipvsadm-a-T192.168.0.1: A-R192.168.1.3-g at this point the re-test will find that a user's access to a service during the timeout period is redirected to the same realserver. Firewall tag Persistent connection based on firewall tag for e-commerce sites, users use 80 ports to navigate through the selection of goods, When the payment is through 443 SSL encryption, of course, when the user selects the product payment, we certainly do not want the HTTPS 443 jump to another realserver, it is clear that the same realserver is right, This is the use of a persistent connection based on the firewall tag, through the definition of the relationship between the port to achieve the first two realserver to configure SSL, it is clear that the certificate should be the same as the definition of the port affinity relationship, Make the same firewall tag configuration for 80 and 443 ports implement firewall-based tag Lvsyum install MOD_SSL Two servers are installed on REALSERVER1 above. That's 192.168.1. 2 The following Operation CD/etc/pki/tls/certs/make Httpd.pem #此种SSL证书的生成方式只为测试, build SSL in practice see the full steps in front of my blog to fill in the relevant information, host name must be and VIP on the Internet to resolve the DNS name consistent CP Httpd.pem/etc/httpd/vi/etc/httpd/conf.d/ssl.conf # Find the following line to enable and modify DocumentRoot"/var/www/html"ServerName www.test.org:443#这里修改为你得服务器名称 # Modifying the certificate path SSLCERTIFICATEFILE/ETC/HTTPD/HTTPD.PEM Sslcertificatekeyfile/etc/httpd/httpd.pem SCP Httpd.pem192.168.1.3:/etc/httpd scp/etc/httpd/conf.d/ssl.conf192.168.1.3:/etc/httpd/conf.d/Firewall Add markiptables-T Mangle-a prerouting-d192.168.0.1-P TCP--dport the-j MARK--Set-markTen(0-99 range) Iptables-t Mangel-a prerouting-d192.168.0.1-P TCP--dport443-j MARK--Set-markTenIpvsadm-a-FTen-S Wlc-p -Ipvsadm-a-FTen-R192.168.1.2-g-w4Ipvsadm-a-FTen-R192.168.1.3-g-w2This article is from the "Grow Full record" blog, so be sure to keep this source http://lymrg.blog.51cto.com/1551327/684681about Arp_announce and arp_ignorehttp://lymrg.blog.51cto.com/1551327/660925VS/dr or vs/Tun Application of a model (all machines are in the same physical network), all machines (both director and Realserver) use an additional IP address, the VIP. When a client makes a connection request to the VIP, the request must be connected to the director's VIP, not realserver. Because the main goal of LVS is to have the director responsible for dispatching these connection requests to realserver. Therefore, after the client sends a connection request to the VIP, only the director will respond to the client (or the routing device directly connected to the director), and the director updates its IPVSADM table to track the connection. It is then forwarded to one of the realserver at the back end. If a client responds to a request by a realserver when it requests a connection to the VIP, the client establishes a VIP-to-realserver pair in its Mac table, and then communicates later. At this point, the client seems to have only one realserver to be aware of the existence of other servers. To solve this problem, you can do this by setting its forwarding rules on the router (the static Mac-IP bindings). Of course, if you do not have permission to access the router and make the appropriate settings, you can only resolve the problem by using a traditional local approach. These methods include prohibiting Realserver from responding to ARP requests to VIPs, hiding VIPs on realserver so that they are not able to learn ARP requests on the network, based on transparent proxy (Transparent proxy), or Fwmark (firewall mark) "; Prohibit ARP requests from being sent to realservers; Traditionally, resolving ARP problems can be based on network interfaces or host-based implementations. Linux uses a host-based approach because it works well in most scenarios, but LVs does not belong to one of these scenarios, so it has been quite a hassle to implement this functionality in the past. Now you can set arp_ignore,arp_announce, which is a lot easier. Linux2.2 and 2.4（2.4The kernel of the. 26 version) solves the "ARP problem" differently and is more cumbersome. Fortunately,2.4In the. 26 and 2.6 cores, two new flags for tuning the ARP stack are introduced (device flags): Arp_announce and Arp_ignore. Based on this, in the Dr/tun environment, all IPVS related settings can use arp_announce=2 and arp_ignore=1/2/solve the "ARP problem". Here is the official note: arp_annouce:

Define different restriction levels forAnnouncing the local source IP address fromIP PacketsinchARP requests sent onInterface；0- (default) Use any local address, configured on anyInterface.1-Try to avoid local addresses that is notinchThe target's subnet for this interface. 2-always use the best local address forthis target. Arp_ignore:define different modes For sending replies in response to received ARP requests that resolve local target IP address. 0-(default): Replyfor any local target IP address, configured on any interface. 1-reply only if the target IP address was local address configured on the incoming interface. 2-reply only if the target IP address was local address configured on the incoming interface and both with the SENDER's IP address is part fromSame subnet on This Interface.3- DoNot reply forLocal address configured with scope Host,only resolutions forGolbal and link addresses are replied.4-7-Reserved8- DoNot reply forAll local addresses

Arp_announce defines a network card that declares its own Mac -the limit level of IP time, there are three values:0: The default value, regardless of which NIC receives the ARP request, as long as the local computer has a response to this Mac1: Try to avoid the response ARP request in the MAC is not the network card, a host has a number of network cards, one of the network card received the ARP request, found that the requested Mac is another NIC of the computer, this time received the ARP request of this NIC to try to avoid the response2: Always use the most appropriate network card to respond, a host has a number of network cards, one of the network card received the ARP request, found that the requested Mac is another piece of network card, this time received the ARP request of this network card must not respond, only to find the request of the MAC is its own to respond

Arp_ignore defines the response level of the NIC in response to an external ARP request, there are 8 values, but we only use 20: The default value, regardless of which NIC receives the ARP request, as long as the local computer has a response to this Mac1: Always use the most appropriate network card to respond, a host has a number of network cards, one of the network card received the ARP request, found that the requested MAC is the other network card, this time received the ARP request of this network card must not respond, Only the Mac that finds the request is its own to give the response on Realservers, the VIP is configured on the local loopback interface lo. If the packet that responds to the client is routed to the Eth0 interface, the ARP advertisement or should be implemented through eth0, so the following configuration needs to be defined in the sysctl.conf file: Vim/etc/sysctl.conf Net.ipv4.conf.eth0.arp_ignore =1Net.ipv4.conf.eth0.arp_announce =2Net.ipv4.conf.all.arp_ignore =1Net.ipv4.conf.all.arp_announce =2The  above options need to be done before the VIP is enabled, otherwise you will need to empty the ARP table on Drector to use LVS properly. the packet that the connection trace arrives at the director is first passed through the prerouting and then routed to the address of the local interface, so the packet is then sent to input (local_in HOOK). At this point, the Ipvs (always monitoring local_in HOOK) running in the kernel discovers that the packet is requesting a clustered service because the destination address is a VIP. As a result, this packet's original arrival (Director) target trip was changed to be sent via postrouting hook to Realserver. This process of changing the normal travel of a packet is done according to the Ipvs table (defined by the administrator through the Ipvsadm). If there are multiple realserver, in some scenarios, the Director also needs to be based on the "Connection tracking" implementation will be the same client's request is always sent to its first assigned to the Realserver to ensure the integrity of its request. The function of its connection tracking is implemented by hash table. Hash table size and other properties can be viewed by the following command: Ipvsadm-LCN to ensure its timeliness, the "Connection tracking" information in Hash table is defined as "Time to live". LVS defines three timers for recording "Connection timeout": Idle TCP Session, TCP session after the client is disconnected properly, UDP packet not connected (recording its two time interval for sending packets), and the default values for the three timers above can be modified by a command similar to the following The values that follow correspond to the three timers above: Ipvsadm--Set 28800  -  -when a packet is sent from DIRECOTR to Realserver, only the destination MAC address is changed (it becomes the MAC address of the Realserver). Realserver packets are routed to the local loopback device based on the local routing table after the packet is received, and then the service listening on the VIP of the local loopback device handles the incoming database and then responds to the RIP, but the original address of the packet remains the VIP. Dip to configure on the interface, VIP to configure configuration on the front end server on the interface alias (configure VIP) ifconfig eth0:0$192.168.0.1Broadcast $192.168.0.1Netmask255.255.255.255Up Route Add-host $192.168.0.1Dev eth0:0Route Add-host $192.168.1.1Dev eth0 Echo1>/proc/sys/net/ipv4/ip_forward The configuration rip above the Realserver to be configured on the interface, the VIP is configured to define kernel parameters on the LO alias, and suppresses the response to ARP broadcast requests to the VIP echo1>/proc/sys/net/ipv4/conf/lo/arp_ignore Echo1>/proc/sys/net/ipv4/conf/all/arp_ignore Echo2>/proc/sys/net/ipv4/conf/lo/arp_announce Echo2>/proc/sys/net/ipv4/conf/all/arp_announce Configuration vipifconfig lo:0$192.168.0.1Broadcast $192.168.0.1Netmask255.255.255.255Up Route Add-host192.168.0.1Dev Lo:0# Make sure that if the requested destination IP is $VIP, then the source address of the outgoing packet will also be shown as $VIP configure and start the service on the front-end server Ipvsadm-a-t192.168.0.1: the-S WLC ipvsadm-a-T192.168.0.1: the-R192.168.1.2-g-w4Ipvsadm-a-T192.168.0.1: the-R192.168.1.3-g-w2Ipvsadm-l-N ab-c-N10000http//192.168.0.1/index.html watch-n 1 ' ipvsadm-l-n '

LVS Persistence working principle and configuration