LVS fullnat + ecmp [4] the backend nginx successfully parses the real IP Address

LVS Configuration:

[[email protected] keepalived]# cat keepalived.conf ! Configuration File for keepalivedglobal_defs {   notification_email {     [email protected]   }   notification_email_from [email protected]   smtp_server 192.168.200.1   smtp_connect_timeout 30   router_id LVS_DEVEL}local_address_group laddr_g1 {  #172.16.98.11  172.16.98.10}virtual_server_group shanks1 {  2.2.2.3 80}virtual_server 2.2.2.3 80 {    delay_loop 6    lb_algo rr    lb_kind FNAT    protocol TCP    syn_proxy    laddr_group_name laddr_g1    #local address group    #alpha    #omega    #quorum 1    #hysteresis 0    #quorum_up " ip addr add 10.255.255.123/32 dev lo;"    #add    #quorum_down "ip addr del 10.255.255.123/32 dev lo;"    #del vip    real_server 172.16.97.30 80 {        weight 100        TCP_CHECK {            connect_timeout 3            nb_get_retry 3            delay_before_retry 3            connect_port 80        }    }}include /etc/keepalived/hosts/*.conf[[email protected] keepalived]# 

Note:

Global_defs: This part is not important. You do not need to fill in the content, but you cannot leave it alone. Otherwise, the startup may fail.
Local_address_group: Source Address of SNAT. Multiple IP addresses can be written here. One IP address can be converted to 65536 sessions. The address written here must exist in the NIC.
Virtual_server_group: Both VIP and vport must be declared here.
Virtual_server: configure the VIP ing between VIP and RealServer.

Nic multi-IP configuration method:

[[email protected] keepalived]# cat /etc/sysconfig/network-scripts/ifcfg-em2DEVICE=em2HWADDR=90:B1:1C:5A:37:4ETYPE=EthernetUUID=2b1947f4-1725-4197-abf5-6c8fed750adcONBOOT=yesNM_CONTROLLED=yesBOOTPROTO=staticIPADDR=172.16.99.11NETMASK=255.255.255.0IPADDR1=2.2.2.2NETMASK1=255.255.255.255[[email protected] keepalived]# 

Or in ospfd, add a command through the command (the command is similar to a Cisco command. For details, refer to Baidu)

Enable keepalived

service keepalived restart
Run the following command to view the output information:

[[email protected] keepalived]# service keepalived restartStopping keepalived:                                       [  OK  ]Starting keepalived:                                       [  OK  ][[email protected] keepalived]# ipvsadm -lnIP Virtual Server version 1.2.1 (size=4194304)Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConnTCP  2.2.2.3:80 rr synproxy  -> 172.16.97.30:80              FullNat 100    0          0         [[email protected] keepalived]# ipvsadm -GVIP:VPORT            TOTAL    SNAT_IP              CONFLICTS  CONNS     2.2.2.3:80           1                                      172.16.98.10         0          0         [[email protected] keepalived]# 

Server to view nginx log files

Three tests were conducted, not in
1. Enable the Toa module on the backend
2. The toa module is not enabled on the backend, and LVS configures local_address_group laddr_g1 {172.16.98.11}
3. The toa module is not enabled on the backend, and LVS configures local_address_group laddr_g1 {172.16.98.10}
Logs in three cases.
When the Toa module is enabled, the source address after SNAT is identified if the real IP address is identified correctly and the Toa module is not enabled.
The following are the captured log files:

172.16.99.4 - - [30/Aug/2018:16:13:24 +0800] "GET /nginx-logo.png HTTP/1.1" 200 368 "http://2.2.2.3/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134" "-"172.16.98.11 - - [30/Aug/2018:16:15:39 +0800] "GET /poweredby.png HTTP/1.1" 200 2811 "http://2.2.2.3/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134" "-"172.16.98.10 - - [30/Aug/2018:16:16:58 +0800] "GET / HTTP/1.1" 200 3700 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134" "-"

LVS fullnat + ecmp [4] the backend nginx successfully parses the real IP Address