LVS Detailed notes

1. Be a reliable, trusted person.

2. Self-trivial to start, diligent, hard-working attitude, do not care, do not complain, good habits to start now

3. Don't feel Yourself

4. Concentrate your "All" on one point and you will succeed

5, concentrate all of your time to learn

ARP protocol: Obtain host physical MAC address via IP address

The IP address of the + bit

The MAC address of the bit

ARP principle:

The principle is to convert the destination host's 32-bit IP address to the destination host 48-bit Ethernet address (MAC address)

The IP address is the logical address, and communication between the hosts is communicated through the physical MAC address

The ARP protocol requires both hosts of the communication to be in the same network segment (i.e. LAN environment)

The ARP cache table is a double-edged sword

1, ARP cache table can speed up the resolution of ARP

2, also gives hackers the risk of attacking the server, commonly known as ARP spoofing and ARP attack

The OSI model divides the network into 7 layers, and the compartments do not deal directly with each other, and are called through the AIP.

IP address on layer three, MAC address on level two

ARP is a three layer protocol, but works on two levels

ARP Spoofing principle

An ARP attack is an ARP spoofing that is implemented by forging IP addresses and MAC address pairs.

How to prevent:

Arp-s 192.168.0.11 mac

#################### #LVS +keepalived

Cluster: A cluster of clusters relies on the network to work together

Java Architect: 50W start

Java Service cluster

Java EE program architecture

JVM Bottom

Tomcat, Java Packaging

Program decoupling (programming, tuning, Java process)

Why use cluster??

Performance, availability, scalability, scalability, security

1. High performance (performance) evaluate a website the direct feeling of good or bad

2, price availability, small machine (IBM) extra expensive

3. High Availability 7x24 Run availability

4. Scalability can dynamically add new machines scalability Database scaling is difficult

Classification of clusters:

1.LB Load Balancing cluster

2.HA High Availability Cluster

3. High Performance Computing

4. Grid Computing

The role of load balancing:

(1): Share requests and data traffic to increase throughput, concurrency

(2): 24x7 Service for the cluster

(3): For the Web server, the database from the

Typical load Balancing software: Lvs,f5 (4 OSI) Nginx (7osi) haproxy (4+7osi)

Typical high-availability software: keepalived heratbeat

Internet Business Cluster Hardware: F5, NetScaler redware. A10

Lvs:linux virtural Server

Lvs:

1, the real load balancing function is Ipvs

2,ipvs work at the kernel level, implement scheduling, cannot directly access

3,ipvsadm manage Ipvs, or use keepalived to manage Ipvs

LVS Professional Terminology:

VIP: Virtual IP address virtural IP address used to provide service to client computers

Rip:real IP, IP address used by node under cluster, physical IP address

Dip:director IP is used to connect an IP address on a physical network card, and he is the IP on the load balancer.

Cip:client IP Client User computer requests the IP address of the cluster server, which is used as the source IP address of the request sent to the cluster

########### #在LVS框架中 provides IP virtual server software with three IP load Balancing technologies:

IP Virtual Server Software Ipvs

Kernel Layer-7 switch based on content request distribution Ktcpvs

Cluster management software

In the existing IP load balancing technology, Ipvs Software realizes these three kinds of IP load balancing technology

Three modes of LVS:

1,nat ()

2,tun (tunneling)

3,DR (DIRECT ROUTING)

4,fullnat

Currently the most popular mode DR-skilled use

Features: The target MAC address of the request message is rewritten, the request is forwarded to the real server, and the real server returns the response processing directly to the client for use.

This Dr mode does not have the overhead of IP tunneling, and there is no need to support the IP tunneling protocol for real servers in the cluster

Dr mode requires that the scheduler lb and the real server RS have a NIC connected to the same physical network segment, that is, must be in the same LAN environment

Dr Mode:

Working principle:

Key points:

1, bind VIP to lo, avoid IP conflict problem, solve the problem that real server can't recognize client's message

2, the bound VIP will actively respond to ARP broadcasts, so to suppress ARP, tell real server not to rob LVs ARP packets, only receive packets

Can say the second kill 5-12k around

Summary of Dr Model

1, by modifying the destination MAC address of the packet on the scheduler lb to implement forwarding,, note that the source IP is still CIP, the destination IP address is still a VIP

2, the requested message through the Scheduler and RS response processing of the message does not need to go through the scheduler lb, directly returned to the client therefore, the concurrent access volume is very high efficiency

3, because Dr Mode is forwarded through the rewriting mechanism of MAC address, all RS nodes and scheduler lb can only be in one LAN LAN (small Disadvantage)

4, need to be aware of the RS node's VIP binding (LO:VIP.LOL:VIP) and ARP suppression issues

5, emphasis: the RS node default gateway does not require the scheduler lb dip, and directly is the IDC room assigned to the superior router IP (this is the case of RS with an extranet IP address)

6, since the DR Mode scheduler only makes the destination MAC address rewrite, the scheduler lb cannot change the destination port of the request message.

7, the current scheduler lb supports almost all UNIX Linux systems, but currently does not support windows, but the real RS can be windows.

8. In general, the DR mode is very efficient, but the configuration is troublesome, so the company can use Haproxy, Nginx, this is in line with the principle of operation and maintenance: simple, easy to use, efficient day pv2000w, concurrent with 1 W Haproxy,nginx (LVS Nat mode) just fine.

9, direct external access to the business, such as Web services to do RS node, RS best Use the public IP address (directly return the request to the user), if not directly external business, such as MySQL, storage system, it is best to use internal IP address

Nat mode: Understand, basic use of Dr Mode in work

Summarize:

1,nat technology will request a message (DNAT) and a response message (SNAT), rewritten through the dispatcher address and then forwarded to the internal server. When the message returns, it is rewritten as the original user requested address.

2, only ouch on the scheduler LB configuration Wan public IP, the scheduler should also have LAN IP and internal RS node through the

Letter

3, each internal RS node gateway address, must be configured as the scheduler lb private LAN within the physical network card address

。

。

。

Omitted

####################### #IPVS调度器实现了如下十种负载调度算法: LVs scheduling algorithm

10 Kinds of scheduling algorithms

Fixed scheduling algorithm: Rr,wrr,dh,sh

Dynamic scheduling algorithm: Wlc,lc,lblc,lblcr,sed,nq.

Three most commonly used RR WRR WLC

10 scheduling algorithms are shown in the following table:

RR: Polling scheduling, he will request to assign a different RS node, RS Server averaging request, this algorithm is relatively simple, but only suitable for the RS node difference performance is not small situation

WRR: Weighted polling scheduling, which will be based on the weights of different RS nodes assigned tasks, the high weight of RS will take priority to obtain the task, and the number of connections assigned to the lower weight than the RS node more. RS with the same weights get the same number of connections

Dh:destination hashing find a static hash table with the destination address as the keyword to get the required RS

Sh:source hashing find a static hash table with the source address as the keyword to get the required RS

Dynamic Scheduling algorithm:

WLC: Weighted minimum number of connections dispatch (weighted least-connection) assumes that the weights of each RS are in turn wi (I=1...N). The current number of TCP connections is Ti (I=1..N), followed by Ti/wi as the smallest RS as the next assigned RS

LC

Lblc

Lblcr

Sed

NQ are not used.

################ #LVS调度算法的生产环境选型

General network Services HTTP mail MySQL common scheduling algorithm is RR,WRR WLC

################ #LVS集群的特点

The features of the LVS cluster can be summed up as follows:

Function

There are three kinds of Ipvs software that implement IP load balancing technology and eight kinds of connection scheduling algorithms. In the internal implementation of IPVS, an efficient hash function and garbage collection mechanism are used to correctly handle the ICMP messages associated with the dispatched messages (some commercialized systems can not). There is no limit to the number of virtual service settings, and each virtual service has its own set of servers. It supports persistent virtual services such as HTTP cookies and HTTPS, and provides detailed statistics such as the processing rate of the connection and the traffic to the message. Three defense strategies were implemented for large-scale denial of service (deny) attacks.

There is an application-layer switching software Ktcpvs based on content request distribution, which is also implemented in the Linux kernel. With the relevant cluster management software to monitor the resources, can timely fault shielding to achieve high availability of the system. The master, Slave scheduler can periodically synchronize state to achieve higher availability.

Applicability

The backend server can run any TCP/IP-enabled operating system, including Linux, various Unix (such as FreeBSD, Sun Solaris, HP UNIX, etc.), Mac/os and Windows nt/2000, and so on.

The load scheduler can support the vast majority of TCP and UDP protocols:

Agreement Content

TCPHTTP,FTP,PROXY,SMTP,POP3,IMAP4,DNS,LDAP,HTTPS,SSMTP, etc.

UDPDNS,NTP,ICP, video, audio stream playback protocol, etc.

You do not need to make any modifications to the client and server to apply most Internet services.

Performance

The LVS server cluster system has good scalability and can support millions of concurrent connections. Configure the 100M network card, using Vs/tun or VS/DR scheduling technology, the throughput of the cluster system can be as high as 1gbits/s, if the Gigabit network adapter is configured, the maximum throughput of the system is close to 10gbits/s.

Reliability

LVS server cluster software has been well applied in many large and critical sites, so its reliability is well proven in real-world applications. There are many schedulers that have been running for more than a year and have not been restarted.

Software license

The LVS cluster software is a free software issued under the GPL (GNU public License) license, which means that you can obtain the source code of the software, and you have the right to modify it, but you must ensure that your modifications are also distributed in GPL form.

Official information

LVS Project Description: http://www.linuxvirtualserver.org/zh/lvs1.html

The architecture of LVS cluster http://www.linuxvirtualserver.org/zh/lvs2.html

IP load Balancing technology http://www.linuxvirtualserver.org/zh/lvs3.html in LVS cluster

LVS Load Dispatch http://www.linuxvirtualserver.org/zh/lvs4.html

######################### #安装LVS

1,5.x System Download ipvsadm-1.24.tar.gz

6.X system download ipvsadmin-1.26.tar.gz suitable for kernel 2.6.28 after

：

Http://www.linuxvirtualserver.org/software/ipvs.html

Installation Preparation:

Lsmod | grep Ip_vs Lsmod is a view of the kernel module

Uname-r

If you do not see if ls/usr/src/kernels/2.6.32-504.30.3.el6.x86_64/exists, then

Ln-s/usr/src/kernels/2.6.32-504.30.3.el6.x86_64//usr/src/linux

If not, install Yum install kernel-devel-y

################ #开始安装LVS

Tar xvf ipvsadm-1.26.tar.gz

CD ipvsadm-1.26

Make--This step if error, install yum install libnl* popt*-y

Make install

Lsmod | grep Ip_vs--not at this time

/sbin/ipvsadm or Modprobe Ip_vs

IP Virtual Server version 1.2.1 (size=4096)

Prot Localaddress:port Scheduler Flags

Remoteaddress:port Forward Weight activeconn inactconn

[Email protected] ipvsadm-1.26]#

[Email protected] ipvsadm-1.26]# Lsmod | grep Ip_vs

Ip_vs 125220 0

LIBCRC32C 1246 1 Ip_vs

IPv6 317340 142 Ip_vs,ip6t_reject,nf_conntrack_ipv6,nf_defrag_ipv6

[Email protected] ipvsadm-1.26]#

LVS Installation Summary:

1,centos 5.x Installation LVS, use version 1.24, do not use 1.2.6

2.centos6.x Install version 1.26 and install yum install libnl* popt*-y First

3. After installing LVS, execute ipvsadm to load the Ip_vs module into the kernel

########### #手动配置LVS负载均衡服务

1, resolve VIP to domain name www.xxxx.com 192.168.0.200

2, configure LVS virtual IP (VIP)

Ifconfig eth1:0 192.168.0.200/24

3. Manually perform configuration add LVS service increase two RS

Ipvsadm-c Clear teh whole table

Ipvsadm--set 5 Set TCP Tcpfin UDP timeout

Ipvsadm-a-T Vip:80-s RR add virtual service with options-s scheduling algorithm

Ipvsadm-a-T vip:80-r 192.168.0.221-g-a:add Real Server with options

-g:gatewaying (Direct routing) (default) LVS mode

Ipvsadm-a-T 192.168.200:80-r 192.168.0.221-g-W 1

Ipvsadm-a-T 192.168.0.200:80-r 192.168.0.222-g-W 1

[Email protected] ~]# ipvsadm-l-N

IP Virtual Server version 1.2.1 (size=4096)

Prot Localaddress:port Scheduler Flags

Remoteaddress:port Forward Weight activeconn inactconn

TCP 192.168.0.200:80 RR

-192.168.0.221:80 Route 1 0 0

-192.168.0.222:80 Route 1 0 0

########## #删除:

1. Delete a node

[[email protected] ~]# ipvsadm-d-t 192.168.0.200:80-r 192.168.0.222 Delete node

[Email protected] ~]# ipvsadm-l-N

IP Virtual Server version 1.2.1 (size=4096)

Prot Localaddress:port Scheduler Flags

Remoteaddress:port Forward Weight activeconn inactconn

TCP 192.168.0.200:80 RR

-192.168.0.221:80 Route 1 0 0

2. Delete a service

ipvsadm-d-T vip:80

You cannot access the Web through the VIP at this time

4. Manually binding on the RS side

(1), each RS side performs the following steps

(01) Bind VIP to Lo

Ifconfig lo:0 VIP/32--note that at this time the mask is 32 bits

[Email protected] ~]# ifconfig lo:0 192.168.0.200/32

lo:0 Link encap:local Loopback

inet addr:192.168.0.200 mask:0.0.0.0

Up LOOPBACK RUNNING mtu:16436 metric:1

Route add-host 192.168.0.200 Dev Lo Add host route

[Email protected] ~]# route-n

Kernel IP Routing Table

Destination Gateway genmask Flags Metric Ref use Iface

192.168.0.200 0.0.0.0 255.255.255.255 UH 0 0 0 Lo

0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth0

5. Manually suppress ARP at the RS end

echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore

echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce

echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

Detailed introduction of ARP technical parameters

Arp_ignore

1, which indicates that only the destination IP address is the ARP query request to access the network interface local address

Arp_announce: The ARP response to the local IP address on the network interface, to make the appropriate level of restrictions

2, which means the most appropriate local address is used for the query target, in which case the source address of the IP packet is ignored

All of the above are temporary configurations, restart failure

################## #LVS集群分发请求RS不均衡生产环境实战解决

Production environment Ipvsadm-l-N found two RS load imbalance, one has a lot of requests, one does not. After testing found that no requested RS service is normal, LO:VIP also, but there is no request

TCP 172.168.1.50:3307 WRR presient 10

172.168.1.51:3307 Route 1 0 0

172.168.1.52:3307 Route 1 8 12758

Cause of the problem:

Persistent 10 for the reason that persistent session remains, when Clienta visited the site, LVS was distributed to 52., then Clienta click on the other operation request, will also send to 52 this machine

Workaround:

Comment out persistent 10 in keepalived and then/etc/init.d/keepalived reload and load will be balanced on both sides.

To implement a session-preserving scenario:

http://oldboy.blog.51cto.com/2561410/1331316

########### #LVS故障排错原理

1. Make the bundled VIP a lo configuration file (Ifcfg-lo)

2. Triangle Troubleshooting theory for load balancing and reverse proxy clusters

LVS Detailed notes