Analysis of the principle of LoadBalancerv2

OpenStack is a load balancing task that uses a variety of open source available load balancing projects directly, using HAProxy by default. LBaaSv2 Essentially, it is also based on the user's load balancing requirements, generate a compliant HAProxy profile and start HAProxy, and then load-balanced by HAProxy.

High Availability Proxy (HAProxy)?

? HAProxy is a well-known open source software TCP (four layer)/http (seven layer) load balancer and proxy software that can be run on systems such as Linux,solaris and FreeBSD. At present, it has been adopted by many large companies, including GitHub, Imgur, Instagram, and Twitter. It is similar to Nginx, with a single-process and event-driven model, which uses low-volume and stable memory to handle a large number of concurrent requests.

Here I simply list the haproxy configuration. For more information, see:

Http://www.cnblogs.com/wanstack/p/8392332.html

The Haproxy configuration is divided into five sections, as follows:
? Global: Parameters are process-level, usually related to the operating system. These parameters are usually set only once, and if configured correctly, you do not need to modify them again.
? Defaults: Configure default parameters that can be used with the Frontend,backend,listen component.
? Frontend: The front-end virtual node that receives the request, frontend can more regularly specify the backend that uses the backend directly.
? Backend: The configuration of the backend service cluster is the real server, and one backend corresponds to one or more entity servers.
? A combination of Listen fronted and backend.

Neutron LoadBalancerv2 configuration file in/etc/haproxy/haproxy.cfg

########## #全局配置 ######### Global log/dev/log local0 #[log output configuration, all logs are recorded on this machine via local0 output] Log/dev/log loc AL1 Notice #定义haproxy log level [ERROR warringinfo debug] chroot/var/lib/haproxy stats SOCKET/RUN/HAPROXY/ADMI N.sock mode 660 level admin stats Timeout 30s User haproxy group Haproxy #可以由配置项 user_group specified, default is n Ogroup Daemon #以后台形式运行harpoxy # Default SSL material Locations Ca-base/etc/ssl/certs Crt-bas        E/etc/ssl/private # Default ciphers to use on ssl-enabled listening sockets. # For more information, see Ciphers (1SSL). This list was from: # HTTPS://HYNEK.ME/ARTICLES/HARDENING-YOUR-WEB-SERVERS-SSL-CIPHERS/SSL-DEFAULT-BIND-CIP Hers ecdh+aesgcm:dh+aesgcm:ecdh+aes256:dh+aes256:ecdh+aes128:dh+aes:ecdh+3des:dh+3des:rsa+aesgcm:rsa+aes:rsa+ 3des:!anull:! md5:!       DSS ssl-default-bind-options no-sslv3 ####### #默认配置 ############ Defaults log global mode HTTP Option httplog option dontlognull Timeout Connect-Timeout client 50000 timeout server 50000 errorfile 400/etc/haproxy/errors/400.http errorfile 403/etc/haproxy/errors/403.http Errorfi Le 408/etc/haproxy/errors/408.http errorfile 500/etc/haproxy/errors/500.http errorfile 502/etc/haproxy/er Rors/502.http errorfile 503/etc/haproxy/errors/503.http errorfile 504/etc/haproxy/errors/504.http

　　

We created the QLBAAS-0F66315F-0CCF-43CB-ABCA-2BB6F51E8FB2 in advance and we looked at its configuration file

[Email protected]:~# IP netns LISTQLBAAS-0F66315F-0CCF-43CB-ABCA-2BB6F51E8FB2

　　

This load balancer is a 1 loadbalance corresponding to 1 listener,1 pool.

In the/var/lib/neutron/lbaas/v2/0f66315f-0ccf-43cb-abca-2bb6f51e8fb2/haproxy.conf

# Configuration for Loadbalance1global daemon user Nobody group Haproxy log/dev/log local0 log/dev/log Lo CAL1 Notice stats socket/var/lib/neutron/lbaas/v2/0f66315f-0ccf-43cb-abca-2bb6f51e8fb2/haproxy_stats.sock mode 0666 Level userdefaults Log Global retries 3 option redispatch Timeout Connect-Timeout client 50000 time  Out server 50000frontend 62ac018e-f6fc-4d60-80df-13b1e4cdc6f6 option tcplog maxconn option forwardfor bind 2.2.2.20:80 mode http default_backend 8d28b2c3-9c44-46e5-b2eb-7bd9b8d5faf6backend 8d28b2c3-9c44-46e5-b2eb-7bd9b8d5f Af6 mode HTTP Balance roundrobin timeout check 1 option httpchk get/index.html Http-check expect Rstatus 2 01|200|202 server d56fc582-33cd-4fc7-b95f-16534c8a4860 2.2.2.5:80 weight 1 check inter 1s fall 5 server cc2230bf-f3b 8-4beb-8584-71b0f3a0ba5c 2.2.2.4:80 weight 1 check inter 1s fall 5 server b490cadb-cff1-4e7a-92c7-a134c0f8b321 2.2.2.6: Weight 1 check inter 1s Fall 5 

　　

LBaasv2 can be seen as the virtualization of OpenStack Neutron for a variety of physical load balancers. Its concept can be analogous to the concepts in HAProxy:

the concept of HAProxy	the concept of LBaasv2	Description
	Driver	LBaas v2 is also a load balancer that takes a driver model to support multiple physics. LBAASV2 implements the HAProxy driver by default, and it also supports multiple other Vendor driver. Manufacturer Driver
Frontend	Listener	The LBAASV2 uses listener mode to transfer traffic to member in different pool.
Backend	Pool	Represents the virtual machine pool on the back end of the load that listener listens on.
Backend server	Member	The Member corresponds to an OpenStack Nova VM that handles network requests within the pool.
Health Check	Health Monitor	It is used to monitor the status of member in the pool, and supports various detection methods such as HTTP, TCP, and Ping. In Nuetron this is optional, if there is no health monitor,pool will always assume that all member are active state, so that all member will always appear in the VIP distribution list, even if the member corresponding instance can not respond to the network Network requests. This results in a load-balanced response exception in the actual application.

LoadBalancerv2 's usage scene?

? As can be known, a LoadBalancerv2 can correspond to multiple pool, we also set up a pool as follows:

In the/var/lib/neutron/lbaas/v2/0f66315f-0ccf-43cb-abca-2bb6f51e8fb2/haproxy.conf

# Configuration for Loadbalance1global daemon user Nobody group Haproxy #可以由配置项 user_group specified, default is Nogroup log /dev/log local0 log/dev/log local1 notice stats socket/var/lib/neutron/lbaas/v2/0f66315f-0ccf-43cb-abca-2bb6f51e8 Fb2/haproxy_stats.sock mode 0666 level user Defaults #不用管 log global retries 3 option Redispatch Timeou T connect, 50000 timeout client, server 50000frontend 62ac018e-f6fc-4d60-80df-13b1e4cdc6f6 option tcpl og maxconn option Forwardfor # when mode is "http", set the forwardfor so that the original source IP address is saved via X-forward-for header bind 2.2.2.2  0:80 #监听Listener的vip:p ort mode http #监听Protocol default_backend 8d28b2c3-9c44-46e5-b2eb-7bd9b8d5faf6 #对应的监听池frontend BF144F31-CDBB-4426-B90B-4BDBC67501F1 option tcplog maxconn option forwardfor bind 2.2.2.20:100 mode HT TP default_backend 8b50ed30-5290-421c-9d31-fb3751a26be2backend 8b50ed30-5290-421c-9d31-fb3751a26be2 mode HTTP Bal ance Roundrobin ServeR bef852d0-9164-46ee-ace5-92462e8d89ef 2.2.2.14:100 weight 1 server 8aeb5cc2-7301-4931-ac3b-e0d0ca891e88 2.2.2.15:100 Weight 1 server 250a919f-dfc1-41b6-8378-2b4015f1acd0 2.2.2.16:100 weight 1backend 8d28b2c3-9c44-46e5-b2eb-7bd9b8d5faf6 mode HTTP Balance roundrobin timeout check 1 option httpchk get/index.ht  ML Http-check expect rstatus 201|200|202 server cc2230bf-f3b8-4beb-8584-71b0f3a0ba5c 2.2.2.4:80 weight 1 Check Inter 1s fall aa member1 configuration, including Ip,port (Member service port, no check port specified at this time, so it is also a health check TCP port), Weight;check designated to do health check;
# inter Specifies the interval between two consecutive checks, default 2s (1s), fall specifies Max retries, or the number of successive checks failed to think member is down (5) server d56fc582-33cd-4fc7-b95f- 16534c8a4860 2.2.2.5:80 weight 1 check inter 1s fall 5 server b490cadb-cff1-4e7a-92c7-a134c0f8b321 2.2.2.6:80 weight 1 Check Inter 1s fall 5

　　

Access to Wget-o-http://2.2.2.2:80 and Wget-o-http://2.2.2.2:100 were successful.

Above is VIP and pool of members with a subnet, below we verify that VIP and pool of the same subnet.

We create a new loadbalance and a LISTENER,VIP address of 7.7.7.7, then create a pool, note that a virtual machine can join multiple pool, so we also add the above virtual machine to the new pool. Then connect via router subnet7.7.7.0/24 and subnet2.2.2.0/24. That is to say vip7.7.7.7 can and member2.2.2.4,2.2.2.5,2.2.2.6 is unicom.

Configure the following/var/lib/neutron/lbaas/v2/5081116f-8928-40d7-8aaa-e30c336ca713/haproxy.conf

# Configuration for Loadbalance3global    daemon    user Nobody    group haproxy    log/dev/log local0    Log/ Dev/log local1 Notice    stats socket/var/lib/neutron/lbaas/v2/5081116f-8928-40d7-8aaa-e30c336ca713/haproxy_ Stats.sock mode 0666 level userdefaults    log Global    retries 3    option Redispatch    Timeout Connect 5000    Timeout Client 50000    Timeout server 50000frontend 84800dd3-0507-4628-b54b-a23226bec4f8    option Tcplog    maxconn    option forwardfor    bind 7.7.7.7:80    mode http    default_backend 3583deda-e9ca-40bb-ba23-0fec204c099fbackend 3583deda-e9ca-40bb-ba23-0fec204c099f    mode http    balance Roundrobin    Server 48b36860-8e4d-476e-9196-ad052c317f44 2.2.2.5:80 weight 1    server f8732b2a-bfaa-4e5f-b8bb-f88c9fed899b 2.2.2.4:80 weight 1    server 004f7950-4031-4de3-98b2-ca30e39c4e4e 2.2.2.6:80 weight 1

　　

In other words, as long as the VIP and member can communicate, not necessarily in the same subnet.

In addition, if you want to access from the extranet, you also need to create a floating IP and associate it to the VIP of lb. The namespace in the Haproxy actually has only one IP address, which receives the external connection and the connection between the members respectively.

LOADBALANCERV2 Multi-agent mode?

? The LOADBALANCERV2 service can be deployed on the server independently, including 2 services, Neutron-openvswitch-agent and neutron-lbassv2-agent. Assuming that 2 nodes are deployed with the LOADBALANCERV2 service, when Neutron-server makes a create request, it chooses one of the two nodes to create the corresponding namespace space.

LOADBALANCERV2 's process analysis?

? We use QLBAAS-0F66315F-0CCF-43CB-ABCA-2BB6F51E8FB2 as an example to analyze this process.

IP netns exec qlbaas-0f66315f-0ccf-43cb-abca-2bb6f51e8fb2 IP addr1:lo: <LOOPBACK,UP,LOWER_UP> MTU 65536 Qdisc noqu Eue State UNKNOWN Group default Qlen 1    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet    127.0.0.1/8 SCO PE host lo       valid_lft forever Preferred_lft Forever    Inet6:: 1/128 scope host        Valid_lft forever Preferred_lft F OREVER9:TAP83F82FCF-D1: <BROADCAST,MULTICAST,UP,LOWER_UP> MTU qdisc noqueue State UNKNOWN Group default Qlen 1    link/ether fa:16:3e:d1:c8:b1 brd ff:ff:ff:ff:ff:ff    inet 2.2.2.20/24 BRD 2.2.2.255 Scope Global tap83f82fcf-d1< c7/> #vip的地址       Valid_lft forever Preferred_lft Forever    inet6 fe80::f816:3eff:fed1:c8b1/64 scope link        valid _lft Forever Preferred_lft Forever

　　

? The interface tap83f82fcf-d1 hangs on the OvS and is hit on the vlan_id of the network where it resides:

Bridge br-int       fail_mode:secure       Port patch-tun           Interface patch-tun               type:patch               options: {peer= Patch-int}       Port br-int           Interface br-int               type:internal       Port "tap83f82fcf-d1"           tag:1           Interface "Tap83f82fcf-d1"               type:internal

? For the LOADBALANCERV2 creation process (in V2, create a load balancer and create listener are complete, we find that when just completing create a load balancer, there is no namespace, namespace appears when create listener is complete) we do the following:

Ovs-vsctl--if-exists del-port tap83f82fcf-d1--add-port br-int tap83f82fcf-d1--set Interface tap83f82fcf-d1 type= Internal--set Interface tap83f82fcf-d1 external-ids:iface-id=83f82fcf-d141-4774-87a0-ace79196bc88--set Interface Tap83f82fcf-d1 external-ids:iface-status=active--set Interface tap83f82fcf-d1 external-ids:attached-mac=fa:16:3e: D1:c8:b1#iface-id and Attached-mac can be found in the database IP link set tap83f82fcf-d1 address Fa:16:3e:d1:c8:b1ip netns add QLBAAS-0F66315F-0CCF-43CB-ABCA-2BB6F51E8FB2 IP netns exec qlbaas-0f66315f-0ccf-43cb-abca-2bb6f51e8fb2 sysctl-w Net.ipv4.conf.all.promote_secondaries=1ip Link Set tap83f82fcf-d1 netns Qlbaas-0f66315f-0ccf-43cb-abca-2bb6f51e8fb2ip netns exec qlbaas-0f66315f-0ccf-43cb-abca-2bb6f51e8fb2 IP link set lo UPIP Link Set tap83f82fcf-d1 netns qlbaas-0f66315f-0ccf-43cb-abca-2bb6f51e8fb2 IP netns exec QLBAAS-0F66315F-0CCF-43CB-ABCA-2BB6F51E8FB2 IP link Set tap83f82fcf-d1 upip netns exec QLBAAS-0F66315F-0CCF-43CB-ABCA-2BB6F51E8FB2 IP Addr Show tap83f82fcf-D1 permanent scope globalip netns exec qlbaas-0f66315f-0ccf-43cb-abca-2bb6f51e8fb2 ip-4 addr add 2.2.2.20/24 brd 255.255. 255.0 Scope Global Dev tap83f82fcf-d1ip netns exec qlbaas-0f66315f-0ccf-43cb-abca-2bb6f51e8fb2 ip route list Dev tap83f82f CF-D1 scope Linkip Netns exec QLBAAS-0F66315F-0CCF-43CB-ABCA-2BB6F51E8FB2 route add default GW 2.2.2.1ip netns exec Qlbaas -0F66315F-0CCF-43CB-ABCA-2BB6F51E8FB2 arping-u-I tap83f82fcf-d1-c 3 2.2.2.20

　　

LOADBALANCERV2 interpretation of the source code?

? The code structure for LOADBALANCERV2 is as follows:

1.Create a Load Balancer

2.Create a listener

3.Create a pool

4.ADD Member

5.Create A Health Monitor

Analysis of the principle of LoadBalancerv2