Load Balancing Detailed Talk

As developers of back-end applications, we often develop, debug, test and publish our applications to the production environment, and users have direct access to our applications. But for Internet applications, there is a low profile or thick or thin layer of load balancing software between your application and users, and they do not show the dew silently playing an important role, so much so that we often neglect their existence. Because the load balancer layer is usually not in the general developer's problem domain, and they are generally out-of-the-box and mature solutions, so that we are habitually ignored and considered to be lackluster. In fact, this article writes about my knowledge and understanding of the load balancing hierarchy.

Hard load

The so-called "hard load" is the use of hardware devices to provide load balancing.

Seven or eight years ago, when I was doing Java enterprise Software development, the enterprise Java applications developed were deployed in application containers like Weblogic. And this kind of application container software runs on the Unix small machine. The hardware and software packaged as an enterprise application solution to sell to customers. The deployment of such applications is simple and the hierarchy is relatively shallow. To ensure reliability, deploy one Weblogic Server on each of the two sets of small machines, using a hardware load balancer like F5 in front of the app service, as shown in.

Because the minicomputer and the front F5 load balancing hardware are more expensive, so for the reliability, maintainability and cost of comprehensive consideration, the general application deployment of two sets of running on two small machines, in front of sharing a F5 do load balancing. And the general F5 and minicomputer such hardware devices are at least 5 9 reliability guarantee, so the overall system reliability is basically guaranteed.

After entering the Internet era, application development embraced open source, deploying the use of cheaper PC servers and free open-source application containers. Load balancing also gradually changes from hard load to soft load, because of the massive features of Internet applications and the rapid expansion of deployment scale, front-end load balancing is also beginning to become rich.

Soft load

After entering the Internet company, when we first started to develop the application, the business scale small user quantity is not big, the machine quantity also is few (<10). So the initial load balancing structure is also very simple, similar to the hard load just replaced the hardware with free open source software and ran on the availability of 3 9 of the cheap PC Server.

In front of a LVS followed by several application services, and later in order to facilitate the division by the domain name and suitable for cutting traffic on-line, the middle of a layer of Nginx.

This becomes a two-layer soft-load structure, LVS is responsible for 4 layers, Nginx is responsible for 7 layers. But nginx only responsible for the single-machine multi-instance load balancing, here is mainly because the PC Server is a physical machine, CPU 16/32 core, memory 32/64g, in order to more fully utilize resources, a physical machine has deployed a number of application services instances, and considering that Nginx work in The 7-tier overhead is much higher than the LVS/DR mode, so there is no more than 10 instances hanging behind an Nginx.

But with the growth of business and user traffic, the machine scale is also expanding, resulting in a network segment of the IP is not enough, this load structure has encountered a scale-out bottleneck, because the LVS/DR mode can not cross the network segment. So later in the LVS and Nginx added a layer of HAProxy, the load structure becomes the following.

In fact, after adding the HAProxy, it is also working on the 7 floor, so Nginx this layer does not seem very necessary. However, the three-layer load structure can support a larger cluster, but originally in the Nginx layer to do a set of easy to research and development of cut traffic on-line operation and maintenance management system, so sacrifice a bit of performance in exchange for the current maintainability and future scalability, Nginx this layer has been retained. and Nginx compared to HAProxy is not a pure load balancer, it can also provide the cache function, for some HTTP requests actually only go to Nginx this layer can be returned by cache hit.

DNS Load

As the business developed, the company began to build multiple IDC, taking into account the IDC-level disaster recovery, the cluster began to deploy to multiple IDC. The cross-IDC load Balancing scheme can be implemented simply through DNS polling, but the controllability is not good. So instead of using this, we use a master plus multiple subdomains to implement dynamic domain name scheduling and load based on business scenarios. The primary domain is actually a dynamic traffic scheduler that spans multiple IDC deployments, and for HTTP requests is based on redirection of the sub-domain name, which is requested to allocate the actual connected subdomain each time the TCP method establishes a long connection, as shown in.

CDN Load

Finally, coupled with the necessary CDN for Internet applications to offload the load of static resource requests, the entire load hierarchy is complete.

Changes in load structure caused by SSL

With the popularization of the Internet, security problems become more serious, originally only bank network silver and so on using HTTPS way access, now e-commerce Web site also began to enable the full station HTTPS. What is the impact on the load structure after the introduction of SSL? SSL belongs to the application layer protocol, so only on the 7 layer to do, and HAProxy is also support SSL protocol, so one way is simply let HAProxy turn on SSL support to complete internal decryption external encryption processing.

However, HAProxy's authors do not agree with this scenario because the introduction of SSL processing has additional performance overhead. So in the case of determining the flow, assuming that the original M-HAProxy, after the SSL is switched on, M + N units may be required HAProxy. As traffic grows, the scale-out cost of this approach is high (after all, SSL certificates are charged by the number of servers). The solution he gave was to separate the layer of SSL proxy caches as follows.

The Independent SSL proxy cache layer between L4 and L7 is only responsible for processing the SSL protocol, converting HTTPS to HTTP, and checking if the local cache is hit. The Load balancer layer is applied to the L7 layer of the backend if it misses the forwarding request to the back end. The benefit is that each level can be scaled independently based on traffic, and the SSL layer can obviously be shared across multiple applications, saving cost. If we re-adjust the load-balancing hierarchy in front of us in this way, it will evolve into the following.

In fact, at this point I think the application of the front layer of Nginx may seem superfluous, not necessary. But if the reality of this evolution is likely to have such a layer of redundant things for a long time, this is the gap between ideal and reality.

Summarize

Okay, here's the end of this article. As a background development I actually do not know how to configure, tune and manage the various types of open source software mentioned above, which belongs to the problem domain category of operation and maintenance development. But this does not prevent me from understanding the entire environment in which I am developing the application, knowing more about what and why outside the boundaries of your area of work, and sometimes helping us to better design and solve problems within our own problem areas, and not to limit ourselves and ultimately restricting.

Originally thought that the load balance this ancient topic has stereotyped, in writing this article also sees the news, in recently held the 13th session network system design and realizes the USENIX Seminar, the engineer from Google also shared its self-developed Maglev load balancer. Just under the paper has not looked, looked back to write.

Load Balancing Detailed Talk