P9 Architect explains the evolution of large Web site system architectures from single-machine to billion-tier traffic

Stage one, single-machine building web site

At the beginning of the site, we often run all our programs and software on a single machine. At this point we use a container, such as Tomcat, Jetty, Jboos, and then use the Jsp/servlet technology directly, or use some open-source frameworks such as Maven+spring+struct+hibernate, maven+spring+ Springmvc+mybatis; Finally, select a database management system to store data, such as MySQL, SQL Server, Oracle, and then connect and manipulate the database through JDBC.

All of the above software loaded on the same machine, the application ran up, but also a small system. At this point the system results are as follows:

Phase II, Application server and database separation

With the site on-line, the number of visits gradually increased, the server load slowly increased, when the server is not overloaded, we should be ready to improve the load capacity of the site. If our code level has been difficult to optimize, without improving the performance of a single machine, the increase of the machine is a good way, not only can effectively improve the load capacity of the system, but also cost-effective.

What is the added machine used to do? At this point, we can split the database, Web server, which not only improves the load capacity of a single machine, but also improves the capacity of disaster tolerance.

After the application server is separated from the database, the schema is as follows:

Phase III, Application server cluster

As traffic continues to increase, a single application server is already unable to meet the requirements. Assuming there is no pressure on the database server, we can turn the application server from one to two or more, spreading the user's requests across different servers, thereby increasing the load capacity. There is no direct interaction between multiple application servers, and they are dependent on the database for their external service. Famous for failover software has keepalived,keepalived is a similar to the Layer3, 4, 7 switch system software, he is not a specific software failover of the exclusive product, but can be applied to a variety of software products. Keepalived with the Ipvsadm can also do load balance, it is an artifact.

We have added an application server as an example, the additional system structure diagram is as follows:

As the system evolves here, the following four questions will appear:

Who forwarded the user's request to a specific application server

What is the forwarding algorithm?

How the application server returns a user's request

How to maintain session consistency if the user is not the same server each time they access it

Let's look at the solution to the problem:

1, the first problem is the problem of load balancing, there are generally 5 kinds of solutions:

1, HTTP redirection. HTTP redirection is the application layer's request forwarding. The user's request has actually been to the HTTP redirect Load Balancer server, the server requests the user to redirect according to the algorithm, the user receives the redirect request, asks the real cluster again

Pros: Simple.

Disadvantage: poor performance.

2, DNS domain name resolution load balancer. DNS domain name resolution load balancing is when a user requests a DNS server to obtain the IP address of the domain name, the DNS server directly gives the server IP after the load balancer.

Pros: To DNS, we do not need to maintain the Load Balancer server.

Cons: When an application server hangs, unable to notify DNS in a timely manner, and DNS load balancing control in the domain Name service provider, the site can not do more improvement and more powerful management.

3, reverse proxy server. When the user's request arrives at the reverse proxy server (which has arrived at the Web site), the reverse proxy server is forwarded to the specific server based on the algorithm. Common Apache,nginx can act as a reverse proxy server.

Pros: Simple deployment.

Cons: Proxy servers can be a bottleneck for performance, especially when uploading large files at once.

4, the IP layer load balancing. After the request arrives at the load balancer, the load balancer implements the request by modifying the destination IP address of the request, thus enabling the load balancer to be forwarded.

Advantages: Better performance.

Disadvantage: The bandwidth of the load balancer becomes a bottleneck.

5, Data link layer load balancing. After the request arrives at the load balancer, the load balancer does load balancing by modifying the requested MAC address, unlike the IP load balancer, which returns the customer directly after the request has been made to access the server. Without having to go through the load balancer again.

2, the second problem is the cluster scheduling algorithm problem, the common scheduling algorithm has 10 kinds.

1, RR polling scheduling algorithm. As the name implies, poll the distribution request.

Advantages: Simple Implementation

Cons: Regardless of the processing power of each server

2, WRR weighted scheduling algorithm. We set the weight value for each server, the load Balancer Scheduler weight The server according to the weights, the number of times the server is called is proportional to the weight value.

Pros: Consider the difference in server processing power

3, SH original address hash: Extract the User IP, according to the hash function to derive a key, and then according to the static mapping table, the corresponding value, that is, the target server IP. The target machine is overloaded, the return is empty.

4, DH target address hash: Ibid, just now extract the destination address of the IP to do the hash.

Advantages: Both of the above algorithms can achieve the same user access to the same server.

5, LC Minimum connection. Prioritize requests to servers with few connections.

Advantage: The load on each server in the cluster is more uniform.

6, WLC weighted least connection. On the basis of the LC, weights are added to each server. The algorithm is: (Number of active connections *256+ inactive connections) ÷ weight, the computed value of a small server priority is selected.

Advantage: You can assign requests based on the capabilities of the server.

7. The shortest expected delay of SED. In fact, SED is similar to WLC, except that the number of inactive connections is not considered. The algorithm is: (the number of active connections +1) *256÷ weight, the same calculated value of the small server priority is selected.

8, NQ never queue. Improved SED algorithm. We think about the situation in order to "never queue", that is, the number of connections to the server is 0, then if there is a server connection number of 0, the equalizer directly forwarded the request to it, without the SED calculation.

9, LBLC based on the least local connection. Based on the destination IP address of the request, the equalizer finds the server that the IP address was recently used, and forwards the request, if the server is overloaded, the least number of connection algorithms are used.

10, LBLCR with replication based on the least local connection. Based on the destination IP address of the request, the equalizer finds the most recently used "server group" of the IP address, noting that it is not a specific server, and then uses a minimum number of connections to pick out a specific server from the group and forward the request. If the server is overloaded, then based on the least connection number algorithm, locate a server in the cluster's non-server group server, join the server group, and then forward the request.

3, the third problem is cluster mode problem, general 3 kinds of solutions:

1, NAT: The load balancer receives the user's request, forwards to the specific server, the server processes the request to return to the equalizer, the equalizer returns to the user again.

2, DR: Load balancer receives the user's request, forwards to the specific server, the server comes out to play the request to return directly to the user. Requires the system to support IP tunneling protocol, difficult to cross the platform.

3, TUN: Ibid, but no IP tunneling protocol, good cross-platform, most of the system can support.

4, the fourth question is the session question, generally has 4 kinds of solutions:

1, Session Sticky. Session sticky is the same user in a certain session of the request, are assigned to a fixed server, so we do not need to solve the cross-server session problem, the common algorithm has the Ip_hash method, that is mentioned above two hashing algorithm.

Pros: Simple to implement.

Disadvantage: The session disappears when the application server restarts.

2, Session Replication. Session replication is to replicate the session in the cluster, so that each server is saved with the session data of all users.

Pros: Reduce the pressure on the Load Balancer server and do not need to implement the ip_hasp algorithm to forward requests.

Disadvantage: High bandwidth overhead when copying, large amount of access, the session takes up large memory and is wasted.

3, session data centralized storage: Session data centralized storage is the use of database to store session data, to achieve the session and application server decoupling.

Pros: Compared to the session replication scheme, there is a lot less pressure on the bandwidth and memory between clusters.

Disadvantage: The database where the session is stored needs to be maintained.

4, the cookie Base:cookie Base is the existence of the session cookie, there is a browser to tell the application server my session is what, the same implementation of the session and application server decoupling.

Advantages: Simple to achieve, basic maintenance-free.

Cons: Cookie length limitation, low security, broadband consumption.

It is worth mentioning that:

The load balancing algorithms currently supported by Nginx are WRR, SH (consistent hash supported), fair (I think it can be attributed to LC). However, Nginx can be used as a static resource server together with the equalizer.

Keepalived+ipvsadm is more powerful, currently supported algorithms are: RR, WRR, LC, WLC, LBLC, SH, dh

Keepalived supported cluster modes are: NAT, DR, TUN

Nginx itself does not provide a session synchronization solution, while Apache provides session sharing support.

Well, after solving the above problems, the structure of the system is as follows:

Phase Iv. separation of database Read and write

Above we always assume that the database load is normal, but with the increase of traffic, the load of the database is also increasing slowly. Then maybe someone immediately thought of the same as the application server, the database is a two load balance. But for the database, it's not that simple. If we simply split the database into two, and then the database requests, respectively load to a machine and B machine, then it is obvious that the two database data is not uniform problem. So in this case, we can first consider the use of read-write separation method.

The database system structure after read and write separation is as follows:

This structural change will also bring two problems:

Synchronization of data between master and slave databases

Application selection issues for the data source

Problem Solving Scenarios:

We can implement master-slave replication using MySQL's own master+slave.

Use third-party database middleware, such as Mycat. Mycat is from the Cobar development, and Cobar is Ali Open source database middleware, and later stopped development. Mycat is a good domestic MySQL open source database sub-Library sub-table middleware.

Stage five, using search engine to alleviate the pressure of reading library

Database to do reading library, often to fuzzy search, even if the separation of reading and writing, the problem has not been solved. As an example of our trading website, the products are stored in the database, and the most common function of the user is to find the product, especially according to the title of the product to find the corresponding product. For this kind of demand, we are usually implemented by the like function, but the cost of this method is very great. At this point we can use the search engine inverted index to complete.

Search engines have the following advantages:

It can greatly improve the query speed.

The following costs are also incurred when the search engine is introduced:

Bring a lot of maintenance work, we need to implement the index building process, design a total/incremental build way to deal with non-real-time and real-time query requirements.

Need to maintain search engine clusters

Search engine does not replace the database, he solved some scenarios under the "read" problem, whether to introduce a search engine, need to comprehensively consider the needs of the entire system. The system structure after the introduction of the search engine is as follows:

Phase VI, buffer to relieve the pressure of reading library

1, background application layer and database layer cache

As the amount of traffic increases, there is a growing number of users accessing the same part of the content, and it is not necessary to read from the database every time for these more popular content. We can use caching techniques, such as using Google's Open source caching technology guava or using Memcacahe as the application tier cache, or using Redis as the database tier cache.

In addition, in some scenarios, the relational database is not very suitable, for example, I want to do a "Daily password error limit" function, the idea is probably when the user logged in, if the login error, the user's IP and error, then the number of records, then where is this data? If put in memory, then obviously occupy too much content, if put in relational database, then need to build database table, resume corresponding Java bean, write SQL and so on. The analysis of the data we want to store is nothing more than a key:value data like {Ip:errornumber}. For this type of data, we can use NoSQL database instead of traditional relational database.

2. Page Cache

In addition to the data cache, there are page caches. For example, use HTML5 's localstroage or cookies.

Advantages:

Reduce the pressure on the database

Dramatically increase access speed

Disadvantages:

Need to maintain cache server

Increased complexity of coding

It is worth mentioning that:

The scheduling algorithm for the cache cluster differs from the application server and database mentioned above. It is best to use a "consistent hashing algorithm" to improve the hit ratio. This will not start speaking, interested in the relevant information can be consulted.

The structure after the cache is added:

Phase VII, database horizontal split and vertical split

Our website has evolved to the present, the data of the transaction, the commodity, the user is still in the same database. Although the way to increase the cache, read and write separation, but as the pressure of the database continues to increase, the database bottleneck more and more prominent, at this time, we can have data vertical split and horizontal split two choices.

7.1. Vertical data splitting

Vertical splitting means separating different business data from the database into different databases, and the current example is to separate the transactions, the goods, and the user's data.

Advantages:

Solves the stress problem of putting all the business in a database.

Can be more optimized according to the characteristics of the business

Disadvantages:

Need to maintain multiple databases

Problem:

Need to consider the original cross-business transactions

Join across a database

Problem Solving Scenarios:

We should try to avoid cross-database things at the application level, and if you want to cross-database, try to control it in code.

We can use the third party to resolve, as mentioned above Mycat,mycat provides a rich cross-Library join scheme, details can refer to Mycat official documentation.

The structure after the vertical split is as follows:

7.2, the level of data splitting

Data split horizontally splits the data in the same table into two or more databases. The reason that the data is split horizontally is that the amount of data in a business or the amount of updates reaches the bottleneck of a single database, and this table can be split into two or more databases.

Advantages:

If we can support the above issues, then we will be able to do a good job of increasing the amount of data and write volume.

Problem:

The application system that accesses the user information needs to solve the problem of SQL routing, because now the user information is divided into two databases, need to know where to operate the data in the data operation.

The processing of the primary key is also different, such as the original from the Add-on field, which cannot now be simply used.

If you need paging, it's a hassle.

Problem Solving Scenarios:

We can still solve third-party middleware, such as Mycat. Mycat can parse our SQL through the SQL Parsing module, and then forward the request to a specific database based on our configuration.

We can use the UUID to guarantee a unique or custom ID scheme to resolve.

Mycat also provides a rich paging query scheme, such as first paged query from each database, and then merge data to do a paged query and so on.

Structure after the data has been split horizontally:

Phase VIII, application of the split

8.1. Split Application

With the development of business, more and more business, the application is more and more large. We need to think about how to avoid making apps more bloated. This requires the application to be disassembled, from one application to two or more. Or in our example above, we can split the user, the goods, the transaction. Become a "user, commodity" and "user, trade" two subsystems.

Post-split structure:

Problem:

After this split, there may be some of the same code, such as user-related code, goods and transactions require user information, so in both systems to retain the same code to manipulate user information. How to ensure that the code can be reused is a problem that needs to be solved.

Solve the problem:

By taking a service-based route to solve

8.2, take the road of service

In order to solve the problem of the above split application, we split the public services into a service-oriented model called SOA.

Using the system structure after service:

Advantages:

The same code will not be scattered across different applications, and these implementations are placed in various service centers, allowing the code to be better maintained.

We put the interaction of the database in the various service centers, so that "front-end" Web applications more focused on the work of interacting with the browser.

Problem:

How to make a remote service call

Workaround:

We can use the following to introduce the message middleware to solve

Phase nine, the introduction of message middleware

As the site continues to evolve, sub-modules that are developed in different languages and subsystems deployed on different platforms may appear in our systems. At this point we need a platform to pass reliable, platform-and language-independent data, and can be load-balanced transparent, can be called in the process of collecting call data and analysis, speculate on the site's access rate and so on a series of needs, how the site should grow to make predictions. The open source message middleware has Ali's Dubbo, can match the Google Open Source distributed program Coordination Service zookeeper realizes the server registration and the discovery.

Structure after introduction of message middleware:

X. SUMMARY

The evolution of the above is only an example, not suitable for all sites, the actual site evolution process and its own business and different problems have a close relationship, there is no fixed pattern. Only serious analysis and continuous exploration, to find the right structure for their own site.

P9 Architect explains the evolution of large Web site system architectures from single-machine to billion-tier traffic