Web site Architecture Evolution of frontend learning HTTP

Previous words

This article will describe in detail the evolution of the Web site architecture

Initial stage

Large websites are developed from small Web sites, and the same is true of Web architectures, which evolve from small site architectures. Small sites initially do not have too many people to access, only a single server is more than enough, then the application, database, files and other resources are on a single server. Usually the server operating system uses Linux, the application uses PHP development, then deploys on Apache, the database uses the MySQL, gathers the various free open source software and a cheap server to begin the website development path.

Separation phase

With the development of the website business, a server is gradually unable to meet the demand: more and more user access leads to worse performance, and more and more data leads to insufficient storage space. Then you need to separate the application from the data. After the application and data are detached, the entire site uses three servers: the application server, the file server, and the database server. These three servers have different requirements for hardware resources, the application server needs to handle a lot of business logic, and therefore need faster and more powerful CPU; The database server requires fast disk retrieval and data caching, so faster hard disks and larger memory are required, and file servers need to store a large number of user-uploaded files. So you need a bigger hard drive.

Cache phase

Site Access features follow the 28 law of real-world wealth Distribution: 80% of business access is focused on 20% of the data. Since most of the business access is concentrated on a small amount of data, if this small amount of data is cached in memory, it can reduce the access pressure of the database, improve the data access speed of the whole website and improve the writing performance of the database.

The cache used by a Web site can be divided into two types: a local cache cached on the application server and a remote cache that exists on a dedicated distributed cache server. Local caches are faster to access, but are subject to application server memory limits, with limited cache data, and scenarios where the application is competing for memory. The remote distributed cache can be clustered in a way that deploys large memory servers as a dedicated cache server that can theoretically be cached without content-constrained caching services

Cluster stage

After using the cache, the data access pressure is effectively mitigated, but the single application server can handle the limited request connection, during the peak site visit, the application server becomes the bottleneck of the whole website.

The use of clustering is a common means for web sites to solve high concurrency and massive data problems. When a server has insufficient processing power and storage space, it can share the existing server's access and storage pressure by adding a server. As far as the site architecture is concerned, as long as the load pressure can be improved by adding a single server, the server can continuously improve the system performance in the same way, thus achieving the scalability of the system. Application Server cluster is a simple and mature one in the design of scalable cluster architecture of Web site.

Load Balancer Dispatch server, can distribute the access request from user's browser to any server in the application server cluster, if there are more users, add more application servers in the cluster, and the load pressure of application server will not become the bottleneck of the whole website.

Database separation

Web site after using the cache, so that most of the data read access can be done without a database, but still have some read operation (cache access is not hit, cache expires) and all write operations need to access the database, the site after the user reached a certain size, the database because of high load pressure and become the site bottleneck

At present, most of the mainstream database provides master-slave hot-standby function, through the configuration of the network database master-slave relationship, one database server can synchronize data updates to another server. The website uses this function of the database, realizes the database reads and writes separates, thus improves the database load pressure

When the application server writes the data, it accesses the primary database, and the primary database synchronizes the data updates to the slave database through the master-slave replication mechanism, so that when the application server reads the data, it can obtain the data from the database. In order to facilitate application access to the database after read and write, it is common to use specialized data access modules on the application server side to make the database read/write separated to the application transparent

Reverse Proxy and CDN

With the continuous development of the website business, the scale of users is increasing, because of the complex network environment, different regions of the users to visit the site, the speed difference is also great. To provide a better user experience and to retain users, websites need to speed up website access. The main means are using CDN and reverse Proxy

The basic principle of CDN and reverse proxy is the cache, the difference is that the CDN is deployed in the network provider's room, so that when the user requests the website service, can obtain the data from the nearest network to provide the opportunity room, and the reverse proxy is deployed in the central room of the website, when the user requests to reach the center room, The first server to be accessed is the reverse proxy server, which is returned directly to the user if the resource requested by the user is cached in the reverse proxy server

The purpose of using CDN and reverse proxy is to return data to users as soon as possible, on the one hand, speed up user access and reduce the load pressure on the backend server.

Distributed Systems

Any powerful single server can not meet the continued growth of the business needs of large sites. After the database has been read and written separated, split from a server into two servers, but with the development of the website business still can not meet the demand, then need to use the distributed database. The same is true for file systems, which require the use of Distributed file systems

Distributed database is the last method of database splitting, only used when the scale of single table data is very large. Less than necessary, the site is more commonly used for database splitting is a business sub-library, the different business databases deployed on different physical servers

NoSQL and search engines

As the website business becomes more complex, the need for data storage and retrieval is becoming more complex, and Web sites need to adopt non-relational database technologies such as NOSQL and non-database query technologies such as search engines

Both NoSQL and search engines are technical tools derived from the Internet and have better support for scalable distributed features. The application server accesses various data through a unified data access module, alleviating the hassle of application management of many data sources

Business Split

Large web site in order to deal with increasingly complex business scenarios, through the use of divide and conquer the entire site business into different product lines, such as large shopping transaction site will be home, shops, orders, buyers, sellers and other split into different product lines, divided into different business teams responsible for

specifically to the technical, according to the product line division, a site is split into many different applications, each application independent deployment maintenance. Applications can be linked through a hyperlink (each of the navigation links on the first page points to a different app address), or through Message Queuing, or, of course, by accessing the same data storage system to form an associated complete system

Distributed services

As business splits are getting smaller, storage systems become larger, the overall complexity of application systems increases exponentially, and deployment maintenance becomes more and more difficult. Since all applications are connected to all database systems, the tens of thousands of server rules
Web site, the number of these connections is the size of the server, resulting in insufficient database connection resources, denial of service

Since every application needs to perform many of the same business operations, such as user management, commodity management, and so on, these shared services can be extracted and deployed independently. These reusable business-connected databases provide shared business services, while application systems only need to manage the user interface to perform specific business operations through distributed service invocation of shared business services

The architecture of large web sites evolved here, and basically most of the technical problems were solved. Real-time data synchronization across data centers and issues related to specific website business can also be addressed by combining improvements to existing technology architectures

But the development of things to a certain stage, will have their own development impulse, out of its original intention, towards the direction of their own more powerful development. Since large-scale web-site architectures address the management of massive data and the processing of high-concurrency transactions, these solutions can be applied to businesses outside of the site itself

At present, many large-scale web sites have begun to build a cloud computing platform, the calculation as a basic resource for sale, small and medium-sized sites do not need to care about technical architecture issues, only need to pay on-demand, you can make the site with the growth of the business gradually get more storage space and more computing resources

Resources

The core principle and case analysis of large-scale website technology Architecture Hae

Web site Architecture Evolution of frontend learning HTTP