Starting from the evolution of website architecture

 

 

Some time ago I saw the evolution and knowledge system of haowen's large-scale website architecture, and told me about the evolution from a simple website to a large complex website architecture:

The first step in Architecture Evolution: physically separating webserver and database;

Step 2 of Architecture Evolution: add page cache; involve these knowledge systems:
Front-end page cache technology, such as squid. If you want to use it well, you must have a thorough understanding of squid implementation methods and cache failure algorithms.

Step 3 of Architecture Evolution: add page fragment cache; involve these knowledge systems:
The page fragment cache technology, such as ESI, also needs to master the implementation method of ESI if you want to use it well.

Step 4 of Architecture Evolution: data caching; involves these knowledge systems:
Cache Technology, including Map data structures, cache algorithms, and implementation mechanisms of the framework itself.

Step 5 of Architecture Evolution: Add webserver; involve these knowledge systems:
Server Load balancer technology (including but not limited to hardware Server Load balancer, software Server Load balancer, load algorithms, linux forwarding protocols, and implementation details of the selected technology) master-slave technology (including but not limited to ARP spoofing and linux heart-beat), status information or cache synchronization technology (including but not limited to Cookie technology, UDP protocol, status information broadcast, the implementation details of the selected cache synchronization technology, etc) shared File technology (including but not limited to NFS) and storage technology (including but not limited to storage devices ).

Step 6 of Architecture Evolution: Database sharding; involves these knowledge systems:
This step requires a reasonable division of the business to achieve database sharding. There are no other requirements for specific technical details;
However, with the increase in data volume and database sharding, database design, optimization, and maintenance must be improved. Therefore, high requirements are raised for these technologies.

Step 7 of Architecture Evolution: Table sharding, DAL, and distributed cache; these knowledge systems are involved:
Table shards are also business partitions. Technically, dynamic hash algorithms and consistent hash algorithms are involved;
Dal involves many complex technologies, such as database connection management (timeout and exception), Database Operation Control (timeout and exception), and database/table sharding rule encapsulation;

Step 8 of Architecture Evolution: add more webservers and involve these knowledge systems:
At this point, as the number of machines continues to grow, the amount of data continues to grow, and the requirements for system availability are getting higher and higher, we need to have a deeper understanding of the technology we are using, more customized products are required based on website requirements.

Step 9 of Architecture Evolution: data read/write splitting and low-cost storage solutions; these knowledge systems are involved:
Data read/write splitting requires an in-depth understanding of database replication, standby, and other strategies, and requires self-implemented technologies;
The low-cost storage solution requires a deep understanding and understanding of OS file storage, and a deep understanding of the implementation of the language used in the file.

Step 10 of Architecture Evolution: enter the era of large-scale distributed applications and the dream age of cheap Server clusters. There are many knowledge systems involved, requires an in-depth understanding and understanding of communications, remote calls, and message mechanisms, all requirements are clearly understood in terms of theory, hardware, operating system, and language.
O & M involves many knowledge systems. In most cases, you need to master distributed parallel computing, reports, monitoring technology, and rule policies.

 

The knowledge points mentioned in the article:

Squid Chinese authoritative guide

ESI meets the challenge of dynamic web content

Cache Technology

Introduction to the cache technology of ASP. NET 2.0