(2) experiences in large-scale Internet website architecture: and, for example

 

Experience of large-scale Internet website architecture II: and, for other

First of all, I declare that this article was transferred from the master lovecherry. I will share it with you and give myself a learning opportunity.

 

 

The "points" mentioned last time is a relatively large principle and also a relatively high-level principle. This time I want to talk about the other two principles: and change.

 

And

 

Why? It is because we want to improve the system's load-carrying capability by dividing. What is that? I thought about the following aspects:

 

1. Merge user requests. The most basic thing is to merge CSS/images/scripts and merge pages. However, merging may result in a waste of traffic and requires a balance point.

2. the granularity of the merge interface. For distributed applications, we may not directly access the database but call the interface provided by the application layer. Because it is a network call, the cost is relatively high, therefore, when designing an interface with coarse granularity, we should try our best to provide a coarse-grained interface. A single call will return a large amount of data, instead of refining the interface to add, delete, and modify layers.

3. for the deployment of the merge interface, data redundancy can be considered for frequent cross-machine calls, so that cross-network service programming processes can communicate or even be transferred to the client. For example, you can directly call the interface provided by the application layer (cross-machine) to filter dirty words when posting a forum, but it may be costly. You can deploy this interface on the local machine using IPC.

 

Change

 

Time for space and time for space are common practices. Specifically:

 

1. cache. The importance of caching is early in computer hardware. For websites, there are many types of caches, which can be the client resource cache, the page output cache, or the application layer data cache, with the same purpose, it can reduce the number of server requests, the request processing process, or the number of database accesses. Of course, generating static files can also be regarded as a cache. It is impossible to access a disk, but we need to greatly reduce disk access opportunities.

2. Sometimes, in order to get extremely fast responses, we will use repeated computing at any cost. For example, a certain operation may slow the response due to network problems or other reasons, and a unified processing interface can be provided during the design, the interface is distributed to different servers for asynchronous implementation of this operation. When the server returns the result, we use this result and then kill the redundant operations of other servers.

3. websites generally pursue fast responses, and generally do not use time for space at a relatively high level. However, some user-specific data processing algorithms may still consider space savings.

4. Sometimes we use some aggregate tables to store aggregated data, that is, to perform some pre-computing to Improve the Performance of complex computing (such as reports. Of course, building a multi-dimensional database for data analysis is also a good choice.

 

A lot of netizens commented that there was nothing specific about it. I think it is difficult to say how to implement the architecture. Because the implementation depends on the situation and there is no perfect thing, the architecture is usually balanced, it is very likely that the implementation of the architecture will be affected by a different emphasis. I hope my articles will give you a tip. If you think "I didn't take this into consideration, but pay attention to it later", it may be the greatest help, next I would like to talk about some other problems, each of which is scattered. It is a supplement:

 

1. whether to use the existing items or do it on your own requires detailed consideration. using other things may be relatively stable, but your own control is a little less. Using your own things can be flexible, however, there may be many problems. In any case, when we adopt a third-party framework, we must conduct a thorough investigation to see its shortcomings. Otherwise, the project may be restricted by this framework in the future. Otherwise, when you decide to build a framework, you need to see what other frameworks cannot provide.

2. Data can be compressed during transmission, but the CPU resources are required for compression and decompression. There is a balance between I/O (disk, bandwidth, and transmission capacity) and CPU.

3. The ideal scalability architecture allows you to add or replace servers without downtime or great adjustments. When using a unified scheduling center to schedule these servers and allocate requests, we need to consider how much traffic the scheduling server can withstand.

4. Is there a large number of cheap servers or a small number of high-end servers? How to combine servers to maximize their role as needed.

5. For the distributed architecture, we try to keep each node simple logic and minimize the dependencies between nodes at the same level. A unified place is required to manage all nodes.

6. Function Decomposition, asynchronous integration, failover, and failure protection.

7. the architecture upgrade of software is very similar to the architecture upgrade of computer hardware. It may take some time for us to gradually improve our overall capabilities, which has increased several times in two years, then we found that we could improve our capabilities by dozens of times only through some thorough architectural changes. After the upgrade, we may encounter other problems. Like the CPU, it is a simple way to increase the clock speed or completely change the architecture.

8. Data: read/write splitting, database separation, function division, cache, and image.

9. The architecture on the hardware network is very important, but some details in software development cannot be ignored. A good architecture does not mean that code reviews are not required.

 

The writing is a bit messy. Please discuss it.