Website construction-horizontal database Scaling

During the construction of the website, a highly scalable architecture design ensures that the existing system can be expanded at a low cost when the website business volume and access volume reach zero threshold. The most difficult part of the website system is database or persistent storage. The most popular RDBMS databases today, such as Oracle and SqlServer,

During the construction of the website, a highly scalable architecture design ensures that the existing system can be expanded at a low cost when the website business volume and access volume reach zero threshold. The most difficult part of the website system is database or persistent storage. The most popular RDBMS databases today, such as Oracle and SQL Server,

Horizontal database Scaling

During website construction, a highly scalable architecture design ensures that the existing system can be expanded at a low cost when the website's business volume and traffic reach zero threshold. The most difficult part of the website system is database or persistent storage. The most popular RDBMS databases, such as Oracle and SQL Server, are relational databases. Most databases have ACID-based foreign keys, as shown in the following table,

Attribute

Description

Atomicity)

Either all the operations in the transaction are completed, or none of them are executed.

Consistency)

When the transaction starts and ends, all data in the database must be in the same state.

Isolation)

A transaction is like a unique operation performed on a database.

Duration)

When the transaction is completed, the operation cannot be changed

No matter which type of foreign key is used, the principle is that as long as the write and read operations occur at the same time, there will be two situations,

The read data is different from the real data because the read data has changed;

The read and write operations must not take place at the same time. The operation can only be performed after another operation is completed;

Based on this principle, when the number of website visits reaches a certain level, the data may be significantly different or the operation wait time may be too long depending on the ACID foreign key attribute used, therefore, we must expand the database to solve this problem.

In more than 90% of website construction projects, database read operations are far greater than write operations, especially for enterprise website production, 99% of which are read operations, the database is written only when the Administrator adds, deletes, modifies, and queries the website information. For such websites with a high read/write ratio, you can easily expand these systems by creating read-only database copies. This technology of Horizontally scaling databases by creating database copies is called "read/write splitting ".

Time sensitivity

Based on the time sensitivity of data, there are two ways to distribute read-only copies of data. What is time sensitivity? Why do we care about it? The time sensitivity refers to the number of new read-only copies compared to the database's write copy, or full synchronization. For read/write splitting databases, full synchronization of read/write copies can be achieved theoretically, but the cost is very high. For stakeholders, this is often not the cost-effectiveness they want. Therefore, we will discuss how much data difference you can tolerate.

Query Cache

This method creates a cache layer on the data or application layer. Each query can first check whether an object already exists in the cache, rather than reading the database every time. After a write operation, the data must be marked as expired. The transaction database needs to be queried to obtain data and update the cache for the next read. There are so many excellent key-value storage systems that can be used as cache systems, such as memcache, redis, and couchbase. Therefore, we strongly recommend this method first.

Cache Mechanism

Copy Database

Apart from creating a cache layer, you can also split data by copying a database. Most relational databases have some replication functions. SQL Server implements the replication function through the master-slave Database concept. The so-called primary database is a database that only performs write operations, and the slave database is a read-only copy of the primary database. The primary database records updates, inserts, deletes, and other operations in binary logs. Each slave database requests these logs from the primary database and repeat these operations on its own. For example,

Database read/write splitting

As mentioned above, in more than 90% of website construction projects, database read operations are much larger than write operations, which means we can use such database extensions, A small number of write operations are completed in a primary database, while a large number of read operations are undertaken by multiple slave databases, which can allocate User read operations to multiple slave databases, this greatly reduces the burden on a single database. Theoretically, you can use the unlimited expansion of the number of databases to adapt to the infinite increase in Website user traffic.

Most website construction projects can use these two methods to enhance website scalability, not just databases, but web servers can also be easily expanded through similar principles, in this way, the transaction can be evenly distributed to multiple servers for horizontal scaling. Unfortunately, there is almost no existing site building system or site building software native support for such horizontal expansion. programmers must perform secondary development on the Code and database before implementation.

However, this method is not applicable to some data that must be synchronized in real time. Caching customers' private data in the memory or special data of different functions may cause expansion service bottlenecks, and it is difficult to expand these services without affecting the customer response time. To solve this memory limit, we need to extend the system in depth.