Pay attention to scalability when designing Web Applications

Max indelicato, a software development director and former chief software architect, recently published an article on how to design scalable Web applications. He proposed to select the right deployment and storage solution, select Scalable Data Storage and mode, and use the abstraction layer.

Suitable tools

The first recommendation of indelicato is to "select the right tool for your work". To achieve this, select one of the following Architecture solutions:

• Use cloud Deployment Solution
• Use Scalable Data Storage solutions such as MongoDB, couchdb, Cassandra, or redis.
• Add a cache layer, such as memcached.

He believes that these solutions are not necessary at the beginning of application development, but it is wise to choose a Scalable Data storage solution at the beginning, because it will avoid switching later. Deploying applications in the cloud brings us some benefits, especially for startups, because they cannot accurately determine how many people will use their applications after they are enabled. After deploying the application department to the cloud, the application can be elastically scaled as demand increases. Many software architects have told us that they have to scale their applications, and they will introduce high-speed slowdown.
Storage layer, which solves most problems. However, we should consider the corresponding solutions at the design stage. This is easy to implement later.

Scalable Data Storage

Next, we recommend that you select data storage that supports partitioning, replication, and elasticity, including MongoDB, Cassandra, redis, Tokyo cabinet, project Voldemort, or MySQL as a relational database. This is necessary, because in any case, partitions are necessary in the application lifecycle. For scalability, partitioning is not necessary, but it is necessary to "ensure high availability. Flexibility allows us to quickly add more nodes. This may be due to traffic peaks or hardware faults or upgrades.
In the case of changes to the scaling mode of a level or a large scale or any need to bring the node offline, maintenance is required for the node ."

Scalable Data Mode

We recommend that you create a mode for indelicato so that we can easily perform data sharding. It also provides the following example of temporary components: user and userfeedentry:

Collection (or table, or entries, etc) User {userid: guid, unique, key Username: String passwordhash: String lastmodified: Timestamp created: Timestamp} collection (or table, or entries, etc) userfeedentry {userfeedentryid: guid, unique, key userid: guid, unique, foreign key body: String lastmodified: Timestamp created: Timestamp}

Then we recommend that you partition by userid:

By partitioning the user set and userfeedentry set based on The USERID field, we place two related data blocks on the same node. All userfeedentry data with userid XXX-xxx and user data with userid XXX-xxx will be contained in the same data segment.

Why is this scalable? Because our requirements for this application are completely for data distribution. When each visitor accesses the user information page, the system sends a request to the data segment to obtain the detailed information of the user in the user column, then, send a request to the same data segment to obtain the user's userfeedentries. One of the two requests will obtain one piece of data, and the other will obtain multiple pieces of data, which are included in the same data segment. Suppose that most users have the same number of accesses to the information during the day, we have designed a scalable mode that will support the needs of our web applications.

Use Abstraction Layer

The last suggestion of indelicato is to use one of the following abstract layers, but not limited to these: metadatabase (repository), cache, and service. When creating the metadatabase layer, we recommend that you:

Do not name a method that is unique to your abstract data storage. For example, if you abstract a relational database, we generally define select (), insert (), delete (), and update () functions to execute SQL queries and commands. Do not do this. Instead, you should make your function names less specialized. You can use fetch (), put (), delete (), and replace (). This will make sure that you better follow the metadatabase mode and work easier when you need to switch to the underlying database.

If possible, use interfaces (or abstract classes) to pass these interfaces to a higher layer in the application, so that you will never directly reference the specific inherent Implementation of the metabase. This is also great for building and unit testing, because you can write other inherent implementations that carry data related to test cases in advance.

Encapsulate all the special code for storage into a class (or module, etc.), and the real metabase will reference or inherit it. Only the necessary details (query statements and so on) are placed in each function ).

Remember that not all metabases need to abstract the same data storage solution. As long as you want, you can store users in MySQL, and store userfeedentries in MongoDB. metabases must be implemented in this way. They support this without incurring too much cost. The previous three suggestions also indirectly help us to achieve this.

Indelicato said that for the cache layer, at the beginning, it often uses "simple page (or view, etc.) cache or service layer cache, this is because two regions do not change their status frequently."

Indelicato considers that the service layer needs to be sufficiently abstracted, so that when the demand increases, we can easily switch from the internal implementation of the Service to the implementation outside the process.

Some people think that scalability does not need to be considered when building an application, because it will be emphasized when necessary. But if we want to consider scalability from the very beginning, do you have any suggestions?