"Go" how to build a highly scalable Web site?

This article summarizes the following by reading the 50 principles of the High Extensibility website.

On the one hand Bo Master no actual architectural experience, on the other hand, the knowledge is not broad enough, so can only summarize the main points of the book, and according to their own understanding to do some induction.

Main content

This book from a number of aspects of the high scalability of the 50 recommendations, a highly scalable web site with the development of business, users, the free expansion of the architecture, so as to easily cope with the rapid development of the site. Here's a look at the details of this book:

Equation of simplification

1 Don't overdo the design

Excessive design is equivalent to adding complexity and maintenance costs to the system. But these excessive design, in the normal use, but does not have the very big function. is often the designer's own thought is very important or icing on the cake function, the actual use is not big.

2 Design with extensibility in mind

In the design to follow the design principle: 20 times times the design of capacity, the implementation of the 3 times-fold capacity, consider the deployment of 1.5 capacity. When a project expands, the temporary expansion creates difficulties.

3 simplification and simplification of the scheme

Should follow the Pareto law, 20% of the design did 80% of the work, so 80% of the time, should be placed on the 20% design.

The main function of a product is in fact focused on a few points, the design of these points, and others are some additional functions. So the core business must be simple enough to ensure that it is easy to use.

4 Reducing DNS queries

Each file under a different domain needs to be queried for DNS when it is loaded. For example, cnblogs.com and i.cnblogs.com belong to different domains. Then when querying DNS, it will be queried two times. When the volume of business is large, it will have a certain impact.

5 Minimizing objects

The object needs to be loaded when it is accessed by the browser. So consider reducing the number of requested files (the number is related to the number of concurrent browser loads) and merging some objects as much as possible. Compared to the standard class of files, can be combined into a large picture. A reasonable number of files will speed up the browser's access load.

6 using the same brand of network equipment

Due to an HTTP request, many physical devices may be passed. such as load balancers, switches, routers. So try to use the same brand of equipment, will avoid some unexpected situation.

Distribution work

7 x axis, transverse copy

This is the simplest kind of service extension that is implemented through cloning or replication, such as your application being placed on multiple servers for service. Common such as clustering, load balancing and so on, the database read and write separation.

8 y-axis, split different things

In large-scale systems, different functions are split, such as registering, purchasing, querying, and cloud disks. Wait a minute

9 Z-axis, split different similar things

For example, according to the user level, or the user's geographical location and so on split.

Scale-out design

10 Design Scale-out solutions

Extensions include landscape, portrait. Horizontal is through the replication of cloning applications, the use of minicomputer cluster expansion. Vertical is to improve the server hardware and network facilities.

Through a lot of cases can be found that the simple upgrade of hardware to achieve vertical expansion, only to solve a little realistic pressure. And through the horizontal cluster expansion, but can be free to achieve scaling.

11 Adopt Economical System

Similar to the above principles, the use of high-priced servers does not guarantee good performance in the future. The normal minicomputer cluster extension should be used.

12 scale-out data center

Data centers have a lot of design options, such as

Hot-Cold Station configuration: Use the Heat station to provide service, when the heat station crashes, use the cold station to continue service.

It is recommended to use multiple live sites with lower cost and dynamic invocation. The disadvantage is that it increases the difficulty of operation and maintenance.

13 Design with Cloud technology

Cloud computing is a bit of virtualization, which can be a resilient expansion device when business peaks. And in daily processing, return the extension.

The disadvantage is that it improves the coupling applied to the virtual environment. The subsequent mention of the use of physical devices to isolate the business, in virtualized cloud computing, may cause some disruption to business isolation errors.

Use the right tools

14 Rational use of the database

There are many database versions, such as traditional relational database Oracle, MYSQL, and newer non-relational database nosql, such as MongoDB, Memory Database Fastdb, and aerospike for SSD SSDs, and so on.

But to the selection of the time, or to a personal business needs to be determined. See if your database requires speed, security, and so on.

15 firewalls, all over the firewall.

Firewalls can intercept and filter for invalid access. Usually put some CSS, static files, pictures, JS etc do not use firewalls, and the key business involves personal information when used. Reasonable design of the firewall, will also have a certain impact on the performance of the site.

16 active use of log files

Monitor your business in real time with a variety of logs and tools. It is not just monitoring the server's memory CPU, it should also monitor the data on the business. such as Splunk (provide log collection, storage, search, graphical display).

Don't do repetitive work.

17 Do not immediately check the work you have just done

For example, just write the data, do not read it immediately. Although some customers need to ensure the integrity of the data, can not be lost. But it can be done through the log and other records, written to check this practice, or not recommended.

18 Stop Redirection

Redirection consumes a certain amount of delay to compute resources. Should try to avoid

19 Relaxation timing constraints

Most relational databases stress acid properties, which can cause a certain amount of distress when extended. Therefore, some business-appropriate relaxation of timing constraints can improve the performance of the site.

For example, a station in the booking hotel, the user after booking, will wait for the hotel audit. For example, a treasure, in the withdrawal, the scope of time to confirm. This is the expansion of timing constraints, which in turn improves website performance and transaction security.

Active use of caching

20 Leveraging CDN

You can use a CDN to save your customers ' data and content. The general process is that when the user visits the website, they go to the CDN server, the CDN performs DNS queries, and the user requests are distributed to different servers. There are a number of CDN service providers that provide such services.

21 Using Expired Headers

For different object types, use expired headers to reduce object requests. Common HTTP Correspondence properties are: public No-cahe max-age, etc.

22 Caching Ajax Calls

Correctly modify the HTTP header last-modified Cache-control expires and other properties.

23 using Page Caching

Cache response before the winter request, reducing the load on the Web server.

24 using the app cache

For some special users, for example, cache their request data.

25 Using Object Caching

Applies to data objects that are used repeatedly for querying. such as a shopping site, cache hot product data.

26 placing the object cache on its own layer

Easy to scale and maintain with a separate ease layer.

Learn from your mistakes

27 Positive Learning

A company has a learning atmosphere, it will be derived from a better product. The content of the study includes the customer's business knowledge, on the one hand from the technical and operational fields.

28 don't rely on QA to find fault

Employment testing or quality assurance personnel, the greatest purpose is to detect the correctness of the product. It reduces costs and increases developer speed because developers do not have to be concerned with the correctness of the code and can be passed on to QA for testing.

But QA is only responsible for discovering problems, how to avoid them or to rely on developers.

29 Design Without fallback is a failed design

The fallback here refers to the fallback of the product release. If you run into some version of the bug, you may need to deliver the previously available version, and you will not be able to deliver the product without a fallback at this time.

It is recommended to learn about continuous integration.

30 discuss failure and learn from it

Should not fail on the same issue two times, each failure more summary is indispensable.

Database principles

ACID Properties for relational databases:

Atomicity: A transaction is either fully executed or not executed,

Consistency: When transactions start and end, all data states are consistent,

Isolation: The performance of a transaction is the only operation that the transaction has on the database,

Persistence: The transaction is complete and the operation cannot be changed.

31 pay attention to high-cost relationships

The structure of the design table should be perfected at the design stage, and when development starts, it can be costly to add some columns.

32 using the correct database lock

The database has many lock concepts, such as implicit lock, explicit lock, row lock, page lock, range lock, table lock, Database lock, and so on.

Unreasonable use of locks can affect the throughput of the site.

33 Do not use multi-phase commit

For example, two phase commit: vote first, submit. This reduces extensibility because there is no other action to make until its commit transaction is complete.

34 Do not use the Select for update

Because the FOR UPDATE clause causes the row to be locked, the transaction speed is reduced.

35 do not select all the data

such as SELECT * from XXX;

The first is not open with the expansion of data, for example, there would have been four columns of data, business processing code to write directly to death. When a column of data is added, an error occurs, and unnecessary data is queried.

or inset into XXX values (xxxx);

This is also an error when the column information does not match.

Fault-tolerant design and fault control

36 "Swimlane" with isolated faults

There are many kinds of services and data partitioning, such as containers, clusters, pools, shards, and lanes. Swimlane means that each business has its own domain and cannot be called across lanes.

37 Do not trust single point of failure

There are many systems designed to be single-point mode, when the whole system just use the module, when there is a single point of failure, the entire system will collapse.

38 Avoiding system concatenation

For example, a system has a lot of components, each component 99.9% security, when the concatenation of 3 components, the overall system availability becomes 99.7%.

39 to enable/disable features

For some shared libraries, third-party services should provide the ability to turn on or off.

Avoid or Distribute status

40 Striving for Statelessness

Implementation status limits scalability and increases costs

41 maintaining sessions as much as possible on the browser side

On the one hand, reduce the server pressure, on the other hand any request can be sent to any server.

42 storing state with distributed cache

Use a separate cache layer to facilitate scaling. There are many distributed caching schemes, such as memcached.

Asynchronous communication and message bus

43 using asynchronous communication whenever possible

Asynchronous communication ensures the independence of each service and layer, so that it is easier to grow early and to reduce coupling.

44 Ensure that the message bus can be extended

Use the y-axis or z-axis extension as much as possible, expanding by business requirements and capabilities. Because a simple copy or clone, it will increase the number of listeners each message subscribers. Business pressure can be separated according to business isolation.

45 avoid getting the message bus over-crowded

Measure the value and the cost of the message.

Other principles

46 caution with third-party solution extensions

If there is a problem, then looking for a third party can solve the urgent situation. But it is not a long-term solution, because the provider of solutions has a lot of customers, your crisis is not their crisis, so it is impossible at the critical moment, dutifully. Therefore, enterprises should have a certain degree of control (the word is really tall on!) ）。

47 Purge, archive, and cost-appropriate storage

There are some unnecessary data that should be deleted on a regular basis. Some of the slightly valuable data is periodically archived for immediate deletion. Some very valuable data that should be backed up and quickly accessed.

48 removing business intelligence from transactions

The product system should be separated from the business system to improve the extensibility of the product.

When you avoid business expansion, you are constrained by the system architecture.

49 designed to monitor the application

The overall monitoring strategy should be designed to ensure that the answer

"Is there a problem?" “

"Where has the problem happened?" “

"What happened?" he went on. “

"Is there a problem?" “

"Can I fix it automatically?" “

50 to be competent

The best architectures should be involved in each design and not rely entirely on third-party solutions.

A simple and excellent architecture, are small and refined, if simply rely on open source solution architecture, although solve the problem, it will lead to the application of bloated.

Reference

"1" The 50 principles of the High-extensibility website

Original: http://blog.jobbole.com/86118/

"Go" how to build a highly scalable Web site?