Scalable Web architecture and distributed system (I)

Open-source software has become a basic component for building some large websites in recent years. Along with the growth of websites, the best practices and guidelines around their architectures have been revealed. This article aims to address some of the key issues that need to be considered when designing large websites and some components used to achieve these goals. This article focuses on Web systems, but some of them are also applicable to other distributed systems. 1. Web distributed system design principles construction and operation 1

Open-source software has become a basic component for building some large websites in recent years. Along with the growth of websites, the best practices and guidelines around their architectures have been revealed. This article aims to address some of the key issues that need to be considered when designing large websites and some components used to achieve these goals. This article focuses on Web systems, but some of them are also applicable to other distributed systems.

1. Web distributed system design principles

What exactly does it mean to build and run a scalable website or application? From a basic perspective, it is to connect users and remote resources over the Internet-to make them scalable, these resources are distributed across multiple servers. Like a lot of things in life, it takes time to plan a web service in advance, which will help the service to run for a long time; understanding the considerations and trade-offs behind some large websites will help you make more informed decisions when creating small websites. Below are some key guidelines that will affect the design of large-scale scalable Web systems:

Availability

The normal running time of a website is crucial to the reputation and functions of many companies. For some large online retail websites, even a few minutes of unavailability may lead to tens of millions of dollars in revenue loss, therefore, designing their systems for uninterrupted availability and elastic recovery is both a basic business and a technical requirement. The high availability of distributed systems requires careful consideration of key component redundancy, fast recovery of some problematic systems, and elegant degradation in case of problems.

Performance

Performance has become an important indicator for most websites. The speed of a website affects usage and user satisfaction, as well as the search engine ranking, which is directly related to website revenue and user retention (stickiness ). Therefore, the key is to create a system with fast response and low latency.

Reliability

A system must be reliable so that requests for (fixed) data always return the same data. If the data changes or is updated, the same request should return new data. Users need to know that once some data is written and stored in the system, the system will be persistent (the data) and can be retrieved at any time with trust.

Scalability

For any large distributed system, the system scale is only one aspect of scalability. It is equally important to increase the capacity to handle the work required for a larger number of loads, which is often mentioned in terms of system scalability. Scalability involves many different factors in the system: how much additional traffic the system can process, whether it can easily increase storage capacity, and how much more transactions it can process.

Manageability

Designing an easy-to-operate system is another important consideration. The manageability of the system is equivalent to the scalability of operations: maintenance and changes. Manageability needs to be considered: when a problem occurs, it is easy to diagnose and understand, easy to change and modify, and easy to operate the system. (For example, can the system perform routine operations without causing failures or exceptions ?)

Cost

Cost is an important factor. This obviously includes hardware and software costs, but we also need to consider some other aspects to deploy and maintain the system, such as the development time required to build the system and the O & M workload required to run the system, even the required training needs to be taken into account. Cost refers to the total cost of ownership of the system (the original sentence is Cost is the total cost of ownership .).

Each of these guidelines provides the basic principles for deciding when designing a distributed web system architecture. However, they may also conflict with each other. for example, to achieve a certain goal, they sacrifice another one. A typical example: when focusing on system capacity, the cost of simply adding more machines (scalability) is (increased) manageability (you need to maintain more servers) and cost (more server prices ). When designing any web application, these key principles need to be considered. even if you have to admit that a design may sacrifice one or more of them.

2. Basic principles

For the system architecture, there are some things to consider: What are the correct components, and what are the correct trade-offs for how these components work together. Investment prior to the real need for scalability is generally not a wise commercial proposal, however, foresight in some designs will save real time and resources in the future.

This section focuses on some factors that are very important to almost all large Web applications: services, redundancy, staging, and failure handling. Each factor includes choices and compromises, especially in the context of the guidelines mentioned in the previous section. To explain this in detail, the best way is to start with an example.

Example: Image hosting applications

You may have uploaded an image online. For large websites hosting and transmitting large numbers of images, there will be many challenges in architecture construction, such as cost effectiveness, high availability and low latency (fast retrieval ).

Imagine a system where users can upload their images to a central server, and the images can be requested through a web link or API (application interface, it's just like that of Flickr or Picasa. For the sake of simplicity, we assume that this application has two key parts: the ability to upload (write) an image to the server, and the ability to query an image. Although we want to upload images faster, we are most concerned about the ability of the system to quickly distribute images requested by users (for example, videos can be requested for a webpage or other applications ). These are the servers used by a web server or CDN edge server (CDN) to store content in many locations, so that the content is closer to the user in terms of geography and physics, the functions provided are very similar.

Other important aspects of the system

• There is no limit on the number of images to be stored. Therefore, the storage scalability should be considered for the number of images.
• Low latency is required for image downloads/requests.
• If a user uploads an image, the image always exists. (Image data reliability)
• The system needs to be easy to manage (manageable ).
• Because Image hosting does not bring high profits, the system needs to be cost-effective.

Image 1.1: simplified architecture of image hosting applications

In this image hosting example, the system must be able to perceive fast, store data reliability and all those highly scalable features. Building a small-sized application hosted on a single server is too simple and meaningless. This chapter has no fun. Let's assume that we want to build something that can grow into something like Flickr.

Service

When considering the design of a scalable system, the (service) helps function decoupling and think about each part of the system through a clearly defined interface. In practice, the system design of this method shows that it has a service-oriented architecture (SOA ). For these types of systems, each service has its own specific functional context, and any interaction with the context is carried out with an abstract interface, in particular, it is the public interface of another service.

Splitting a system into a complementary service set decouples those operations. This abstraction helps to establish clear relationships between services, potential (running) environments, and consumers of services. These clear representations help isolate problems and allow each part to expand independently of each other. This kind of service-oriented system design is somewhat similar to object-oriented programming.

In our example, all the requests for uploading and retrieving images are processed on the same server. However, if the system wants to be scalable, it is wise to split these two functions into their respective services.

Fast forward, assuming these services are widely used; such scenarios will be very easy to see for a longer time (the original article is how longer writes will impact the time it takes to read the images) the write operation affects the image reading time (because these two functions compete to share resources ). Under such an architecture, such influences actually exist. Even if the upload and download speeds are the same (not necessarily for most IP networks, because most of them are designed as a ratio of download speed to upload speed of), files are usually read directly from the cache, write data must eventually arrive at the disk (it may be written multiple times in the eventually consistent scenario ). Even if everything is read from the memory or disk (such as SSD solid state drive), the database write operations are always slower than reading (Pole Position, an open-source database evaluation tool
Http://polepos.org/http://polepos.sourceforge.net/results/PolePositionClientServer.pdf ).

Another potential design problem is that a web server, such as Apache or lighttpd, can maintain the concurrent connections online (about 500 by default, but can be adjusted higher ), in addition, the write operation will soon consume all resources ). Since read operations can be performed asynchronously, or other performance optimizations such as gzip compression or chunked transfer encoding can be used, the web server can be converted to faster service read operations and faster client switching, thus, more requests are served than the maximum number of connections per second (Apache's maximum number of connections is set to 500, but generally thousands of requests can be served per second ). Write operations, on the other hand, tend to maintain an open connection during the upload process. it takes more than one second to upload a 1 m file on most home networks, therefore, the web server can only process 500 write operations at the same time.

Figure 1.2 read/write splitting

Splitting Image read and write operations into their respective services is a good solution to this bottleneck, 1.2. This allows us to expand them independently (we usually read more than write), and helps to make the progress of each point clearer. Finally, we can separate future concerns and solve problems such as slow read operations and achieve scalability.

The advantage of this method is that we can solve the problem independently (without affecting others)-we don't have to worry about writing and reading new images in the same context. The two services still affect all images, but they can be optimized through the service-appropriate method (such as queuing requests, or cache popular images. For more information, see the following ). From a maintenance and cost perspective, each service can be independently and scaled on demand, because if they are combined and mixed, in the scenario discussed above, A service may inadvertently affect the performance of other services.

Of course, when you consider two different points, the above example can work well (in fact, this is similar to the implementation solution of some cloud storage providers and CDN ). Although there are still many ways to deal with these types of bottlenecks, each has a different trade-off.

For example, with the increase of users, Flickr solves the read/write problem by distributing users in different regions. for example, each partition only processes a certain number of users, the cluster has more partitions (refer to the Flickr scalable report, http://mysqldba.blogspot.com/2008/04/mysql-uc-2007-presentation-file.html ). In the first example, the hardware can be more easily scaled based on actual usage (the number of read/write operations on the entire system). However, Flickr is based on its users (however, it is mandatory to assume that the user's usage is equal, so there is still additional capacity ). For the former, a power outage or a service problem will reduce the functionality of the entire system (for example, no one can write files). However, a power outage in a partition of the Flickr will only affect the users of this partition. In the first example, it is easy to operate the entire dataset, for example, upgrading the write service to include new metadata or searching all the image metadata. However, in the architecture of Flickr, each partition needs to be updated or searched (or a search service needs to be able to sort out relevant metadata-in fact they do ).

For these systems, there is no right or wrong. Instead, it helps us to return to the principles mentioned at the beginning of this chapter and determine the system requirements (whether there are many reads, writes, or both, concurrency, cross-dataset query, search, sorting, etc.), checks different trade-offs, understands why the system fails, and has a reliable plan to cope with failures.

Redundancy

To handle failures elegantly, the Web architecture must ensure service and data redundancy. For example, if there is only one file on a single server, losing that server means losing that file. Data loss is rarely a good thing, and the common solution is to create multiple redundant backups.
This criterion also applies to services. If an application has a core function, you can avoid SPOF by ensuring that multiple copies (multiple similar service instances) or versions run simultaneously.

Creating redundancy in a system removes single point of failure and provides a backup or replacement feature in case of emergency. For example, if two instances of the same service are running in the production environment and one of them fails or is downgraded, the system can (start) failover to the healthy service. Failover can automatically occur or requires manual intervention.

Another key aspect of service redundancy is the creation of a non-shared architecture (the translator understands it as a stateless architecture ). Through this architecture, each node can operate independently, and there is no central "brain" to manage the status or coordinate the activities of other nodes. This is very helpful for scalability, because new nodes can be added (to the cluster) without special conditions or knowledge ). However, the most important thing is that there is no single point of failure in these systems, so they can face failure more elastically.

For example, in our image service application, all images will have redundant backups on the hardware in another place (ideally in a different geographic location, in case of disasters such as earthquakes or data center fires, the image access service is also redundant, and all (services) may request services. (See. 3) (the server load balancer can turn it into reality. for details, see the following .)

. 3: Image hosting applications with redundancy

Partition

A single server may not be able to put down massive data sets. It is also possible that an operation requires too much computing resources and consumes performance, making it necessary to increase the (system) capacity. In either case, you have two options: scale vertically or scale horizontally ).

Vertical scaling means adding more resources to a single server. Therefore, for big data, this means adding more (larger capacity) hard disks so that a single server can accommodate the entire dataset. For computing operations, this means that a computing task is handed over to a large server with a faster CPU or more memory. For each scenario, vertical scaling can achieve the goal through its own (individual) ability to process more ways.

On the other hand, for horizontal scaling, more nodes are added. For large datasets, another server may be used to store some datasets. for computing resources, operations are decomposed or loaded on some additional nodes. To make full use of the advantages of horizontal scaling, the translator considers that the system supports horizontal scaling. Vertical scaling does not need to be modified for applications. Generally, you can upgrade the machine to achieve the goal. Horizontal scaling requires the application architecture to support this type of scaling, because data and services are distributed and must be supported at the software level, in this way, the data and service levels can be expanded .) It should be naturally included in the system architecture design guidelines, otherwise it will be quite troublesome to achieve this by modifying and isolating context.

For horizontal scaling, one of the common methods is to split and partition your services. Partitions can be distributed, so that each logical function set is separated. partitions can be divided by geographical boundaries, or other standards such as paid/unpaid users. The benefit of these designs is that they make it easy to increase the capacity of services or data storage.

In Our Image Server example, you can replace a single image storage server with multiple file servers, each of which saves their own separate image sets. (See. 4) This architecture allows the system to store images to file servers and add additional servers when the disk is full. This design requires a naming mechanism to associate the image file name with the server. The name of an image can be generated through the consistent Hash mechanism between servers. Alternatively, you can assign an incremental ID to each image. when a client requests an image, the image search service only needs to maintain the ID range corresponding to each server (similar to the index ).

. 4: Image hosting applications with redundancy and partitioning

Of course, distributing data or functions on multiple servers poses many challenges. One of the key issues is data locality. in a distributed system, the closer the data is to an operation or computing point, the higher the system performance. Therefore, data distribution across multiple servers may be problematic, and any need (data) may not be local, so that the server must obtain the required information through the network.

Another potential problem is inconsistency. When different services read and write the same shared resource, it may be another service or data, and there will be opportunities for competition. when some data is updated, but read operations occur before updates-data inconsistency occurs in such scenarios. For example, in the image hosting scenario, a client sends a request to update the title of the puppy image from "Dog" to "Gizmo, but another client is reading the image at the same time. In this case, the second client does not know whether the received title will be "Dog" or "Gizmo ". (The translator does not understand the author's original intention, because both distributed and standalone environments may allow simultaneous read and write operations, the returned results depend on the scheduling of the underlying storage to process the requests received at the same time, which may be read before writing, and vice versa. so the example here is used to explain whether inconsistency is inappropriate ?)

It is true that there are still many obstacles to data partitioning. However, through data, load, and user usage modes, partitions break down each problem into easy-to-process parts. This helps scalability and manageability, but it is not without risks. There are many methods that can be used to reduce risks and handle failures. however, to simplify the process, this chapter will not cover (these methods. If you are interested in learning more, read the articles on fault tolerance and monitoring published on my blog.

In the next article, we will introduce how to build quick and Scalable Data Access Components.

Note: this article