Practice in building a highly scalable Web architecture and Distributed System

Http://www.csdn.net/article/2013-01-10/2813545-Building-Scalable-Web-Architecture

Kate Matsudaira, the author of this article, is a beautiful female Vice President of engineering who once worked in top IT companies such as Sun Microsystems, Microsoft, and Amazon. She has rich working experience and team management experience, and has worked as a programmer, Project Manager, product manager, and personnel manager. Focusing on building and operating large Web applications/websites, she is currently focusing on SaaS (software as a service) applications and cloud computing (as we call it big data ).

This article describes how to build a Scalable Distributed System in aosa.

Open-source software has become a basic component of many large websites. As these websites grow, their website architecture and guidelines are also open to developers, provide practical guidance and help.

This article focuses on Web Systems and applies to other distributed systems.

Principles of Web Distributed System Design

What exactly does it mean to build and operate a scalable Web site or application? Initially, only users are connected through the Internet and remote resources are accessed.

As with most cases, when building a web service, you need to plan ahead. Understanding the considerations behind the creation of large websites and learning to balance them will bring you more informed decisions. The following are some core principles that need to be paid attention to when designing a large web system:

• Availability
• Performance
• Reliability
• Scalable
• Easy to manage
• Cost

The above principles provide some basic and theoretical guidance for the design of distributed Web architecture. However, they may also face each other. For example, the cost of achieving this goal is to sacrifice costs. A simple example: select the address capacity and add more servers (scalability). This may be easy to manage (You have to operate on additional servers) and cost (server price ).

No matter which type of Web application you want to design, these principles are very important, and even these principles will be mutually constrained. It is also important to make a good balance between them.

Basic

When it comes to system architecture problems, you must consider these things clearly: What modules are appropriate? How to combine them? How can we properly weigh them? Before expanding investment, it usually requires not a smart business proposition. However, some design with foresight can help you save a lot of time and resources in the future.

The focus of this article is to build the core of almost all large Web applications: services, redundancy, partitioning, and troubleshooting capabilities. Every factor here involves selection and compromise, especially those principles discussed earlier. The best way to explain these core points is to give an example.

Image hosting applications

Sometimes you upload images online, while some large websites need to host and transmit a large number of images, which is cost-effective, highly available, and low-latency (quick retrieval) architecture is a challenge.

In an image system, users can upload images to a central server, and request these images through network connections or APIs, just like Flickr or Picasa. Simply put, we assume that this application only contains two core parts: Upload (write) images and retrieve images. It is best to be efficient when uploading images, and the transmission speed is also our biggest concern. When someone sends a request to an image (such as a web page or another application ). This is a very similar feature that provides Web services or content delivery networks (a CDN server can store content in many places, so it is closer to users both geographically and physically, this results in faster performance.

Other important aspects to consider for the system:

• There is no limit on the number of image storage, so storage should be scalable. In addition, image computing needs to be considered.
• Download/request requires low latency
• When a user uploads an image, the image should always be there (the reliability of the image data)
• The system should be easy to maintain (easy to manage)
• Because image hosting does not have a high profit margin, the system needs to be cost-effective.

Figure 1 is a simplified function Diagram

Figure 1 simplified structure of the image hosting System

In this example, the system must be fast and the data storage must be reliable and highly scalable. Building a small application is insignificant, and a server can be hosted. In this case, this article will be uninteresting and attractive. Suppose that the application we want to do will gradually grow to be as large as the size of the Flickr.

Service

When we consider building a scalable system, it should be helpful for decoupling. each part of the system can be used as its own service and has a clear interface definition. In practice, such a system design is called a Service-Oriented Architecture (SOA ). For such systems, each service has its own unique function. Through an abstract interface, it can interact with any external content, usually another public service API.

The system is decomposed into a set of complementary services, and these operation blocks are decoupled from each other. This abstraction helps to establish a clear relationship between services, basic environments, and consumer services. This decomposition can effectively isolate problems, and each block can also be scaled to each other. This kind of service-oriented system design is very similar to the object-oriented design.

In our example, all upload and retrieval requests are processed on the same server. However, because the system needs to be scalable, it makes sense to break these two functions and integrate them into your own services.

Fast forward and assuming that the service is being used in large quantities; in this case, it is easy to see how much time the image writing time will affect the reading time (the two functions compete for sharing resources ). According to their respective systems, this impact will be huge. Even if the upload and download speeds are the same (this is not possible, for most IP networks, the download speed is: the upload speed is at least). Generally, files can be read from the cache and written, it is written to the disk eventually (it can be written several times in the case of eventual consistency ). Even when reading data from a cache or disk (similar to SSD), Data Writing is slower than reading (pole ).
Position, an open source tool and result for Open Source dB benchmarks ).

Another potential problem with this design is that Web servers such as Apache or Lighttpd usually have a maximum number of concurrent connections (the default value is 500, but more), which may result in high traffic, writing may quickly eliminate all data. Since reading can be asynchronous or use other performance optimizations, such as gzip compression or block transfer code, Web services can quickly switch between reading and client to serve more requests, the maximum number of connections per second is exceeded (Apache's maximum number of connections is set to 500, which is not common and can serve thousands of read requests per second ). On the other hand, writing usually tends to keep an open link for continuous upload. Therefore, it may take more than 1 second to upload a 1 MB file over the home network, such a server can only meet 500 write requests at the same time.

Figure 2: Read splitting

A good way to plan this bottleneck is to separate reading and writing, as shown in figure 2. In this way, we can expand them separately (read more than write all the time), but it also helps to understand the meaning of each vertex. This separation is easier to troubleshoot and solve problems with scale, such as slow reading.

The advantage of this method is that we can solve the problem independently of each other-in the same case, no write or retrieval operations are required. These two services still use the images of the global corpus, but they are free to optimize performance and service methods (such as queuing requests or caching popular images-more here ). From the perspective of maintenance and cost, each service can be expanded independently as needed, but if they are merged or intertwined, it may inadvertently affect the performance of another service, as discussed above.

Of course, if you have two different endpoints, the above example may run well (in fact, it is very similar to the implementation and content delivery network between several cloud storage vendors ). Although there are many ways to solve these bottlenecks, everyone will have different trade-offs, so it is most important to adopt the method that suits you.

For example, to solve this read/write problem, you can distribute users across different fragments. Each shard can only process one group of users, but as the number of users increases, more fragments will also be added to the group set (see the Introduction to the extension of Flickr ). In the first example, it is easier to expand based on the actual usage of hardware (the number of reads/writes in the entire system), while Flickr is based on its user base (
Forces the assumption of equal usage when SS users so there can be extra capacity ). In the previous example, any interruption or problem will reduce the overall system function (for example, no one can perform a write operation), and one interruption of Flickr will only affect the number of users of its fragments. In the first example, it is easier to operate through the entire dataset-for example, update the write service, including new metadata or searching through all the image metadata-and each fragment of the Flickr architecture needs to be updated or searched (or a search service needs to be created to collect metadata-in fact, ).

When talking about these systems, there is actually no very correct answer, but it helps us to look back at the principles at the beginning of the article. Determine System Requirements (a large number of reads or writes or both are performed, level concurrency, cross-data query, range, type, etc ), select different benchmarks, understand how the system goes wrong, and make solid plans for future failures.

Redundancy

To correctly handle errors, services and data in a Web architecture must be properly redundant. For example, if only one copy file is stored on this independent server, the file will be lost if this server suffers a problem or is lost. Data loss is not a good thing. A common way to avoid data loss is to create several more files, copies, or redundancy.

The same applies to servers. If an application has a core function, ensure that multiple copies or versions are running at the same time to avoid Single Node failure.

Create redundancy in the system. In case of system crisis, you can eliminate single point of failure and provide backup or backup functions if necessary. For example, there are two identical service examples running in the production environment. If one of them fails or falls, the fault tolerance of the system is transferred to the healthy copy. Fault Tolerance transfer can occur automatically or manually.

Another important component of service redundancy is to create a non-shared architecture. In this architecture, each node can run independently of each other, and there is no so-called Central "brain" to manage the status or coordinate other nodes. This is very helpful for system scalability, because new nodes are added without special requirements or knowledge. However, the most important thing is that these systems do not have single point of failure, so the failure elasticity is even greater.

For example, in our image server application, all images have redundant copies on the other hardware (ideally in different geographic locations, to avoid some fires, earthquakes, and other natural accidents in the data center), service image access will be redundant, all potential service requests. (See figure 3: Using Server Load balancer is the best way to achieve this. More methods will be introduced below)

Figure 3 image hosting application Redundancy

Partition

The dataset may be very large and cannot be installed on a server. It is also possible that an operation requires too much computing resources, lower performance, and increased capacity. In either case, you have two options: vertical scaling or horizontal scaling.

Vertical Scaling means adding more resources to a single server. Therefore, for a very large dataset, this may mean adding more (or greater) hardware devices to enable a server to accommodate the entire dataset. In computing operations, this may mean moving computing to a larger server with a faster CPU or larger memory. In various cases, vertical scaling can be done by improving the processing capability of a single resource.

On the other hand, horizontal scaling adds more nodes. In large datasets, this may use the second server to store some datasets. For computing resources, this means the split operation or cross-node loading. To make full use of horizontal scaling, it should be an internal system architecture design principle, otherwise modification or splitting will be very troublesome.

When talking about horizontal scaling, the most common practice is to partition or fragment services. Partitions can be distributed, so that the functions of each logical group are independent. Partitions can be completed by geographic boundaries or other standards, such as non-paid and paid users. The advantage of these solutions is that they will provide a service or data storage as the capacity increases.

In our image server case, a single file server used to store images may be replaced by multiple file servers, each containing a set of unique images. (See figure 4) This architecture will allow the system to fill every file/image server. When the disk is full, additional servers will be added. This design requires a naming scheme to bind the image file name to its corresponding server. The image name can form a consistent hash scheme and map it to the entire server; or assign an incremental ID to each image. When the client sends a request to the image, the image search service only needs to retrieve the IDS mapped to each server (such as indexes.

Figure 4 hosting application redundancy and partitioning

Of course, there are still many challenges to partition data or functions across multiple servers. The key issue is data localization. In a distributed system, the closer data operations or computing points are, the better the system performance. Therefore, it may also be a potential problem when data is distributed across multiple servers. Sometimes the data is not local, the server must be forced to obtain the required information through the network, and the acquisition process will be designed to reduce the cost.

Another potential problem is inconsistency. When multiple services perform read and write operations on a shared resource, another server or data storage may be involved as a condition for the campaign-some data needs to be updated, but the read priority is higher than the update-in this case, the data is inconsistent. For example, in the image hosting solution, the possible inconsistency is: if a client sends an update "dog" image request, rename it, and change "dog" to "Gizmo ", at the same time, another client is reading this image. In this case, the title is unclear. "Dog" or "Gizmo" should be received by the second client.

Of course, there are some obstacles in data partitioning, but partitions allow you to split each problem into a management group through data, load, usage mode, and so on. This is helpful for scalability and ease of management, but it is not without risk. There are many ways to reduce risks and handle faults. However, for the sake of simplicity, it is not described in detail in this article. If you are interested, you can visit my blog.

Summary

All of the above are the core elements to consider when designing a distributed system. The principles of availability, performance, reliability, scalability, ease of management, and cost are very important. However, in practical applications, another principle may be achieved at the expense of a certain principle, in this process, we must do a good job of balancing, so as to make sure that it is time-consuming.

In the practice of building a distributed system, we will introduce how to design Scalable Data access, including load balancing, proxy, global cache, and distributed cache.

Http://www.csdn.net/article/2013-01-10/2813545-Building-Scalable-Web-Architecture