Reading notes 7 extensible architecture of on-demand Web sites

One, scalable and extensible-silly division is not clear

Previous notes we learned about scalable architectures, but in real-world scenarios, including many architects, are often confused with scalability and extensibility, with extensible representation of scalability. So here, follow the author we have to clarify these two concepts, to avoid the future we will not be confused about it.

( 1 ) Extensibility ( extensibiltiy )

　　refers to the ability of the system function to be continuously expanded or enhanced in the case of minimal impact on existing systems . We can't help but think of one of the main principles of object-oriented: Opening and shutting, open to expansion, closed to modification. It also says that when a new feature is added to the system, there is no need to modify the structure and code of the existing system.

( 2 ) Scalability ( Scalability )

　　refers to the ability of a system to increase (or reduce) its own computational transactions by increasing (or decreasing) the size of its own resources . In the Web site architecture, usually refers to the use of clustering to increase the number of servers, thereby increasing the overall transaction throughput capacity of the system.

The core idea of designing a Web site Extensible architecture is: modularization , and on this basis to reduce the coupling between the modules, improve the reusability of the module. In a large web site, these modules are distributed and deployed in a way that separate modules are deployed on separate servers (clusters) to physically decouple the coupling between modules, further reducing coupling and thereby improving reusability.

Second, the use of distributed Message Queuing to reduce system coupling

above, we mentioned that to separate the coupling between the modules, if there is no direct call between the modules, then the new module or modify the module has the least impact on other modules, so that the system scalability is undoubtedly better. So, is there an architecture that is based on such considerations? So we turn our eyes to an architecture called " event-driven " .

2.1 Event-driven architecture

based on the event-driven architecture ( Event driven Architecture ) is defined by transferring messages between low-coupling modules to keep the modules loosely coupled, and by communicating with event messages to complete the inter-module collaboration . The typical EDA Architecture is the common producer- consumer model in the operating system. In the large-scale Web site architecture, there are many implementations, but the most common is distributed Message Queuing .

as shown, Message Queuing leverages the release - The subscription mode works, the message sender publishes the message, and one or more message recipients subscribe to the message. A message sender is a message source that is sent to a distributed message queue after the message is processed, and the message receiver continues processing after it obtains the message from the distributed message queue. It can be seen clearly that there is no direct coupling between the sender and the recipient, and the message sender simply sends the message to the distributed message queue as the end of the operation, and the message recipient only needs to get the message from the distributed message queue and does not need to know where the message came from. Therefore, for the new business, as long as the interest in such messages, you can subscribe to the message, the original system and business has no impact, so as to achieve the extensible design of the website business .

2.2 distributed Message Queuing

A queue is a first-in-one-out data structure, and distributed Message Queuing is seen as the deployment of this data structure to a stand-alone server , where the application sees the use of distributed Message Queuing through the remote access interface for message Access operations to implement distributed asynchronous calls.

As shown, we can identify three steps:

The ① message producer application pushes the message to the Message Queuing server through the remote provider, and the Message Queuing server writes the message to the local memory queue and returns a successful response to the message producer.

② the Message Queuing server finds the consumer application that subscribes to the message based on the message subscription list, and sends the message in the message queue to the consumer application via the remote communication interface in accordance with the FIFO principle;

③ The consumer application receives a push-over message and carries out a series of related processing, the process terminates;

PS : Then, is there a situation where the message queuing server causes the message to be lost after the outage . In fact, this situation does exist in the actual operation and maintenance process. So, how do we avoid it? At this point, the author gives a scenario: If a Message Queuing server goes down causing a message to be lost, messages that are successfully sent to Message Queuing are stored in the message producer server, and the message is actually processed by the message consumer server before the message is deleted . After the Message Queuing server goes down, the producer server chooses to publish messages from other servers in the distributed Message Queuing server cluster.

In addition, there are practices on distributed Message Queuing that can be used NoSQL products to build, such as Redis provides a queue data type that makes it easy to build distributed message queues, and if you are interested, you can also refer to my other blog post: using Redis as a Message Queuing service scenario case

Third, the use of distributed services to create a reusable business platform　　

If distributed Message Queuing decomposes the system coupling through the message object , the different subsystems process the same message, then the distributed service decomposes The system coupling through the interface, and the different subsystems make the service invocation through the same interface description.

problems with the application system of 3.1 big mac

The evolution of the website from small to large, performance for the entire site is a single system gradually expanded development and change, with the increasing complexity of website function, the website application system will gradually become a Big Mac, as shown. As you can see, a large number of applications and service components are aggregated in an application, and the Big Mac is a huge hassle for the development of the entire Web site (difficulty compiling, Code Branch Management), maintenance (new business difficulties), and deployment (deployment difficulties).

3.2 split, split or split

The solution is also the split we have mentioned many times, and the modules are deployed independently to reduce system coupling. Split is divided into: horizontal split and vertical split. Here we look again at both of these ways:

( 1 Split vertically: Split a large application into smaller applications, and if the new business is more independent, deploy its design directly as a standalone Web application System;

( 2 Horizontal split: Split the reusable business and deploy it as a distributed service, and the new business only needs to call these distributed services, without having to rely on the specific module code. If the business logic within the module changes, the business program and other modules will not be affected as long as the interfaces remain consistent.

3.3WebService and Enterprise-class distributed services

Wbservice have the disadvantage :

Advantages :

1. Unified Interface Service Specification

2. Multi-frame support

3. Easy to develop

Disadvantages :

1. Bloated registration discovery mechanism
2. low-efficiency XML serialization means
3. overhead compared to higher HTTP Remote Communication
4. Complex deployment and maintenance tools

3.4 requirements and features of distributed services for large-scale Web sites

2. Load Balancing
4. Fail-over transfer
6. Efficient remote communication
8. Integration of heterogeneous systems
10. corresponding with minimal intrusion ( supports centralized deployment and distributed Deployment )
12. Version Management ( Multiple versions released ,)
14. Real-time monitoring

3.5 Distributed Service Framework Design

Large sites need simpler and more efficient distributed service racks to build their SOA, with Facebook allegedly using THIRFT to manage its distributed services, and open source distributed services with Ali's Dubbo

Duboo's Distributed Service Framework architecture schematic diagram:

The Service Framework client module loads the list of service providers through the service registry , finds the required interface and sends a service call request to a day provider server based on the configured load balancing policy, and the service call fails, and the client module automatically requests a service from the list of service providers to provide a service to re-request services for automatic failover of the service to ensure that the service is available.

Iv. Scalable Data Structures

traditional relational database in order to guarantee relational operations (through sql statement), when designing the table structure, you need to make the table's 1nf , 2nf , 3nf rigid data structures are difficult to meet the challenges of changing requirements , and some system designers are able to cope by pre-designing some redundant fields (I have seen this design many times in my internship year, though it can solve the problem, But from the design, really good shit ), But this is obviously a bad database design.

So, is there a way to do scalable data structure design with wood? Is it possible to add fields without modifying the table structure? The answer is yes, the columnfamily(column family) design that many NoSQL databases use todayis a solution. columnfamily was first used in Google 's BigTable , This is a sparse matrix storage format for column families. It may be said that people still do not understand, but can be understood by:

This is a Student basic information table, different students contact different, elective courses are different, and in the future there will be more contacts and courses to join this form, if according to the traditional database design, regardless of the pre-set how many redundant fields are not enough, stretched, tired to cope. Instead of using the columnfamily structure of a NoSQL database, when creating a table, you only need to specify columnfamily name without specifying a field (Column) that can be specified when the data is written, in this way, the data table can contain millions of of the fields, Allows the application's data structure to be arbitrarily extended .

Five, the use of open platform to build the website ecological Circle

the value of the site is to create value for his users, large-scale web site in order to better serve their users, will develop more value-added services, will be the site's internal services encapsulated some of the calling interface open for external third-party developers to use, this open interface platform is called an open platform. Third-party developers use these open interfaces to develop applications (apps) or websites that provide value to more users. As a result, Web sites, users, and third-party developers rely on each other to form an ecosystem of sites that provide more value to users and improve the competitiveness and profitability of websites and third-party developers.

currently BAT such as the domestic internet giants have their own open platform, trying to use their huge user base to attract third-party developers, to create a larger aircraft carrier battle Group, in the market competition in an invincible position.

Vi. Summary of Studies

Web site constantly on the new product is its survival instinct, who can be faster and better exit more new products, who will live more moist. Marx's theory of labor value in the IT industry has been confirmed that the intrinsic value of the product lies in the time of labor, and the labor time is not the individual pay the labor time, but in the industry generally working time , Capitalists will only pay for the industry general labor time, if you are less efficient than the general industry time, sorry, please your resources overtime.

and I think of me in cdeic Internship this year, add a lot of classes, now it seems that if our system has a more scalable system architecture, can be faster to quickly send new products, maybe we will be able to work on time, have a girlfriend to accompany their girlfriend to dinner to see movies and so on, Do not have a girlfriend to develop and develop a girlfriend or something (seemingly I am still single is too much overtime, no time to find a girlfriend?) , or read books, listen to songs, take a Walk, to the moon when the song, tell the Geometry of life.

Reprinted from http://www.cnblogs.com/edisonchou/, slightly amended.

Reading notes 7 extensible architecture of on-demand Web sites