NetEase Aesthetics-System Architecture Series 1-Distributed and serviced

This article from the NetEase cloud community

Li Yuanhong

1. Background information

NetEase Aesthetics in August 2016 on-line, is based on products and technology as the core, to a wealth of beauty products library, users use a collection of ideas for content carrier community-based products. Like most start-up Team technical architecture selection, in order to quickly go online, aesthetics in the beginning to choose a single application mode, all the functions are packaged in a war package, basically no external dependencies, the application is deployed in a Tomcat container, including Client,web, operating background and other logic, The situation continued until December 2016, when refactoring began with a "hodgepodge" background.

2. Aesthetics of single Application Mode

Single application is more suitable for smaller projects, the advantages are summarized as follows:

• Development is straightforward and can be centrally controlled

• Functions are in one application, without the overhead of distributed management (distributed transactions, cross-domain performance, data consistency)

• Integration of MVC, simple structure

But its shortcomings are also very obvious, after the expansion of the new American business, the structure is more and more limited

• All development in one application to change code, resulting in low development efficiency

• Build for a long time, any small demand changes, to the overall compilation of packaged deployment

• Interactions between businesses, modifying the functionality of a module, and possibly affecting another module that is unrelated

• Low stability, strong reliance on application server and database processing power

In view of these problems, the new United States in the design of the micro-service architecture, the single application in accordance with the business module to effectively split, to achieve agile development and deployment.

3. Preparatory work

3.1 Purpose of Service

3.2 Technical Preparation

3.3 Service Specification Development

• Service protocol layer (define service interface, data transfer object, exception handling mechanism)

• Service implementation layer, which can be a crud for a database or a network call to a third-party system

• Service dependency layer, is to assemble each service, realize the specific product business logic.

• Avoid cross-layer calls

4. Process of service implementation

4.1 Business Modeling

This step is the basis of service, the need to split the number of sub-services, inter-service dependencies, all need to be expressed through the business modeling.

New beauty is the application of the beauty community as a business carrier, for women's cosmetic products, users can produce a large amount of content, and thus produce simple social behavior.

4.2 How services are provided

In the New American micro-service architecture, each service has multiple copies to do load balancing, a service can be offline at any time (release time), or may respond to temporary access pressure to add new service node (online activity plan), which involves the problem of service discovery, Through zookeeper as the distributed management of Service registration information, new beauty maintains long connection through heartbeat detection, and updates service information in real time.

4.3 How clients invoke services

Originally single application, the service is local, the UI can be called directly, according to the service split, the service is scattered in a separate Java virtual machine, the background has more than one service, the client will remember more than one service address? This dissatisfaction with the purpose of service splitting, we have established a number of business gateway between the service provider and the client (APP,WEB,WAP) to provide a unified service invocation portal, make microservices transparent to users, and aggregate back-office services for unified management and maintenance.

4.4 How to communicate between services

So far, services have been dispersed into separate jvms, and the various invocations of single-application scenarios have not been satisfied after microservices.

Because the service provider layer needs to adhere to the "avoid cross-layer invocation" guideline, synchronous calls between services are banned.

However, the business needs to solve the problem similar to "User registration, send system notification", to deal with this scenario, we use asynchronous message calls to solve the problem of direct call between services, which also indirectly introduces the problem of distributed transactions, idempotent processing and eventual consistency.

Using asynchronous messages can reduce the coupling between invocation services and can be a buffer between invocations, ensuring that the message backlog does not burst the caller.

4.5 Service Downgrade Scenario

Service downgrade and governance New America is still under construction, mainly to deal with how the system should respond after a few services have been hung up.

The present more mature response plan is more:

• Retry mechanism

• Traffic limit

• Client demotion (client-side local code implementation to ensure core functionality)

5. Smooth Online Service

There is a lot of discussion on how to prepare services, how to implement, but the real problem is how to do a good job of the old application of smooth migration to service applications, without causing online failure.

Here are some simple ideas, based on years of refactoring experience, that involve working closely with QA and OPS teams.

• Service provider layer to do unit testing, new and old applications to execute the same use cases, to ensure that all can pass, so that the service provider on-line switching basically can guarantee that no problem. However, there is a difference between a service invocation and a local call, which is to consider the handling of service invocation exceptions, serialization errors, and call timeout errors, which also need to be guaranteed by writing separate use cases during the single test.

• The client calls the API to do integration testing and full-scale regression, this part of the main work in the QA team, need in peacetime business development process to do this part of the accumulation, new American QA usually do this part of the work to increase our service switch success confidence.

• On-line planning must be fully prepared, including data migration failure how fast fallback, service validation error How to quickly switch to the old application.

• Separate application test environment, import online data into the test library, do several more online analog switching operations.

• It is best to do grayscale switching and release verification, let some users first use the new application, verify OK after the full amount of switching, such as the new United States in accordance with the Office network and external network to divide user groups.

6. The summary of the new American service

Overall, the new United States service is a big move, in the Spring Festival this period of business relative idle time, did a relatively thorough structure adjustment, the duration of 1.5 months, due to the more fully prepared beforehand, the overall switch is relatively smooth, does not involve the release of the outage, the user no sense, precipitation out of some general solutions.

• Business Modeling DDD

• Distributed Deployment architecture

• Final consistency of distributed transactions

NetEase Cloud Gift Pack: Www.163yun.com/gift

This article from the NetEase cloud community, by the author Li Yuanhong authorized release.

Related articles:
"Recommended" practice for performance monitoring of kubernetes using Prometheus+grafana
"Recommended" Use Quic