Great payment microservices Practices

This is a creation in Article, where the information may have evolved or changed.

Have a great experience in technology development

All of the company's businesses (transactions, commodities, UMP, payments, etc.) were completed in a single application in 2014, using PHP development to meet the company's rapid development (we call it v1.0).

Between 2015 and No. 2016, as business flows grew, the existing architecture model faced challenges, and the company began to move towards business splitting and service orientation. Starting with Java as the development language, the service framework uses the company's improved Dubbox to support the Nova Framework (v2.0) for cross-language service invocation.

2017 on a service-based basis we are a step closer to the microservices architecture. Embrace the rich components (v3.0) provided by the community.

The problem we're having

With the development of the business, the size of the team is also growing, at this time the v1.0 monomer architecture encountered a challenge

• The system becomes complex and fragile, and the architecture needs to be upgraded
• High cost of development, high learning costs, increased merge conflict, queue waiting, failure rate, etc.
• Test cost is high, modify one place, also need regression test
• The application is difficult to do horizontal expansion, difficult to carry out capacity rules

The first question is tricky, so let's start by saying what the schema upgrade will do .

To understand what to do, the first thing to consider is what the goal is? The goal of the software architecture is to design a software system to solve the problem, so what the architecture wants to do from an abstract dimension is:

• Define the boundary of the system according to the problem domain (Eric Evans domain driven design, delimit bounded context)
• Segmentation of the system, the purpose of segmentation is division of labor and collaboration (can be parallel to achieve efficiency improvement)
• Establishing principles and mechanisms for collaboration and communication between the segmented parts
• Merge the various parts of the connection into a single whole to complete the system's objectives

Above is the large abstract principle, more specifically, the architecture is designed to be structural, at different dimensions and levels:

• High dimension: The segmentation and interaction structure of a system, subsystem, or service
• Medium Dimension: The division of domain modules within a system or service
• Low Latitude: Is the code structure, data structure, table structure, technical options, development with the cool uncomfortable

There may be some new problems in the implementation of the architecture, which are not considered in the original architecture design, and need to be analyzed and judged, and the new decision-making adjustment should be formed. Other problems, perhaps, are the aliasing in the process of execution, leading to deviations from the original decision. Architects need to take all these concerns into account and develop a variety of options to address these concerns with development engineers, and take appropriate action in the right circumstances, depending on the situation in the real world. Sometimes we call these actions "refactoring or optimization." When an old system does not have this kind of action for a long time, we will have to take another action, which we call the architecture upgrade.

A software system or architecture, unlike a building that naturally consumes decay due to the passage of time, only rots because of change. At first, a clear and tidy architecture and implementation are constantly becoming cloudy and chaotic as demand changes. Computer science loves to borrow the term "entropy" of physics, which expresses the degree of confusion in the system, and the "entropy" of software systems can easily inadvertently become higher as demand changes.

Software system "Entropy" has a critical value, when the threshold is reached and exceeded, the life of the software system is basically the end. At this point, we are going to take that compelling action. The legend shows the life cycle change of the "entropy" value of the software system.

So, not all large systems are well designed, it is very difficult to design good one mega-systems, and with the addition of business functions, the original design will be overwhelmed by the code piled up to break the original design. What we can control is a system with a specific boundary, so it is one of the most intuitive and effective ways to split the system into a bounded context (bouded context) based on the business attributes. This is what the domain-driven design pursues. At the DDD European Conference, Eric also recognized that the popularity of the microservices architecture in recent years has a great advantage, the service granularity is appropriate, the service is physically isolated , and the "entropy" problem of a single service is confined within a single microservices. The cost of replacing and reconstructing a single micro-service is very limited, so that the "entropy" problem is localized, so it is not easy to infect the whole world and get out of control. Of course, there is a premise, that is, micro-service splitting and interface interaction to be reasonable, reasonable test standard is to change with the demand, always realize the change or interface new, rather than always adjust interface interaction. Architecture starts at the beginning of system life and accompanies the entire system life cycle. Each change in demand changes should be done once or for a large or small re-architecting process. The focus of the architecture is to control the change of the "entropy" value when the software system changes.

After splitting the business area, it has been able to meet the development of the business well. But v2.0 is too heavy on developers, and needs to do some work to make it easier for the architecture to perform

• The existing Java framework does not allow developers to focus only on business implementations, and the framework itself does not provide some out-of-the-box, three-way, and company components that need to be heavily configured
• The framework does not provide some common patterns to guide developers in writing code
• The confusing version depends on
• Project structure is not standardized
• Application health checks are not standardized
• Write tests that are complex and difficult to integrate continuously

Why Choose Spring Boot

• Develop a physical examination to greatly improve
• Make application Configuration Simple
• Make coding simple
• Make it easier to write test code
• Local start-up for easy development and commissioning
• Make deployment simple, inline containers
• Simple and powerful SPI mechanism to easily expand your custom Autoconfiguer
• Spring-boot-starter-actuator makes monitoring easier
• Complete ecosystem with seamless integration of mainstream frameworks
• Community Active, iterative fast
• Meet our microservices goals to facilitate future container delivery

Solve the problem

Youzan Pom and Youzan-boot-parent

Draw on spring BOM practice, establish Youzan BOM, version unified management, completely solve the problem of version confusion. Set up youzan-boot-parent and eliminate duplication for each application with duplicate configuration issues, eliminating the need for inter-application copy. Additional benefits are added to facilitate a unified upgrade.

In addition, we have standardized 4 sets of environments for our existing operations environment: development, testing, pre-production, online.

We have made strict restrictions on the Publish API jar deploy to the MAVEN repository, the API jar should have only a few dtos and some interfaces, but because the door opener is often copied and pasted, it also puts a variety of unwanted dependencies (such as spring, Various log frameworks, etc.) are added to the API, causing the application of the jar to have a dependency conflict, by taking unnecessary time to find the conflict and resolving the conflict, we want to do the last line of defense by technical means, from the source to resolve the dependency conflicts caused by relying on other system API jar.

Youzan Application

We want the app to be simpler to use for some open source components and the components that the company develops itself, reducing access costs. For this we have expanded the spring boot Autoconfiger and added various starter.

• Youzan-boot-dependencies
• Youzan-boot-parent
• Youzan-boot
• Nova-spring-boot-starter
• Nsq-spring-boot-starter
• Druid-spring-boot-starter
• Mybatis-spring-boot-starter
• Chameleon-spring-boot-starter
• Youzan-boot-starter-test

For example, if we want to use the Nova framework, just one annotation @enablenova can

Application of standardized health checks

curl http://127.0.0.1:8080/health  

{    status: "UP",    diskSpace: {        status: "UP",        total: 249779191808,        free: 61591195648,        threshold: 10485760    },    redis: {        status: "UP",        version: "3.0.3"    },    db: {        status: "UP",        database: "MySQL",        hello: 1    },    refreshScope: {        status: "UP"    },    hystrix: {        status: "UP"    }}

Failure-oriented design (Circuitbreaker)

In Distributed system design, there is a very important principle is: Design for failure, the error will certainly occur. In order to prevent the system from failing, we need to make certain restrictions on the resources that the dependent services can use to protect the application itself. Usually the following objectives are to be considered:

• Protects callers from issues that depend on services (especially network services) that affect the caller's application (high latency and failure)
• Block cascading failures in complex distributed systems
• Instant failure (fail fast) and fast recovery
• Fallback and graceful downgrade, if possible
• Real-time monitoring, dynamic adjustment of parameters and related operations

The following is a brief introduction to the implementation of the principle of hystrix, please refer to the official documentation

• Use Hystrixcommand or Hystrixobservablecommand to wrap external system calls, usually in a separate thread
• Each dependency sets a timeout call, can customize the time-out, or it can be dynamically adjusted, usually with a slightly higher time-out than the value of the measured section 99.5个百.
• Maintain a small thread pool for each dependency, and deny requests directly when the thread pool is full
• Measurement record request success, number of failures, number of timeouts and rejected
• Trigger a circuit breaker, for a specific service block all requests for a period of time (can be triggered manually or automatically triggered, the automatic trigger rule is this dependency error percentage exceeds the threshold)
• Execute fallback logic when the request fails, times out, or is shorted
• Monitoring of measured values and quasi-real-time configuration changes

Embodied in the code.

@HystrixCommand(groupKey = "RiskGroup", commandKey = "RiskClient-containsSensitiveWord", fallbackMethod = "fallback",      commandProperties = {          @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "3000")      })  public boolean containsSensitiveWord(List<String> words) {    if(skipSensitiveWordValidation()){      return false;    }    PlainResult<Boolean> result = sensitiveWordFilter.containSensitiveWord(words, RECEIPT_SCENE);    LazyLogs.info(logger, "RiskClient.containsSensitiveWord({}), result={}",                  () -> words,                  () -> JSON.toJSONString(result));    return result.getData();  }  /**   * 风控接口调用出现异常，则降级，是弱依赖   */  public boolean fallback(List<String> words, Throwable e) {    logger.warn("RiskClient.containsSensitiveWord({}) fallback", words, e);    return false;  }

Hystrix parameters can be dynamically controlled by the configuration center

About updating the API documentation

As a consumer of the API, developers often find that documents are outdated or even wrong, and it is said that programmers do not like to write documents because they like to write code, so it is best to automatically generate documents by code. With spring Restdocs, you can automatically generate documents by testing code, and if you add or decrease fields in an interface, if you do not synchronize the update test, the test will not pass, so that the document is always up to date.

Test code

  @Test public void Withdrawsummary () throws Exception {given (Withdrawqueryservice.querywithdrawstatus ()). Wil    Lreturn (Plainresults.success (Withdrawstatus.getstatusmap ())); This.mockMvc.perform (Get ("/withdraw/querywithdrawstatus")). Andexpect (Status (). IsOk ()). Andexpect (Content (). ContentType (Mediatype.application_json_utf8)). Andexpect (JsonPath ("Success"). Value (True)). Andexpect (Jsonpat                        H ("code"). Value (0)). Andexpect (JsonPath ("message"). Value ("")). Anddo (Document ("Withdraw-status",                        Preprocessrequest (Prettyprint ()), Preprocessresponse (Prettyprint ()),                            Responsefields (Subsectionwithpath ("RequestID"). ignored (). Optional (), Subsectionwithpath ("Success"). Description ("request Result"), Subsectionwithpath ("code"). Descriptio N ("error code, 0 means no error"), Subsectionwithpath ("message"). Description ("error message, if there is an error"), Subsectionwithpath ("data"). Description ("Withdrawal status list")  ))); }

Generated API documentation

About the API provider's troubles

Within a company or organization, the biggest annoyance for API providers is not finding out what consumers are, and how consumers use their APIs. Not to mention the situation after some company personnel changes. There is a real case where a developer corrects a field word spelling error, and when the result is released, there is a relying party that uses that field to render the relying party service unavailable. In fact, this scene and experience after many times, the developer will be afraid of the foot, the original some unreasonable design and error will not improve it, do nothing.

In fact, the root cause of this problem is the problem of the mode of collaboration between service providers and consumers, and we hope there is some mechanism to alleviate this problem. If the service consumer is able to notify the service provider of the scenario using the API and implement it on the test code, then it is not possible for the service provider to perceive the various relying party API usage scenarios. In fact, this is a contractual spirit, service providers and relying parties to communicate more and communicate the results of communication to the test code. Of course, there are some powerful tools and frameworks to support our assumption that the contract test (contract testing) is to achieve these goals. Please refer to the Spring Cloud contract for the specific use of the documentation

Continuous integration

The benefits of continuous integration do not have to say much, the key is to carry on.

More Benefits

• Log level dynamic adjustment

curl -i -X POST -H 'Content-Type: application/json' -d '{"configuredLevel": "DEBUG"}' http://localhost:8080/loggers/com.youzan.pay  

• More convenient to collect system metrics data, easy to monitor
• Spring Cloud Config/consul Configuration Center
• Service Discovery/consul
• Spring Cloud Zuul API Gateway
• Spring Cloud Sluth & distributed Tracing/zipkin Distributed tracing

Reference

Eric evans-tackling complexity in the heart of software

Spring Boot

Spring Cloud

Https://martinfowler.com/microservices

Microxchg 2017-juven xu:aliexpress ' to MicroServices

MicroServices at the Netflix scale

https://jenkins.io/

Boundedcontext