SOA Architecture design experience sharing-architecture, responsibility, data consistency

Read the catalogue:

• 1. Background information
• Architectural Hierarchy of 2.SOA
  • 2.1. Application services (Atomic services)
  • 2.2. Portfolio Services
  • 2.3. Business Services (Orchestration Services)
• Refactoring of 3.SOA
  • 3.1. Preserving the service space for future combinations of services
• 4. Use of ddd+grasp for analysis and design (to prevent subjective judgments that lead to false assumptions)
• Data consistency under 5.SOA distribution
  • 5.1. Distributed transactions (DTC-based distributed transactions)
  • 5.2. Transaction compensation (providing forward or reverse operations to make the data consistent in business)
  • 5.3. Asynchronous EDA (Implementation of flexible distributed transactions based on asynchronous event streams)
• 6. Summary

1. Background information

The most recent period of time is to do system analysis and design work, the business is a typical heavy-duty enterprise application direction. Suddenly found that many of the previously thought simple problems become less easy to imagine, the biggest problem is how responsibilities are assigned. On the system architecture design of the biggest problem, in fact, is the allocation of responsibilities, allocation of reasonable, the implementation will be very flexible, the contrary will make the structure is very chaotic.

The life cycle of software can be summarized into four basic processes, analysis, design, implementation, testing, of course, this is only the most sketchy expression. Different methodologies have different ways of using these processes. RUP uses a fast iterative process to properly output some process artifacts in these sub-processes, each iteration being the same analysis, design, implementation, and testing. In scrum, it is not advocated to export any document in the form of process products, but also have the above several processes, emphasizing the human-centric, through communication to solve most of the problems.

Can not use good and bad to determine which methodology, can only be based on the current situation of comprehensive trade-offs. There are several key artifacts in each iteration of RUP that are important for system analysis and design, such as: glossaries, Business rules documents, use cases, and domain sketches. These products are important for analysis and design, and it is necessary to extract the design model from these products in order to get the final landing. This is mainly used in business complex application systems. While scrum is more lightweight, it can be used in Internet projects, where business is not too complex.

In fact, I have to emphasize the software engineering and development methodology, because I recently found that the design is based on the analysis, but there are many problems. Large enterprise-level applications, and can not be obtained through a one-time analysis of accurate and full demand, the initial phase of the establishment of the requirements of 70% are inaccurate. So the architecture needs to have the ability to analyze and is based on the appropriate development-side analysis, when to use RUP, when to use scrum, when the use of XP are very fastidious. Both analysis and design require an execution context, and different contexts have different requirements for analysis and design execution.

There is a saying that I think is very enlightening to architects: analysis is to do the right thing, design is right to do things. Architecture is not the language with the platform, after all, architecture is a sub-process in the design process, I think if your design is unreasonable, you can use any language platform can not solve the problem. (The schema context here refers to: Enterprise Application architecture is not a system architecture for infrastructure)

Architectural Hierarchy of 2.SOA

Architecting the SOA type requires a clear understanding of the SOA architecture model. It is not possible to simply split the system into a simple way, it is necessary to figure out what the SOA architecture model is, what each piece is for, so that the requirements of the analysis phase output can be properly divided into responsibilities.

If the SOA architecture is simply understood to be that the integration between multiple subsystems is a bit too simplistic, there is no real understanding of the SOA architecture model. In accordance with the correct approach of SOA to the target model, in fact, SOA in the implementation of the architecture, the need to consider the combination of services, the continuous reuse of existing services, so that enterprise applications can be gradually integrated, rapid implementation of business iterations. In fact, this is the sectionto of the service, through the layering of services according to the type of distribution, the upper layer of services to the lower service packaging, the lower-level service is responsible for atomic operations, the upper layer services to the lower service business combination.

Let's look at the roles and main responsibilities of each layer.

2.1. Application services (Atomic services)

Application services are such as: order services, warehouse services, sales services, customer management services, these services directly correspond to different application systems, directly serve the atomic operations of these application systems. The order service directly atomically inserts the order, without any branching logic across the other services. Warehouse Services Keep your own warehouse logic in mind. The same is true of other application services, which are just as good as their responsibilities, to eliminate calls to other services.

Figure 1:

App service is located between the UI and the background, we can think of it as a heterogeneous system or a database. The location of the app service is between the front-end and back-end, acting like a service API, but the service in the SOA is much more than this application service, and if there is only one type of service in our SOA architecture, then this will increase the coupling of our system, because you do not have a hierarchical division of the system's services, Your business function will go directly to a service on one of the application lines and continue looking down.

2.2. Portfolio Services

Combination service is a combination of application services, depending on the size of the actual project, not necessarily physical isolation, at the code level of service is also possible, in the future, the need for another day of physical separation, after all, physical separation has serious costs and costs, the stability of the system brings many challenges. So experience tells us to split when necessary. "The first principle of distributed system design is to try not to be distributed," said Martin Fowler, who now understands the true empathy.

Figure 2:

The combination service combines the application services of the lower layer, completes a basic business action, and the application service is the basic fundamental atomic operation. But with complex business requirements, most business functions need to cross multiple application lines to complete one outermost business action. Submitting an order may require crossing many application lines, order management, warehousing, finance, and so on. So here we also need a business service that can orchestrate the composition service at the outermost layer. This orchestration service can be fully automated, with a combination of workflow engine automation, which makes sense for automated processes in enterprise applications.

2.3. Business Services (Orchestration Services)

The business service is the outermost service, and the Portfolio service is orchestrated down. Business services are at the top level, and when you need to have a business that spans multiple application lines, the business is put into business services. such as the submission of orders, first check inventory, deduct inventory (frozen inventory), and then place a single, and then notify the financial, and then inform the logistics and so on is a complex enterprise service line. This outermost business logic if you don't do SOA layering and then put it into the outermost business service, you put it into any application line will make the system call confusing. So the problem is that you need to have a vertical hierarchy of divisions. If there is a hierarchy of SOA, there will be no confusion with each other. In fact, we can refer to the service design method of Ali. (A large-scale Internet architecture and practice written by Hae also describes the hierarchy of services to be divided)

Figure 3:

When business logic is executed in a business service, it needs to be done across multiple application lines. This part of the logic is also said to be a duty, if not put in this position, which application line is not suitable, put in which application line will make the system call chaos. In fact, the problem here is that we can not use a dimension to the design of the SOA system, the service has a combination of features, so the appropriate level of upgrading services is beneficial, but the application services and composite services can be built at the code level, and business services called Orchestration Services are required to be physically isolated, After all, it is worth considering the complexity and stability of the system. In the troubleshooting, system performance, stability, and so on, the physical isolation has a certain role, after all, business services are originally to combine multiple application lines, so that will make the whole system architecture is clear.

Refactoring of 3.SOA

SOA implementation, most of the time the existing system is reconstructed after consideration, the initial stage of enterprise development in order to quickly prototype as the primary goal, only when the system bottlenecks will consider the use of SOA to solve. But at this time there are a lot of historical baggage can not be solved, SOA restructuring in fact, the cost is very high, and very dangerous, for some complex logic to say that the reality is powerless. If all can be solved by refactoring this technology, then we are too naïve. The refactoring-Martin Fowler book is about refactoring at the code level, with two concepts of system-level refactoring. There is not much mature methodological support for system-level refactoring. In particular, new technologies are emerging, each has advantages, can be very good use of these technologies, methodologies, processes to reconstruct large enterprise-class systems, the difficulty is very large, which requires the entire company to invest a lot of manpower, resource costs. Back to think about it, in fact, it is necessary to consider it in the early stage, so as to reduce the late many technical debt.

Here I only summarize my idea of SOA reconfiguration in analyzing and designing a business system of a company. Refactoring is a continuous iterative process that cannot be strode across. With a lot of scaffolding support, so that the system slowly over to the new SOA architecture, since the implementation of the SOA architecture, it is important to the business logic of the migration of the appropriate collation, what business logic should be put into the "business services", what logic should be placed on the code level "Composite Services", There is an "app service" in the code plane for basic operations.

3.1. Preserving the service space for future combinations of services

When the system is split, the call to the current backend is properly planned, divided into two categories, one is the application service, the other is the combination service, and the two services can be abstracted at the code level. The focus is on those calls to other systems, the need to put it into the business services, this logic is more complex, difficult to extract, the need for appropriate combination of "data landing" thinking to consider comprehensively. Sometimes it is possible to get some data into the local to improve the overall simplicity and stability of the system, but consider a life-cycle nature of the data.

In the process of migration may also have some new features in parallel development, both new logic needs to be put into the new SOA services, there will be migration logic, the two processes at the same time is actually very painful, as far as possible to avoid this at the same time, but the reality is not possible, normal is a parallel advance. If the two processes are the same team responsible for the fact that it is OK, after all, the code, business are more understanding.

The purpose of this section is to emphasize the current system migration to consider the level of service, do not just a simple move code, this is a good opportunity to refactor the code, the Division hierarchy of the hierarchy, the read and write separation to read and write separation, to focus on those "business services", Logic that needs to span multiple application lines.

By the way, there is also a focus on the migration of data storage to consider the migration, the optical code level of migration is only the first step, the second step also requires a data-level migration. Of course, these two big steps are done through many iterations, and it's a good opportunity to sort through the business, code, and sort it out.

In the service of the system to consider the service layer, if there is no business service logic so keep the service space, at least to understand that in the service layer there is such a space to be reserved, when there are other application lines need to interact with you can smoothly enter your service area, rather than directly to your application.

4. Use of ddd+grasp for analysis and design (to prevent subjective judgments that lead to false assumptions)

System design is most afraid of the responsibility is wrong, which will make the system suddenly complex structure, and the system architecture is difficult to change or can not change the decision. So I pay attention to this piece, sometimes you understand the business in the familiar still will be mistaken responsibility for this piece of light by subjective judgment is not long-term, no replication, can not spread, can not be written.

I do not introduce more to DDD here, it is grasp to emphasize. The use of DDD can be very good to help us to strategic observation of the area of the enterprise sitting, I still advocate ddd in the company, not to mention the "tactical design" methodology in DDD, said that its "strategic design" methodology is still a great role, so that we can build a strategic model in mind. Specifically, the level of the code should not be landed this depends on the actual situation. And a lot of good ideas in DDD can be borrowed from, including the domain of the common language, with the field of the common language team between the communication and exchange to save a lot of costs. For new people, you can quickly understand some of the company's general business, and the "glossary" in fact, there is a difference.

It says that many of the duties are divided by experience and subjective judgment, and there is no other objective evidence. So there is not a good methodology or model to guide us to solve such problems, in fact, there are, because in foreign people this aspect is already very mature.

Grasp is a set of patterns that can help us with objective design responsibilities. In which application to put this piece of data, in the end, you should put this logic into which service, there is guidance, including the design of the object level can still be. We may have an understanding of the "information expert model", but in the past we may have used it only in object design, not in a system-level perspective. That's because we might not have met a complicated assignment scenario in the past, and only if we had a problem could we really grasp the quality of something.

DDD only combines grasp to be objective and accurate in a certain area of responsibility, whether it is strategic design level or tactical design level, is a good balance standard, not because the technical staff subjective interest tendency leads to a wrong responsibility allocation decision, and this wrong decision ultimately is the developer to pay the bill.

Data consistency under 5.SOA distribution

Traditional distributed systems are different from contemporary SOA-oriented distributed systems, and conceptually SOA is service-centric, and since service-centric there are many service-oriented design principles. The traditional distributed system does not have the concept of service, there is no so-called everything is the principle of service. However, the principal principle of contemporary SOA is service-centric, and there are many service design principles for service design.

SOA also divides services into categories that are categorized according to the level of application of the service: Business services, portfolio services, application services, packaging services, and so on. Then according to the management and operation of the level of classification: control services, scheduling services, monitoring services and so on. Traditional distributed systems do not have these, we are talking about the contemporary SOA of the distributed system, so we emphasize the service-centric, service design principles for the design of the guidance requirements, contemporary SOA is an iterative evolution of traditional distributed systems, not a product of the Times, SOA has increased the emphasis on service as the primary principle and has been elevated to another more advanced level.

In this section, we exchange data consistency issues in contemporary SOA distributed systems, which are mainly concerned with two levels in SOA, one is service level, one data persistence layer. In accordance with the basic requirements of consistency, can be divided into: Read consistency, write consistency, session consistency, final consistency, real-time consistency, and several other dimensions, of course, there are several other dimensions of the consistency requirements.

Here we focus on some of the more difficult data consistency issues and solutions encountered in implementing SOA in enterprise applications, and have important reference values for the consistency requirements of several dimensions just mentioned.

5.1. Distributed transactions (DTC-based distributed transactions)

In the past, many projects have tended to use DTC to handle distributed transactions, most of which are suitable for general enterprise applications where the business, traffic, and data volume requirements are not very high. It is convenient to use DTC, transaction automatic propagation, transaction Auto-sensing, transaction automatic rollback and commit, this is the central DTC to help us manage.

Because of the unified coordination of central DTC, seems to help us to solve a lot of the problems we need to consider, but it is also the entire platform of the fatal bottleneck, once the DTC due to a problem error, and this error is a system-level error, many problems we are powerless. If there is a problem, the entire application platform will not be able to complete any cross-service business process, which is actually very dangerous, you do not control the stability of the system.

This concludes that the DTC is used for general small business applications and is not recommended for medium-sized enterprise applications, not to say that this thing is bad, but that it cannot be controlled.

5.2. Transaction compensation (providing forward or reverse operations to make the data consistent in business)

The books written by world-class SOA experts refer to the use of "compensate" operations to complete the inconsistency of data, when we write a service method A, we need a service method A1 compensation interface to complete a service compensation operation. But in real business situations it's hard to implement a design that looks like it's beautiful and flexible. Without practice there is no voice, our company's technical team has implemented this kind of solution, but it is not ideal, it is not the technology itself and the technical team, but our platform business is too complex, it is difficult to "compensate" an already done operation.

This certainly depends on the project you are facing, the quantitative change caused by qualitative change, if you have all the amount of up, the "compensation" scheme is not practical, and it is difficult to "compensate" at the data level. In short, this is not a medium-to long-term solution.

5.3. Asynchronous EDA (Implementation of flexible distributed transactions based on asynchronous event streams)

EDA is referred to as "event-driven architecture". Multiple systems drive the operation of the entire business by propagating "events". Operations that do not have a tightly coupled synchronous call between systems are notified to the next business segment by issuing an asynchronous "event".

Perhaps you will have a question, asynchronous operation, is not the system delay between the very long, in fact, there are many mature message middleware inside the network is almost the millisecond level of delay, as for the cross-room to see the physical distance.

There are many benefits to asynchronous operations, and here I don't waste time repeating those benefits. Using EDA to achieve a loose transaction relationship between the system, to control the quality of the project, the non-functional requirements of the system, the number of bugs and so on may affect the business disruption of the place to establish an appropriate mechanism, so that these problems at the earliest possible online solution. For example, can implement unittest, continuous integration and other agile methodologies.

6. Summary

The same tool is used by the people, the real craftsmen are using a very simple tool to carve the impossible works of art, this is the craftsman feelings. Recently to the craftsman feeling more and more deeply, always thought oneself is a more like special person, this is not deviate from a big direction rushed into a small alley, until recently I realized, this is actually "software craftsman" spirit. But this does not mean not to consider the overall situation, this is only a kind of feelings, an attitude, for the architecture is also, for the code, do not think those seemingly insignificant problems ignore it, with the artisan spirit carved it.

Reference books: SOA Practice Guide, SOA Concepts, technology and design, lean software, UML and pattern applications, the art of software pre-construction

SOA Architecture design experience sharing-architecture, responsibility, data consistency