Lightweight micro-service architecture and best Practices _ architecture

first, micro-services will become lightweight

Architecture needs to be designed by people, who are called architects. Many people may not have given the architect the title, but they are doing the work of architecture. We believe that the structure of this work will always need to be done by people, may not be in the short term by the machine to replace. If we do not understand what the architecture is, or if we are puzzled by the responsibilities of the architect, it is difficult to get the architecture work to the ground effectively. We will recognize the architecture in this section and redefine the architect's responsibilities. In addition, the evolution of the architecture is a tortuous process, but it is not difficult to see the development of the structure of the law, and even can speculate on the future development of the structure of the trend. We believe that micro-services must not be the end of the architecture, it may just be the structure from the heavyweight transition to a lightweight bridge, we are the engineers who designed and built this bridge.

Now we start with the architecture and architect perspective to open the architectural adventure of lightweight micro-service.

1. Architecture and architect

Perhaps the vast majority of programmers want to be a good architect, every day to engage in the technical framework of the relevant work, edit point frame code, Draw point architecture diagram, write a ppt, handsome stand on the platform for programmers technical training. It is generally agreed that the architect's code is less than others, but the salary is more than others, the architect is the technical team of the most skilled people, the other people's technical problems, the architect's eyes are a piece of cake.

Is such a person really an architect?

We believe that they are not architects, but technical experts. In fact, architects and technologists are fundamentally different, from their focus on the technical direction, the architect is more inclined to technical breadth, and technical experts are more inclined to technical depth, as shown in Figure 1-1.



Figure 1-1 The difference between architects and technologists

In other words, architects need to have a strong comprehensive ability, they need to contact a wide range of technology, but they do not have the technical expertise of technical experts so deep. If we want to be an architect, then we should not devote all our energies to a certain technical field, but we should learn to distract ourselves from the level of our attention, so that we have to have a certain depth in many technical fields.

In addition to the technical needs of the architect of the "hard skills", but also need to constantly improve their "soft skills", such as communication, organization, learning and other skills. Sometimes soft skills can be more important than hard skills, and even soft skills can affect career development. Without better soft skills, architects will not be able to successfully hand over their designs to programmers and instruct them to truly fall into the ground. It is through their comprehensive ability that architects lead the technical team to solve the emerging technical challenges.

What is the architect's responsibility.

Our answer is: the development of norms + guidance to the ground.

We refer to this specification as a framework for the reasonable and landing technical specifications of the architect based on the business requirements.

Make the architecture work as if we were walking on two legs, the left leg was taken to say "norm-setting", and the right leg was followed by "guiding the ground", as shown in Figure 1-2.


Figure 1-2 Architect's responsibilities

If the left leg is taken and the right leg is not followed, it is not an architect, it may be someone who needs a crutch. However, we are surrounded by such unqualified architects, they only know how to make norms, but ignored the guidance of the landing. If the schema cannot be landed, then it cannot be called a schema.

In addition, some architects believe that the architecture is only a technical aspect of the problem, the structure of their own design should be used in the market the most popular new technologies, such as other companies in the use of micro-services, then their own companies to use. If you leave the architecture work out of business requirements, we don't think it's about architecture, it's about playing with technology. Leaving the business to design the architecture is a disrespect to the architecture. Micro-service is an application system architecture that requires architects to design around the business.

However, we must not go to micro services for micro services.

Architects who work on micro-service architectures have more comprehensive skills than traditional architecture architects. They are not just system architects, they are business analysts, and their responsibilities are huge and challenging.

In the big direction, the micro-service architect needs to have the following basic responsibilities. Analyze business requirements and segment micro service boundaries. Define schema specifications and document standards. Ensure a smooth landing of the micro-service architecture. Improve the micro-service architecture and improve development efficiency.

Responsibilities and challenges are often inseparable, and micro-service architects must confront and overcome these challenges. Architecture needs to adapt to changing business requirements. Architecture with stability, scalability, security, fault tolerance and so on. Enable the technical team to understand the concept of micro-service deeply. Demonstrate the value of a micro-service architecture.

We believe that the transformation of traditional architects into micro-service architectures requires a deep understanding of the business rather than a superficial understanding of the requirements. Business and requirements are not the same thing, the back of the business reflects the needs of the customer, and demand is often the product manager according to the business needs to specify the solution. As a micro-service architect, we need to understand the nature of the business through the representation of requirements. The second thing to do is to optimize the architecture to make the architecture simpler and lighter. We will be the best performance of yesterday, as the minimum requirements today, only the technical requirements of their own, in order to make the architecture better.

2. Architecture Evolution Process

In other words, the world trend, a long time must be combined, for a long time, for the evolution process of the architecture, also in line with this law.

The earliest applications actually did not have any architecture, because the business was simpler and no architecture might be reasonable, as shown in Figure 1-3.


Figure 1-3 No schema

But as the business continues to become more complex, we realize that the architecture can be layered horizontally, such as the performance layer, the logic layer, the data layer, and so on, we can achieve each layer on the different layer of the content of concern, we call it "separation of concerns." But at this time the architecture is more like a "plate", each layer can not be separated, so we also call such a framework "monolithic architecture", as shown in Figure 1-4.


Figure 1-4 Horizontal layered Architecture (Single block schema)

More complex changes have taken place from this architecture, the hierarchy is getting deeper, and is no longer limited to horizontal hierarchies, and in fact more and more applications are being implemented around different business requirements, and there is a "vertical hierarchy" in which each vertical application is actually a separate subsystem, Together they form the entire application system. However, these subsystems can generally be deployed on different servers, which can be distributed across different geographies, and we call them "distributed architectures," as shown in Figure 1-5.


Figure 1-5 Vertical layered architecture (distributed architecture)

Business has not stopped developing, and so is the evolution of architecture. With more and more calls between distributed applications, the complexity of the system increases dramatically, and people want to find a way to reduce the coupling between distributed applications. There is a service-oriented architecture (SOA) that is expected to become a "silver bullet" for solving the complexity of distributed application systems, but the reality backfired. Not only is the complexity of the application not resolved, but it also complicates the architecture and also creates a large number of SOA business products that make people more fearful of SOA as a complex and expensive synonym, as shown in Figure 1-6.


Figure 1-6 Service-oriented architecture (SOA)

As the Internet industry continues to develop, users of the product experience requirements are more and more high, the value of the front-end is gradually highlighted, at this time the "front and rear separation" trend, the front-end engineers focus on the interface and data rendering, back-end engineers focus on business logic and structure, the front and back end only through rest The API interacts, the division of work is clearer, and the development efficiency is more efficient, as shown in Figure 1-7.


Figure 1-7 Front and back separation architecture

It seems like a long time ago, no technology can be compared with the SOA of the year, in addition to the micro-service architecture. Since the concept of micro-service was first proposed in 2014, it has been the core topic in the field of application architecture in recent years. But it is often easy to think of SOA in the year, thinking that micro services have the same purpose as SOA, but the implementation details are not the same, as shown in Figure 1-8.


Figure 1-8 Micro-service architecture

What is the difference between a micro-service and an SOA?

We believe that micro-service is a kind of landing plan for SOA.

SOA is a service-oriented architecture idea, and micro-service advocates the same idea. Micro-service is a large monolithic architecture, split into multiple fine-grained service architectures. What's more, the micro-service test is to understand the business and make a reasonable segmentation of the service boundaries. The concept of micro-service is more likely to fall than SOA because it is not conceptual innovation, but a breakthrough in technology, especially the popularization and application of container and automated operation and maintenance technology.

We have always believed that micro-service is not the end of the architecture, perhaps the beginning of the new era of architecture.

3. Development trend of micro-service architecture

The scope of the micro-service architecture is quite extensive, so we might as well speculate on the development trend of the micro-service architecture from multiple perspectives.

From the perspective of micro-service development, we believe that the development framework of micro-services will become more diversified.

Developers can use a more appropriate development framework to achieve a micro-service business implementation, rather than adhere to a certain programming language, just to ensure that the external provision of a unified API interface. You can even separate queries from modification operations, and query operations can be implemented in a more lightweight programming language, while modification operations involve transactions that are generally guaranteed by the transaction characteristics of the development framework.

We firmly believe that micro-services will adhere to the lightweight technology route.

What exactly is lightweight.

We believe that lightweight must contain three features: ease of use, speed, and stability.

We hope that the technology involved in the micro-service architecture can be quickly started, the runtime does not occupy too much system resources and performance is outstanding, and can be long-term stable operation.

From a micro-service deployment perspective, we believe that the deployment process for micro services will become more automated.

Deploying a micro service is no longer done by hand, because it is inefficient and error prone, and we tend to use software tools to automate it. To achieve this goal of automating deployment, we are often unable to one-step, the most reasonable way is "let it run first, then let it run faster." In other words, early automated deployment may not be complete enough, more or less there will be some human involvement, in fact, these are all normal, but we need to continue to optimize, and strive to replace repetitive manual operations through Automation technology.

Finally want to say, although the automation is good, but do not automate to automate, perhaps some links by hand processing is the most effective way.

II. Preliminary preparation of micro-service architecture

To build a micro-service architecture is not an easy thing, we should not only have a profound understanding of the concept of micro-services, but also to study a large number of technical tools, and master their strengths and weaknesses, and ultimately need to combine technical team to understand the extent of technology, choose the most appropriate technology selection. These pure technology tools are just micro-service infrastructure, and we need to make a reasonable segmentation of the micro-service boundary around the real business scenario on this infrastructure. We will introduce the iceberg Model of micro service in this section, you will see the full picture of the micro-service architecture from this iceberg, and then we will dive into the world below the iceberg to explore the eight centers of the micro-service infrastructure, and finally we will introduce some principles and techniques for splitting the micro-service boundaries.

Let's start with the micro-service iceberg model, which seems to be a lot bigger than we thought.

1. Understanding the micro-service architecture Iceberg Model

Some people think that the use of the spring Boot development Framework is to have a micro-service, in fact, such a recognition is not correct. Spring boot is just a micro-service development framework, and can only be used in Java applications, it is undoubtedly the tip of the iceberg of micro-services. In addition, we recommend that you combine your own business scenarios and choose a more appropriate programming language and development framework to implement micro-services rather than a programming language.

Some people think that the use of Docker into the micro-service era, in fact, this is not true understanding. Docker is just a container technology that encapsulates a micro-service application, changing the way applications are delivered and accelerating the speed at which a micro-service architecture can be landed. To be sure, if there is no Docker container technology, perhaps today we will not be able to hear the concept of micro-service.

If the technology stack involved in a micro-service architecture is likened to an iceberg, the most obvious part of the iceberg is the development framework and container technology of the micro-service, and the technology that Spring boot and Docker all belong to the iceberg.

What exactly is the technology under the iceberg?

We believe that the technology under the iceberg is the cornerstone of the entire micro-service architecture, which forms the infrastructure of the entire micro-service architecture. For example, the Zookeeper service registry, the Node.js service Gateway, the Jenkins continuous deployment system, which we learn in the previous book, are all part of the iceberg.

We can use a picture to describe the micro-service architecture, as shown in Figure 1-9.


Fig. 1-9 The iceberg Model of micro-service architecture

Because the service registry centrally manages the service configuration of a micro-service, we also call it a "registry", because the service gateway is the only portal to the front-end application, so we call it "call center", and we also call the continuous deployment system the "Deployment Center." These centres are all pooled under the micro-service iceberg model, and there are other centres of function, which together form the infrastructure of micro-services.

2. Micro-service infrastructure under the iceberg

The micro-service infrastructure under the iceberg actually includes eight centers. Registration Center: The Center for registering information about the configuration of micro service, we choose Zookeeper implementation. Call Center: Used to provide a unified portal to the front-end call, we choose Node.js implementation. Deployment Center: For compiling and packaging the micro-service source and deploying it to the Docker engine, we choose Jenkins implementation. Log Center: Used to collect and manage the logs generated in a micro-service application, described in detail in chapter 2nd. Monitoring center: To monitor the real-time operation of micro-service, the 3rd chapter will be described in detail. Tracking Center: The call trajectory for the final micro-service, described in detail in chapter 3rd. Message Center: Used to decouple the invocation relationship between micro-services, which is described in detail in the 5th chapter. Configuration Center: The configuration parameters needed to manage the micro-service application are described in detail in chapter 7th.

Perhaps when you see the above eight centres, there will be some doubts: many people say that micro-services are to be centralized and why we have to provide these centres.

In our view, the centre is divided into two categories: one is the centre of business meaning, the other is a centre that does not contain business significance (it is just the technical aspect).

In a micro-service architecture, we should remove the center of business meaning rather than remove the broad center at the technical level. For example, there is no business process within the call center we are designing, it is a purely technical framework. The same is true for other centers, which do not contain any business, otherwise we should remove it.

3. Segmentation of micro-service boundaries according to business

As anyone who has studied micro-service knows, we need to segment the service boundaries according to the business. The truth may be understood, but you may still not know what to do. For example, what are the key steps in splitting a micro-service boundary and what are the important principles involved?

After a lot of micro-service practice, we summarize the following five steps to help us to effectively segment the micro-service boundary. The first step: comb the business process.

The first thing we need to do before we slice the micro service is to comb the business process. Find business experts to consult and communicate with them to understand the real business process and draw it into a flowchart. For overly complex business processes, we can also draw a separate flowchart and add related process instructions. It is also possible to provide a statechart diagram that describes the changing process of the state involved in the business process.

Spending more time analyzing business processes is not too much, and it's worth every minute of it now. Extraction of public services.

In a business process that is not very relevant to the business, we can consider stripping it out and forming a public service. For example, mail delivery, file upload, other third-party interfaces, and so on. Each public service corresponds to a micro service, each of which has its own API, and each API has its own input and output. These APIs must be documented so that other services can be invoked.

Under normal circumstances, the extraction of public services are not very changeable, we must find ways to change things from the changing world to extract. Step three: Define business services.

When the public service is extracted, the remaining part of the business process is the business service. It is recommended that when you start implementing micro services, do not cut the boundaries of business services too thin, you can consider "a large number of pieces", but you need to ensure that each service as far as possible with no dependencies. In other words, each service is independent, although the size of the service may be larger at this time.

Let's make sure these big services are running on the micro-service infrastructure, and then refining them to smaller services. Step Fourth: Design the data model.

To drill down into each business service, the first thing we need to do is define the data model that is involved in the underlying layer, also known as the "domain model." This will involve database table structure design, and data model and relational design. Design at the data level is critical, and if the part is not designed, it will increase the cost of realizing the micro-service later.

The design of the data model also needs to be documented, which will guide the backend engineers to complete the micro service implementation smoothly. Step Fifth: Define the Service interface.

After the underlying data model is designed, we convert the perspective to the top-level service interface. The service interface is actually a set of APIs that require a single responsibility and need to recognize its business meaning by name. It is recommended to ensure that each API is named globally unique, and that each API has its own version number, which can be reflected in a growing way.

Service interfaces also need to be documented, which are typically written by a backend engineer and provided to the front-end and test engineers for reading.

third, lightweight micro-service architecture diagram

As a micro-service architect, we need to understand not only the business, but also the service boundary accurately, and we need to understand the micro service and choose the technology selection reasonably. In our view, the micro-service architecture is actually divided into two parts, one of which corresponds to the deployment phase and the other part corresponds to the runtime. These two parts contain a lot of technical tools, we need to combine various factors to consider, choose the most appropriate technology selection to build a micro-service architecture, and ensure that it can maintain lightweight.

This section focuses on the two phases of deployment and operation of micro-services, describing lightweight micro-service architectures graphically.

1. Lightweight micro-service deployment Architecture

When the developer completes the details of the micro-service, the first thing to do is to ensure the availability of the code they write, and they often use the Unit test tool to ensure that the link is not problematic. When they submit the source code and push it to the repository, the deployment Center takes the source code from the repository and performs the compilation and packaging operations.

Not only this, the deployment Center also needs to obtain configuration parameters for the corresponding running environment from the configuration center, generate the appropriate configuration files, and copy the configuration files to the Docker mirror with the application, and eventually upload the mirror to the mirrored warehouse so that the specified mirrors can be downloaded from the mirrored warehouse later. Thus running the corresponding Docker container.

In addition, the deployment center scans the source code and automatically generates API document sites so that other technicians can view the API documentation contained in the latest deployment services from the document site at any time. Of course, we can also only provide a simple code implementation of the required micro services, this allows the output time of the micro-service document to be as early as possible so that other technicians can understand the relevant API information of the micro-service earlier, then complete the more detailed code implementation, which is our recommended method of work.

When the Docker Mirror is uploaded to the mirrored warehouse, the deployment Center can launch the corresponding Docker container based on a specific mirror in a different operating environment. For ease of description, we refer to the container as a "service container" that contains the applications that host the micro services and their configuration files.

When the service container is started, its configuration information is automatically written to the registry, and at the same time, the deployment center connects to the registry and sets the version number of the service to make it possible to identify the currently available services based on the version number on subsequent invocation of the service.

We can draw the above process as a schema diagram, as shown in Figure 1-10.


Figure 1-10 Lightweight micro-service deployment architecture

Visible, in the micro-service deployment phase, the deployment center is the protagonist, it dominates other components, so that services can be successfully deployed, we need to ensure its stability.

2. Lightweight micro-service operation Architecture

When a user accesses an application system through a browser or mobile end, the request first enters the service gateway because it is the center of all request calls and we call it the call center. It's a center without any business, but we need to make sure that it does little enough to make it less of a call bottleneck for the entire application system.

The call center will then connect to the registration center, and the IP address and port number (that is, the service address) where the service is obtained from the registry through the service name, known as "service Discovery," where the call center can invoke a specific service container based on the service address as a reverse proxy, a process called a "service invocation".

Events may be triggered in the service container, and these events will be written to the message center as messages, so that other services can listen to the message center and get the appropriate message from it. This scheme can solve the coupling problem between services, and simultaneously can switch the synchronous call to asynchronous call, and improve the throughput of the whole application system.

A large number of logs are generated when the service container runs, and we can write these logs uniformly to the Log center and query specific log information on the console provided by the Log center. In addition, the log Center can also help us to quickly locate and analyze the abnormal state of the system.

To see if the service container is functioning properly, we can use the graphical data that the monitoring center outputs to judge. The monitoring center will continuously collect the running state in the service container, including CPU, memory, hard disk, network, and JVM memory usage of the application.

Since micro service is difficult to cut cleanly, there will inevitably be a small number of calls between the services, we can write the information generated by each call to the Tracking center, and through the Tracking Center provided by the graphical interface to see the call between the service and the resulting call latency, so that the performance bottlenecks resulting from the service invocation can be analyzed.

We can draw the above process as a schema diagram, as shown in Figure 1-11.


Figure 1-11 Lightweight micro-service runtime architecture

Visible, in the micro-service runtime, the call center is the protagonist, the registry as its data source, service container as its calling target, it needs to have good performance and high availability characteristics.

3. Lightweight micro-service global Architecture

We use a map to consolidate the lightweight micro-service deployment architecture with the running architecture, which is the full picture of the lightweight micro-service architecture, as shown in Figure 1-12.


Figure 1-12 Lightweight micro-service global architecture

The architecture diagram shown in Figure 1-12 includes 12 components, the code warehouse, deployment Center, registry, Mirror Warehouse, call center, service container, 6 components are described in the previous book, the remaining configuration center, document site, Message Center, log Center, monitoring Center, Tracking Center These 6 components will be explored in depth in the next chapter of the following book.

Iv. Summary

In this chapter, the lightweight micro-service architecture is described macroscopically, which provides a clear blueprint for the future exploration process. First, we start with architecture and architects, briefly review the process of architecture evolution and the development trend of micro-service, we believe that the emergence of micro-services is inevitable, and it will be in the direction of lightweight development. We then explored the work to be done before building a micro-service architecture, and the "iceberg model" of the micro-service architecture, which focuses on the micro-service infrastructure below the iceberg, and introduces the methods and techniques for splitting the micro-service boundaries, which we believe It is one of the functions of the micro-service architect to split the micro-service boundaries reasonably. Finally, we looked at the micro-service architecture from two aspects of deployment and operation, and ended this chapter with an architectural panorama, followed by a section on the relevant part of the architecture diagram, and we will choose the most appropriate technology selection to build this lightweight micro-service architecture.

This article turns from: http://geek.csdn.net/news/detail/234078