Read the story between Spring Boot, microservices architecture, and big data governance

Micro-Service Architecture

The birth of MicroServices is not accidental, it is the product of the rapid development of the Internet, the rapid changes in technology and the traditional architecture can not adapt to fast changes, such as the impetus of the emergence of multiple factors. In the Internet era, the products usually have two kinds of characteristics: rapid demand changes and large user groups, in this case, how to build a flexible, easy-to-expand system from the perspective of the system architecture, quickly respond to changes in demand, at the same time, with the increase of users, how to ensure the scalability of the system, high availability, become the system architecture

If the development of a large and full-scale system has been difficult to meet the needs of the market for technology in accordance with the traditional development model, then the idea of divide and conquer has been raised, so we have evolved from a separate architecture to a distributed architecture, from a distributed architecture to an SOA architecture, services are constantly being split and decomposed, and the granularity is getting smaller. Until the micro-service architecture was born.

Micro-service architecture is the inheritance of SOA architecture, but one of the most essential differences is that microservices are truly distributed and decentralized. Put all the "thinking" logic including routing, message parsing and so on inside the service, remove a unification ESB, light communication between services, is a more thorough split than SOA. The focus of the microservices architecture is on the need for a thorough component and service of the business system, with the original single business system split into multiple small applications that can be independently developed, designed, run, and operated, interacting and integrating between these small applications through the service.

Since about 2009, Netflix has completely redefined its application development and operational model, starting with the first step in microservices exploration, and until March 2014 Martin Fowler wrote an article microservices In a more understandable form, we define what a microservices architecture is. Martin Fowler The idea of a microservices architecture as an architectural model that advocates dividing a single application into a small set of services that are coordinated and co-ordinated to provide the ultimate value to the user.

Each service runs in its own separate process, and services and services communicate with each other in a lightweight communication mechanism (usually HTTP-based RESTful APIs). Each service is built around a specific business and can be independently deployed to a production environment, a class production environment, and so on. In addition, we should try to avoid a unified, centralized service management mechanism, for a specific service, should be based on the business context, choose the appropriate language, tools to build it.

Micro-service architecture and data governance

With the advent of the microservices architecture, the micro-service architecture has been found to improve the development model, but also introduced a number of issues, in all of these problems, the most important is immediately facing a problem is the problem of data. In the MicroServices architecture, we emphasize the complete component and service, each micro-service can be deployed and put into production independently, in fact, it means that a lot of micro-services have their own independent database.

The entire business data is scattered in the sub-service will bring two most obvious problems: 1, the business management system of data complete query, such as paging query, multi-conditional query, etc., how to integrate the data after fragmentation? 2, how to further analyze the data mining? These requirements may require analysis of the full amount of data and cannot affect the current business at the time of analysis.

From the technical solution, we generally have two options to deal with these problems, the first is to process the data online, the second is to process data offline.

The process of processing data online is done according to the standard interface of the microservices, and the backend needs the data of which system to invoke the interface provided by a microservices to obtain. Back-end management platform according to the requirements of the front end to different microservices system to , and then processing the returned data after the data returned. This scheme has two drawbacks: 1) on the one hand, the micro-service data side needs to provide data interface, on the one hand, the data users need to write the calling method, and the caller needs to write a lot of code for data processing, 2) in the various micro-services to adjust the data will affect the normal business performance of microservices.

Offline processing data solution, is to synchronize the business data in a quasi-real-time to another database, in the process of synchronization of data integration processing to meet the needs of business parties to data, data synchronization, and then provide another service interface professional responsible for external output data information. This scheme has two features: 1 The data synchronization scheme is the key, the technology selection has many, how to choose the technical solution to suit the company's business; 2) offline data processing has no effect on the normal service of microservices.

Both programmes have been implemented in my previous work and individuals are more inclined to use the second option.

MONGDB and data analysis

MongoDB is called the developer's friendliest database, no longer emphasizes rows and columns in traditional relational databases, the entire table can be thought of as a Json document, and MongoDB is considered to be the most like NoSQL database in a relational database in NoSQL, Databases, collections (Collection), document objects (documents) that resemble relational databases are preserved.

MongoDB is currently one of the most popular non-relational databases, in the latest database rankings MongoDB ranked fifth, in all non-relational database ranked first, very widely used in domestic and foreign internet companies.

The biggest feature of MongoDB is the support of the query language is very powerful, its syntax is somewhat similar to the object-oriented query language, almost can achieve similar relational database single-table query most of the functions, but also support the indexing of data. MongoDB's implementation of high-availability and read-write load balancing is very concise and friendly, and MongoDB comes with the concept of a replica set that can be highly available, read-write, and load balanced by designing the right set of replicas and drivers.

These features of MongoDB are very handy for high-performance querying of data, and MongoDB supports Aggregate and Mapreduce using the idea of divide and conquer to handle large-scale data analysis. Spring Boot's support for MongoDB is very friendly, and with spring boot it is very convenient to handle MONGODB queries and operations, and spring boot also provides a component package to support the use of MongoDB.

MongoDB 4.0 announces that it will formally support ACID transactions, and future mongodb imagination space is even greater! Therefore, Mongdb + Spring Boot is one of the best choices for data analysis in the MicroServices architecture.

Let's talk about Spring Boot.

Spring Boot is a new framework provided by the Pivotal team designed to simplify the initial setup and development of new Spring applications. The framework uses a specific approach to configuration, which eliminates the need for developers to define boilerplate configurations. With Spring Boot, you can greatly simplify the development model and all the common frameworks you want to integrate with the corresponding component support.

Spring boot is based on spring, and spirng boot itself does not provide the core features and extensions of the spring framework, but is for rapid and agile development of a new generation of spring framework-based applications. That is, it is not a solution to replace spring, but rather a tool that works closely with the spring framework to enhance the spring developer experience. While it integrates a large number of commonly used third-party library configurations (such as Redis, MongoDB, Jpa, RabbitMQ, Quartz, and so on), these third-party libraries in the spring boot application are almost zero-configurable out-of-the-box, most of the spring boot Applications require very little configuration code, and developers can focus more on business logic.

Spring boot has been sought after by the open source community, and spring boot officially offers many starters to facilitate the integration of third-party products, and many mainstream frameworks have been actively integrated, such as Mybatis. Spring official attaches great importance to the development of Spring Boot, on the spring homepage of the main recommendation of the introduction, is currently one of Spring's official focus on the development of the project.

Spring boot itself has grown particularly fast, and since the release of Spring Boot 1.0 in April 2014, the version has been updated very frequently, and I used it in 2016 as 1.3.X, and Spring Boot 2 has now been released. The 0,spring Boot 2 integrates many of the latest technology and new features, and greatly optimizes the Spring Boot 1.0 API. Spring Boot has quickly become a popular technology once it is launched, and it can be seen from this conclusion:

For the 2014 to 2018 Spring boot Baidu Index, it can be seen that the launch of Spring Boot 2.0 triggered a search peak.

Spring Boot and MicroServices architecture

With the continuous development of Spring, more and more areas involved, project integration needs to be combined with a variety of documents, slowly become less easy to use, contrary to the original idea, even the person to configure hell. Spring boot is an abstraction of the development framework in such a context, in order to make it easier for everyone to use spring, easier integration of common middleware, open source software, on the other hand, spring boot was born, is in the micro-service concept is slowly brewing, The development of Spring Boot incorporates the concept of microservices architecture, enabling the technical support of the micro-service architecture in the Java domain.

Spring Boot is a brand new framework that comes from the spring family, so spring has all the features it has and is easier to use, and spring boot has a lot of settings that are set by default, and most spring boot applications Only a small Spring configuration is required. Spring Boot has developed a number of application integration packages that support the vast majority of open source software, allowing us to integrate other mainstream open source software at a very low cost.

Spring Boot Features:

• Use the Spring Project Guide page to build a project in a few seconds
• Easy to export various forms of services, such as REST API, WebSocket, Web, streaming, Tasks
• Very simple security policy integration
• Support for relational and non-relational databases
• Supports runtime inline containers, such as Tomcat, Jetty
• Powerful development package, support hot start
• Automatic management of dependencies
• Self-bringing application monitoring
• Supports a variety of IEDs, such as IntelliJ idea, NetBeans

These features of Spring Boot are very convenient and fast to build independent microservices. So our use of the Spring Boot development project will bring great convenience to our traditional development, so if you have used spring boot to develop a project, you will no longer be willing to develop the project in the same way.

To summarize, using Spring Boot can at least bring us the following improvements:

• Spring boot makes coding easy, and spring boot provides a rich solution to quickly integrate solutions to improve development efficiency.
• Spring boot makes the configuration simple, spring boot provides a rich starters, and the integration of mainstream open source products often requires simple configuration.
• Spring boot makes deployment simple, and spring boot itself launches the container, with just one command to start the project, and the combination of Jenkins and Docker Automation operations is easy to implement.
• Spring boot makes monitoring simple, spring boot comes with monitoring components, and actuator easily monitors the status of the service.

To summarize, Spring Boot is one of the best microservices architecture technology in Java.

Micro-service architecture, Spring Boot, Big Data governance

After understanding the MicroServices architecture, Spring Boot, and big data governance, we found an interesting thing: microservices Architecture is an architectural idea, an inevitable result of the continuous development of architecture, with the characteristics of flexible, easy to expand, fast application, scalability, high availability, etc. Micro-service architecture idea of the introduction of the technology has a higher demand, in this context spring boot bred, spring boot born, from the beginning to stand in a relatively high starting point, and after the development of these years, the ecology is perfect, Spring Boot has been well-deserved to become The hottest technology in the Java world.

Micro-service architecture, the data is separated into N independent microservices, how to deal with the market, the business of large amounts of data query, analysis is very urgent, using Spring Boot and MongoDB can easily solve this problem, by technical means to split into N microservices data synchronization to MongoDB cluster, in the process of synchronizing data cleaning, to meet the company's business needs. Spring Boot's support for MongoDB is very friendly, on the one hand, spring Data technology pre-generated a lot of common methods for ease of use, on the other hand, Spring boot package of distributed computing related functions, you can let us in a more concise way to achieve statistical query.

Spring Boot is the best-in-breed technology for Java-domain microservices architectures, and the spring BOOT+MONGODB solution is one of the most optimal solutions for data governance under the MicroServices architecture.

Of course, if we are unfamiliar with the microservices architecture, Spring Boot, and MongoDB, we may need to go a lot of detours. The wrong technical solution will be the late micro-service landed very big trouble, increase the additional development workload, I have deep experience, the appropriate technical solution can save more than 60% of the workload.

To this end, I wrote a special column, "Micro-service technology architecture and Big data governance," to explain the micro-service architecture and big data governance technology landing. In the work to practice the most practical representative of some of the content, in the form of 51CTO columns to show. The course content will take a holistic view of micro-service practices and data analysis, explain the micro-service architecture, Spring Boot, MongoDB, synchronization scheme, data analysis and other content, to help the vast number of technical friends less detours, master micro-services and data analysis technology.

This column contains a total of 18 lessons, which focus on four main blocks:

• A, micro-service architecture and Spring Boot;
• Second, Spring Boot and MongoDB real-combat detailed, introduce the typical problems encountered in combat;
• Third, the micro-service structure of data governance, data synchronization, real-time change;
• Iv. experience and lessons learned in the practice of micro-service architectures.

After studying this column, you will have a better understanding of microservices architecture practices, Spring Boot and MongoDB usage, data processing under microservices architectures, and the pain points of using Spring boot to develop microservices projects and leverage related technologies to address data governance in microservices architectures.

Address: Micro-service technology architecture and big data governance combat

Read the story between Spring Boot, microservices architecture, and big data governance