DevOps is committed to the unified integration of software development and software operation and maintenance. It is characterized by a strong advocate for comprehensive automated monitoring of all aspects of building software (from integration, testing, release to deployment and infrastructure management). The goal is to shorten the software development cycle, increase the frequency of deployment and more reliable release, consistent with business goals.
In the actual operation process, for many companies, there is no clear boundary between developers and operators, and even if the responsibilities of operations and development are separated, it does not mean that the career development directions of the two will be very different. . In fact, in the actual operation process, developers and operation and maintenance personnel will still use the same tools, unified processes, consistent management ideas, and optimize the process through a series of management methods and tools.
What is continuous integration and continuous delivery?
Continuous Integration and Continuous Delivery are also the two most basic enterprise-level R&D and delivery activities in DevOps.
Continuous integration comes from the idea of agile project management. Its core is that team members should often integrate their work, usually requiring integration once a day, of course, team members can also be required to integrate multiple times a day. After each integration, continuous construction tools will be used to automatically run automated construction methods (such as compilation, unit testing, integration testing, and system testing), and the results of the integration will be verified, thus enabling the early discovery of unknowns in the enterprise management process. The goal of the problem. In the traditional software delivery, all problems may be backlogged into the overall system test or even UAT (user acceptance test) link, which makes the software testing time longer, and even causes the software problems to flow to the customer site, allowing the customer to become The condition of the little white mouse happened from time to time.
In fact, the seemingly simple integration is not simple. It should be an iron law in the process of enterprise management. Only strict implementation can ensure that the software is always available; otherwise it means the completion of the so-called delivery process. What percentage is only an empty vernacular.
Continuous integration is often inseparable from continuous delivery. In the book "Continuous Delivery" translated by Mr. Qiao Liang, the author Jez Humber said: Continuous delivery is a capability, that is, it can be safely and quickly delivered in a sustainable manner. Code changes (including features, configurations, defects, and trials) are deployed to the production environment for users to use. The author of this book also pointed out in the last chapter of the book: It (continuous delivery) is not only a new software delivery methodology, but also a brand-new paradigm for software-dependent businesses.
Continuous delivery and continuous deployment seem to be similar, but there are differences. The former often refers to pushing the environment to the user so that the user can reach and use them; while deployment simply installs the software package on the target computer, the user may also Cannot be used directly.
For Internet-oriented software companies, a complete set of continuous integration/continuous delivery processes has often been established in the past few years, but for some companies in the rapid development period, it is still relatively backward. The main reason is that the human and material resources behind the rapid development of the company in the past can be sufficient to ensure that the output of the company can meet the needs of the company’s development. However, when the team size cannot keep up with the needs of the company’s business development, repeated labor and The seemingly worthless waiting period, the backlog of test tasks and the realization of software functions that cannot be effectively measured will actually cause the company's management costs to increase further.
Where can I get a systematic methodology?
For developers who need to build a complete environment, the online information is very complete. This article also tries to summarize all aspects of the content as much as possible, and strives to provide developers with an out-of-the-box operation process, but for those who want to For developers of systematic learning continuous delivery or knowledge in the field of DevOps, you are not only satisfied with putting the environment together, then you should read the book.
There are a lot of excellent works in the field of continuous integration and continuous delivery, and I think that the work "Continuous Delivery 2.0" from Mr. Qiao Liang is a fine product. Mr. Qiao Liang is an experienced industry expert and has accumulated in his career In addition to the very rich product development experience related to this field, he has also personally participated in the continuous delivery process optimization process of many enterprises. These experiences have allowed him to analyze continuous delivery issues from a more comprehensive perspective. This book introduces many management methods, project cases, and tools that can be used directly, which can bring a lot of thinking to developers who need to make a difference in this field.
In essence, for a company that wants to
practice continuous integration/continuous delivery, building tools is not the core difficulty. The difficulty still lies in how to use the idea of agile project management to achieve fine-grained task splitting of software and the ability to perform individual tasks. Better testing, maybe TDD is a good model, but it may put forward higher requirements for developers' basic skills, which will cause TDD to fail to land.
How to quickly verify product requirements? In "Continuous Delivery 2.0", a series of methods, such as decorative windows, minimum feasible feature method, special zone method, directional search method, scarecrow method, and minimum feasible product method, were established. By establishing a rapid verification model, Enhancing the entire process of software from demand to realization can bring a lot of convenience to the enterprise.
In the research and development phase, you can use feature branches and feature switches to use the beauty of the git source code management tool branch to organically couple requirements and code, and at the same time rely on project management tools to achieve from requirements => implementation = 》The complete closed loop is released, which provides double insurance for the verification of the demand; the feature switch I first saw in the book "Cheetah Action-Agile Transformation" by Liu Hua teacher, by using the form of soft switch to avoid undeveloped Going online early poses a huge risk, and this method is also mentioned in this book. Feature branching components can also be used in .NET, and I will try it later.
As for how to reduce the waiting period, the methods mentioned by the author are:
1. Let the value flow through "pulling". For example, if it is a stranded production line, by expanding the processing capacity of the bottleneck, more demand can be delivered quickly. This approach seems to temporarily improve the processing capacity of the link and protect the team. The output is obviously not a good measure. A more reasonable measure is to determine the upstream processing capacity based on the downstream production capacity, and the downstream will drive the upstream demand. This objectively requires that the granularity of tasks and requirements be further homogenized, and the requirements are divided into smaller requirements that are easier to execute and have a similar workload, making the development process smoother.
2. Task self-service: that is, let the team master certain general skills so that when other people are blocked, they can complete related tasks synchronously, avoiding the lag of the overall process caused by the blocking of certain key tasks.
I won't describe the essence of the book here too. Those who are interested can start a book with absolute value for money.
2. Overall process and environment deployment
Next, I will enter the topic of this article. First, I will build a simple enterprise-level continuous integration/continuous delivery management process, and then introduce the process implementation process in more detail.
1. For the code submitted by the dev branch, after the process of code editing, static code scanning, and automated unit testing, after running through, there are testers to test the code, and after the code test passes, submit to the master branch through a pull request Of reviewers conduct code review and merge.
2. The tester confirms the code submitted by dev, and the master branch code reviewer conducts code review. After passing, the code is confirmed, and a generation package for release is generated.
Components and instructions involved
In the process, the following tools are used, which can be installed in sequence.
1. Install OpenJDK
2. Install Jenkins and related plugins
3. Install PostgresDb
4. Install SonarQube
5. Install dotnetsdk3.1
6. Install git
7. Install the nexus package manager for windows version to implement package management.
8. Install Qy Wechat Notification or HTTP Request to implement enterprise WeChat reminder.
Well, the environment installation will not be introduced. .
Supplementary instructions for installation
1. The version installed by Jenkins is 2.190.3, and the version installed by OpenJDK is openjdk12.0. After installing jenkins and openjdk, you need to set environment variables.
2. The version installed by SonarQube is 7.9.1. According to the instructions on the official website, the recommended database includes: sqlserver\oracle\postgresdb. In 7.9.1 and later versions, mysql is no longer recommended.
3. SonarQube uses an embedded database based on H2 memory database by default, which can be used in a test environment, but it is not recommended for production environments.
5. After installing sonarqube and the database, you need to modify the database configuration address in the sonarqube/conf/sonar.properties file and restart the sonarqube service.
In the windows system, click InstallNTService.bat in the sonarqube-xxx\bin\windows-x86-64 folder to install the SonarQube service, and StartNTService.bat is used to start the SonarQube application service. If the database configuration fails, SonarQube will fail to start with the following error:
[sonar-1510653879773] exception caught on transport layer [[id: 0x346b46fb, /127.0.0.1:59330 => /127.0.0.1:9001]], closing connection
java.io.IOException: An existing connection was forcibly closed by the remote host
6. Because the self-built Nuget package source manager is used, when building, it will prompt an error and jenkins will use
Jenkins project types
In Jenkins, there are different types of projects such as free style, single pipeline, multi-branch pipeline, and multiple configuration projects. You can choose according to the actual situation. In my attempt, I have summarized the three different types of projects that are applicable. Scenes:
