Introduction:
DevOps is a collaborative process between software developers and IT staff. The goal is to efficiently automate software delivery and infrastructure change processes. In the cloud-native era, how can enterprises use
DevOps to achieve rapid, stable, efficient and safe iteration of products to unlock business value?
What is cloud native
In order to solve the problems of slow upgrade of traditional applications, bloated architecture, inability to quickly iterate, failure to locate quickly, and failure to resolve problems quickly, the concept of cloud native was born.
Pivotal is the originator of cloud-native applications, and launched Pivotal Cloud Foundry cloud-native application platform and Spring open-source Java development framework, becoming a pioneer and pathfinder in cloud-native
application architecture.
DevOps is a collaboration between software developers and IT operations. The goal is to automate software delivery and infrastructure change processes. It creates a culture and environment in which software can be built, tested, and released quickly, frequently, and more reliably;
Continuous delivery allows individual application changes to be released as soon as they are ready, without having to wait for events such as bundling with other changes or waiting for maintenance windows. Continuous delivery makes the release behavior dull and reliable, so companies can deliver frequently with lower risk and get feedback from end users faster until deployment becomes an integral part of business processes and enterprise competitiveness;
Microservices are architectural methods for developing applications as a collection of small services, where each service can implement business functions, run in its own process, and communicate via HTTP APIs. Each microservice can be deployed, upgraded, expanded, and restarted independently of other services in the application, usually running as part of an automated system, and can frequently update the application in use without affecting the end customer;
Compared to standard virtual machines,
containers provide both efficiency and speed. A single operating system instance uses operating system-level virtualization to dynamically divide between one or more isolated containers, and each container has a unique writable file system and resource quota. The lower cost of creating and destroying containers, combined with the high packaging density in a single virtual machine, makes containers the perfect computing tool for deploying individual microservices.
the establishment of the Cloud Native Computing Foundation (CNCF) defines cloud native as:
"Cloud Native (Cloud Native) technology helps companies and institutions build and run elastically scalable applications in new dynamic environments such as public, private, and hybrid clouds. Representative technologies for cloud native include containers, service grids, and micro Services, immutable infrastructure, and declarative API; these technologies can build loosely coupled systems with good fault tolerance, easy management, and easy observation. Combined with reliable automation, cloud-native technology allows developers to easily perform frequent and Major changes to the forecast."
At present, the biggest driver behind cloud native is CNCF. The key technologies include containers, microservices, service grids, devops, declarative APIs, and so on.
Comparison of cloud-native applications and traditional applications. Cloud-native applications can make full use of the advantages of the cloud, flexibly distribute applications among various cloud vendors, release enterprise productivity, and focus on business innovation instead of spending more time on adaptation and expansion On different infrastructure platforms.
New DevOps challenges in the cloud-native era
First of all, we have to know clearly, from the perspective of the enterprise, in such an era of fast business, what do enterprises need most?
Only fast and unbreakable. There are two layers of meaning that can be interpreted here. One is that business applications come online quickly, which is conducive to seizing market opportunities. The second layer is how you can give sufficient computing resources when your business has explosive growth. Guarantee, at this time, in fact, adding huge IT investment to purchase software and hardware may not keep up with the rapid development of the business. This is actually a question of enterprise R&D efficiency;
Seek change in stability. The stability of the business or application is always the first. How to ensure the "steady state" of the business and meet the "sensitive state" needs of fast business, such as the launch of new services and the change of applications. This is a problem of enterprise IT architecture;
Save resources, how to save
computing resources, automatically expand and shrink according to whether the business is peak, this is a problem of cloud platform construction;
Pioneering and innovative, developing integrated operation and maintenance, micro-service architecture.
The initial emergence of DevOps broke the barriers and gaps that have always existed between developers and operators, and strengthened the communication, collaboration, and integration between development, operations, and quality assurance personnel. In the post-DevOps era, we can use container technology to iterate and launch applications faster.
The following is the general process of application release. Developers push the code to trigger the build. The build process is to pull the source code, package the application, push the container image, and deploy.
In fact, there are many places in this model that make full use of the advantages of cloud native, such as container technology, Kubernetes, and dynamic allocation of slave pods. But there are still some challenges.
How to apply security promotion between environmental stacks
How to manage permissions and security approvals for application publishing
How to improve the average deployment time and average recovery time of an application
How to quickly locate, reproduce and roll back online applications
The way of DevOps in the era of cloud native
First of all, we must make full use of the advantages of cloud-native technology. Cloud-native can improve the efficiency of application development, change the organizational structure of an enterprise, and even directly influence a company's decision-making at the cultural level. In the field of containers, Kubernetes has become a community standard for container orchestration and management. By abstracting application services into multiple resource types, such as Deployment, Service, etc., it provides a portable model for cloud-native applications.
In this context, how to practice more efficient DevOps in a cloud-native environment to achieve more productive performance has become a new issue and demand.
The following are the construction goals of an enterprise application platform:
Based on this PaaS platform, we designed the GitOps secure release model to solve some of the challenges we mentioned earlier.
When designing the GitOps release model, there are the following core demands:
Version management. We hope that the version number of each published application can be associated with git commit id. This has the advantage that every change has a history query, and faster fault location and repair;
Baseline management. Facilitate problem reproduction and quick rollback;
Secure release. Including content management and security approval;
Quick feedback. Improve R&D efficiency.
The GitOps release model has the following characteristics:
The Git repository is the only input source for any CICD process
Declarative application orchestration, build deployment model
Application of no difference between the environment stack, automatic promotion
PR/MR triggered pull pipeline process
Quick feedback mechanism
The following is an architectural diagram of using GitOps to manage applications published to different Kubernetes clusters.
The first is the separation of the application source code and the build source code. We can see the two source code items framed in orange. One is our application source code application-java-demo. The source code item on the left is used to store the build source code. For example, the Jenkinsfile of the preview pipeline, the Jenkinsfile of the staging pipeline, and the Jenkinsfile of the production pipeline. In addition to the Jenkinsfile, there may be some sensitive information about dynamically creating a test environment, connecting to a pre-release environment, or a production environment. These sensitive information can also be stored in the database Here, and then save the connection information of
the database.
This common application application-java-demo has different branches in the Git repository, each branch has a certain correspondence with the Kubernetes cluster environment, such as our settings here, the master branch corresponds to the production environment, the latest branch Corresponding to the pre-release environment, the other development branches correspond to the test environment. The dynamic creation and destruction of the test environment and the deployment and release of the application re-test environment are self-service services of the development testers, but the application wants to be deployed to the pre-release environment and production The words in the environment need to be approved by the administrator for security.
The authority of ordinary developers is only the authority to create new code branches and create merge requests. In addition, the rest of the other parts are only authorized by the administrator. The green area is the pipeline task of Jenkins. Of course, you can also be Use other cicd engines to build this pipeline task. Ordinary developers do not have permission to create jobs and build jobs in the Jenkins environment, nor do they have permission to change the configuration. All he has is permission to view logs for building jobs.
Finally, there is a sequence diagram, demonstrating a complete process of an application from development and testing to business online iteration:
The developer submits a new feature branch feature;
The developer creates a Merge Request requesting merge code to the latest branch;
The action of the developer to create a Merge Request automatically triggers the construction of a Jenkins pipeline task called preview-pipeline;
The preview-pipeline pipeline task pulls the preview-pipeline source code project from the Git server, and runs source code compilation, testing, container image construction and push, application deployment to the preview container cluster, and nail notification according to the declarative script in the Jenkinsfile file of the project. Process
The administrator checks the preview connection of the application on the Merge Request page of the Git server and verifies whether the application can be merged into the latest branch. If it passes the verification, it accepts the merge of the Merge Request and triggers step 6. If it does not pass, the developer is notified to update and submit the code. , Return to step 1;
The action of the administrator accepting Merge Request will automatically trigger the construction of the Jenkins pipeline task staging-pipeline;
The staging-pipeline pipeline task pulls the staging-pipeline source code project from the Git server, and runs source code compilation, testing, container image construction and push, application deployment to Staging's container cluster, and nail notification according to the declarative script in the Jenkinsfile file of the project. Process
After the
application services in the Staging environment pass the test and verification, the administrator can merge the latest branch to the master branch;
After the administrator merges the latest branch into the master branch, it will automatically trigger the construction of the production-pipeline Jenkins pipeline task;
The production-pipeline pipeline task pulls the production-pipeline source code project from the Git server, and runs source code compilation, testing, container image construction and push, application deployment to the production container cluster, and nail notifications according to the declarative script in the Jenkinsfile file in the project. Process.
