The change of DevOps system

Editor's note: The development and operation as a whole to look at the DevOps engineering thought gradually in the hearts of the community, this article explores the origins and development of DevOps, from the physical machine/stand-alone virtual machine deployment, to the deployment based on IaaS, and then last year, just the emergence of container based deployment. The writer, Jason, from a newly established Hangzhou Flying Cloud Information Technology Co., Ltd., its brand "Fit2cloud" (@fit2cloud) is a hybrid cloud management and DevOps tool built on IaaS to provide configuration, monitoring, Automate and manage services.

The origin and developing course of

DevOps

Over the past few decades, in order to deliver software products and services on time, there is a growing awareness that the traditional practice of separating development from operations is not suited to the needs of modern product and service development. Therefore, the development and operation as a whole to look at the DevOps engineering thought gradually popular, there is also a gradual need for a devops system, in the hope that there is a platform or tool to unify support for the development and operation of the delivery and subsequent environmental management work, that is, a series of continuous integration, continuous delivery, Automated deployment, automated test monitoring, automated scaling, automated recovery systems to improve deployment efficiency during development test operations, simplify the management of development test operation processes, reduce delivery risk, and reduce communication costs and operating costs.

Broadly speaking, whether it's a cloud management platform tool (such as Rightscale), a variety of PAAs platforms (Cloudfoundry,heroku etc), or automated deployment tools such as Chef, Puppet, and Ansible, They are essentially part of the DevOps system and are designed to address issues of delivery in the development process and operational management issues after delivery, namely

during the development and testing process, help develop testers to build and manage the environment so that changes can be deployed to test after the change, and during operations and support, help the operation support personnel upgrade the system, extend the rebuild recovery system, and continuously master the whole system and the stack status, and monitor the system from all levels. Telescopic system, recovery system.

Over the years, as cloud computing and container technology progressed, as well as the demand for it capabilities in the product business, the DevOps system grew faster, and its roles and concepts became clearer and more independent. Looking back on the path of development and the process of change, we think that the basic can be divided into three generations: based on the physical machine or stand-alone virtual machine deployment era, based on the IaaS Programmable resource deployment era and container based deployment era. With the improvement of these three generations, the overall capability of the DevOps system is getting stronger. Here we first look at the characteristics and capabilities of each generation of devops systems, and then further classify the DevOps system to help us choose the right devops system.

The change of

DevOps and its key enabling technology 1.

of deployment times based on physical machine/Independent virtual machine

This is the first generation of devops system, characterized by static configuration + manual coordination + application only part of the automatic deployment.

In the process of building the whole application system, it is necessary to create the resource environment (such as host, network, storage, etc.) for running the application outside the DevOps system, and the DevOps system has no control over this part, only responsible for automating the deployment application after the resource environment is built, The construction of the resource environment and the subsequent application deployment process are separated, the need for artificial hand-coordinated control, that is, such as the resources and environment, by people to control the opportunity, waiting for the resource environment ready to manually modify the configuration (such as a variety of host IP address, login password key information), and then manually run automated scripting tools, such as the Shell,python,ruby script, the installation of the application to upgrade the deployment, and after the addition or reduction of nodes, but also by the person to run automated scripts to configure the system, can not implement including resource environment creation or node changes to the application deployment of the entire process of a key deployment, that is, cluster awareness + Automatic coordinated control + dynamic configuration + full stack automation.

At present, it can be said that most of the devops system still stays at this stage, because the DevOps system does not realize the automation of the resource environment creation and the coordinated automation based on the cluster sense, then the ability of the devops system of this phase can have the following influence and consequence:

Create system resources environment inefficient, time-consuming, high risk, especially the creation of complex system components complex structure, the creation of system resources and environment process requires specialized network engineers, system engineers, can not realize the development test operators self-service, the more complex system, communication costs, the development of operation and maintenance process management is more complex Creating the entire system requires network engineers, system engineers, developers of the joint participation and cooperation, system components more complex structure, communication costs, the development of operation and maintenance process management is more complex, time-consuming and laborious, coordination trouble, high risk and error prone; When the system resource environment changes, such as after increasing the reduction of the host, Manually coordinated control by people, manual static configuration of the deployment of the upgrade required IP, login password or key information, resulting in a high risk and low efficiency of the change process, especially when the system is large and complex; the delivery process is high risk, the development of testing products are not unified environment, often appear in an environment of normal operation, Another phenomenon of abnormal environment.

One thing to mention here is that although many organizations are already using IAAS (such as Aliyun) to create virtual machines to build the resource environment for application systems, but do not realize cluster awareness, the automation of the system's entire environment is still in the semi-automatic phase (for example, starting a package of annual monthly virtual machines, Then manually configure the deployment script required IP address, login password, login key information, and then manually run automated script deployment, so this way still belongs to the first generation of the DevOps system. At the same time, this is also the current situation of most organizations in China, the devops of automation and efficiency is enormous.

2. Deployment era based on IaaS

This is the second-generation DevOps system, characterized by cluster awareness + Automatic coordinated control + dynamic configuration + full stack automation.

Thanks to the programmable nature of cloud computing IaaS resources, this generation of DevOps system realizes cluster perception, automatic coordinated control, dynamic configuration, full stack automation, which realizes a key creation and deployment of the entire process from the creation environment to the deployment installation application component, and realizes the cluster perception after the creation phase ( Cluster-aware), which automatically deploys and configures the system automatically according to changes in the environment. For example, when a Web site grows in volume, it needs to expand when you add a Web computing node, you can automatically install the Web Components after the newly added web virtual machine is started, and configure each virtual machine Web service registration to the load Balancing service, which is automatically removed when shrinking, and this process does not require human coordinated control. The DevOps system can automatically configure clusters according to the changes in the cluster.

At present, to achieve this level of devops system is relatively small, because this phase of DEVOPS system Automation management covers the creation of environmental changes, application component deployment automation, as well as environment creation, cluster awareness and application component deployment of the process of automated coordinated control, So this phase of the DevOps system compared to the first generation will give the development and operation of the work of the following very significant improvements:

Development test operators are able to create environments and deploy systems themselves, the more complex the system, the more communication costs are reduced, the more the risk of developing operational management complexity is reduced, for example, only by engineers with specialized knowledge, if engineers are not available when needed, it is troublesome; create environments and deploy efficiently, automate, Fast, less time required, low risk; When the system resource environment changes, such as scaling, after increasing the reduction of host, can realize cluster awareness, dynamic configuration cluster, improve the process of change efficiency and reduce risk, especially the system components are large and complex; be able to quickly create the environment on demand to meet a variety of testing, demonstration, online expansion needs To be able to create a startup shutdown development test environment, save costs, and improve the efficiency of development testing and delivery;

3. Container-based deployment times

This is the third generation DevOps system, which is characterized by the addition of the application across cloud mobility on the second generation basis. Based on container technology.

With the help of the programmable features of cloud computing IaaS Resources and Linux container technology, not only realizes the cluster perception, the automation coordination, the dynamic configuration and the whole stack automation, but also realizes the application across the cloud mobility and the elasticity expansion, eliminates the development, the test, the production environment inconsistency, causes the application not to be locked on some IaaS , so that all infrastructure services IaaS and physical machines have become a common resource pool, but also to improve resource utilization, which gives it the development and operation of more and more space to imagine, which is docker,kubernetes now very hot reason.

For example: if we want to migrate a service from AWS to Azure, we will have to start from scratch to create a set of virtual machine images and virtual machines, and configure the system or application components, and if the system is complex, the process can still take a lot of time and labor, and relies on certain engineers with this knowledge, but with the support of container technology and related container tools, the process becomes a very quick and easy process, becoming a standard virtual machine that needs to be started automatically on the target cloud like azure, then downloading container mirroring, configuring Startup containers, configuring DNS, and so on, Real easy to move across the cloud, and flexible dynamic expansion services.

For example, at present, Google Open source container management system kubernetes can be said to be widely recognized and supported by industry, when we have done the application system docker images, so long as the various IaaS on the support of Docker environment, such as the Kubernetes cluster, we can quickly and easily migrate applications on different IaaS, or expand, the following figure shows the Fit2cloud based on cloud deployment and management solutions, We hope that future users will be able to use Fit2cloud to create kubernetes clusters on several different IaaS, through kubernetes management and deployment of application systems. After that, we will have a new article to share how Fit2cloud is created and transported to the Kunerbetes cluster.

In the above section, we introduce the characteristics and capabilities of the DevOps system in different times, so is it possible for us to choose the most powerful third generation DevOps system? Do you choose a devops system to kill? The answer is no, each devops system is not a silver bullet, we need to be managed according to the needs of the system to choose the appropriate devops system or tool, in the next section, we will answer this question.

three, how to choose a suitable devops system?

At present, DevOps system can be said that a lot of diversity, functional differences, the application of different scenarios, then how do we choose the appropriate DevOps system? Here we propose a selection method based on target system classification. We classify them according to the target application or system type to be managed. For the target system, we can divide it into three categories first, namely IaaS Service System, PAAs service system, application system, application system can be divided into simple Web application system, complex distributed system, then with this classification, We choose DevOps systems and tools to be relatively easy and clear.

1. IaaS Service System

Since the creation of the IaaS system is based on the physical machine itself, the applicable devops system or tool for such systems is the automated operational management system developed by the Shell,chef, puppet and IaaS service providers themselves. Only the first generation of DevOps systems can be selected.

2. PAAs Service System

If a PAAs is not deployed above IaaS, it is not, in essence, a PAAs because it does not have elasticity and automatic scaling. The real PAAs system is deployed on IAAS, providing services for the development of test operators, and the applicable devops tools can be chosen Rightscale,scalr,cloudformation, The second generation of Opsworks and Fit2cloud based on the IaaS programmable resources of the DevOps system, of course, can also choose the third generation of container based DevOps system, but the third generation is still in development, not as good as the second generation of maturity.

3. Simple Web Application System

For a simple Web application system, the salient feature is the simple structure of the application, such as only a Web component and database, caching, or some common middleware services, not including a lot of distributed components, then for such a system can choose the traditional PAAs of the container class, That is, Cloudfoundry,heroku,openshift.

4. Complex Distributed Application System

For complex distributed Application systems, container-class PAAs cannot be used to manage, only through custom devops tools or systems, or by using cloud management Rightscale,scalr,cloudformation,opsworks, Fit2cloud some or some combination of such tools, the second generation of DevOps systems based on IaaS programmable resources, can also choose a third generation container based DevOps system. Because such tools provide users with more control over the IaaS host, and provide a callback interface for each deployment process, it realizes cluster awareness and automatic coordinated control of each deployment process, that is, full stack automation.

SOURCE Link: DevOps system changes (Zebian: Zhou Xiaolu)