"What should I do with automatic deployment?" 】

Anatomy of AWS CodeDeploy

Author Liu Tao published on May 13, 2015 | Note: GTLC Global Technology Leadership Summit, 500+CTO technical leadership redefined! discussion

• Share to: Weibo facebooktwitter Youdao Cloud Note email sharing
• Read later
• My list of reading

At the end of 2014, AWS released three new deployments, management Services Codedeploy, Codecommit and Codepipeline at the "re:invent" conference. Since AWS has already provided deployment and management services such as Beanstalk,opsworks,cloudformation, why is AWS still continuing to deploy and manage services? What are the user's problems that have not been solved well? This article will dissect one of these three services in depth: Codededploy, analyzing the problems that Codedeploy solves, and explaining our understanding of the principles and ideas behind them. Hopefully this will absorb Amazon's experience and apply to improving and accelerating our development delivery process.

1. What is Codedeploy?

Codedeploy is an automated deployment service provided by AWS to its users, enabling AWS users to quickly and easily deploy applications to EC2 instances. Accelerate deployment, control deployment cadence, reduce the complexity of application upgrade updates, and reduce the errors and risks of manual deployment operations by standardizing and automating deployment processes. This ultimately allows users to quickly release new features while ensuring the quality of the deployment and avoid disruption of service during deployment. On a support scale, the service is capable of handling thousands of node-sized application deployments that meet the deployment size requirements of most users. The service is currently open only at AWS East Virginia and West Oregon.

2. Sources of Codedeploy

In November 2014, Amazon CTO Werner Vogels revealed the source of Codedeploy and the story behind it in his blog (The stories of Apollo-amazon's Deployment Engine).

Related Vendor Content

Qcon Global Software Development Conference Shanghai Station, October 20-22nd, Shanghai Marriott Hotel Bao Hua! Apmcon China Application Performance Management Conference, August 18-19th, give you professional answers! GTLC Global Technology Leadership Summit, August 29 – 30th, high-end technology leaders event! Cnutcon Global container Technology conference, September 9 – 10th, Docker application Practice! New version of the campaign base!

Related Sponsors

More offers, more free events, richer online courses, click to learn more!

Over the years, Amazon has made adjustments to the system architecture and development organizational structure from the corporate level to speed up the delivery of research and development. The entire system architecture shifts to SOA, splitting large systems into smaller, independently-running SOA services. Development organizations are also tuned to small autonomous teams, each of which is solely responsible for managing the development and operation of its SOA services, rather than separating development and operations from different teams. After this change, they quickly discovered that the deployment process became a new bottleneck, and many teams solved the bottleneck by automating their deployment process. Initially, when the system deployment node is small and the deployment requirements are relatively simple, the deployment problem and resolution become complex as the size of the system deployment nodes increases, the requirements for cross-datacenter deployment and service SLAs are higher. To prevent teams from repeatedly solving the same problem, Amazon built an on-premises system, Apollo, so that the team no longer reduces the speed of releasing new features because of deployment. It is reported that thousands of engineers per day in the Amazon to deploy services through the Apollo, 2014 deployed more than 50 million times.

At the same time, many of its customers outside of Amazon are experiencing the same problem, and they want Amazon to share relevant hands-on experience. Amazon then released its public version, Codedeploy, based on Apollo.

Speaking of which, we can't help but ask, how does Amazon's Apolllo and its public service Codedeploy solve deployment problems in DevOps? AWS itself also needs to be deployed every day. So complex a system, with a strict SLA, can do without downtime upgrade, behind the principle of what? How is it designed? Let's take a look at the specific problems that Codedeploy solves, and then look at the design and implication behind Codedeploy.

3. Codedeploy Resolved Issues

The main problem with the Codedeploy solution is to adapt to the Amazon organizational structure and system architecture design, accelerate the delivery of the business, and handle the deployment delivery of the application, so that this link is no longer the bottleneck that affects delivery speed.

For deployment, many people feel very simple, not much use, not worth spending time on it. But in fact, it can be said that the deployment process for the entire development delivery operations process, delivery speed quality impact is very large, especially for distributed systems, the system is more complex, more components, deployment node size is very large, the system has SLA requirements, its demand scenario connotation and extension is very broad, The processing scenarios include different application types and architectures, different application component code packaging methods, different target deployment environments, different deployment process requirements, and so on. Here are some scenarios for deployment to process.

1) different application types and schemas

• Different sizes (small applications, large-scale distributed applications, etc.);
• Architectures are different (simple, complex, various business areas, Web applications, platform systems, etc.).

2) Application component code packaging is different

• The application system code is packaged in a different range (e.g., all components are bundled together or packaged separately);
• Different application build libraries (S3,nexus,git, SFTP, etc.);
• The application code structure is different (Java,c++,python,ruby, etc.).

3) different target deployment environments

• The base resource environment of the deployment is different (aws,azure, physical machine, etc.);
• Different use types (development, test, commissioning, product, demo);
• Node size is different (several, dozens of, hundreds of, thousands of);
• Geographical range (single region, Trans-region, transnational);
• Operating system and Operation Environment are different (Ubuntu,centos, library package, etc.);
• Legacy system compatibility requirements are different (you can import hosts and need to be newly created).

4) The deployment process requires a different

• The frequency of each component deployment is different (different components in the same phase, one component different stages);
• Deployment Workflow Rhythm is different (different types of components are deployed simultaneously, sequentially, one component is deployed in batches);
• Service allow interrupt time SLA requirements are different (can be interrupted a few seconds, a few minutes, a few hours, can not be interrupted);
• Deployment takes different time requirements (seconds, minutes, hours);
• The range of people that can be deployed is different (only some people have the ability to deploy and everyone can deploy);
• Permission Requirements for deployment (only certain people have permission, no restrictions);
• Visualization of the deployment process;
• Post-deployment validation.

For Codedeploy, with the exception of binding the environment to AWS, the design concept is general enough to handle most of the above scenarios. So why Codedeploy can solve these problems, eliminate deployment bottlenecks, and remain flexible and generic? Let's take a look at its design principles and ideas.

4. Codedeploy design principle and thought core Design 1: For SOA design, flexible universal, local independent deployment

The traditional approach is to adopt a holistic approach to deployment. In our actual development and operations process, the simple application of the components are usually relatively small, deployment scenarios are relatively simple, suitable for the overall deployment. However, for large applications with many components, we find that often each deployment upgrade involves only a few of these components, and the most complex part of the deployment process is the connection configuration between the components, the control of the upgrade sequence, and the upgrade of the database table structure. In an SOA approach, a large and complex system is decomposed into smaller, autonomous SOA services, which is equivalent to simplifying the problem. On the one hand, the complexity of each system is reduced, and the connection configuration between components is greatly simplified, which simplifies the deployment and makes it easier to handle and guarantee the quality of the deployment.

Codedeploy focuses on the decomposition of large systems into multiple SOA services and the deployment of an entire system by deploying a decomposed SOA service. The equivalent of the entire system deployment into multiple local deployments, divide and conquer, this is the concept of microservices, SOA in the deployment of the embodiment.

Core Design 2: Application code is integrated with deployment scripts

Traditional application code and deployment scripts are decoupled, developed based on many different deployment tools, such as chef,puppet,ansible, or Shell,python deployment scripts that developers write themselves. Because the development and operation of a system is entirely up to an autonomous team, it is natural to put code together with deployment. It is also possible to see the devops philosophy of eliminating the gap between Dev and ops and unifying the goals and deployment of Dev and OPS. In addition, the integration of application code with deployment scripts simplifies the management of code and deployment scripts, avoiding the need for code versions and deployment script versions to correspond to problems. In fact, this design also simplifies the user's use of the process, there is no need for additional deployment script version management.

Core Design 3: Event-based deployment process

Codedeploy defines an event-based deployment process interface that defines multiple deployment file copy source target deployment mappings (files, source-> destination) in the interface definition, and the order of execution between the steps and steps in the deployment ( Applicationstop, Beforeinstall, Afterinstall, Applicationstart, Validateservice, The individual steps to execute the script, the execution time-out and the execution of the user. As shown in 1, the right part is a deployment interface definition, in which the developer defines the stop application step using the code root directory under the scripts directory Stop_ server.sh script, before deploying the application code, i.e. Beforeinstall, Beforeinstall step executes install_dependencies.sh script under scripts directory to install various dependencies, start application steps using Start _server.sh scripts, and finally verifies the deployment using validate_service.sh script validation.

It can be said that the design of this interface is very flexible and universal, suitable for a very wide range of applications and deployment scenarios, such as whether the application component code packaging is together or separate, regardless of whether the application architecture is SOA, the adaptation of the implementation of a variety of scenarios to the application developers, by the developer for different scenarios on demand implementation. Instead, Codedeploy handles the management of the application version information, deploys the group management, automates the coordinated control of the various steps of the deployment process, performs the scripting of the specified steps, and visualizes the deployment process.

Figure 1: Event-based deployment interface definition

Core Design 4: Open API

Since Codedeploy only handles the deployment of EC2 instances of the underlying environment and is only for one application (component), and in the actual process a system contains multiple components, the entire system lifecycle management process needs to be automated, So Codedeploy also opens the API interface as well as the CLI so that application developers can integrate Codedeploy services into their own development processes for continuous delivery.

5. Application scope and limitation of Codedeploy

Application lifecycle management includes configuration management, resource management, environmental management, deployment delivery management, automated testing, monitoring alarms, backup recovery and capacity scaling, and we see that Codedeploy only handles code deployment issues, does not handle application configuration management, resource management, Environmental management and subsequent monitoring and recovery, stretching and other aspects. which

• Code versioning in configuration management is handled by Codecommit or GitHub in AWS services;
• Application build storage is handled by S3;
• Environmental management, infrastructure resource management processes are handled by AWS Ec2,cloudformation.

Therefore, the deployment of applications using Codedeploy also requires integration of multiple services, such as AWS Ec2,iam,s3, to complete the process of static code-to-online services. For example, before using Codedeploy, you need to start running an instance of your application with AWS EC2, configure the deployment group type for the instance (for example, by playing tag), configure access to the instance with the S3 permission, and authorize Codedeploy operation instance permissions, and so on. Thus:

• To implement continuous automated deployment of the system, users still need to integrate their own development, such as the need to implement the new version of the package and upload to S3, and then call the Codedeploy Rest API to upgrade the new version;
• Codedeploy can manage only apps on AWS, not apps that run on other IAAS infrastructures, and does not support cross-cloud migration and management of applications.

6. Summary

"You build it,you run it." is the concept that Amazon CTO believes in, and this tool once again embodies this. Of course, it also embodies the idea of DevOps, which embodies Amazon's SOA architecture. While Codedeploy can only be used to manage the deployment of applications on AWS and not be used to manage our domestic cloud applications, we are able to draw on many ideas and designs, adapt our system architecture and organizational structure, unify the goals and tools of Dev and ops to speed up our delivery, Improve the efficiency and quality of operation and maintenance.

About the author

Liu Tao is an AWS Certified Solution architect and co-founder of Fit2cloud. Fit2cloud not only provides one-stop application delivery and operations management tools, but also provides methodologies to help organizations get through the code-to-service channels for continuous delivery and automated operation of cloud applications. Fit2cloud's code deployment features are similar to AWS Codedeploy, and are compatible with the AWS Codedeploy APPSPECS.YML Interface Specification, while Fit2cloud also enables users to import external hosts for unified deployment and management.

Resources:

Fit2cloud Overview of the continuous integration and deployment process: An overview of the Http://docs.fit2cloud.com/v1.1/cmguide/ci-overview.html#fit2cloud continuous integration and deployment process

Anatomy of AWS Codedeploy:http://www.infoq.com/cn/articles/analysis-of-aws-codedeploy

"What should I do with automatic deployment?" 】