Build jobs for continuous delivery management

The process of continuous delivery (CD) development to product delivery is simpler for continuously evolving products. Continuous integration (CI), which is at the beginning of the ongoing delivery process, plays an important role in this process by defining the software development process.

There is a lot of information about continuous integration tools available on the books and on the web, but there is not much information on the core build operations in the ongoing integration process.

A typical continuous integration process is as follows: Developers build and test the source code manually on their own machines. They then submit the changes to a source control management system. The build tool then runs the job to compile and test the code. The built artifacts are then uploaded to a central repository for subsequent development and testing.

Therefore, the role of the job in the continuous integration tool is to manage the source code constantly modified, run the test, and through the release channel to manage the logistics of the workpiece transfer.

There are few job tasks to build, many thousands, and all jobs perform different functions. Fortunately, we can manage these jobs in a more effective way.

Automating the creation of build jobs

So why do we have to install a facility that automates the creation of other jobs?

The build tools describe how to create a build job in the user's manual, but rarely explain how to manage the jobs, although these tools are perfectly capable of doing these things.

Often developers want to understand how their applications are built, and then gain exclusive control over the creation and configuration of the build jobs. However, there are some drawbacks to this process:

Software architecture constraints: Because of architectural considerations, applications are likely to have different ways of building. The developer may need to demonstrate that he differs from another developer in building the application, so there is always a slight difference in the build configuration of one job and another job. Therefore, if all configurations are different, then a large number of jobs will become extremely difficult to maintain.

Human factors: Creating a job by hand introduces the risk of error, especially by copying and modifying it to create a new job (that is, copying an existing job and then modifying the copy to get a new job).

Job Timeline control: Typically, the job configuration for each build is not saved, which means that if a job's configuration is modified, it is possible to break the previous build.

So considering the above points, if there is no consistent way to create jobs, and developers are going to build with their own devices, the end result will be extremely difficult to maintain, continuous integration building systems and headaches management. This violates the principle of establishing a continuous Integration System: Lean and maintainable.

In fact, most build tools have the facility to automate the creation, maintenance, and backup of build jobs through API scripting. However, although the build tools mentioned these features in the user's manual, they are often overlooked and not adopted.

Automating the benefits of creating a build job

To simplify the continuous integration build process, you can use a master build job to automate the creation of multiple jobs, which has several advantages.

Create configurations for all build jobs through job templates or scripts. They can also be placed under configuration control. You can create a new job very simply by using a stencil or running a script. This can significantly reduce the job creation time (from several minutes to a few seconds). In the future, you can also modify the job configuration more easily; Modifying the configuration of the build job template ensures that all newly created jobs will inherit these modifications.

All existing build jobs are configured in a consistent way, which makes it easier to update all configurations globally with tools or scripts, compared to clutter-inconsistent systems.

Developers no longer need the detailed knowledge of building tools when they build jobs.

The ability to automate the creation and disassembly of build jobs is part of the lifecycle of the agile continuous integration build.

Let's explore some of the following points:

Point 1: Configuration control for job configuration

It is a good practice to have every component of a continuous integration system under configuration control. These components include not only source code, but also basic build systems and build jobs.

Most build tools keep the job configuration in a file form on the main system. These files are set to normal XML format and can be accessed directly through a REST API front-end interface. Some build tools have built-in features that can be used to save job configurations to any location in a file format.

Because the job configuration can be saved as a file, the job configuration can be saved in file form in the configuration Management (CM) system. Whether you modify the file directly, upload it to the build tool, or modify the build tool's job configuration, save the file and upload it manually to the configuration management system, and any changes made to the job configuration in either of these ways will be recorded. Which method is actually implemented depends on how easy it is to access the job configuration file directly from the build tool.

There is also an important reason to save job configuration in the configuration management system, if a catastrophic failure occurs and all job configurations are lost, the build system can recover more quickly, restoring all jobs, build logs, and build history back to the last known good state.

Point 2: Maintenance of the job

It goes without saying that it is a headache to maintain thousands of different configurations, and that is why the standardization of job configuration is particularly important. If you need to make a change and it affects a lot of similar jobs, it's easier to write a script or create a specific task to modify the jobs.

For continuous integration, there are thousands of jobs in the build system that are not the best strategy in any case. We will discuss this in more depth in point 4 below.

Point 3: Developer control

While developers are encouraged to work autonomously, they also build applications using a centralized build system, which they need to follow in accordance with the guidelines for the build process.

Developers want their code changes to get quick feedback, so they don't want to waste too much time on build tools and job configurations. If you use a button to provide a "self-service" solution for developers, you can automate building jobs and help them complete development tasks faster.

Point 4: Lean Continuous Integration System

Although software development teams employ agile methodologies in product development, they do not necessarily use the build tools in the same way at work, which means that a build tool that has been properly configured should not contain too many long-term build jobs. Ideally, the job should be seen as part of building a continuous integration lifecycle, created on demand. A common approach is to re-create the build job with any history job configuration, keep the build trace, the day log, the artifact, the test report, and so on, and then keep a small number of jobs in the build tool.

Of course, this solution (that is, on-demand-for-disassembly jobs) may be an impossible goal, but the main point here is that the number of jobs in the build tool should be kept at a manageable level.