Dry Goods | JENKINS2.0 Pipeline Framework (IPipeline) Optimization of Practice (I.)

Click on the "ZTE developer community" above to focus on our

Read a first-line developer's original article every day.

Overview

The original CI system of XXX project is set up by the project, using Jenkins traditional job way to realize Verifyci and Mergeci as well as dailybuild. With the increasing size of the project, more and more branches, the frequency of the code is increasing gradually, the existing system presents a lot of inconvenience. To solve these problems, the project attempts to introduce a series of off-the-shelf company-level devops development tools and combine Jenkins2.0 's pipeline new features to transform the pipeline process of this project.

Pipeline as Code is the essence of the Jenkins 2.0 version and is the key tool to help Jenkins achieve a gorgeous turn from CI to CD. The so-called Pipeline, in simple terms, is a set of operating on the Jenkins workflow framework, the original independent operation of a single or multiple nodes to connect the task to achieve a single task difficult to complete the complex release process. Implementation to the code level, that is, we only need to focus on writing jenkinsfile files, and then together with the code base hosting, Jenkins can be based on jenkinsfile to quickly pull the project CI process, convenient and efficient.

However, with the implementation of several projects, the landing, found that based on Jenkins2.0 pipeline powerful, but the specific application to each project there are the following several pain points:

Each project in the preparation of their own characteristics of the jenkinsfile, a lot of time to copy and modify the code to achieve, Jenkinsfile redundancy is very high;

According to the thinking of jenkins1.x times, custom-made their own CI/CD process, mail notification, report output, the process of conversion slow and painful;

Lack of capacity to address technical problems, use of outdated methodologies and tools, and other project teams that have addressed technical issues or adopted more advanced methodologies and tools that are not shared in a timely manner;

In order to solve these problems, after the project repeated investigation and mapping, and finally adopted the open source of the Jenkins2.0 Pipeline library IPipeline (also known as PLLL Library) to assist this project to reconstruct CI process.

IPipeline is a toolset for simplifying CI pipeline deployments, a function library for developers and CI configuration administrators, encapsulating the common functions of Jenkins 2.0, integrating Gerrit, product libraries, cloud CI, metrics, Alarm collection, mail notifications, In addition, the Toolset for Docker encapsulation (complexity, Klocwork, metric analysis, metric import, etc.) is provided. Using it can help us save a lot of energy and avoid duplicating the wheel, so it's very satisfying for our project needs. Problem description

The IPipeline framework has indeed improved a lot of efficiency and happiness in the course of this project, but in combination with some practical usage of this project, it is found that there are still some optimization and improvement points. To this end for our project needs, we have done some of the following optimization and practice ipipeline.

Question 1

With the Pdocker interface provided by the PLLL library, it is convenient to run the specified Docker container on the specified node to complete the related CI task. For example, this project cyclomatic complexity check has been docker, so you can use the Pdocker interface, through the configuration of photogenic name, mapping directory and the need to run the instructions and scripts to perform the loop complexity of the code check.

However, in the actual use of the Pdocker interface found that the PLLL library will go to the node pull mirror each time, and then run, which is already existing in the image of the local node is not necessary, so need to transform.

Question 2

Currently, the update interface provided by the PLLL library only supports code checkout updates for Gerrit code warehouses, but the messages needed to run certain use case tests are stored on the SVN repository, so it is necessary to extend the original interface of the PLLL library to support the code update of the SVN library

With these two questions, we have made some corresponding optimizations and practices for the PLLL library.

Optimization Practice

To optimize whether the 1:pdocker interface requires a pull mirror to be customized by the user

Transform the Pdocker interface, add parameters to control the need for pull mirroring, the interface code fragment is as follows:

/**

* Tool Name: Docker execution

* Tool Description:

* Image-Full path to the mirror

* Needpull-Need pull mirroring

* Before_cmd-the command line executed with root, executed before executing user_cmd

* Volumes-Path mapping list, "A:b,c:d"

* Params-Execution parameters

* Profile-Environment file, executable

* User_cmd-the command line executed by the current user (requires mirroring support AddUser command)

* After_cmd-the command line executed with root, executed after User_cmd execution

* Local_dir-directory that holds temporary files (requires permission to map to containers)

* Shared_dir-directory where code and output files are stored (requires permission to map to containers)

* Sh_exec-shell process, the default is SH

**/

def call (image, Needpull, Before_cmd, volumes= ', params= ', profile= ', user_cmd= ', after_cmd= ',

Local_dir=env. Local, shared_dir=env. SHARED, sh_exec= ' sh ') {

// ... Here the code omitted ...

if (Needpull = = "No") {

echo "No need to pull image!!! The image exists in local! "

WriteFile File:docker_entry_file, Text: "" "

Set-e

Docker run--rm ${params} ${volumes} ${image} ${sh_exec-x-c}

"""

else if (Needpull = = "Yes") {

WriteFile File:docker_entry_file, Text: "" "

Set-e

Docker Pull ${image}

Docker run--rm ${params} ${volumes} ${image} ${sh_exec-x-c}

"""

}

// ... Here the code omitted ...

}

As the code can see, by adding the Needpull parameter, the user can customize whether a pull mirror is required.

Use examples:

For example, this project calls the Pdocker interface to complete the code cyclomatic complexity check using a self-made docker mirror:

Pnode ("${env.") Node_name} ") {

Plll. Check ("Ccn_docker", "Ccn_docker", [

run_execute:{Pdocker (

/* Image * * "DOCKER.ZTE.COM.CN:5000/10010891/LIZARD:V1",

/* Need pull * * "no",

/* cmd * * cd/home/code/&& chmod-r 777 * && CD script/verifyci/ccncheck/&& sh +x Liz Ard.sh ",

/* Volumes *-V ${env. Sharespace}/${env. Xxxxx_dir}:/home ",

* * params * "--privileged",

)

},

param:[

Report_file:[]

]

]);

}

Since Docker.zte.com.cn:5000/10010891/lizard:v1 already exists on our plug-in node, the Needpull parameter here can be controlled without the need to pull mirroring inside the pdocker.

Optimization 2: Expand the framework update interface to support SVN code updates

Optimize the PLLL library code and join the UPDATESVN interface to support the code checkout and update of the SVN library

/**

* Function Name: Update

* Functional Description: Update the code via SVN

**/

def updatesvn (name, desc, args) {

Logdebug ([DEBUG] Update: ${name}, ${desc} ")

Args.run_execute = {

Dir ("${args.scm.path}") {

Svn_checkout (Args.scm.keyid, Args.scm.repo, Args.scm.path)

}

}

/* Call Function Adaptation * *

Functionadapter ("Update", "Update", name, desc, args) {}

Return

}

/**

* Tool Name: Svn_checkout

* Tool Description: SVN update code

* Parameter Description:

*-Credentials ID of KEYID:SVN Library

*-REPO:SVN Library repo Path

*-Path: Code download path

**/

def svn_checkout (keyID, repo, path) {

Checkout ([

$class: ' SUBVERSIONSCM ',

Additionalcredentials: [],

Excludedcommitmessages: ',

Excludedregions: ',

Excludedrevprop: ',

Excludedusers: ',

Filterchangelog:false,

Ignoredirpropchanges:false,

Includedregions: ',

Locations: [

[Credentialsid: "${keyid}", Depthoption: ' Infinity ', ignoreexternalsoption:true, Local: ".", Remote: "${repo}"]

],

Workspaceupdater: [$class: ' Updateupdater ']

])

Return

}

Use examples:

The user only needs to configure:

Credentials ID of the SVN library

SVN library Repo Path

Code Download path

To complete the SVN code checkout and update, the examples are as follows:

Configure the SVN correlation parameters first.

Env. svn_key_id = "89a6fe98-8f0c-4fe6-829e-6d1cbda188e1"

Env. Case_dir = "/jenkins_ci/case_test/path"

Env. Case_svn_url = "svn://xxx." Xx.xxx.xxx/xxxxxxx/case/xxxx "

Call the UPDATESVN interface to complete the SVN library code update

Plll. UPDATESVN (' svn_up ', ' Update SVN escalation text ', [

Scm:[keyid: "${env." SVN_KEY_ID} ", Repo:" ${env. Case_svn_url} ", Path:" Env. Case_dir "],

Run_dir: "${->output_path}"

) Promotion proposal

This article related optimization improvements can be extended to projects that require the use of Docker mirroring to complete related CI tasks and code control involving SVN libraries

The next article will be released tomorrow, the main introduction of the report/message can be extended, please look forward to ~

Expand your Reading

Preliminary study on dry |docker swarm cluster