A Web continuous integration work practice

A continuous foundation of web practice:

Https://mp.weixin.qq.com/src=3&timestamp=1494325174&ver=1&signature= Wfvc0e6ylksnscynhs8xlmdrtmtwbu8qmw4ycsnoryvciagd8hpcnocaxb5wisygrmeovujvd1n2frjv3ohyuwutdugmghkdpxalvd6t0rtnsivqrmrgof1kj cnzravztykjurszdpjk8wr5vq8asuoarm9mfluadtp8csvbgm=

　　

background

I joined a profitable start-up company in October 2015, responsible for the Web technology direction. In order to survive in the entrepreneurial process, are hard to spell fast, it is hard to choose the hard rough fast method of work. As businesses and teams grow, the challenges are becoming increasingly challenging. I gradually introduced some of the automation tools and methods into my daily work to get some benefit from the team.

This article summarizes the lessons I have learned this year to do continuous integration. Because I have limited technical field of vision, inevitably considered ill-conceived, welcome colleagues treatise.

What is continuous integration (continuous integration)?

Personal understanding: Continuous integration is through the Platform series of various development links, the realization and precipitation of work automation methods.

Continuous integration is widely used in agile development and is almost standard for a variety of projects.

I think continuous integration is the approach that the head of the research and development team must understand and master.

Why do we have to do continuous integration

Various methods are introduced to solve various problems.

Problems encountered before the introduction of continuous integration

• Offline code and code warehouse are out of sync, impacting iterations and team collaboration

After joining the company, I found that the online deployment is to upload the code directly via FTP, using the file comparison tool for code merging. Because the configuration is different, the modified people are different, often resulting in code warehouse and online code is not uniform. Every time before the online code to do a line and then manually merge.

• Static resource release relies on labor, wasting development manpower

transcoding, compression, and deployment of image resources require human intervention. CDN links for static resources also need to be manually replaced.

• Lack of automation testing, product quality is not guaranteed

Each time the online only relies on manual testing, the test case is difficult to cover all the affected functions, often a primary interface problem, until the product on-line user feedback to find the problem.

• Copywriting simple modification on-line, need technical intervention

Products that are re-operated and have operations on holidays. The copy of the activity page needs to operate classmate repeated deliberation, frequent changes are accustomed to. Every time you modify a copy, you need to develop a student intervention to deploy. To modify a word, research and development need to accompany the operation to stay up late.

the need for continuous integration

In order to solve these problems, we urgently need to change the working methods, combing the requirements as follows.

Automation

Programmers who do not want to throw tasks to machines are not good programmers! (except for moon cakes)

• Automatic compilation

Automatic introduction of various dependencies (development dependencies, package dependencies, configuration dependencies). Automatic transcoding, merging and compression of resources. Automatic processing of configuration files.

• Automatic deployment

Static resources are automatically uploaded to the CDN server. Application files are automatically uploaded and synced to the application server.

• Automated testing

Automated unit testing, integrated environment testing.

• Automatic monitoring

Build exceptions, test exceptions, and run exceptions to automatically notify the responsible person concerned.

Team Collaboration

Continuous integration is not only the need for research and development positions, testing, products, designers, operations and other positions as needed.

• Designers update picture material

Designers can directly update the image resources, image automatic cutting, transcoding, on-line

• Team members (including: test, product manager) can test and experience the latest version in the quickest possible time

Deploy beta versions in the first time and automatically notify team members

• Operations planning can directly modify activities, help copy

User-oriented documentation, such as only the text changes do not need to intervene in the development of manpower, you can complete the online update.

• Numerical engineers can update the table of values directly

A numerical engineer is a person who designs equipment, attributes, and hierarchical numerical relationships in a game scene. A numeric configuration is typically an Excel file. Need to be automatically compiled, updated, and deduced.

adaptable to a variety of operating environments

• Native Environment Local

The application can rely on the computer to run at least. Ability to modify and preview code in a timely manner. Ability to simulate a running environment (interface or data).
Simulating Ajax, we recommend using Mock.js

• Development environment Develop

Before the general Web project goes live, there will be a development environment for the local area network for team members to test and experience. The development environment has complete sandbox data and on-line isolation. Easy to print full logs and provide privileges.

• Online Environment Online

The online environment is also called the production environment, directly facing the user. Access to real-world data, testing and experience should be very cautious.
Multiple versions are usually available on-line for easy testing and rollback.

Agile Development

• Time: Step fast, advance each iteration speed, sedimentation working method

• Space: Automate the collection and concatenation of jobs in various positions

How to do continuous integration the tools you need

工欲善其事 its prerequisite. Today, continuous integration is so widely used that there must be a lot of mature tools to choose from.

• A Unified Code Warehouse

GitLab is strongly recommended, similar to a private GitHub. Code warehouses, milestones, members, static resources, documentation, continuous integration, static sites, and so on, cover almost every function needed for software development.

• Continuous integration Platform

We are using the old-style continuous integration platform Jenkins, of course, there are many rising stars such as Travis Ci,gitlab itself has a CI service.

• Build tools

Web Building tools are also blossoming. We chose FIS3 and Gulp. Mainly in addition to the front end, we also have to deal with PHP, NodeJS, Go and other operating environments.

• Deployment Tools

Since the FIS release is used in most cases, it expands the FIS deployment sink itself. However, from the performance, security recommended rsync old Synchronization tool.

• Other

To accommodate team members using both Windows and MAC development, test, compile, and use cross-platform scripts such as NodeJS and PHP. In the build platform, we use the Shell to write build tasks that can be used across languages. The operating environment required for various tools is not to be mentioned.

Tools are constantly evolving, and they need to be selected by their own team.

Continuous Integration Work

Create a CI project

If using GitLab + Jenkins combination reference: GitLab Hook Plugin

• Basic settings

Project name, description, task type, etc.

• Specifying the code warehouse and related authorized users

Specify code warehouse access links, authorized users with permission to pull related code

• Setting up build triggers

Specifies the trigger type to add the time and address of the trigger build request at the same time on the Code warehouse platform (for example: GitLab).

• Setting up the Build script

To set up a build script, we recommend using the Shell
According to the characteristics of their own projects, we have agreed on a number of commonNPM Scripts

npm run online  # 构建到线上环境npm run develop # 构建到开发环境npm run stable  # 构建到内网稳定版npm run debug   # 启动本机调试

• Setting up build Exception notifications

Specifies the message for the build owner, notifying the owner when a build exception occurs and the fix is taken.

Benefits

The exploration and implementation of continuous integration in succession, there are some tangible benefits

Reduce labor costs

Repetitive and tedious work can be automated. Team work processes can be continuously refined and precipitated.

Improved product stability

Pre-deployment testing, post-deployment testing, and test case coverage of each basic function. Test discovery and user feedback bugs can be converted into use cases to continuously enhance test coverage.

Reduce Code maintenance costs

The build process automates the configuration of a variety of operating environments, maintaining only one set of code warehouses throughout the development process.

Speed up product iterations

Each code document change produces an experienced version that accelerates testing and experiences the time of product intervention.

Building an instance

Give a few practical examples

Web game material resources automatically launched

Dress up games, often add clothing decorations. The designer provides PNG footage, which is automatically transferred from the build tool to the WEBP resource to the CDN.

Daily activity copy Update to Operation

Add the students who are operating to the GITLAB project members. Operating classmates do not need to install other software, directly in the browser to modify the GitLab project file (usually the copy in HTML), save the Instant update online.

Cluster service automatic deployment and testing

Highly concurrent WEB applications, often with many shards (which can be understood as multiple hosts). The code needs to be synchronized to each shard, and each shard may be slightly different, not necessarily every time the code iteration is working. We propose a test port for each shard, and each shard is covered by a test case before going online to ensure the stability of the integration service.

Cost of Use

Solving the old problems will bring new problems as well.

Learning and use costs

Continuous integration covers almost every aspect of the development process and the operating environment, and members of the general project team are not necessarily accessible. So I give the group of students to devolve more intranet environment rights. Of course, we can also install the relevant environment ourselves.

Online Environment Privacy Protection

The operation of the online environment requires great care and some configuration is highly confidential. Includes not limited to: third-party payment authorization code, third-party application authorization code, file deployment authorization code, database user identity, namely: a variety of important private configuration.
Our approach is to prepare another set of code warehouses dedicated to managing on-line configurations that are only available to administrators.

Differentiate between different operating environments

The local operation, development environment (personal development environment, stable version, development version), online environment (pre-line, Gray Line), all need to be differentiated by configuration or environment variables.

Build process Self-exception

Exceptions may also occur on the build itself. such as: Build machine hardware and software exceptions (network interruption, disk full, compile dependent upgrade failure). And holidays are not in the office environment.
Need to prepare a short-term manual intervention maintenance program, such as: Reserve a System upgrade page, can buy time, not easy to reduce the user experience.

Risk of errors covering items on the line

Sometimes in order to catch the project progress, the use of the programmer must kill skill Ctrl+C、Ctrl+V . Cloning a build task can also be risky, because having the same deployment configuration will overwrite previous online code and lead to online accidents.
To avoid this problem, we added a piece of code before the build to check the build project name

node -e ‘if(require("./package.json").JOB_NAME!==process.env.JOB_NAME){console.error("JOB_NAME Error.");process.exit(1)}‘

Practical ExperienceDefining the Project

Whether it's a front-end project or a back-end project (PHP, NodeJS, Go), we package.json define it. Easy to unify the project name, version, the source of the build script.

The build process uses cross-platform scripting

You can choose PHP, NodeJS, Python and other cross-platform scripts, easy to run into a variety of environments. Do not build VBScript scripts that are used or JScript that can only be run directly in Windows.

Writing small cost test cases

Writing test cases does not necessarily introduce a heavy-duty test framework, as long as the process exits in a non-0 state, the build process can be interrupted. For NodeJS use process.exit(1); , PHPexit(1);

Building Project Specifications

There are no rules inadequate surrounding area, the use of unified norms can be better team collaboration.
For example, build a project with a package.json declaration item, a NPM Scripts write build portal script, and a JOB_NAME field check.

Build manually

To avoid interfering with the development Member deployment project, assign each member a sexual port. The code does not need to be submitted to the warehouse, by manually deploying the corresponding project.

Configuring different operating environments using jdists

Finally Amway: jdists-Powerful code block preprocessing tool to easily adapt to a variety of operating environments.

<!--remove trigger="release" desc="仅本机加载，构建时移除"--><script src="node_modules/mockjs/dist/mock.js"></script><script src="mock/mock.js"></script><!--/remove-->

Reference

• "MOCKJS + jdists" Example items used in conjunction with

• FIS3 user Documentation

A Web continuous integration work practice