Software development process automation principles and techniques (complete example)

Principle and technology of software development process automation a simple and complete example of Automation 1 overview

About this article, initially just want to write some about the software Automation test Development article, but later wrote wrote, found not first in the macro software development process to introduce, will not cause you to the formation of automation technology understanding and attention. Therefore, this paper introduces the macro-level of software engineering, and makes some comparisons with the traditional method of implementation, and attach some code, so that interested friends have some preliminary understanding of the idea of automation and the concrete realization of technical means.

Since it is to be automated then surely is directed at efficiency . Before formally starting a systematic study of automation technology, let's start with a complete example of an overall understanding of the concept of automation .

2 Usage Scenarios

You can first define the simplest scenario: a product that provides a third-party front-end JS library (for example, jQuery).

People who have used jQuery can be sure that the product has the following characteristics:

• The static file supports the online reference of CDN mode, which can not be downloaded as an offline file
• JS file is followed by a version number identification
• A static file is an incremental release, not an overwrite replacement

Here are some of the jquery notes that make it easy for the reader to understand the specific manifestations of some of the features mentioned earlier.

How jQuery 's CDN is referenced online

<script src= "Https://code.jquery.com/jquery-2.1.4.min.js" ></script>

Each jQuery library file has a version number, for example: jquery-2.1.4.min.js

If a new file version is released, add a file with the new version number.

Basically such product development and on-line steps are as follows:

1. Development Engineer to develop JS project locally, and to do unit testing and functional self-test.
2. Use the build tool to package a JS file (to simplify the problem, this assumes that there is ultimately only one file)
3. The release engineer receives the built-in files and publishes the static files to the public network.
4. Check that the static files are successfully deployed and are properly accessible by the public network

3 Traditional Solutions

In the previously defined steps, the traditional scenario is:

1. Developer completed, simple self-test, and manual function testing

2. Developers use the IDE to manually package

3. The release engineer manually copies the built files to the public server specified directory

4. The publishing engineer manually enters the public link of the file in the browser to make sure that the appropriate resources can be accessed.

   There is a possibility of public file permissions or server configuration problems, which can cause the file to be accessed even though it is in that location. Like a 403forbidden situation.

This process is to be done manually by the programmer, in the now pay attention to the rapid iterative software research trend, the long-term manual repetition of the consequences is:

1. Always manual operation leads to inefficiency
2. Repetitive work kills creativity.

For projects that have stable product lines that will continue to do addition iterations, it is best to be able to automate the operation, on the one hand, to liberate the programmer and, on the other hand, to improve efficiency to meet the objective needs of today's rapid development and iteration.

4 Automation Solutions 4.1 Automated Unit Testing

In fact, developers in the development process, is to allow the program to achieve the desired goal, then there must be a certain self-test process. But only then the self-test is not possible, but also for the core of the algorithm functions of the unit testing code to ensure that the subsequent iterations and refactoring, there will be no demolition of the east of the situation.

At this point the situation is:

• Core functions---- automated Unit Testing
• interface function---- manual functional self-test

In general, functional testing refers to end-to-end direct user-oriented interface testing, because there are too many uncertainties in the interface, this piece is not suitable for writing automated test code, although for the interface automation has been a part of the script recording tools or development tools, but not advocated.

Attention

Automated testing involves the writing of automated code, which is paid back at the time of regression testing , and the more the number of regressions, the smaller the marginal cost. So only a fairly stable function has a return value.

Due to the development of many products, are no longer independent systems, often there are some external call interface, so before the automated packaging build, but also in the test environment for the interface test. Here in the Automation program, the basic principle and unit testing is similar, so this article omitted not table, the following will be dedicated to the content of the automated testing section.

4.2 Automated Packaging Build

There are quite a few ides that provide a number of packaged visual manipulation tools. However, these tools need to be manually in the IDE according to the wizard, to do a step-by-step click operation, the advantage is to reduce the packaging construction of the threshold, ordinary people can not understand the principle, without writing the building code, but also to complete the corresponding construction work. The downside, of course, is that unattended automation is not possible.

Basically now all kinds of languages have their own corresponding mature building tools, the example of the front-end development in this article, there is grunt such a package build tool.

The automation tasks that can be accomplished are:

• Remove the comments inside the JS source code
• Compression JS
• Confusing JS
• Merging files

By writing the corresponding configuration file and running the corresponding parameter command of grunt, it is very good to realize the automation workflow of the development and construction stage.

4.3 Automation release

There are many kinds of specific implementation techniques for automated publishing.

Script-like languages (PHP,PYTHON,NODEJS) can use version management tools such as Git , invoke shell commands, or third-party operations libraries (such as the Python language's Gitpython ) enables automated deployment of code.

For some builds the product itself is a large binary file, such as an EXE file, or Android APK app, the hundreds of M, is obviously not suitable for use of git such a refined version management tool to publish. You can use FTP or SSH's third-party programming interface for automated publishing. Here you can recommend a Python-based, three-party extension Fabricthat can perform remote file transfers and command-line operations on the remote server.

In this paper, the jquery static JS publishing scheme uses the whole file to upload to the public server (using the Fabric tool) to the basic process as follows:

• Directory for scanning automation builds
• Use fabric to upload files to the appropriate directory for n specified servers
• Use fabric to manipulate n servers to set static file permissions

Here is a sample code that distributes file uploads from a local directory to n servers and makes simple settings:

#!coding:utf8 "" "Automatically upload files to a static server, and test server Above" "" from __future__ import with_statementimport sysimport osfrom Fabric.api Import lcdfrom fabric.operations import put, runfrom dtlib.dtlog import dlogserver_folder_path = '/static_folder ' Local_ Folder_path = Server_folder_pathuser_name = ' xxx ' #服务器登录用户名server_ssh_pwd = ' xxxxxxxx ' #服务器登录密码lcoal_dirlist = []# List of login information for all servers that need to upload files server_list = [# (Host,user,password) (' 192.168.1.201 ', user_name, server_ssh_pwd), (' xxx.xx X.xxx.xxx ', user_name, server_ssh_pwd)]def Scandir (): "" "Upload a local directory to the server's specified directory: return:" "list = Os.listdir (             Local_folder_path) for i in List:dirpath = Os.path.join (Local_folder_path, i) if Os.path.isdir (Dirpath): Lcoal_dirlist.append (i) dlog.debug (lcoal_dirlist) for J in Lcoal_dirlist:dlog.debug ("SubDir:" + j) with LCD (Local_folder_path): Run (' uname-s ') put (J, Server_folder_path, Use_sudo=true) # Upload local file to server # Batch modify file permissions on remote machine run (' chmodA+RW%s/js/jquery-2.1.4.min.js '% (Server_folder_path)) # Modify file permissions for read-write run (' CP%s/js/jquery-2.1.4.min.js%s/js/jquery-2.     1.4.min.js '% (Server_folder_path, Server_folder_path)) Print '%s have synced '% jif __name__ = = ' __main__ ': Currentpath = sys.path[0] for item in SERVER_LIST:FA        B_cmd = ' fab-f scp_static.py-h%s-u%s-p%s scandir '% (item[0], item[1], item[2]) dlog.debug (fab_cmd) Os.system (Fab_cmd)

4.4 Automated Inspection Release results

If a static file for the specified version is published successfully, a final test is required to implement the closed loop . Of course, depending on the requirements, the test case will be different. In general, if the previous process is more normative, there is not much need to detect the functionality, but the release results need to detect: Detection of the specified version of JS can be successfully accessed by the public network.

The average person may not understand, has not already completed the release, why superfluous here? Experience has taught us that the results of an open-loop system are often unpredictable and often untrustworthy. Especially when publishing remotely, the stability of the network environment, the server's hardware quota (disk capacity is full), the Web server configuration (permissions issues) will be the cause of the publication failure. Only a closed loop can form a reliable delivery.

The purpose of the release is not to perform the release process, but ultimately to enable the development outputs to provide normal service .

This article has designed two test cases for the success of JQuery :

• Capable of HTTP request to normal JS source code
• jquery's header information supports cross-domain

The manual detection method is to enter the link in the browser:

Https://code.jquery.com/jquery-2.1.4.min.js

Watch the browser display results.

Obviously:

• The contents of the Browser content box shows the normal return JS content

• The browser's debug box can also be seen inside the header information is supported across the domain

  Of course, the use of a slightly advanced data-plane detection method, if you do not understand the principle of HTTP, may also be dedicated to a test page, to see the normal loading to the JS file.

This is too time consuming, so this article recommends an automated solution. The above-hand approach, in essence, is to use the browser to specify the HTTP link to initiate the request, and then use the eyes to determine the results of the returned data, which can be achieved through the program.

The primary ability to do automation is to look at the essence through the phenomenon , that is, through the interface to see the data , the above two use cases of the main technical principles:

• Request the JS resource, the status code returned by HTTP is 200. Of course, if you want more refinement, you can make further rigorous judgments about what they return.
• The value of the access-control-allow-origin field in the HTTP request header data is * (asterisk wildcard character)

Below is a section of the Python pyunit Framework for Automated detection code:

# coding:utf-8 "" "Test whether the static server file is published successfully" "" Import requestsimport unittest__author__ = ' Harmo ' class Staticservertest (unittest . TestCase): "" "" "" "    static file Request    " "Def setUp (self):        pass    def tearDown (self):        pass    def test_ Jquery_js_release (self): "" "        http returned normally: return:" "        test_js_url = ' https://code.jquery.com/ Jquery-2.1.4.min.js '        res = requests.get (test_js_url)        self.assertequal ($, res.status_code, msg= ' Check for normal return ')    def test_http_static_allow_origin (self): "" "        should not support cross-domain under HTTP        : return:        "        "" Test_ Js_url = ' Https://code.jquery.com/jquery-2.1.4.min.js '        res = requests.get (test_js_url)        Self.assertnotequals (' * ', res.headers["Access-control-allow-origin"], msg= ' http not allowed across domains ')

The above code is two small automated test cases, in order to compare, deliberately made a successful example of running (successful request to file) and a run failure detection example (requires file support cross-domain, in fact, jquery should support cross-domain reference).

For the sake of simplicity, run this test code under the IDE and view the results:

Detects if the static library file above the server is loaded successfully. Of course, the normal result should be such a state of complete success:

If the test passes, it proves to be a successful release.

5 Summary

The purpose of this article is to illustrate what automation and automation benefits, the scope of the foregoing is not limited to "test automation", but the final goal is to be automated testing and now advanced testing Responsibilities: Continuous integration .

Automated Testing also has the following areas that require further research:

• Abstract data from different test cases to achieve the goal of automation

  This requires a solid basic knowledge of the computer

• Automated test scripts that can be organized into scale (e.g., thousands of tens of thousands of) through code

  This requires a certain Software engineering foundation and system development capabilities

• Ability to master and integrate with other systems to achieve continuous integration of fully automated software production processes

These skills are not simple words here can be done, this article can only be used as a primer to the subsequent content of the preheating bar.

Continuous Integration is also established in the previous introduction of the various aspects of the formation of automation, and then use a certain technical means, this series of events to be deducted to trigger the next event, thus interlocking, forming a stable software production automation line . To form a continuous and stable software deliverable.

As for the benefits of continuous integration , a manufacturing example can be used to describe:

In 1913, Ford used the assembly line to assemble the car, and the first line made the assembly time for each T-car to be reduced from the original 12 hours to 28 minutes to 10 seconds, and the productivity increased 4,488 times times!

In the modern software industry, it is also necessary to automate, then continue to integrate, to achieve rapid iterations to produce huge productivity, in line with modern software engineering expectations. We hope that the relevant practitioners work together to enhance their knowledge structure of competitiveness, but also to enhance the overall productivity of the industry.

(not finished, to be continued ...) ）

Author:	Harmo ha mo
Author Introduction:	Https://zhengwh.github.io
Technical Blog:	Http://www.cnblogs.com/beer
Email:	[Email protected]
Qq:	1295351490
Time:	2015-11
Copyright Notice:	Welcome to learn to exchange for the purpose of the reader to reprint, but please "annotated source"
Support this article:	If you are inspired by the article, you can click the button in the lower right corner of the blog to "recommend"

Software development process automation principles and techniques (complete example)