"Everything is code" for continuous integration"

This article has been published in the "continuous integration" column on the infoq Chinese site. For more articles and video resources on "continuous integration" and "continuous delivery", visit "continuous delivery Chinese site ",
Http://www.continuousdelivery.info/index.php/resources /.

In the previous article "software self-identification", we discussed how to enable software self-identification to promote automated deployment and Version Detection. With the rapid development of the Internet and the improvement of infrastructure, more and more businesses are put on the "Cloud" side. Managing thousands of servers and different application versions is already a common transaction. So if you manage these machines and code well? This article will introduce some best practices to help you better complete related transactions.

I. The test code is not a second-class citizen.

Business pressure makes the team seem a little nervous. One afternoon, people are working hard and the new version is coming soon. Suddenly, the conversation between two colleagues caught Joe's attention.

"Hi, Sam. Let's take a look. I have an automated test failure ." Jared, a tester who just joined the team, called Sam.

"Why? I don't have this test locally, but I have updated my local code ?" Sam was stunned.

"Oh, right? No. I also just updated the code from SVN ." Said Jared.

"Jared, you should update it again. Maybe you updated it before I submitted it... Oh, the results are the same. Go to my workstation and check it out ."

"Oh, you put the test here ." Sam suddenly realized, "We two use different test suite versions. According to user feedback, we modified a small feature of the product. Therefore, the original automated Acceptance Test logic of this feature is incorrect .". The directory structure 1 of the svn code repository is shown in.

Figure 1 product code and test code separation

At this moment, Joe also came together. "Well. We should modify the organizational structure of our code in SVN. Put the test code and product code in the same code library, so that the product code and test code are the same ."

Jared asked: "Why? When I was in a company, the test team also managed functional test cases on a self-developed test case management platform. When you need to perform regression testing for the old version (such as V1.0), you only need to check the test cases corresponding to this version on this platform, and then click the 'execute 'button. When a new version, such as V1.1, is available, you only need to copy all test cases of V1.0, mark them as V1.1, and modify them. The organization of the test code is often to use the directory structure to separate versions ." 2.

Figure 2 manage test cases in directory Mode

Joe replied: "I have also met the practices you mentioned in other companies. This approach is common in teams that use the traditional waterfall development model, namely, the separation of the development phase and the test phase. In the development phase, you will not run the relevant automated tests frequently. These automated tests serve testers. Here, automated testing serves everyone, and developers can run the testing at any time. In fact, most of our tests have been placed in the same code repository as the product code. This part of the code is old, and the requirements have not changed. The automated test has been running successfully, so there is no motivation to migrate this part of the test code ."

Joe paused, took a sip of coffee, and said, "When we achieve the same product code and test code, we only need to check out the source files of a specific version from a SVN code repository, the product code and the test code corresponding to the version are also obtained. Therefore, there will be no version inconsistency, or you need to manually select test cases. There are a lot of servers now, and applications use the gray scale release method. There may be multiple versions in the production environment. If a running version has a problem and needs to be fixed, we can easily get all the corresponding code. Therefore, we should not take the test code as a second-class citizen in the code, but should get the same attention as the product code. The directory structure in our code library is like this ." 3.

Figure 3 same test code and product code

Ii. configuration information is also code

Alex also came together at this time and said, "Not only do you need to treat the Code equally, but also put the configuration information in the code repository. In this regard, we also have a painful experience. I remember that a few months ago, our products were not very large at that time. For rapid adjustment, we often directly modify the configuration information in the production environment, such as database connection, function toggle, or IP address. Result: each time you prepare for the test before going online, you need to download a configuration information in the production environment. Once, a problem occurred to a machine in the production environment. As a result, all the information on the machine was lost, which caused us to look for information everywhere, it took a long time to reconfigure the machine. Now it's much more reliable to store them in the code repository ."

"What does configuration information mean ?" Jared asked.

Joe replied: "The configuration information includes application-related configurations and related configurations during application deployment. Application-related configuration is a configuration item that affects application behavior, such as a function switch, database connection, and cache size. Related configurations during program deployment refer to how program components deployed on different machines are connected to each other, such as IP addresses ."

"Then some configuration information is dynamically generated. How can we put it in SVN ?" Jared went on to ask.

"We must first separate dynamic information from static information. Generally, for dynamic information, you only need to save the changed rules in SVN ." Alex replied. "For example, if some IP addresses or internal domain names are automatically configured, put the automatically configured scripts in SVN. In this way, once a problem occurs, we can know exactly what the allocation rules are ."

"What should I do if the configuration in the production environment is different from that in the test environment ?" Jared is desperate.

"This is easy to do. You only need to save three configurations, corresponding to three different environments ." Joe smiled. "So, the directory structure of our code is like this ." He picked up the pen and drew it on paper, as shown in figure 4.

Figure 4 business-centric organization Product Structure

3. The script file is also the source code

"Hi, Joe. What is in the script directory here ?" Jared asked.

"Oh, this directory contains all the scripts we use ." Alex said with a smile, "In the past, people wrote a lot of scripts for testing and deployment, and used them in their own work. Now we put it in this directory, so that everyone can use the same script to do the same thing. For example, when a developer executes the deployment script autodeploy, the developer reads the relevant configuration from the Development Configuration directory (CONF/Dev) and deploys the development and debugging environment. When the tester uses the same deployment script autodeploy, it reads the relevant configuration from the test configuration directory (CONF/test) and deploys the test environment, the production environment deployment also runs the same script, but only the production environment configuration."

"This is good. Our scripts have been tested many times before they are finally deployed to the production environment ." Jared laughed.

4. Is data also code?

"Let's get back to the test question we first encountered ." "In addition to minor functional changes, the test fails, and there are some exceptions in the log, as if the data format is incorrect," Jared said ."

"Well, the data used in this test is stored in a fixed shared directory. However, although the file name remains unchanged, the data format has been changed. That is to say, this test code is inconsistent with the test data ." Said Joe.

Alex said, "Well, now we have stored the data in the code repository ."

Jared turned a suspicious face and asked: "How can we put the data so big in the code repository? It is not efficient for SVN to store big data, and it also occupies a large amount of space ."

"Alex is only half done. In fact, we put big data in a self-developed version control system. When you place a copy of big data on it, the system returns a unique ID, which is placed in the conf directory of the product code. In this way, when you check out the code of a specific version, you can directly obtain the corresponding big data. If the big data is modified, you just need to put it in the big data storage system again, and then update the unique identifier returned to the correct conf directory ." Joe added. "If you do not have such a large database version management system, you can also use a shared directory, except that each subdirectory stores a copy of data and stores the subdirectory address in SVN ."

"Another way to modify the database structure is to use tools like dbmaintain or dbdeploy. Write each database structure change to a database script and put them together with the code. In this way, the upgrade process will be completed by these tools ."

5. Everything is code

"Oh, I understand ." "It is about managing all the versions of our services or products," Jared said. In this way, all content has only one source, and everyone can get the same information, and automation is very easy ."

"Yes. Moreover, after this step is completed, everything will naturally become traceable ." Joe smiled and said, "This is an extra benefit ."