Virtualization--Product development accelerator in the Internet age

High-tech competition in the Internet era, the delivery of products gradually shortened, and the quality of delivery requirements gradually increased, a variety of new ideas, new products, market-allowed product launch cycle is also getting shorter, the traditional software development model has been unable to keep up with the current needs, efficient, convenient, The iterative product development pattern is becoming more and more popular, and virtualization technology is the most important tool to embody this development pattern.

From the functional point of view, the advantage of virtualization is to improve the utilization of resources, the second is to provide a variety of configuration management; The third is to provide a snapshot of the preservation and recovery capabilities, four is to provide product dynamic expansion of the ability, these are also the Internet product development model needs the important characteristics.

I compare the project experience of a year ago with the current project experience of applying virtualization technology to say how virtualization technology works in the development, testing, and deployment processes.

Briefly introduce the development conditions at that time with the present.

A year ago, my project team had 6 development machines, 2 testers, 6 developers and 2 testers, the machine is shared by the team, each developer is assigned a separate account named after his or her name, and the developer uses this account to log into the system and develop the work. Believe this is also the standard configuration of most companies.

Now my project team has 10 development tester, there is no special distinction in the function, a total of 24 development and testing personnel, no specialized testers. Developers work by logging on to their own virtual machines.

The role of

development phase

The role of virtualization in the development phase has two key points:

first, the rapid environment to build

Initial development requires a process of resource allocation, developers are often unable to obtain the necessary flexible hardware resources, generally can get an account, and a machine to determine the operating system, there are three reasons, first of all, the hardware resources are limited, can not guarantee that each developer has a stand-alone development machine, Only the use of public machines, through different accounts to isolate the way; second, because of the common use of machines, many people at the same time development, so can not according to their wishes for environmental adjustment; The third is because the server makes operating system changes expensive. If the development does need to customize the environment, need to change the operating system, in our previous team, need to submit a separate change orders, after a lot of review to system engineers, system engineers to Friday scheduled next week, so from the request to the operation of the general to go through a week. Even into the actual operation phase, the system will take a long time to reload, the installation of the system takes about 30 minutes, the server system restart time in 8-15 minutes, the time to download and configure the software in 1 hours-3 hours. Overall, rebuilding a system environment takes at least half a day.

In this case, developers do not have the desired development environment at the start of development and can only dance in chains throughout the development process.

Looking at the current situation, we use Xen virtualization technology to solve these problems well. At the beginning of development, the developer submits the required environment configuration list, the Configuration Manager chooses the corresponding virtual machine image from the mirrored library according to the development request, and then find a suitable physical machine to generate the virtual machine from the mirror and deliver it to the developer for use, although the hardware resources are still limited, but because the virtual machine takes up less physical resources , you can ensure that each developer has its own independent control of the environment. The entire virtual machine is created in 1-5 minutes.

With a stand-alone virtual machine, the resources are completely isolated from the rest of the developers, and developers can make their own system configurations. When needed, you can also reload the request at any time, the reload operation is very simple, delete the virtual machine, from the mirror Coussen into a new virtual machine, the original need for more than a week to provide the initial system is now in 1-5 minutes to complete.

Figure I: Currently a developer's development tester

second, independent operating environment

In the previous configuration, because the system is shared by many people, or has been through the development of several projects, many system environment parameters have been many rounds of configuration, leading to the entire development environment undercurrent raging, hidden many conflicting factors. At the same time, because multiple accounts are used at the same time, the modules belonging to different accounts will also produce conflicts, the most common such as port conflicts, which often through the developers to coordinate with each other module used by the port number to solve, but also inevitably left some problems, such as we have encountered a failure, is to develop a communication port that forgets to modify an internal module when submitting a product. Once again, developers spent a lot of time looking for reasons after a program error, and the last sad discovery was that another developer had a tool library upgrade on the system. In addition, it is common for a developer to run the machine dead because of a program problem that causes other programs to run.

The problem of sharing a development environment can also be solved by migrating to a virtualized environment. Because of the independence of the virtual machine itself, each virtual machine is a complete and isolated system, the virtual machine has adequate isolation, developers do not have to worry about port problems, system parameters, and so on, with their own machine natural to the system all changes are clear to the heart, when there are bugs, Developers can also focus their time and effort on debugging program functions.

The role of

test phase

The importance of virtualization becomes more apparent after the completion of development into the testing phase, and the importance of virtualization in testing is embodied in four aspects:

First, flexible environment choice

Testers first need a good test environment. In the past test, it is necessary for the tester to put forward the environment application, reload the system on the test machine, and configure the system parameters according to the test requirement. In the physical machine operation, generally about 1 days, testers can get the environment they need. But because of the diversity of the tests, if the development phase is acceptable for a day, it is very intolerable for testing because the test emphasizes the diversity of the environment. For example, our products need to provide support for the Windows environment, Linux environment, and various distributions of Windows and Linux. In this case, each product test is a painful process, the system engineer needs to reload the system at any time according to the Test schedule in order to provide a different environment configuration of the machine, and the test will wait for the system engineer to provide a good environment before the next environment test.

Even if the environment has been provided, each environment will need to add a different combination of applications, such as front-end products need to test the support of mainstream browsers, browsers including Ie,firefox,chrome,safari,opera, and so on, for each browser also need to consider different versions, such as IE6, Ie7,ie8. What is more frightening is that many browsers of the same type cannot be installed in a single system at the same time, IE series is so, this situation has caused the test environment changes extremely frequently, the system engineer is often overwhelmed, the test engineer complains the need system environment to delay to provide.

The problem no longer exists in our current project team. The virtualization system has already created templates that correspond to multiple different operating systems, such as CentOS 5.4,ubuntu 10.04,windows 2003 server,windows 2008 Server,windows Xp,windows 7, etc. The corresponding virtual machines can be quickly generated by the template when the test is needed, and each virtual machine is generated in 1-5 minutes. At the same time, corresponding to the combination of different applications and environments, but also a separate template, such as Windows 2003 server+iis templates, Windows XP + IE8 templates, when needed to tap to generate virtual machines, a set of available environments appear. The form of the template is shown in Figure two and figure three.

Figure II: Truncated windows partial mirrors

Figure three: captured Linux partial mirrors

second, resource allocation on demand

The distribution of resources on demand is one of the most important features of virtualization. If there are enough physical resources, then you can deploy a separate set of environments on each physical machine for developers to use, but this way consumes a lot of resources, such as we have currently saved a mirrored environment of 56, if the use of 56 physical machines to build the environment is a huge waste of resources.

Because of the limited machine of the test environment, it is necessary to virtualize the real resources on demand. Virtual machine only in the runtime will occupy CPU, memory, network resources, when the virtual organization, its consumption is only mirrored disk resources, at present, each of our initial image size within 1G, 56 mirrors occupy space is not more than 50g.

At the same time, engineers in the test also developed a good habit of use, stop the virtual machine when the test, so that the existing 10 test machines can play 20, or even 40 of the role of machines.

Another benefit of resource allocation is that testers can easily retain their previous test environments, a year ago, our next door team had a cluster of about 20 machines, prepared for a long time before, and the product was still in maintenance, so the test environment was used about once a month, but because of the complexity of the configuration, Resources no one dares to recycle, only by it has been idle.

After using virtualization technology, we made a backup of the test environment corresponding to the release version of each module in the project, turned off the virtual machine after the test was completed, and made the image save. When needed, the virtual machine is tested by a mirrored build. The cost is just a little more disk space.

third, efficient snapshot playback function

The snapshot playback feature is extremely important for testing, and it's actually one of the most popular features for test engineers. Its main application scenarios are two points:

The first is the scene for branch detection. The product has two selection orientations from point a A1 and A2, the test engineer chooses the path A1, and when A1 test is completed, it needs to go back to point A to perform the A2 test. In the past, we did it by going back to a and then doing the next step. This is a time-consuming task when the path is complex, and because two operations cannot be exactly consistent (such as different operating delays per step) can lead to the inability to actually go to point a, which reduces the reliability of the test.

Virtualization snapshots are the best solution to this problem. From our practical experience, in the face of branch selection, only need to do a snapshot at point A, then you can rest assured that the A1 branch of the detection. When the A1 detection, restore the virtual machine to a point snapshot of the state, and then the class can go smoothly A2 branch. The entire process is very process, reducing the operation to restore a state of the work, the snapshot creation and recovery in 1-2 minutes, greatly saving the test time, but also improve the test reliability.

The second is the fault site reservation. In the past, testers who discovered a product bug were in a dilemma, and the bug was likely to fail to replicate every time, so if continuing the test would disrupt the locale, if you kept the site called Rd to find the cause, you might not find Rd, and the RD location problem itself would take time, In this case the environment is occupied, further testing work is delayed.

With virtualization snapshot technology, testers only need to take a snapshot at this time, save the full virtual machine environment, either restore the snapshot to Rd when Rd is free, or simply throw the virtual machine image to Rd,rd to generate the virtual machine from this mirror and debug work. The original test work can also go down smoothly.

For Rd, there is a save the environment after the bug, you can rest assured that a variety of debugging work.

the simulation

of special environment

Due to the hardware limitations of the test environment, in many cases can not simulate the product's real application scenario, such as we are doing a network module development, need to test three network card scenarios, but the test machine only two network cards, can not be simulated. So we created 3 virtual network cards in the same network Bridge of Xen, which solves the problem of this application scene very well.

In addition, we need to do a cluster test for another project, the same is due to the hardware limitations of the test environment, unable to meet the number of clustered machines, so we built a number of virtual machines on a physical machine to solve this problem, the last project is to create 20 virtual machines completed the test.

Finally, the product deployment and operation phase, the significance of virtualization.

deployment, on-line stage first, enhance product pre-release function

In the product should have a pre-release before the process, will be the first product in the pre-release environment on line, run for a period of time stability and then on-line production environment. Therefore, the pre-release environment needs to simulate the system architecture of the production environment, and to ensure the number of machines, due to the limitations of hardware resources, this process is even often canceled, thereby increasing the product risk.

At present, our team, based on virtual machines to build a complete set of pre-release environment, with the production environment to achieve the Basic agreement, in many products on the line has also been a lot of problems, do a preventive.

Second, the dynamic expansion of product service capability

Products on the day after the business traffic is often not a is too distributed, but more extreme, in the morning the lowest flow, in the evening traffic burst very high. The previous business on the line, generally in accordance with the expected maximum flow on-line machine, when the flow of low system resources are very wasteful, such as many physical machines CPU utilization is within 1%. And when the traffic suddenly increased beyond expectations, it is very difficult to deal with, emergency machine time is not enough, it is very easy to make mistakes. This is a problem that makes the operational staff very headache.

We also use virtual machines in the same way, using virtualization technology to solve this problem well. Now, before you go online, you need to have a business-related mirror. Through the flow and performance monitoring, at any time to observe the system load profile, in the low load can choose to shut down a few business virtual machines, when the load burst immediately through the mirror to create more virtual machines to provide services, so as to effectively solve the problem of flow change.

Summary

Virtualization technology plays an important role in the whole on-line process of the product, it is an effective tool for product development in the Internet age, the demand of virtualization technology, snapshot function, isolation function, dynamic expansion ability and so on will provide great convenience for product development.

This article is from: Http://www.infoq.com/cn/articles/dh-virtualization-product-development-accelerator