Experiences in Software Testing Management)

Experiences in Software Testing Management)

A Class A asked: "Will the bug be solved if too many tests are done ?」

A B replied, "It's not easy. As long as no tests are performed, there will be no bugs .」

The above conversations reflect the ideas of many software staff for testing. For most software developers, testing is probably the most annoying job behind maintenance. For software R & D directors, testing is a must-have evil: if you do not do enough, you will suffer from endless troubles, increase costs, and delay business opportunities. Therefore, how to plan and execute the most cost-effective test work is a subject that must be studied by software developers.

Software testing is difficult because it is not only a product test, but also a product design program test. As it is related to design testing, the criteria are not easy to find, and experience may not be reused. The mountain stone also has application limitations, so it is quite difficult. To improve the effectiveness of the test, it depends on the overall planning, implementation, and post-review and improvement actions. Many small software R & D units do not pay much attention to software testing, but many software companies with a relatively small scale are configured with permanent testers, or even the testing and quality assurance department, the test work clearly has knowledge and value.

There is no best practice for testing because different software requires different testing techniques. For example, the practice of small software and large systems is different; the requirements of customized software and software packages are different; the testing of system software is often unable to use the skills used by application software; game software and inventory systems have their own test targets. Therefore, testers must plan the most suitable testing method based on the characteristics and resource constraints of the software and the past experience. With the accumulation of experience and continuous improvement practices, we can find the best test method.

Therefore, we can see that effective testing is not just about hard work. It requires good management to achieve the best results of the entire job. The management of tests can be discussed from the perspectives of organization, planning, implementation and review. I would like to share my rough experience with you and hope to provide you with basic assistance.

Design of test organization

Because human nature always thinks they are the best and the most correct, software developers are solely responsible for testing and are not recommended. In practice, developers often serve as testers due to insufficient economic scale of software development units. However, if feasible, the R & D organization should try its best to set up dedicated testers, especially those independent from the development team. Although it is still controversial whether an independent testing team should be set up, many people even think that software quality can only begin with improving software development programs, however, most software companies in the United States have independent testing or product maintenance personnel or even departments, which shows that the independent testing still has its unshakable status.

Many software R & D units regard testing as a secondary task, so that the secondary personnel are responsible for relevant work. As a result, excellent people cannot participate in the test, and they lack the willingness to participate in the test. Results The software quality is not easy to measure, and the R & D results are often offset by poor quality, which makes software developers discouraged. Whether the supervisor can recognize the importance of software testing is usually the key to success. Software testing is a supporting task, and reasonable resources should be configured to achieve overall results. In the current environment, it is necessary to give more attention to testers.

Test Plan

The plan of the test includes at least two key points: Setting the test target and configuring the test resource. The attack requires the target and the test is also true. The purpose of the test is to identify software problems and provide a reference for improvement. If the target is unknown, the tester does not know how to proceed.

The most important part of the purpose of testing is the software passing principle, that is, when the testing will end. The common situation is that the progress of software development is constantly falling behind, and the last remaining time is only two weeks. Therefore, the tester's goal is to use up the last two weeks, so that everyone can listen to their daily lives. How many complete tests, how many hidden errors, and how productive is the test work? I don't know. There is time for improvement after the product is sold or launched. However, after the product is sold, the cost is often greatly increased. Even if the original developer leaves the company, it is difficult to make up for it. Experience has repeatedly showed that pre-test debugging is definitely more time-saving than post-event maintenance, except for software that cannot be sold or used.

The requirements for testing can be simply divided into two: one is the quality of software set by the target, and the other is to achieve the best results within the established resources. The former requires the mountain to be attacked and never stop if it fails to achieve its purpose. For example, the error discovery rate of the target unit test time must be lower than a certain number. If the unit test time is exceeded, the test must be extended. This method is suitable for software with high quality requirements. As for the latter, the market time has been announced and cannot be changed. The goal is to eliminate the most serious errors. This test focuses on testing preparation, and often sets time limits and quantity requirements for test execution and debugging. The easiest principle to follow is that important functions are always tested first. Different requirements for these two types of tests are sufficient to influence the testing plan, test sequence, and focus. Readers cannot ignore this issue.

As for the suitability of test resource configuration, it is an important reference indicator to assess whether the test target can be achieved. Testers need reasonable testing resources, such as more than 20% of total R & D personnel. More than 1/3 of the total time period. Lack of manpower, testing is in the form, and the time period is too short. It is too late to correct the error to be found. In addition to the proportion of testing in R & D, you also need to pay attention to the allocation of manpower in planning management, specification case setup, test execution, regression testing, and training preparation. Personnel training and equipment arrangements are especially easy and should be noted. Resource Configuration for testing at different stages must also be considered. This prevents testing from focusing on Function Testing and ignoring system testing. The proper arrangement of such work helps to assist the testing work in carrying out the most important and most effective tests from time to time.

Test execution and management

Test execution is managed. First, the test and development personnel should be given priority. Testers often do not consider the test results, but only test according to the convenience of the test. For example, the software has ten modules and each module has 50 test cases. Therefore, he starts the test from the first case of the first module and tests the entire module, test the second module. The execution is complete or cannot be performed. In fact, testing should start with important and common projects.

The debugging of developers often starts from making changes. As a result, 90 were changed for 100 errors, and the main defects of the system were still overcome. Test Management personnel should pay special attention to this matter to ensure the efficiency of testing.

The advantage of testing management is that you can keep abreast of the situation and adjust the testing strategy as needed. For example, after testing for a period of time, if you find that a subsystem has many problems, you can adjust the manpower and enhance the testing. Or some people have poor test performance, you can adjust the assignment of work to achieve the overall effect. Of course, the acquisition of such data depends on the collection of relevant information, including the quantity and time information. If feasible, you can record the labor hours consumed by different testing tasks and calculate the consumption costs, so that you can have more accurate reference data for future testing planning.

For statistics and analysis of relevant data, it is best to use tools to help save manpower and improve results. If the existing test management tools in the market meet the requirements, you can also use them. The statistical data of the test results may be posted to everyone's eyes, so that the test results can be understood by everyone, and can also promote work colleagues to obtain better results. The figure shows a simple statistical chart that shows weekly test results, debugging results, and the number of remaining problems of the product. You can help the supervisor decide the time of the test termination and product release.

Test Result Analysis and Improvement

After the (phase) test is completed, the test Manager can analyze the test results. Differences between the pre-defined objectives and actual execution results can be used as a basis for improving the next Software Test Review. For example, is the number of pre-opened test cases achieved, and the number of execution and pass acceptable? Are there enough manpower for testing or debugging? The workload is calculated based on the standard workload as an estimate for future testing. Through analyzing the life cycle of software errors, you can study shortened methods, such as accelerating the debugging and retest cycles, or reducing the probability of errors in the analysis design phase to shorten the test time.

The test results can analyze the benefits and improvements of different tests. The following table is used as an example. Unit Tests consume most of the manpower and may make integration and system tests incomplete. Based on the number of detected errors, the cost of one error detected by integrated testing is much lower than that of the other two. It can be seen that in a limited human time testing, unit testing has done too much, and integration testing is too small. This means that the human resources consumed for unit testing are overly optimistic or the configurations for testing are not ideal and should be improved.

　
Testing human hours testing human distribution ratio error count error distribution ratio average number of hours/error count
Unit Test 227.104 58.6% 49 39.51% 4.635
Integrated Testing 87.212 23.3% 54 43.55% 1.615
System Testing 70.184 18.1% 21 16.94% 3.342

Total 384.5 100% 124 100% 3.2

In addition to the above test results analysis. If there is a problem, you should analyze the cause of the error and try to solve it from the root cause of the problem. This includes improvements in testing and development processes. In the past, you may consider using adequate training for testers to avoid wasting valuable manpower and time in testing due to poor preparation. The establishment of the testing standard program also helps improve the testing efficiency. The latter can be prevented by studying the cause of the error. For example, problems with design errors that have not been documented in demand changes, or software design that has not been fully considered before the writing of programs, resulting in a large number of errors caused by poor design should be prevented, this is expected to fundamentally solve software problems.

Conclusion

To improve software quality and productivity, you must first master the current situation. Testing is both necessary and evil, and we need to develop the best way to deal with it. Although there are a large number of books and articles on software testing methodology, they must be adjusted according to their own situations in terms of application. Dai Ming, a quality control master, believes that quality control cannot rely on tests to improve work processes. Therefore, testing is just a start. The goal is to improve software quality and productivity through testing. I wish you all the defenders of software quality a smooth move forward in your position, win the honor of testing, and enjoy your successful products.