Software development and test management

From Computer World News

To ensure that the software project is completed on schedule, it depends not only on developers, but also testers.

One of the mistakes that a project manager often makes is to think that only software engineers can be hired, and other people are not necessary, or that software engineers can make a high proportion of the entire team. They may think that the more developers, the more programs they write, this is a wrong concept. The purpose of a project is to complete the software, rather than to complete more program code. In the development team, some work is not suitable for software engineers.

To ensure that the software project is completed on schedule, it depends not only on developers, but also testers. Software development seems to be on schedule, rather than playing a symphony: The symphony is harmonious, orderly, and elegant, while software development is more like a pile of jobs. No two notes in the symphony cannot conflict with each other. What the whole shows is a piece of beautiful music. In all uncertain software development processes, let the tester's "bug baton" to let everyone know when to perform and when to move back, it is the best rule that Microsoft has taken the software development process to a climax!

The test group is not an assistant to the development team.

No one seems to pay more attention to the power of testing than Microsoft. In Microsoft's product group, the test group is in parallel with the product planning group, product management group, development group, and user education group. The relationship between testing and software costs is that the earlier problems found in the product, the lower the development cost, the higher the product quality, and the lower the maintenance cost after the software is released. In the four phases of Microsoft's development cycle, the purpose of testing is to ensure the software quality and meet the design requirements and customer needs. Different types of errors are systematically revealed, it also takes a minimum amount of time and effort, and reduces software development and maintenance costs.

During the software development process, developers may be unaware of the actual needs due to some occasional things or some irrelevant inspiration, and temporarily forget what is the most appropriate function of the product, it was the test engineer who pulled them back to the original track. The tester is responsible for working with the entire project team to ensure that the product meets the predefined design objectives according to the predefined schedule. Testing personnel's work is a part of a holistic and continuous software development activity, rather than occasionally coming up with an embellishment. The quality of software products is determined by the resources used for testing, product functions, and project schedules. It is a balance between the three. For any product group, both subjective and objective, the existence of test engineers should be emphasized, which is an important guarantee of product quality.

Rome was not built in one day. No independent test groups were set up in earlier Microsoft development teams. At that time, Microsoft had several hundred people working on several projects. Generally, programmers wrote the program and tested it on their own. Later, Microsoft's projects grew larger and the software became more and more complex. Code Writing and testing were performed in parallel and an independent test group was gradually generated. An important figure is that the ratio of testers to developers in the Microsoft product group is about. In fact, to ensure that the software project is completed on schedule, it depends not only on developers, but also testers.

The advantage of a test group independent from a development group is that programmers tend to think that their own code is good enough, independent Test groups allow testers to test the software from different perspectives without any assumptions, which is conducive to ensuring that the quality of the software is always under control, and reuse and continuously improve test resources. However, the test group is independent from the Development Group, and does not mean that the programmer throws the "wall" after writing the code. When the test engineer finds all the bugs, the tester thinks that his job is to test the program, developers think that their job is to write programs to testers for testing. If you are a project manager, be careful! The superiority of developers makes testers feel that they are discriminated against, which will inevitably affect the quality of software. Programmers must complete basic tests after writing code and correcting problems, such as code path testing, unit testing, and problem verification. Product quality control exists in every step of the development process.

There is a joke that Microsoft's testers have been working for a long time and have the habit of being spider. When they get off work, they should comment on what their wife has done. For Microsoft, the smart, meticulous, conscientious, patient, and perfect products have a responsible attitude towards users, independent and open thinking methods, and have sufficient technical background, however, test engineers who are willing to learn new technologies are more likely to be favored by "bosses.

Stages of software testing

The process of software development is like a team in a rush, but when it is running, it will "Break Away". The establishment of milestones is like a "point ", everyone must repeat the pace to achieve synchronization. In terms of the working mode of the test group, the phase division of the test is consistent with the project progress. That is to say, the tester should keep running at the same pace with other project team members, and cooperate with every milestone of the entire project to complete the corresponding testing work.

Dividing a large project into several sub-projects as a milestone development model is the essence of Microsoft software development management, as described in the previous Microsoft R & D Method (I, each product development cycle of Microsoft has the following milestones: CC (code complete: Code completed), beta testing, and RC (Release Candidate: candidate release ); RTM (release to manufacture: put into production), testers and others in the product group work together to reach every milestone. A complete test cycle should include running all test cases, beta standard tests, bug verification and all bugs, and random tests. A good test cycle ensures a comprehensive and complete evaluation of the software quality. Each milestone must have at least a complete test cycle.

In Microsoft's product development cycle, during the planning stage, when developers plan and compile progress, conduct feasibility studies on function implementation, and provide feedback on the planned functions, the tester should study the specifications and prepare a test plan. In the second stage, namely the development stage, when the developer is writing code, testing and debugging, the tester should begin to design test cases, develop automatic test tools and programs, familiarize yourself with the necessary environments, tools, software and hardware, and constantly enrich test cases until CC (Code completed) milestone-the software can perform an overall test at this time. The user interface may be imperfect but can work, and there may be many obvious bugs.

In the third stage of the development cycle, testers can perform large-scale tests, such as system-level overall testing, interactive testing, and deep testing. After the test, the tester should say "no" to the new function until the bate test milestone is reached. Reaching this milestone means that all critical beta problems have been fixed and closed, all planned functions have been working in the software, the product is stable, and most interfaces are available, although it may be only a part of it, online help and user manuals are available, and even publishing will not have a negative impact.

The purpose of the beta test is to determine whether the product can run normally on a variety of expected hardware platforms and operating systems. Although the feedback from the beta test is of great reference value, unless there are major problems, otherwise, the function set should not be modified. All suggestions and feedback should be included in the next version. After beta testing, we will be stepping into Rc and RTM. Testers should focus on changes after beta testing. When RC is reached, it means that the software quality status is no active bug; there are no pending issues; it has been stable for a period of time, such as few or no changes within a week, or the changes are very small. If no new bugs need to be modified are found in the tests after RC, the RTM can be reached, then the virus is checked, the CD is verified, and the content is checked.

It should be noted that during the entire phase of software testing, quality is not only the responsibility of the testing group. Ensure that the test group is fully dependent on the test group to ensure the quality. Otherwise, the results can only be poor quality and delayed progress. The advantage of adopting the zero-defect policy is that it can ensure the quality, progress, and function set of the product at the same time, and detect and correct bugs in the product as soon as possible, the product status and bug quantity are always controlled within the manageable scope. The programmer should correct all bugs with priority of level 1 (P1) before writing new code.

At the same time, the test group should develop some good test management tools, such as test case management tools and Bug management tools. Test engineers should refine the test sub-areas, pay attention to the quantity and quality of test cases, and respond quickly to changes in the bug status. It is necessary to avoid evaluating the engineer's work by bug.

Bug discovery and Management

In Microsoft's tester's opinion, the problems with which the functions are not implemented or are inconsistent with the specifications are bugs, and those that cannot work (crash or fail to respond) are bugs; the incompatible part is a bug, the boundary condition is not processed as a bug, the interface, messages, and prompts are not accurate enough, and sometimes the unfinished work is also a bug. Microsoft classifies common software bugs into the following types: Functional errors, user interface errors, boundary value errors, initialization errors, computing errors, and memory errors; hardware errors and document errors.

The measure taken by the test engineer after discovering a bug should first be to verify whether it is caused by accidental errors. If not, immediately create a new bug record, including specific reproduction steps, environment, screen, etc. analyze the cause of the bug as much as possible; design the appropriate priority and severity level, remember, the purpose of the tester is not to find more bugs, but to improve the quality of the product. The tester is assigned to a corresponding person based on the priority and severity of the bug, for example, programmers, project managers, and test group owners.

Generally, a bug has three states in the database: active, resolved, and closed ). These three States are usually represented by "red, yellow, and green" Colors in Microsoft. The active status refers to the status when a tester creates a bug. It must be assigned to the corresponding designer, developer, or user education personnel, indicating that the status of the bug is waiting for correction. The resolved status refers to the status after the bug is corrected by the designer, developer, or user education personnel. The status must be re-assigned to the tester who reports the bug, indicating that the bug has been corrected, but wait for verification. The closed status refers to the status after the tester completes verification and closes, indicating that the bug has been corrected and verified. However, if the same problem occurs again, it may also become active again, and another round of State loop starts. The trend of the number of active bugs is that the number of active bugs is usually very small before the code is completed. After the code is completed, it will grow very fast. When it approaches the beta test, it will decrease, and when it approaches the RC, it will go to zero. Therefore, the product quality and milestones can be determined based on the number of bugs created each day, the number of bugs corrected, and the number of active bugs.

There will always be defects that are characteristic of all intellectual activities. Software must have countless disadvantages. The problem is not to judge whether the product is good or not, but to decide which part of the product can be modified so that the product can be accepted or liked by users. Microsoft calls this judgment and modification process "first aid ". In the emergency room, doctors must quickly check all problems faced by the patient, take the most urgent and necessary first-aid medical measures, and then perform rescue based on priority. The same is true for the Microsoft product group. They divide the bug severity into four levels: leading to crashes; major problems, leading to serious problems, such as the failure to implement a small function; small problems, not very serious, if the prompt information is not accurate or a small problem occurs, if there are some typos. The priority of a bug is divided into four levels: corrected as soon as possible; corrected before each milestone ends; corrected if time permits; low priority. Microsoft has a "bug Court", which is usually convened by the program manager and consists of the Project Manager, Development Director, and Test director. It can be divided into specific "bug Emergency Team (bug
Triage Team) ", reviews every bug, selects and modifies the most important errors in the product, and determines the corresponding solution, so that most users can enjoy most of the time. Of course, the frequency of "bug courts" meetings depends on the stage at which they are held.

The test group should have a region dedicated to storing databases that record all bugs. This database should be the central record and control area of the entire product group, and all the records cannot be deleted. For each record, you can only add content all the time. The test group should have a public tool for communication and communication with all members of the project team. In cooperation with bug mail, the testing team can promptly notify the current owner of the Bug based on the current status of the bug; effectively track the progress of the project, provides criteria for product release, and accumulates testing personnel results.

Secret Weapon: Test Cases and test plans

To sum up, the essence of Microsoft testing is: Reusable test cases of the system; test activities centered on problem (Bug) discovery and tracking; independent testers; testing Plan Based on Product Planning and product design specifications; milestone-based software testing cycle in collaboration with the entire project. Testing plans based on product planning, product design specifications, and reusable system test cases are Microsoft's "secret weapon ".

At Microsoft, the test plan is an important tool to help testers manage test projects and discover bugs. It is a programmatic document. The test plan clarifies the test tasks and test content list of the project. The content cannot only exist in the minds of testers, but must be understood by the Project Manager and Development Manager, the test plan must enhance the communication between test tasks and the test implementation process and provide guidance. The test plan must also provide a framework structure for organizing and managing test projects to help control the progress.

The scope of the test plan should include Product Overview, test strategy, testing methodology, testing region, and testing configuration (software environment, hardware environment, and network environment), testing cycle (with project milestones), testing resource planning, risk analysis, and cases. The test plan is not static. Although the test plan was written at the product design stage, it will change with the change in the project schedule during subsequent development.

A good test case has the following characteristics: first, it is the most likely to catch errors; second, it is not repetitive or redundant; third, it is the most effective in a group of similar test cases; finally, do not be too simple or complex. Therefore, when designing test cases, testers should follow the following principles: Reuse personnel changes and new projects; be able to classify; Test content should not be repeated; and be stored in the database of test cases; it can be continuously enhanced during the project process.

When designing test cases, the following points are generally taken into account: Test basic functions based on product specifications; design common user usage plans; design rare or special use plans; cooperation with other components of the system (for example, fax and the Internet may all use a modem, and the sharing of devices should be considered during the test ); consider special cases (such as conflicts between memory and hardware) and design extreme cases (such as memory leakage and destructive testing ).