The cause analysis and countermeasure of the bug can't reproduce

Summary: This article briefly analyzes the possible causes of a Bug that cannot be reproduced, including inconsistent environments, lack of the most accurate descriptions, and inappropriate browser settings. In view of these reasons, this paper gives the corresponding countermeasures. With these measures, you can reproduce many of the bugs that were previously thought to be non-reproducible.
Keywords: reproduce; Bug; environment

InTestWhen a person submits a bug, the last thing you want to see is that they are marked as invalid, although you believe it must be a bug. When developers look at the description of bugs, the least desirable result is that you can't reproduce them, although he uses all the possible methods to reproduce them. Once this happens, the testers will be sad and the developers will have a problem with the testers. This makes the relationship between testers and developers more tense than the relationship would otherwise be. This adds fuel to the tension between testers and developers.
In order to reduce the occurrence of this situation, it is necessary to analyze the reasons why the bug cannot be reproduced. According to my Test experience, the reasons why a bug cannot be reproduced are:
I. Inconsistent environment
This is the most important reason why a bug cannot be restarted. In a bug where developers think this is not valid, it is estimated that at least 80% of the bugs are due to environmental inconsistencies. This includes inconsistencies between the development environment and the test environment, as well as the actual operating environment of the development environment and the system.
There are some reasons for the bug, and it also needs a certain environment to reproduce. If there is no corresponding environment, then you may be difficult to this bug.
Broadly speaking, any factor that guarantees or affects the software is the environment. For example, the configuration of hardware, software settings, network bandwidth, speed, and so on. In general, the software factors in the environment are: the build of the system, the type and version of the application server, the language and version of the operation system, the language and version of the browser, and so on.
Here are some of my interesting experiences: a bug appears only on WebSphere 6.0.2.15, and as the developer asks, the bug will automatically disappear after upgrading WebSphere to 6.0.2.17. Therefore, this bug is caused by inconsistencies in the version of WebSphere.
Several of the images have mysteriously disappeared on a build and are beginning to suspect that the developer has made a mistake by modifying other bugs. Later after careful and careful testing only to find that the original isOperating SystemThe language of the "ghost": the tester used the machine's operating system language is Simplified Chinese, the developers are using traditional Chinese.
The most accurate action to reproduce the bug is missing from the bug's description, which is the page where the test system adds a role:

Figure 1: page with role added
When a tester enters some data and then clicks Save, "accidentally" appearsJava. lang. NullPointerException's error. To be careful, it is because this is a tester who inadvertently discovers, and after this error, you can no longer add any role (of course, you can not reproduce the bug). Worst of all, many of the other function-related points of role cannot be verified. Although it is not known why this error occurred, it is not possible to reproduce the bug, but given the severity of the bug, the tester has submitted the bug to the bug library (which proved to be a very sensible move afterwards).
When testing the next build, I asked the tester to focus on the bug. The real reason for this bug was found with the help of the DBA: After you enter role code, if role name is empty, the page is not checked (the foreground is not checked), but when you click Save, the data needs to be saved toDatabase, role code and role name cannot be empty. As a result, the Java.lang.NullPointerException error message appears.
After I find out why this bug occurred, I recommend that the developer use role code as role name if the user enters role code without entering role name. After testing, this bug will never appear again.
The following is the full history of this bug:
[Step]
1. ADD one role on role Manager;
2. Click on the "Save" button to save it.

[Result & Memo]
One error message appears:java.lang.NullPointerException. As a result, I could not does any operation on "Role Manager" tab. This is a bug appears even after I restart the application server.

-------Additional Comment #1 from Jim 2006-11-25 14:08-------
I cannot reproduce this bug. Describe testing scenario in details + Attach error logs.

-------Additional Comment #2 from Mike 2006-12-15 10:28------
[Step]
1. On the ' Role Manager ' tab, Click on ' Add ' button;
2. Enter one valid role code and click on ' Save ' button. Note:you is forbidden to enter any value for ' Role Name ' on order to reproduce this bug.
Now there was one error message on the page when clicking on ' Role Manager ' tab.

[Memo]
According to the schema tbcn_wf_role, the ' Name ' column was forbidden to be null. But there was no code to check it when such occasion happens. For detail information, please view the attached file.
To avoid this bug happening, I suggest you set the value of "role Code" as "role Name" if such occasion happens.

-------Additional Comment #3 from Jim 2006-12-17 18:06-------
Fixed in Build # >= 484
Second, improper settings of the browser
For Web testing , IE's settings are also important for reproducing bugs. Several of the following pictures illustrate several key settings for the browser:

Figure 2:internet Settings for temporary files

Figure 3: Settings for the advanced option

Note: In the "advanced" option, it is important for the test to "Prevent Footstep debugging" (no selection, i.e. no front tick) and "show notification of each script error" (select it, that is, a tick before it).

These settings play an important role in reproducing bugs that are related to scripting errors. Based on the notification of these script errors, developers can quickly find errors in the code and fix it in a timely manner.

Third, memory leaks

One of the main reasons why some systems run slow and unresponsive after a long run is memory leaks. Some developers do not develop the habit of recovering memory in a timely manner, the result is that the system is constantly applying for memory space, but not a bit of memory release. Such errors will not occur in the short term, but will occur when the system is running for a long time, and this will cause a series of problems. This type of problem can only be discovered after a long run of tests.

Four, the function interface type does not match

This kind of error is difficult to reproduce, and it is difficult to find the real reason for it, generally only after viewing the source code to discover. Some types of data are automatically converted by the system, and there are generally no errors, but some potential errors occur when some data is truncated or cast to another data type. This type of error is most likely to occur during integration testing.

Now that we have analyzed the reasons why these bugs cannot be reproduced, we can remedy the problem:

1. Testers should be aware of the meaning of the test environment and add an accurate description of the test environment to the bug report, especially those environmental factors that affect the recurrence of the bug.

2. Step of the bug to accurately describe the procedure. To reproduce a bug, testers may need to continuously track, test, and locate several builds to create the most fundamental cause of the bug.

3. Set the browser's options correctly.

4. Performance-specific software or systems must perform long-term load testing (Loading testing ) to identify issues such as memory leaks that require long-term operation. According to Microsoft 's testing experience, if the software can pass a 72-hour brute force test, the software may have a slight chance of having problems after 72 hours. So just a 72-hour brute force test of the software.

5. Be sure to check that the function interface type matches when integrating tests.

6. Testers should maintain a good relationship with developers, DBAs, etc. Communicate with them in a timely and proactive manner and listen to their opinions. With their help, you can more easily find out where the problem lies.

Based on these suggestions, I believe most of the bugs that can't be reproduced can be reproduced. Of course, due to the development of the test system language, development platform and other factors, I am not able to enumerate all the reasons cannot reproduce the bug occurred. If you still encounter some serious, but can not reproduce the bug, then do not panic, you can follow the following actions to find the cause of the bug:

1. Remember the symptoms of the bug and all the environmental factors, and don't miss out on the details of shred.

2. Analyze those environmental factors together with developers, DBAs, system designers, project managers, etc. to analyze the important factors that affect the recurrence of this bug, and to test the guesswork by installing the same system on the same environment, based on past experience.

In addition, for some can not reproduce, but the severity of the bug is not very high, you can temporarily only record, and do not have to spend a lot of manpower and material to analyze. If the next time it comes up again, analyze whether you need to track this bug based on the frequency that occurs. If you need to track it, then you must immediately after it occurs to the environment, such as error messages, interfaces, logs and so on. This also facilitates the developer to locate and analyze it, and to fix it accurately and quickly. If conditions permit, testers should immediately protect the existing environment and invite relevant developers and system analysts to discuss the causes and workarounds for this issue.

Reprint: http://www.51testing.com/html/11/n-16611.html

The cause analysis and countermeasure of the bug can't reproduce