Remember a very covert bug.

Today, there is such a problem. The problem successfully went through the first round of tests, the second round of tests and the successful release to the line.

Background

The content of this test is an activity page, the requirements of the scene is to access the link from within the app and go to this activity page, along with the automatic login (do not need to enter user name and password, the user login status of the app to obtain and automatic login).

Problem

Only the first visit to the active page will return 50x, and then into the active page will no longer return 50x, everything is OK.

I couldn't find the cause of the problem.

Because the daily test work to a large number of switch hosts, it is easy to appear due to browser or app cache problems caused. In general, by refreshing and cleaning up the problem that the cache solves, most of our cases are considered environment-led and not part of the development code problem. It is precisely this idea, with this problem logically will only appear once the concealment, resulting in I have not been able to find this problem.

Reflection

To find such a bug, you first have to be very sensitive to the data returned by the interface. We have to catch the packet through the fiddler, find the Problem interface, and the data returned by this interface to the development, let development to find the corresponding background code, that is, the source of the problem to confirm. Only in this way can we find the problem by black-box testing. It is difficult to find this problem youbiaojili without the black-box test of real-time packet tracking with fiddler. In other words, to find this problem, there are only two ways to do it from a simple black box test:

1. A large number of user tests;

2. That's what we're talking about, by cooperating with fiddler for real-time capture, never let go of any suspicious interface that returns data (you must be experienced enough to be sensitive enough.) And it is best to clear the fiddler before each operation to ensure that the captured data you do not think is the result of your actions or actions outside of the action and ignored. Anyway, it's hard to find! You have to stare at the data returned from all the interfaces in the entire test process in real time and judge whether the return value is normal. Once a problem has been identified, it must be reproduced in a new environment, or it will be considered an environmental issue and overlooked due to the inability to reproduce the current environment.

And if the development of the code has enough unit testing, this problem should be not difficult to capture! The importance of unit testing is visible.

Summarize

There are two ways to avoid this problem, one is to know the logic of the Code (unit tests should be able to spot the problem). The other is to find the problem from the table and the Way (black box testing). Because the problem is very subtle, it is difficult to reproduce the problem on a single machine, so you must have multiple terminals to test to find the problem (at least three). Once you find the problem of returning 50x, we must pay attention to it, record the link of the problem and the corresponding repro step. Before the online confirmation, to get at least three test terminals and set up a WiFi proxy connection test environment, respectively, in each terminal to reproduce the problem, if the three terminals reproduce the problem, you can determine the problem exists, through a stable repro step, the problem occurs in the process of capturing the packet to the wrong interface, The interface and interface of the return data to the development, let development to find the corresponding error in the background code logic.

Remember a very covert bug.