Software Test in Black Swan (iii) test of the average Stan and the extreme Stan

1. Breakthrough and non-breakthrough

"Black Swan" talks about the concept of breakthrough and non-breakthrough.

The income of some professions in the world is not a breakthrough, such as a baker, consultants, massage therapists, dentists, etc., whose income is limited by the number of guests served within a given period of time, is to a large extent predictable, and the Baker must bake a new loaf of bread for each client, no matter how expensive the price of the bread he sells, Its income is always limited.

Other professions such as the recording division, the film actor, the writer, speculators and so on, they can only spend a single investment without too much work may increase the income after a few zeros, "Harry Potter's author does not have to read the book every time they want to rewrite it, this occupation is a career of income and breakthrough."

If you think about it, you'll find that groundbreaking work is often done by those who are better at using and organizing creative thinking and innovative techniques, while less disruptive parts can often be stripped away to those who are paid on piecework or paid on time.

Is the test a groundbreaking job? This can not be generalized, the scope of the test is too large, contains too much content, we will continue to discuss this topic in the later part.

2. Average Stan and extreme Stan

This breakthrough and non breakthrough difference is related to two other concepts: the average Stein and the extreme Stan.

Human height, weight and other physical attributes are not a breakthrough, embodies a kind of is called "average Stan (Mediocristan)" phenomenon. Take the example in Black Swan For example: if you randomly select 1000 people from the general population and then take a sample of the heaviest person you can think of, assume that he weighs 5 times times the average weight, and that it accounts for 0.5% of the overall weight, assuming you've picked 10000 people, his body The share of the weight is almost negligible. "In the ideal average Stan, the individual effects of a particular thing are small, and only the group has a large impact." "In fact, for the average Stan," when the sample size is large enough, no single case will have a significant effect on the whole. ”

And people's income is a breakthrough, reflecting a phenomenon called "extreme Stan (Extremistan)." If we consider the income of these 1000 people, if Bill Gates is put into the sample, what is the proportion of his assets to the total assets? Perhaps the sum of all 1000 other people's assets is just a fraction of Bill Gates ' assets. In extreme Stan, the individual's effect on the whole is groundbreaking and can have a disproportionate impact. It is noteworthy that "extreme Stan can create the Black Swan phenomenon".

For software testing, because of the huge impact of black swans, we need to be constantly aware of what is going on in terms of the extreme Stan, which is the average Stan, and using the information we collect to help determine how to avoid the black Swan phenomenon, or to facilitate the occurrence of the Black Swan.

3. The average Stan in the test

Keywords: average stan, large number of laws, N of the square root law

If you're a Test manager and you have tons of data pouring into your brain every day, you need to learn how to judge and use the data to help you better manage your test projects.

If you need to develop a test plan for a new test project, you need to do a test estimate of how many test-months the project needs to be put into. How many manual test cases can you probably execute? How much time does the test plan, test design, and test execution take up? Wait a minute. One approach is to estimate the current version of the test input by referencing the product scale and test inputs for each historical version of the product.

When you do this, you need to know:

You are faced with an average Stan's problem;

Perhaps some historical version of the data appears unusually prominent (large or small), but you do not have to pay special attention to, or even estimate the time will ignore these abnormal point data;

The accuracy of your estimates is influenced by the amount of historical sample data you choose, and if you have only one historical version available for your reference, you may be able to estimate a significant deviation.

What you care about is that the data is generally a reference value to the decision you want to make, rather than pursuing an absolutely accurate estimate, because you know that no matter how hard you try, it is impossible to develop an absolutely accurate test plan.

Similar to the average Stan's example, if you want to understand the average productivity of your test team, such as "Day execution use cases", "Day design Use Cases", "daily submission problem singular" and so on, you may wish to observe a period of time, record data, calculate an average. Of course, you do this because you are more concerned about the impact of collective events, average events, regular events and predicted events on you at this time, and allow for certain errors to exist.

Gerald M. Weinberg's "Law of large Numbers" in "an Introduction to general Systems thinking" is related to the average Stan. Physicists in the 19th century wanted to study the movement of air molecules in bottles, and they had no way of figuring out how each molecule was moving, because the number of molecules was so large that they had to assume that the observational properties they were interested in were some of the average characteristics of a large number of molecules, rather than the properties of any of Because the number of molecules is particularly large, their research also satisfies the so-called "law of large Numbers": the more the number of observations, the closer the observation value to the predicted average. Schr.dinger also gives the "square root law of n" to predict accuracy: