Domestic Software Testing Status investigation report (III)

Investigation Report on current situation of software testing in China (III.) Quality Report of original Zhu Shaomin software Quality Report

Now that two reports have been published, and the focus is not high, and the automation content that may be of greatest concern does not appear, today we will analyze the highlights of this research: the status of automated testing.

1. Half of the domestic software testing team did not use exploratory testing, although as early as 2006 exploratory testing (exploratory testing, ET) on relatively mature, have a clear idea, method, operation process and management. Et first appeared in 1983, the relatively clear concept is 1995 years. If you do not know ET, you can refer to the history of exploratory testing (History of definitions of ET) and what exactly is agile and exploratory testing. Relative Et,st (scripted testing, script based testing) are relatively traditional. The development of key software products team, you can use the St mode, et supplemented, the current part of the One-third, the expected results should account for more than 60%, and the expected results of a larger gap. For most non-critical systems, et can be fully implemented, but also without St, because automated testing also belongs to St. Ideal situation is the ET and St two ways to combine (I hope that ET, st as "test Method"), for example, the new function test can use ET, regression test using St, if you want to better understand its operation, can refer to: Software testing a new formula caused by thinking.

2. Research and development team in the past year, the overall model of automated functional testing shows "rugby mode", two (UT, UI) Small, intermediate (API) large, but also benign. API Test ranked first, followed by UI (user interface) test, Unit Test (UT) is the lowest, if the UI and UT results are interchangeable better. From the ideal pyramid model, UT should be the highest, then the API, and finally the UI. There are no barriers to automated testing of your code, and the code is changing every day. The interface test automation input is small, the benefit is big, should vigorously advocate, but on the one hand requires the API to be stable, because its change can cause the big influence.

From the survey results, there is no obvious difference between the front-end and the back end, but theoretically, the back end is also the focus area of automated testing, can reach more than 90%.

3. For automated inputs (or ROI) patterns, nearly half of the people do not understand that this is normal, engineers focus on technology, call work, may not focus on input and output, and focus on ROI, most of the managers. But to enter the software development of this line, but also need to understand, more should be concerned about whether their work is worthy of greater investment. In addition to the pyramid and rugby models described in the 2nd above, the ice cream pattern is the opposite of the pyramid model, which is to be discarded, and the proportion of the rugby model is close. Cupcakes mode, more UI, although understandable, but also discouraged. The results were significantly higher, and the cupcakes model was much higher than the rugby model, and the pyramid model was close.

4. Function Automation test implementation is not ideal. Before, there are students told me that I am more optimistic, the reality is cruel. No automation accounted for the proportion of not low, 18.53%, very little automated test also up to 30%. And the real automated test was done well (>80%), only 10%. The domestic ratio of below 50% is as high as 75.33% (27.07+29.73+18.53), below the international level of 10% (39+26=65%), both domestic and foreign, the level of automation testing is still low, although everyone thinks that automation testing is important.

The following is the result of an international survey

5. Software projects in the past year, the unit test code line coverage is not required, close to 60%, unit testing is very unsatisfactory. Required to reach the total coverage of less than 5%, more than 80% is reasonable requirements, and only 13%.

6. API Automation testing tools are mainly used JMeter, followed by their own development tools, postman and Soupui also have a high rate of use.

7. Web function Automation Testing tools are mainly used selenium+webdriver, more than half, followed by their own development tools, most Web test tools have no market. "Other" accounted for One-third, need further investigation, or comments recommended.

8. Mobile application automation testing tools mainly used Appium, nearly half (44%), "other" accounted for a larger proportion, higher than 40%, need further investigation, or comments recommended. The official espresso application is very low, calabash lower, but still hope that people pay more attention to these two tools.

9. Performance testing tools relatively concentrated, open source is jmeter, business tools are loadrunner, almost no opponents, "other" accounted for relatively low proportion (18.4%). For special applications, you need to develop yourself (21%). Gatling is good for web testing, but not many engineers who are familiar with the Scala scripting language, but it's not difficult to master Scala.

10. Security testing tools too much, distributed more widely, "other" accounted for a high proportion (66%), including many people do not use security testing tools, the number of security testing, compared to the performance test is much lower, not clear which to choose. In order to understand the security test tool situation, a new survey (HTTPS://ZH.SURVEYMONKEY.COM/R/TT8S3N2), because it is a foreign site, the speed is very slow, it seems to be moved to the domestic web site in the investigation, then need everyone support.

11. Test management and defect management tools are relatively concentrated in--jira and Zen, and their development and "other" share is close to about 23%.

12. The test work that your team will strengthen in the next 1-2 years is mainly focused on the "automated testing" and "Development technology" upgrading. Automated testing is really not satisfying, with almost half of the team working as a focus in 2018, hoping to improve the automated test next year.

How do you feel when you read this report? Welcome to the message discussion.

In the fourth report, we compare the traditional IT companies and the new Internet companies on these 12 items to see how they differ.

Reference:

Domestic Software Testing Status investigation report (i)

Domestic Software Testing Status Investigation report (II.)

Initial report on the status of Agile testing (i)

Initial report on the current survey of Agile Testing (II.)

Preliminary report on the current survey of Agile Testing (III.)

Investigation report on application of software testing technology (I.)

Investigation report on the application of software testing technology (II.)

Investigation report on the application of software testing technology (III.)

Investigation report on the application of software testing technology (IV.)