The path to Software Quality

I have worked with people with very high qualifications but no practical experience. I have also worked with talented engineers who have little or no qualifications, I have had to work with people who don't want to work with me or who are not interested in learning new things. If we say this profession is a piece of paper, these people are like stains on paper. The inferiority of the software development industry cannot be totally blamed on ignorant managers, cunning marketers, and users who are always impatient. In fact, it is largely attributed to some practitioners in the industry, they should engage in some actions that will not cause such great harm as in the software industry even if they neglect their duties, instead of mixing them into the most complex and creative industry that gathers human imagination.

-- Mongowwilson

 

1 Guide

Recently, some colleagues asked me some questions about software quality. I am sorry that I have not provided a good answer. The main reason is that over the past two years, I have been busy learning and summarizing the basic knowledge. Although I have been engaged in a lot of such content, I have not made a good summary and record, I am ashamed of this.

At this project kickoff meeting, I heard that Han's sentence is probably like this: "as an Internet company, we not only need to make a good product, we also need to educate the masses from products and guide the society ". As a member of the industry, I feel that I am shouldering great responsibilities when I see this section by Matthew Wilson.

Therefore, we would like to take this opportunity to summarize the software quality management experience we have experienced over the past two years. In addition, I would like to declare that, first of all, this is just some of my thoughts on this. It may be different from other people's ideas. If you have other ideas, we can discuss and learn, second, personal opinions will inevitably conflict with others. If you offend readers, please forgive me and give me more advice. Finally, I like the discussion very much, because every discussion will have rich gains. I am very happy if you have any students to discuss and learn!

This part may not be consistent with the entire topic (perfectc ++), but each language should have a corresponding quality management method. After all, every software cannot guarantee that there is no bug, therefore, software quality is the most important and fundamental. So I still include it as part of perfect c ++, but the content should also apply to most languages.

 

2. Software Quality

How to measure software quality first? I think the simplest measure most people think of is: How many bugs are there in this software?

Such a simple standard is actually incorrect. First, software bugs cannot be counted, because bugs may exist in different scenarios and situations, but this quantity cannot be measured. Second, quality cannot be measured by "errors" alone. For example, software execution efficiency should also be regarded as one of the standards.

Speaking of this part, I can only reference the relevant part in code Daquan (Chapter 1 software quality Overview ). Among them, software quality is based on external features, that is, the user's perspective, including correctness, availability, efficiency, reliability, integrity, adaptability, accuracy and robustness; from the internal characteristics, from the code perspective, this includes maintainability, flexibility, portability, reusability, readability, testability, and comprehensibility.

 

2.1 external features

We often pay attention to external features, because from the perspective of users, this is the most important part of users. What users care about is naturally what we value most. However, software quality cannot process these details in multiple degrees. If you pay too much attention to a specific detail, it will have a certain impact on other aspects, as shown in the following table:

	Correctness	Availability	Efficiency	Reliability	Integrity	Adaptability	Accuracy	Robustness
Correctness	Bytes		Bytes	Bytes			Bytes	Bytes
Availability		Bytes				Bytes	Bytes
Efficiency	Bytes		Bytes	Bytes	Bytes	Bytes	Bytes
Reliability	Bytes			Bytes	Bytes		Bytes	Bytes
Integrity			Bytes	Bytes	Bytes
Adaptability					Bytes	Bytes		Bytes
Accuracy	Bytes		Bytes	Bytes		Bytes	Bytes	Bytes
Robustness	Bytes	Bytes	Bytes	Bytes	Bytes	Bytes	Bytes	Bytes

Of course, this is only a typical relationship. In different projects, the impact may be the opposite.

 

2.2 internal features

As a developer, you should think about internal features. As far as I know, most people do not pay attention to this part, and this Part has a great impact. This part is the root of external features. From my perspective, internal feature management is in place, which will greatly improve the software quality.

 

2.3 General principles

After talking about this, there may be a lot of people who are not very concerned about software quality, but think about how many lines of code a person writes on average every day? What are you doing at other times? I think most of the time is debugging, locating bugs, solving problems, and testing again.

The best way to improve production efficiency and quality is to reduce the rework time spent on this code, regardless of whether the reworked code is caused by requirements, design changes, or debugging.

In addition, how much time is consumed to add a function to the software? I think adding a function that is unrelated to the whole is much easier than adding a function in a complicated logic.

Therefore, the obvious way to shorten the development cycle is to improve the product quality, thus reducing the time spent on debugging and software rework.

Of course, the above two points are all conclusions on a large amount of data. For details, refer to Chapter 20.5 of code Daquan: general principles of software quality.

 

3. Technologies for Improving Software Quality

It's boring to talk about some impractical things, so hurry and skip the cold dishes and start serving hard dishes!

 

3.1 main methods

Since software quality is divided into so many aspects, how can we improve it? Improvement methods include:

1> define clear software quality objectives. First, we need to have a clear goal. If the goal is not set, it will be like marching in the desert, and finally it will all perish.

2> determine the work to be done to ensure these goals. What work should we do to achieve these goals? There is only a goal and no work plan, which is equivalent to taking a rest in the same place, and the effect is the same.

3> determine a specific test policy. The test cannot improve the software quality (this is the opposite of the current work situation, which makes me unable to speak out). It is just a haystack to evaluate and improve the software quality based on the test results, if you don't get it, you can only see luck, and it will make the testers miserable. Testers must put forward clear testing strategies and cannot use "tests" as the primary method to improve software quality.

4> define clear requirements. The definition of requirements must be clear. In many cases, the definition of requirements personnel is the opposite of what the user wants. Such development is useless.

5> define a clear development project guide. The Development Project Guide includes the above definitions and should control the technical characteristics of software and implement all development activities. For example, code specifications are a typical example.

6> code review. This work is very important. Not only can we find a large number of problems, but it can also improve the technical quality of our staff to a certain extent. However, some review tasks are not in place, so we pay too much attention to details and the investment cost is too high, little gains.

 

3.2 common technologies and bug detection rate

The following are some common technologies and the detection rate of corresponding bugs. You can have an intuitive understanding.

Troubleshooting measures	Lowest detection rate	Typical detection rate	Best detection rate
Informal Design Review	25%	35%	40%
Formal Design Review	45%	55%	65%
Informal code review	20%	25%	35%
Formal code review	45%	60%	75%
Personal Code Review	20%	40%	60%
Unit Test	15%	30%	50%
Component Test	20%	30%	35%
Integration Test	25%	35%	40%
Regression testing	15%	25%	30%
System Test	25%	40%	55%
Small-scale beta testing (less than 10 persons)	25%	35%	40%
Large-scale beta testing (more than 1000 people)	60%	75%	85%

The above data should be understood by every tester. The test Manager does not simply arrange to estimate the number of test cases, allocate and write test cases, arrange for testing, and submit bugs. According to my personal understanding, determining software quality is the primary task of a test Manager. Testing is just one of the most basic tasks. If the test is used alone, then every new graduate student will be competent after several months of training, even a few weeks of training, without improving his skills, and everyone will know the consequences.

The following are some tools and experiences for the above objectives. Of course, this is only within the scope of my experience. You are welcome to come and try again.

 

4 basic objectives of software quality

When talking about software quality objectives, we must first clarify the requirements of qualified code. The main indicators are as follows:

1> total number of lines: the number of lines of code, including empty lines.

2> function circle complexity: the number of executable paths of a function. The following statement contributes 1 to the value of the sphere complexity, if/else/For/while statement, ternary operator statement, and if/For/while judgment condition | or &&, the switch statement, followed by the case statement of the break/goto/return/throw/continue statement, and the catch/fail t statement.

3> function depth: the depth of the function indicates the number of nested branches in the function.

4> Number of statements: In C ++, the statements end with a semicolon. The branch statement if, loop statement for/while, and jump statement goto are all included, and the pre-processing statement # include/# define/# UNDEF is also included, other pre-processing statements are not calculated, and the statements between # else/# Elif/# endif are ignored.

5> branch statement proportion: This value indicates the proportion of the branch statement to the number of statements. Here, the "branch statement" refers to the statements that make the program run in an unordered manner, including if/else/For/while/switch.

6> comment proportion: This value indicates the proportion of comment rows to the total number of rows.

7> Number of functions: number of functions.

8> Average number of statements contained in each function: Total number of function statements divided by the number of functions.

The first software to be introduced is source monitor, which is a basic tool for code quality detection, that is, a tool for calculating the above indicators.

For developers, you can add them to a series of editing tools such as Visual Studio and ultraedit. For details about how to add them, refer to the source monitor document. Testers need to understand how to use the command line mode to create a project and filter out non-conforming targets.

The following is a reference object (from Huawei ):

1> the circle complexity is less than 7;

2> the maximum number of rows is less than 1000;

3> the maximum number of methods for each class is less than 30;

4> each method has a maximum of 50 rows;

5> the maximum depth is less than 5.

 

5. Basic Objectives of code Quality

For C ++, it is very difficult to check its code quality, especially in terms of a large number of pointers and templates. Therefore, there are not many code analysis tools available, most of them use pclint for static code check.

Pclint is a very strict compiler. It can be found that many compilers do not remind you of the errors, so as to improve the quality of the Code. For example, it can be used to identify out-of-bounds memory problems, uninitialized variables, and NULL pointer operations, redundant code and execution efficiency issues.

For developers, each time they submit code, they must execute pclint to ensure the security of the modified Code. For testers, The pclint execution environment should be configured and the code should be scanned and checked regularly to warn developers.

The basic goal of pclint is no compilation error or warning.

 

6 testing objectives Unit 4 testing

The tests mentioned here are unit tests, which are the main concern of developers. However, testers are required to perform management and control.

Theoretically, all functions with a circle complexity greater than 2 should be written in related unit tests. The writing task is generally completed by the author of the function. Of course, there are also many examples of test-driven development, but this is too demanding for general testers, you need to have a good understanding of the overall framework and code, and be able to clearly define the development framework and function to be competent. So I will not go into detail here.

Most of the unit testing tools currently use gtest, gmock, and mockcpp. You only need to select one of the mock tools.

In this step, developers need to add the corresponding test function every time they add a function. Testers should perform smoke tests each time they build to ensure that all test functions pass. In addition, they should combine the benchmark of the circle complexity. If the circle complexity is high but there is no test function, warning should be given.

 

6.2 System Test

This requires the attention of testers. The main tool can use the integrated test project in Visual Studio, record the script, and execute the script to achieve the purpose of system testing.

Generally, you need to write a test case before testing. This task was previously written by the developer. In this way, the amount of code modifications, scope of modifications, and use cases can be written in great detail, which is highly targeted. Reference benchmark objective: Number of test cases/number of lines modified by code = 1/3.

After the test is completed, the standard for the official release should be: number of failed test cases/total number of valid test cases <3%. Of course, when the beta version is released, you can adjust the size of this value for different aspects.

If the version release requirements are not met, You should modify the code for the bug and append the test case until it meets the requirements of this range.

As I am not a tester, I do not know much about system testing, including automated testing using tools. If you have experience with Daniel, please kindly advise.

 

7. Clear requirements and objectives

This mainly relies on product review meetings to organize product personnel, developers, testers, and users to review product requirements. In this way, the requirements can be clearly defined, the requirements can be refined, the development time can be reduced, the iteration can be performed quickly, and the product development can be promoted.

As I have not had a deep understanding of this aspect, I welcome anyone who has a deep understanding to add this question.

 

8. Integrated Construction

The previous sections describe the objectives and methods for achieving the goals of software quality. Of course, there is no relevant part of code review. Generally, different companies have different organizational methods, I will not go into details here. However, to combine the above parts for automatic control, check every day or every code commit, you can greatly reduce manpower, it is not good for everyone to take the time to communicate and learn and continue to improve themselves?

It is also possible to achieve this goal. First, you need to automatically build a management tool: cruise control, which is divided into. NET and Java versions. Of course, you 'd better select the latter, because there are many tools that can be used together. The main function of this tool is to provide a server for you to configure tasks, such as compile, check each target, and finally update tasks by time or code, run the check on each task to notify the user of the failed task.

Next, ant can be used to execute various tasks. ant is a Java-based build tool. ant can be used to specify different tasks during compilation, and then specify these tasks in cruise control, the automatic execution can be achieved.

Finally, the statistics and representation of the results. Cruise control is just a management tool. The specific data needs to be collected by the tester and the corresponding part is counted. For example, the number of cases submitted in 1st day of this month and the number of execution failures can indicate information about the software quality. To achieve this goal, testers generally need to master one or more scripting languages.

9 further thoughts

After so many years of development in the software industry, everyone has followed the process of "agility". I have never walked through the previous era. It cannot be said whether it is "fast" or "slow ", whether the quality is "high" or "low", everyone is desperately trying to catch up with the schedule and catch up with the version.

Recently, I have always wanted to write something, sum up something, and have many topics, but I have a very simple understanding. I feel that the road is getting longer and longer, and I am afraid to say "yes" or "wrong, thanks to these great people, I have opened up this road. Is there the day I have finished?

The times are changing, and technologies are improving. Even if we do not get on the last bus of technological trends, we should move towards the future!

The path to Software Quality