Do we need dedicated QA?

[Reference the comment in a sentence: hurt but true aside from some of the author's radical ideas, the author still needs to test and think about the issues: 1. testcase, testdata, testconfiguration does not implement version control, which is messy and overwrite and complete. It has a low reference value but is time-consuming. 2. The design of testcase is only based on requirements, but does not take into account the system implementation. Due to insufficient understanding of the system implementation, case coverage is incomplete. 3. What can be done in testing besides functional testing? 4. Where does the test help for development and the entire team? Is it just the check at the end of the workflow missing? 5. How to improve the development and testing time ratio? (Tools? Different test forms ?) 6. Is the proposed bug document complete in Bug Management? Is it helpful for reference? Different from the author's point of view, I think a team still needs a full-time test, but it is not just about executing a function test. The test should follow up every step of development to understand the system implementation, develop test plans and use cases based on system implementation and requirements. After the development, You can distribute the use cases to different developers for joint testing or cross-testing, follow up and analyze bugs, and provide development and repair suggestions. after the project is completed, build an automated maintenance system and precipitated management of the testing process. Testing should consider how to improve product quality and reduce work hours. That is, to improve the development and testing ratio on the basis of ensuring the quality of the project. ] Do we need dedicated QA? In April 11, 2012, Chen Hao posted comments, read comments, and read comments by 38,350 people.

This article must be controversial. I have discussed it many times on my Weibo and it is highly controversial each time. It is always a good thing to have different points of view and debate, which can lead to thinking. Therefore, if you agree with me in this blog post, I will be happy. If you think deeply, I will be happy. If you disagree, it doesn't matter, it can be discussed.

Before that, I would like to explain how this "full-time QA" is defined in my opinion.

1. It is a dedicated testing technician established by many companies and is only tested and not developed.
2. These QA engineers are not familiar with or even do not understand software development technologies.

 

I have experienced a number of companies with full-time QA teams (full-time testers). Since my development team in the previous company was screwed up by the QA Department on a project, I am increasingly skeptical about the significance of full-time QA. My opinion is not necessarily correct, but please let me express it clearly --I don't think you need a full-time QA, or even a full-time QA role or department, because developers who don't know how to perform tests are not required. Just as R & D managers who do not understand development cannot manage R & D teams.I feel more and more that Dev should be the most suitable candidate for testing. This must be the future trend (because I have seen the progress of Chinese programmers, compared to 10 years ago, today's programmers are already very comprehensive. In the next ten years, it must prove that my point of view is correct ).

Before I expand the description, I want to reference two articles:

Two articles

The first article is "on testers and testing". The author of this article is Sriram Krishnan, a programmer who has worked in Yahoo and Microsoft and has developed many software programs, I was once reported by The New York Times that I wrote a book. This article is a blog of his. He expressed these points in the article --

Most development teams do not need an independent test role. Even if yes, all the development times should be greater than the test times above 20:1 .. Evidence? Just look at some of the most successful software development teams since ancient times. Whether today's Facebook or the initial nt team 30 years ago, many great products come from teams with no or few testers.

Developers should test their own code. There is nothing to say. The truth behind this is not important. This includes unit testing, full-coverage automated testing, manual testing, or combined testing. If your developers cannot/do not want or think that this is "not my responsibility", you need a better programmer.

 

Another article is Yan Xin's "modern software engineering handout 9 testing QA roles and division of labor", which is a good article. He mentioned in his article the necessity of division of labor, such as a third-party appraisal organization,It also pointed out some problems of division of labor, such as a solid division of labor and a division of labor without clear responsibilities. These problems directly hit the key of division of labor.. I vaguely think that many of my views and content are the same, but there are still differences in form. In addition, my point of view is too clear to be easily oriented to extreme understanding.

You see,We all agree that Dev needs to understand testing, QA needs to understand development, but the division of labor is different. Since you have me and I have you, don't separate them. Let's develop and test together.. (In addition, I personally think that testers who do not understand development cannot perform well)

 

-- Update --{

// After this article is published, some articles on the Internet have been discussed. I will update them here.
[Do We Need full-time QA? "@ Duan Nian-Duan Wentao]
[About "do we need dedicated QA?" @ Jacky Guo]
[Do We Need dedicated QA? (Evaluation) "@ monkey comment Comment comment]
[Do We Need dedicated QA? Read it and read it @ peanut color Magic Uncle]

}

 

My story

Let me talk about my worst QA experience. The QA department of this company only performs tests. Their leader thinks that all the test design and test processes do not require Dev participation, they are independent of Dev departments. They almost don't care about Dev design and implementation. They only care about running test cases they designed. However, when executing test case, Dev Support is required, especially in environment settings, test tool usage, and whether it is a bug, all of which consume Dev resources, most importantly, they are not responsible for any online problems, and Dev will work overtime to solve any problems.

When I review their test cases without permission, I found that many test cases write-"expected result: make sure every thing is fine", WTF, what is "every thing is fine "?! In test case design, what is test environment/configuration? Where is test data? Test Case, test data, and test configuration have no version control, and many test cases are designed to be very redundant (multiple test cases only test one function ), if you do not know how to analyze the function point, do test design. In addition, I don't know why they are so keen to design a bunch of negative test cases, but many positive test cases are not covered. Why? Because they do not know the details of development and design, there is no way to design a valid test case. They can only create Black Boxes Based on requirements and surface.

When testing the performance, Dev needs to teach you how to test the performance, how to find the system performance limit, how to test the system latency, and how to observe the system load (CPU, memory, network bandwidth, disk and Nic I/O, Memory Page feed ......) How to perform soak test, how to observe the resource usage of each thread, how to simulate various network errors by configuring a network switch, and so on.

The test is not very serious. A large number of false alarm instances are environmental problems, such as the failure to restart the service after the new version is installed, the use of new configuration files, network configurations, and so on.

One week before the project was about to go online, I checked their test result without permission. I saw the soak test memory usage keep rising for five days, and the memory leakage is obvious, this happened two months ago, but I never reported it. I had to work overtime with my programmers every day until the early morning, and solved the problem before going online. However, the QA Team still works normally as if nothing had happened. Ah ......

Why? I think there are several reasons (the same as Yan Xin's point of view)

1. All QA testing powers are given, but no corresponding responsibilities are given,
2. QA has never experienced the pain of software quality problems (the stress of solving online problems), so QA will not take the initiative to think about and improve.
3. QA has no knowledge of the dev development process and technology, and adds a lot of communication between QA and Dev.
4. QA does not understand the Design and Implementation points of software projects, resulting in many ineffective tests.

 

Note: I did not mean to devalue QA's ability to work here. I only saw some real problems that QA was not involved in development.

My point of view

Yan Xin provides two solutions to problems arising from division of labor:

• Full authorization and Trust (empower team members)
• Perform their respective duties and be jointly responsible for the project (establish clear accountability and shared responsibility)

My point is, Theoretically correct, the operation is too false. This is like the slogan "serving the people" shouted by our country. There is no specific method and it cannot be implemented at all.

 

I didn't mean to belittle QA's work here, and I didn't mean to go to another extreme because of this. However, there are still many emerging companies in my current company's experience,I feel more and more that software development really does not need full-time QA, and does not need to write code and do not know how to do the test.. The points are as follows:

1) Testing by developers is more effective

• Developers have to test their own software. If developers do not understand the test or are not professional in the test, they are not professional developers.
• Developers understand the entire software design and development process, and developers are the most clear about how to test, including unit testing, function testing, performance testing, regression testing, and soak test.
• Developers know how to test is the most effective. Developers know all the function points, and what tests are required for regression and verification after a bug is fixed. Developers know how to better test their technical capabilities.

Many developers only like writing code, do not like testing, or they say that developers should focus on development rather than testing. This idea is quite wrong. Developers should pay attention to the quality of software and prove the quality of their development achievements.If the developer does not know how to perform the test, the developer is an unqualified developer..

In addition,I still don't understand why QA without development is more professional than Dev in testing? This cannot be said at all..

 

2) reduce communication, wrangling, and pushing

Are you familiar with the following situations?

• QA always requires Dev to review and check the test plan, test case design, and test results.
• In the process of testing, QA always requires Dev to provide guidance on the testing environment, configuration, and process.
• QA will always argue with Dev about whether a problem is a bug or not.
• No matter what problems are found, Dev always solves them, and QA never fixes them.
• We can always hear that, when an online problem occurs, Dev complained that QA and other problems were not tested,
• QA will always complain that the dev code is too bad and does not know how to test it. If it is not tested, it will give hand over to QA.
• QA always pushes Dev. If this bug is not fixed, you will affect my progress.
• And so on.

If there is no QA, there will be no such thing as Dev's own problems, and it's nothing to worry about.

On the one hand, QA says Dev doesn't understand testing, and Dev says QA doesn't understand technology, but we need to isolate them from each other, this is not conducive to flattening the generation gap between Dev and QA.To make Dev understand QA, let QA understand Dev, and reduce public reasoning, we can only communicate on our own standpoint. There is only one way, that is, let Dev do the test and let QA do the development.. In this way, everyone is a programmer.

 

3) Eat your own dog food

Really good development teams should eat their own dog food. The meaning of this sentence is --If you don't know what you're doing and what you're suffering, you don't have the motivation to improve..Without suffering, you will not really think. Without real thinking, there will be no real progress..

In my current company, programmers have to do almost everything from requirement analysis, design, coding, integration, testing, deployment, O & M, oncall, from start to end, because:

• Only by understanding the difficulty of testing can you understand how to write testable software and how to perform testing automation and testing systems.
• Only by operating and maintaining your own systems can you know how to write logs, perform monitoring, and perform statistics in your programs ......
• Only by using your own system can you understand your feedback, users' ideas, and users' needs.

So,Real engineers can truly understand software development, not just coding, but also the entire software engineering. I only understand or only like coding. It's just coders, not engineers.

 

4) Other problems

• About SDET. The full name is software development engineer on test. Positions such as Microsoft, Google, and Amazon are available. However, I don't know the ratio of such jobs to Microsoft and Google, and there are very few such jobs in Amazon. Is there a dedicated test that understands development like this? My answer is yes! However, I was thinking,If a person understands development, why only allow him to perform tests? Is the Division of Labor reasonable for such programmers? Does it make sense to divide programmers into two other citizens? How many developers are willing to only perform test development?Therefore, in actual operations, SDET is mostly for developers who are not familiar with development. In other words, it is difficult for developers to perform tests.

• If you say Dev is not professional in testing, not careful, not seriousSo we cannot guarantee that QA is professional, careful, and conscientious. Problems that may occur on Dev will also occur in QA. If the problem arises, QA will not work overtime, but developers should solve it by themselves. Therefore, if QA does not need to solve the problem, how can qa be really careful and serious?

• If you don't want QA, Dev is not enough.. If you change all the existing QA in your team to Dev, then you can develop, test, and communicate with each other easily, will you think this will be more effective? Did you find that Dev can help QA on major issues, but QA cannot help Dev.

• Third-party neutral, you will say that people are always unable to test what they write, because there is a mindset. Yes, I agree. But what if it is Dev cross-test? You may say that developers will have their mindset. This only means that the developers are not mature yet and they are not qualified yet. It doesn't matter. As long as you eat your own dog food and suffer, you will be responsible.

• Sharpen your skills. If you develop something that you are using, you are your own natural qa. If other teams are using the modules you developed, other teams will naturally help you with the test, and it is the most authentic test.

• You may say that eating dog food is a joke, because if it is me, after I break it down, I quit and let others eat my dog food.. This will indeed happen in reality, and it is also very realistic. But think about it. Why did you let him do it in the first place? In addition, if your team does not take good care of the design review and code review, so that someone can take down the work, the problem caused by the departure of this person will be solved by this team, as a result, the whole team is eating their own dog food, which is fair. Once again, when your team does what it will do next time, it will not dare to drag people out of the box, it will not dare to review the code at will, it will not let people just do one thing. In the end, we did not escape the dog food category.

• About System Integration testing.The so-called integration test means to integrate the modules developed by multiple development teams for testing. Because developers may not be able to see the overall situation, do not know the systems of other teams, and the pace is different, they need to have a dedicated QA with overall management to conduct overall planning and testing. I have no objection to this. In actual operation, it seems that it is more effective to use full-time QA for integrated testing to uniformly schedule various teams. However, this still does not allow me to stop thinking about two issues. 1) if a developer cannot see the global picture, can he develop better software? 2) isn't the full-time QA for integration testing composed of the backbone Dev of each team? 3) isn't the unified scheduling task more like what the project manager is doing?

• About Automated Testing. Automation means that this is a mechanical repetitive task. I want the tester to think about whether you are doing this? If you are doing something like this, you have to think about your value. However, all repetitive mechanical work will be replaced by machines one day.

• About Online Testing. We all know that, no matter what we do in our internal test, there will always be something that cannot be tested on the user's side. Therefore, some companies generate a UAT for user acceptance testing. The company that makes the product is called beta testing. In any case, you always need to go to the production line for real testing. For Internet enterprises, some are playing a/B testing on the production line, and some are playing some user testing. For example, only 10% of new online users can access the product, this will not affect all users due to problems. Developers must be responsible for such testing systems.

Well, I have written so much for the time being. I will repeat your discussions to add my point of view.

 

-- Update 2012/4/11 --

Some people think that I am venting my anger. I can understand why I am misunderstood, but it doesn't matter. Many new things and ideas will always be misunderstood. Aside from these emotional factors, let's simply think about whether the team architecture without full-time QA has positive significance in it?

Let's add another point. Let's think about it. QA ensures quality. But many QA engineers are doing tests. Is software quality tested? If you do not control the requirement analysis, software design, and code implementation, how can you ensure the quality during testing?

(Full text)

(For reprinted articles on this site, please indicate the author and the source coolshell.cn. Do not use this article for any commercial purposes)