Today, I saw a Google speech on test on infoq, which is wonderful.

Http://www.cnblogs.com/lixiong/archive/2010/04/10/1709242.html

 

Today, I saw a Google speech on test on infoq, which is wonderful.
Http://www.infoq.com/cn/presentations/duannian-agile-test

The speaker Duan Nian, also known as guanhe, introduced the following online:
Http://www.qconbeijing.com/Speaker.aspx? Id = 33
Personal website on
Http://www.cnblogs.com/guanhe/

 

I did not expect the domestic community to find such valuable and meaningful things. It seems that it is time to include infoq in my Google Reader list.

 

Many methods and practices mentioned in my speech are actually exactly what I met at work every day. of course, I am still exploring. I didn't expect this 30-minute speech to perfectly summarize many aspects. Its value is equivalent to the contribution of the periodic table of elements to chemistry.

 

The lecture contains two new things for me.

 

The first is "development and testing ". the author did not go into depth. Every time I mentioned development and testing, I simply mentioned that it is independent from UNIT TESTING AND requires testing by developers. in the traditional method, the pressure on developers is more from the progress. therefore, it is very important to promote developers' enthusiasm for code quality and the enthusiasm for product quality after final integration. development and testing is a very important tool to achieve this goal, but the difficulty lies in how to define and execute. is it to allow developers to manually perform pre-designed tasks, so that developers can design and execute a set of automation? Or simply share the work of some testers? I personally think that the development and testing culture is more important than the process and results, while culture is not a simple process,
We hope that the author will be able to introduce in depth how to promote such a culture in the development team.

 

The second is diff sign off. the background of this problem is that all regression tests (also called full test pass) need to be run once in the execution of the large sign off process. Is there any simple method. the author mentioned diff sign off here. that is to say, the important regression test is to run, but we can discard some regression tests based on product changes, especially code changes. for example, if the previous milestone ran the performance test regression, all the modifications in the next month were made for localization, for example, localization, and the same was true after analysis by the Code comparison tool,
Then the tester does not need to run all the performance regression tests. during our testing, we didn't have a specific name, but sometimes we used the same method for analysis. I personally think this is a very efficient and worthy of promotion method, but the actual situation is that no one explicitly proposed this operation. I think it is mainly because our Organization is not agile enough to get used to the vote, so it takes more time to completely return to the end to be safer. the secondary reason is that automatic coverage is indeed comprehensive (of course, the quality of different teams is high and low), so it seems that there is no need for it. there is only one test in 10 Google development projects, which is more challenging than we do now. Therefore, diff sign off is naturally useful.

 

For diff sign off, a similar example of last week is that we upgraded the tool to vs2010. however, the previous milestone release was vs2008. we will discuss whether to leave a vs2008 environment, so we should guard against hotfix. I mean to compile hotfix with vs2010. but the boss means that, in this case, it is not enough to release hotfix light for smoke, and it must be completely regression, because the compiler has changed. I am thinking that the probability of problems caused by the compiler for standard code written by C # is very low. I think the security of the compiler is greater than that of smoke. therefore, diff
The promotion of sign off depends largely on the existing process and experience.

 

In addition, the lecture also contains things that I think should be included but not included.

 

The author emphasizes the historical changes of test roles and mentions the objectives of testing in the new era. however, the tests discussed in this article are more inclined to the reality of after the fact. it emphasizes unit testing, development testing, and other things that require attention from developers, but all this happens after encoding. I think agile test and the entire software test should focus on how to put bugs before coding. BJ generally calls this "Fix the bug before the fact ".

 

This is a lot to talk about. I personally have some of the most effective methods.

 

The first is to allow testers to participate early, at least not later than the developers. with the awareness of "Fix the bug before the fact", no matter how early the tester was involved, there was something to do.

 

The second is to do "psychical test" (I used the term psychical debug to test my thinking ). in general, spec and design should be tested from the very beginning, instead of waiting for the code to come out. early participation of testers is the basis for achieving this. for example, when detailed requirements come out, read the requirement documents carefully. Based on my experience, I can often find many conflicting places. after the design documents are published, the test of thinking is more important, and the key is to grasp the details. I found that most teams will not go into details too deeply when doing spec review. A design, the overall looks okay,
There is no problem with the module division of labor, so it is basically over. the so-called in-depth details require that all possible possibilities, cases, data flows, performance, scalability, and so on be taken as far as possible, regardless of the priority. if there is any problem, immediately write it down and confirm it with the developer. if it is difficult to pass the thinking test, you can immediately write a prototype to prove it. compared with the code, eliminating bugs at this stage requires less than 1/10 overhead. therefore, do not waste the spec/design review time.

 

The third is in-depth code implementation. to put it bluntly, we need to test the product code as well as Dev. I admit that this is highly controversial. It may only represent my personal habits. in-depth code implementation is different from white box testing. the purpose of in-depth code is to understand the risks involved in the specific implementation and arrange the test based on this. for example, you can use regular expressions and your own parser to capture keywords. if the former is used, more attention should be paid to the performance.

 

In addition, I think the author's understanding of UI automation is somewhat extreme. the author thinks that UI automation should not exceed 20%. ui automation is unstable, and UI changes at the front end lead to high UI automation maintenance costs. although I have only conducted tests for three years, all of the tests are UI, winform and WPF, not HTML. the reasons mentioned by the author are correct. I have some deep understanding, but it does not mean that there is no solution.

 

First of all, UI automation has mature and stable windows APIs. compared with the implementation of Windows message and screen location a few years ago, the UI automation through UIA is now very stable, and both are implemented in C. the recording tool can be used in the project, but we find that writing code directly is more efficient, but occasionally the code is automatically generated using the tool. the performance and stability of UI automation mainly depend on the implementation quality. in fact, compared with API automation, I think there are up to 20% more UIS. it is simpler and more stable to test the html ui.

 

Secondly, the maintenance of UI automation depends on the quality of the test code. when the UI changes, I believe it is gradual. as long as the UI automation code is properly designed, this kind of progressive maintenance is actually very easy. for example, assume that login.html has changed. The ideal situation is the resource file for updating the automation code. if the logic also needs to change, you may need to modify the code of the login UI proxy, but the interface is stable. no, the login step does not go through the UI now. It is better to simply modify the config to replace it with a non-UI provider and implement http post. therefore, UI automation is not difficult, and the overhead is not large.

 

The value provided by UI automation cannot be replaced by other automation methods. because the UI is actually used by the customer. in addition, many simple UI operations can cover multiple underlying APIs and achieve the maximum coverage. unlike API automation, many bugs are easily exposed only under UI automation. This is mainly because UI operations are very sensitive to timing. almost all of the UI-related products I have made have the problem of immediate operation resulting in a crash when the main interface appears, and these problems are all exposed during Automatic UI testing.

 

Therefore, the UI automation should not be simply marginalized, but find the correct solution.