Turn: How to Improve the test coverage rate of Test Case Design

Speaking TestUse CasesDesign, I think every test engineer who has been tested will think it is very simple, not that: design according to requirements or outline, get the software function division diagram, and then according to each function, use Case design using equivalence classification, critical value, and cause/effect. But in fact, the design of the test data is not discussed. Just for the test items, we find that experienced testers for the same project, there will always be more test considerations when writing test cases or tests, so as to discover more problems. However, some testers write test cases with only three axes, on the surface, it seems that all the tests on the page have been taken into account. In fact, a large number of test coverage points are missing, which leads to the vulnerability of the tested program. The reason is that I think the test case writing level is not in place, more specifically, the test case coverage is too low. To be honest, I think System TestIt is very difficult for use cases to achieve 100% coverage. It is hard to say that by dividing functions in the design, every function is written into this use case to cover the whole? This is far from enough. Because we know that there are still a lot of internal processing, transformation, business logic, and relationships that affect each other, which are not explicitly required or designed. On the one hand, we need to rely on testers to understand the project itself, and on the other hand, we need to rely on the experience of testers to find and test these hidden points one by one, so as to truly ensure our test coverage. Therefore, this article introduces the test data design method from the perspective of test coverage. Case DesignHow can we take into account more comprehensive, how can we find hidden test items one by one, so that our test is more comprehensive and complete. Although the ideas are good, after all, each test project is different. Our experiences for different projects will also tell us different ideas. These ideas are usually very emotional, it is difficult to use strict logic theory to sublimate it. Therefore, the content of this article is still very simple and immature. I just hope that this article will be used as a brick for everyone to think about and supplement and improve it together, so that our test case design level will continue to improve. Body I. Test Case Design1. functional aspect 2, specific aspect 3, hidden aspect (1), Background function (2), and test the complete business process (3), in a specific situation of the system operation (4), other related systems (5), Function TestingOther Test types Ii. Detailed case design1. Functional surface case design (1), functional testing (2), combined operation testing (3), Gui testing (4) test data initialization (5), whether business requirements are met correctly 2. Test Case Design for implicit function aspect test items: (1) test data integrity (2), background special processing (3), Association and transformation between functional businesses (4), mining and testing from design implementation (5) testing during concurrent operations 3. Specific Section case design 4. Implicit section case design (1), unbounded background functions (2), and business flow-related tests (3) other Test types Iii. Design of test data I. Test Case DesignThe so-called test plane design is actually the division of the test cases. The typical method for dividing test cases is the waterfall model, which is gradually subdivided from top to bottom. Large modules include small modules and small modules include smaller modules. However, this alone is not enough. We need to cut the system into one piece from more perspectives for testing, so as to ensure the integrity of the test items. 1. functional aspect this is the most common aspect. We usually think that a button on the page is a functional aspect. Then, we can compile the use case based on the complexity of the function, by each function, by one function point into multiple pages, or by combining multiple function points into one page. 2. In addition to specific aspect, there is also a specific aspect division method, which is often used in case writing. The specific aspect is to ignore the functional points on the surface and focus on a certain aspect of the test object. For example, our internal management system provides functions such as sales entry import and registration entry import, which correspond to seven or eight functional points in menu division. However, there is a common processing item in the background for these functions to process the Authorization Record generation. In this case, we can separate the "Authorization Record generation" and create a test item, the use cases involved in other test items do not need to be written one by one. In addition, similar operation cases on some interfaces are written into one page separately, which is also a specific aspect. Therefore, if the use case is divided by function points by vertical division, the specific section is the section obtained from the horizontal analysis. By dividing common functions and designing specific sections based on actual conditions, we can make our use cases more readable, understandable, and operational. 3. Use Cases such as implicit cut are the easiest to ignore. It is often not an obvious function item. It may be implicit processing of function items in the background, or association processing between multiple function items, it may even be handled in a specific situation. This requires testers to learn and understand the software for mining. (1) Common background functions such as scheduled and automatic start services and operations automatically executed under certain circumstances. They are often not reflected on the interface, but many of them will be mentioned in the requirement design, and some small functions may be ignored, the tester needs to mine the project based on the understanding of the project. Therefore, a developer familiar with the project and an unfamiliar tester can write the test cases at two levels. (2) We all know that the design of test cases is based on three levels: point, line, and surface. A complete functional item is a line, and a button is a point. Multiple related functions are combined to form a complete business flow as a plane. In reality, such cases are often ignored. In fact, the company's software is originally business-type application software, cut out a variety of functions from the business flow to write separate use cases, there will certainly be a situation involving the overall process. If we do not differentiate them, we will bring together the details and the whole world, which is not only confusing, but also easy to consider. Therefore, in the system test phase, it is recommended that the use case design be divided and combined. For specific functions, only this function should be turned around: in the Business Process test items, consider the use case from the overall business flow perspective. This not only makes it difficult to generate omissions, but also makes it clearer to read and execute the use cases. (3) the design of such cases for system running under certain circumstances is often inseparable from the actual business conditions of the system. For example, financial software usually needs to test the date processing of all related functions at the end of the month, the first day of the month, the end of the year, and the day of the year. For example, a Windows 2000 environment development and testing system, it is necessary to test whether the operating systems such as 98, XP, and 2003 can run freely, and then display the speed of webpages such as a large number of dynamic images and videos at normal network speeds. In short, we should try our best to consider from the perspective of practical application for testing and supplement. (4) other related systems refer to other achievements directly used in the current project, including the company's own system modules, components, and functions, and some functional components purchased or obtained for free. The content should be discussed with the development team lead in advance to determine whether tests are required. If the time is too short or other reasons decide to be unexpected, it should be stated in the test plan. If you want to test the function, you can design it according to the actual situation. You can test a function of the system. In this case, you do not need to separate the test items. If the test item is relatively independent, you can also use a separate test item to test it. (5) other testing types except functional testing include reliability, security, recovery, configuration and installation testing. These testing types are all independent testing items. The so-called good start is half the success. Ensuring that the test items are completely, reasonably, and correctly divided will directly affect the effectiveness of this test. It is generally recommended that this phase take 1-2 days to consider and adjust and supplement the software during the testing process. Do not move the functional model diagram in the analysis design. Ii. Detailed case designAfter dividing the test items, consider the specific test cases for each test item. According to the characteristics of the test item, the design angle of the test case is also different. Next, let's take a look at the common functional test cases from which to design: 1. Functional surface use case design (1). Specific Functional Tests are analyzed and designed based on requirements, design use cases based on various functional items provided on the page and black box testing methods. For example, if the page provides the add, delete, modify, and query functions, I want to verify whether these four functions are correctly implemented. This is the simplest, most basic, and necessary test case. Our Coding staff usually perform self-testing to this extent. Pai_^ (2). the test of combined operations is extended from the previous point of view, which is not tested by the coding staff. Therefore, the testers need to consider more. The so-called combined operation test, that is, to select a few operation items, operate in a certain order, the verification system will not encounter unexpected errors. Of course, it is not necessary or impossible to combine all functional items for testing. So what functions should be combined and what steps should be taken to perform the operation? We still need the tester to design according to the actual situation (so in the IT industry, talent is everything, haha :). In general, we will consider whether the data between function items will be associated. If so, we need to consider this combination. For common query functions, You need to accumulate the conditions one by one to test whether the added data can be changed, whether the added data can be deleted, and whether the added data can be added after deletion. Whether the correct results can be queried between the added data and the added data can be queried; whether the button can be clicked multiple times in a row or not. If there are several operations that require strict order, try to operate them in reverse order, and whether the system can control the operation. Not only is a function extended to multiple Associated Function items, but also has a combined operation test. If the application is complete, report the report. If the application is successful or fails, try again. Of course, this type of use cases can be written to either a functional aspect or a complete business flow aspect, depending on the number of use cases involved. If the relationship is complex, of course, it is better to write it separately. If it is the number of three or five use cases, it will be added directly to the use cases of a function. (3) testing on the GUI is the strength of testers. You do not need to repeat the specific test items, such as the limited length and illegal input. It should be noted that during the test, the actual user's operation habits should be taken into consideration. You need to know that what the UI prototype can determine is the display of the page, and the actual control implementation is still implemented by the coding staff. Even with the coding guide, its scope is very limited. Many coding personnel tend to be less satisfied in terms of user convenience. Therefore, testers must take care of this level. (4) Check whether data that should not be empty is verified during data initialization; whether the default value of data is correct; whether the data generated by referencing other functions will be refreshed in real time; after the page is closed or the system is restarted, data initialization settings are all such cases. (5) Whether the business needs are met correctly is often due to the fact that our requirements are not described in detail, and the requirement of coding personnel is not well understood. As a tester, they naturally need to thoroughly study the needs to control the software implementation. Some common concerns here are: whether the data length and type control are reasonable (for example, the taxpayer identification number can only be a number, but there are letters in the actual business ); business Logic Control is reasonable (for example, a data item does not provide modification, but the data item often needs to be changed in actual business ); whether the provided implementation method is reasonable (for example, the data retrieval function is provided only on a page, but some personnel cannot operate on the page according to the Business Division, but must be able to see the data ); whether the data control is reasonable (for example, you must add data in function a before you can operate in function B, but the opposite operation may occur in the actual business ); whether the data control is complete (for example, the authorization methods include regular monthly subscriptions, buyout, and pay-as-you-go control. When the same enterprise tries to have the above authorization methods at the same time, whether the system can be controlled as necessary ); there are other requirements in operation details (for example, whether the batch operation function is provided for data requiring batch operations, and whether the failed import result file can be modified before direct import ). If the development team lead or demand manager confirms that no modification is made to the unsatisfied requirements, it shall be specified as a software deficiency or limitation in the test report. 2. Design of implicit test items in the function aspect: (1) test data integrity when a data is referenced by other functions; or the current function must reference data from other sources, data Integrity testing is involved. If the referenced data is deleted or the keyword is modified, the referenced data will fail or the data entered through the two channels will conflict or be duplicated; in addition, several related functions are encoded by different persons, resulting in inconsistent controls between them. For example, if data entered by function a is allowed in extreme situations, whether the reference to function B will be abnormal (most common example is that the length of a user name can be 10, but the length of a reference to a list is 8, which will cause an exception ). (2) Special background processing refers to the processing of a function except what you see on the surface. For example, what we see in order input is order storage, however, the background will also include the judgment of duplicate data, the processing of illegal data, the processing of business logic conflicts, and other types of processing specific to the needs. Another example is the backup function. data may be cleared before backup, the backup directory may be cleared, the check of the backup target, and the processing of duplicate backup files. Similar to this, it may not be completely written in the analysis design. It is still necessary for testers to spend more time thinking and mining. (3) Whether data transmission between functions related to association and conversion of function businesses will be affected. For example, a new special character may cause an exception in the query SQL statement. For example, if a download file name contains Chinese characters, garbled characters may occur during the download; when the report is filled in again, the four digits after the decimal point will be ignored when the packet is generated, and so on. This problem is usually caused only by ensuring that the data in the use case can involve various permitted conditions during each function design case, that is, the equivalence class division + boundary value method is fully used to design various "odd" data, and the correctness of the data needs to be verified from the beginning to the end, not just in the current function. (4) Exploring test points from design implementation this is the most difficult bug in our tests. It is often created by the coding staff themselves and not even known by the designers. For example, in an internal management system, the category of a normal product is usually two digits. If it is a module, its category is replaced by the product code. How can we determine that this product is a module? The most safe of course is to check whether the value of the product category field can be found in the product table. A simple method is to directly determine whether the category code is greater than 2 bits or less than or equal to 2 bits. If you know exactly which implementation method is used, you can find the vulnerability. For example, when the length of a product category changes, the system may encounter errors. Even if the implementation method is confirmed not to be changed, the tester should write it into the test report as a limitation ,. Let everyone know that the length of this product category cannot be changed at will. What is depressing is that too many coding personnel handle such implementations as this, which are detailed and concealed and will not be discovered under normal circumstances of system data; in the long way of using software, the original design will be maintained and changed due to reasons such as demand change. This problem will suddenly pop up and make people unable to prevent it. Therefore, to prevent such vulnerabilities, in addition to testing staff to be a groundhog and constantly explore the implementation details of various software functions, the coding staff should also be given the idea of perfect implementation, use complex but highly sensitive code to replace simple but highly dependent code. (5) Whether testing during concurrent operations will cause data confusion when two or more users operate the same function simultaneously. In the C/S structure, this test is usually required if simultaneous operations are possible. In the B/S structure, this problem is usually difficult to solve due to its particularity. Unless a user has used a function and is locked for reuse within a certain period of time, this will also cause inconvenience in actual application. Unless it is a special core data, in general, we will not do this control. Of course, the possible concurrency conflicts will be left over as system restrictions. 3. The specific section of the use case design is actually the use case from another perspective, so the specific case writing method is actually consistent with the functional section. 4. Implicit section design the implicit section is divided into the following situations: (1) Background functions of unbounded sections perform tests on such test items through parameter settings and code calls, however, the specific test design is no different from the general function test. Note that during testing, the front-end and back-end operations are usually performed separately, while the two operations are likely to overlap, during the test, pay more attention to the conflict between the execution of background functions and the execution of some foreground functions? For example, if there is a file shipping service in the background, is it possible that the background will execute file shipping during file generation in the foreground? If possible, pay attention to the problem. (2) the design of such test cases related to business flow needs to design data from a complete business perspective. Theoretically, various data that may occur in various functions should be combined and tested one by one according to the business process. But in fact, we cannot do full coverage. Therefore, when designing such use cases, it is best to have a draft to list all relevant functions one by one according to the business process, and then mark the specific data that may appear in each function one by one, finally, extract the most likely, error-prone, and core data from the graph and combine them into complete business data use cases for testing. In this way, you can find out the most practical and effective test data based on clear ideas. (3) tests of other test types usually have specific methods. If you want to test the reliability, prepare a large amount of data to be continuously executed; if you want to test the security, consider data encryption, data transmission, and data destruction; Recovery generally starts from the network and power supply aspects; for configuration and installation, set up the corresponding environment based on the configurations supported by the system for functional verification. The verification here also requires skills, that is, to test more involved: database read/write, disk file read/write, file upload and download, file encryption and decryption, data statistics, Chart display, printing, and other functions. Iii. Design of test dataEach test idea must eventually be converted into specific data for execution. The methods for designing test data are nothing more than those. Here, I would like to make some mistakes that are often easy to make and put forward some points of attention as a reference for case data design: 1. Try to avoid data that may lead to ambiguous test results: that is to say, the data you designed must only reflect the results you want to test correctly. For example, if a group of test data may obtain result a or result B, then you can use this data alone to test the case with the expected result a, which obviously produces ambiguity. 2. for data that cannot be listed in detail, you must describe its features in detail: Sometimes we save time when designing the use case and do not have to enter a specific value. This is also allowed, but the premise is that you must describe in detail the data features you want to test. For example, if the length of a database field is 20, you can describe it as follows: Try to enter a 21-bit half-width English character; try to enter a 19-bit half-width English character, switch to the Chinese fullwidth and enter a full-width character. Do not write it as follows: Try to enter super-long characters, because this can only be a test scheme. You can write it as a scheme, but it must be specific, clear, and operable in the use case stage. 3. The design of the test data must have a clear purpose: that is, the test data is derived from the test plan. The preceding test scheme is to control the input of ultra-long characters. Therefore, the test data must be input into ultra-long data based on the specific field length, it makes no sense to enter 15-bit long and 16-bit long data. Good test data can be used for multiple test solutions at the same time. In this case, you can specify the test Purpose of the data, because over time, for specific data, you may forget what it is for test. This is very unfavorable for your final summary test and test coverage check. So keep down your ideas and ideas at any time, poor memory. 4. test data can be omitted. The description of test data is based on understanding. Therefore, when writing a use case, when several consecutive use cases are encountered, only a few key data values are changed, and the rest are the same, you do not have to repeatedly describe all data in each use case, after the first use case is complete, only different data is listed in the remaining use cases, and the remaining data is marked as the same as the X use case. In this way, the test data can still be restored during the test, and the purpose of the test case is clear at a glance, increasing the clarity of the test case. To some extent, based on the sequence of test case design, I have gone from the test data aspect design (that is, the test item Division) to the detailed test case design, and then to the test data design, this section describes how to improve the coverage of test cases one by one. Because there are too many specific situations in a specific project, the content described above can only be a glimpse. As for the omissions and errors, I only hope that you can open your ideas after reading them. I will summarize and extract some ideas and ideas from my years of testing experience, and further supplement and improve this document, this allows you to further improve your test case design capabilities.