Exploratory software Testing

Recently found in the first half of last year read a book on the test summary notes, has been placed on my personal USB flash drive, at that time was recorded with Xmind, now re-organize to share to everyone!

James A.whittaker [Beauty] James Whitek (absolute master in software testing) exploratory software testing, Chinese name "Exploratory software test", I remember being fascinated by the book (I don't know how many of my little friends who did the test read the book)! Feeling is a necessary test aspect of the book, instantly feel the charm of testing! Needless to say, direct to dry, hope can give the exploratory test like or interested in small partners a little help!

Testers must answer the following questions when testing.
(1) is the performance of the software running in line with the design expectations?
(2) The user purchased the software for a feature, but did the software implement this feature?
(3) When the software is running, is it fast enough, safe enough, stable enough, and so on?

In many ways, the goal of testing is to do the best in this test, while ensuring that you can do better next time. Just the exploratory test method can provide good guidance.

There are two main guidance methods that can help testers make specific decisions. A method called local exploratory testing, which assists testers in making immediate decisions during testing. The other is called the global Exploratory test, which helps testers design overall test plans and test strategies.
If you combine exploratory testing with manual testing of both substance and script, you can get a third exploratory test, the hybrid exploration test technique. Here are three parts to illustrate:

First, the local Exploratory test method (Exploratoy testing in the small)

This method assists the tester in making immediate decisions during the testing process. testers do not need to know a lot of information to complete certain tests, such as determining the input value of the text box, how to interpret the error message, and how to understand the relationship between the previous input values.

Solve a lot of details of testers, it is not suitable for building a complete test architecture, nor should it be used for the entire design process of test cases.

The following five areas are of major concern:

1. User input input refers to an environment-generated stimulus that causes the application being tested to respond.
Tests include legal input and illegal input, input filters, input checks, exception handling code, default input or user-supplied input, and input selection using output guidance.

2, State software A state is a point in the state space, it is determined by the value of all internal data structures.

The application interacts with its running environment and all incoming input causes the software state to change. Testers are the conditions that test these state changes. Test that it correctly updates its current state? Is it entering a state that it should not enter?
The relationship between input and state is critical, and the following methods are recommended in the local exploratory testing method:
1) Use status information to help find relevant inputs
2) Use status information to identify important input sequences

3, code path A code path is a series of code statements, it starts at the beginning of the software to run the statement, terminates in a specific statement, is often the one that represents the end of the software run statement.

Testers must know exactly what branches are in the program and understand which inputs will cause the software to go this way and not another.

4. User data If you expect the software to access a huge amount of data storage, such as a database or a large number of user files, you need to set up a data store in the test environment. And the data should be as consistent as possible with the data used by the real users of the software.

5, the running environment is the operating system and its current configuration, also includes running on the same operating system and the tested software to interact with some other applications, including indirect or direct impact on the tested software itself or affect the test software run any driver, code, files, settings, etc. It also includes the current network conditions of the software connection, the bandwidth available to the network, performance, and so on.

Second, the Global Exploratory test method (exploratory testing in the large)

This method is used to help testers design the overall test plan and test strategy. A variety of decisions that help testers have to make globally, such as when considering feature interactions, data flow, and how to select different paths in the application's user interface to accomplish some real-world functionality. Instead of examining atomic inputs that serve as intermediate uses on a single input Panel, discuss those inputs that can help test for more important purposes. We actually need to make a global goal at the beginning to guide the subsequent testing process. Using the global exploratory test method, the decision made only affects a single dialog box or a single user interface, which is scoped to the overall software. Rather than determining how to test a single function, it determines the overall direction of how the software is explored.

The main reference is the roaming test, which provides a framework for testing, which helps testers create user scenarios that are more interesting and targeted than using free-to-play methods. It also sets a goal for testers to lead them to try more interesting and complex use paths. Roaming tests can help testers think about how to test applications and help them organize actual tests. Of course, this series of tests can be compiled into a test checklist, which prevents testers from missing a certain type of test, or helps testers match the functionality of the application with the testing techniques appropriate for those functions.

In addition, these tests can help testers make countless decisions about how to determine test paths, how to choose input values, and which parameters to use for testing. In countless decisions, compare and choose different test methods, the essence of the experience. This is the real test guide.

According to the roaming test to the different areas of the software region type, the following for the corresponding test one by one description of different areas:

1. The business district test type focuses on testing the product's selling point characteristics and instructs testers on how to test the software code path for these features. The following test methods are available:

1) The guidelines test method requires testers to perform operations by reading the user manual and strictly following the recommendations of the manual.

2) selling point test testers should look at the sales demos, watch the sales video, and visit the customer with the sales staff. Follow the steps of the product demo yourself to perform it again and see if there is a problem.

3) Landmark test method The testers identified the key software softness in advance, which is the landmark, after selecting the landmarks, you need to determine their order, then proceed from one landmark to another to explore the application until all the landmarks in the list have been accessed.

4) The method used by the testers of the limit test is to ask the software many questions which are difficult to answer. For example, how to maximize the software? Which feature will make the software run to its design limit?

5) Express test method testers focus on data. You should confirm those stored input data and "follow" them around the software.

6) Late-night Test testers are concerned about the operation of code outside the main features, such as various maintenance tasks, should be done automatically by the program. The variant has an early morning test to test the software's startup process and scripts.

7) Traverse test Method The tester accesses all objects contained by the target by selecting a target and then using the shortest path that can be found.

2, the historical area test type mainly for the old function and defect repair code.

1) The bad neighbor test method with the depth of testing, the discovery and reporting of more and more defects, the number of defects can be linked to the product characteristics, so as to track where the product defects occur.

2) Museum test method those old code either accepts the re-modification, or is not changed to put in the new environment to run, it is easy to fail. Therefore, you should also test legacy code and old executable files.

3) Previous test method if the current product construct is an update to a previous version, it is important to run all the scenarios and test cases that were supported on the previous version. You can verify that the functionality that the user is already familiar with is still viable on the new product. If a new version is re-implemented or removed, testers should choose a new version definition method to enter data and use the software. Double-check the test cases that are no longer available in the new version to ensure that the product does not omit the required functionality.

3. Entertainment area test Type This kind of test helps testers test those auxiliary features, not the mainline features, and ensures that the two features can be combined in a practical and meaningful way.

1) supporting test methods encourage testers to focus on certain features that are not primarily used by the user, but will appear on the display together with the main features, and the closer they are to the main function, the more likely they are to be noticed, so attention must be paid to them.

2) Shenxiang test method if the test department tracks the coverage of the code, the test requires testers to find ways to test the code that has not yet been detected. Hybrid test method is to try to mix the most popular and least popular features together.

3) Overnight test testers will keep the program running without shutting it down. That is, the continuous use of certain features or the file remains in the open state.

4, tourist area test type is concerned with the various functions of fast access software.

1) The Collector test method recommends testing the output of the collection software, collecting the more the better. You should ensure that you can observe any output that the software can generate. For example, for a text processor, make sure it can print, spell check, format text, and so on.

2) The long path test is to test the feature as far away from the application start point as possible. For example, which feature needs to be clicked N times to be used? Select that feature, click the past, and then test it. The main idea here is to try to navigate the application as much as possible before reaching the destination.

3) The focus of the supermodel test method is not the real interaction between the function or the test function, but the test interface. Watch the elements on the interface. For example, have they been mapped correctly? Is the performance good? What is the refresh situation when changing the interface? are buttons and controls in the graphical interface compatible with the off-the-line values?

4) test one-off test the case of running multiple copies of the same application simultaneously. That is, run an application at test time, then run another copy of the program, and then run a copy.

5) Scottish Pub test method is suitable for large-scale complex applications. For some parts of the program, testers need to know beforehand how to see them. You can find groups of users and participate in their discussions, read industry blogs, spend a lot of time in-depth understanding of the applications under test.

5. Hotel Area test Type testers let go of the main and most popular features, and test some of the secondary and accessibility features that are often overlooked or are less described in the test plan.

1) Cancel the test method is to start the operation and then stop it. For example, print the file and cancel printing before the file is printed. You can do the same for any feature that provides the option to deselect or that takes a long time to complete. At a minimum, you should be confident that the canceled operation can be executed again and successfully completed.

2) Lazy test method testers do as little practical work as possible, which means that the software accepts all default values ... Focus on whether the software handles default values and whether to run code that handles blank input.

6, dilapidated area test type

1) Destructive Testing methods testers try to exploit every possible opportunity to sabotage the application. A. Forcing the software to do something B. Master the resources that the software must use to successfully complete the operation C. Remove those resources from different programs or restrict the use of those resources. For example, add or delete files, change file permissions, disconnect the network cable, run other applications in the background, and put the application you want to test in a problematic machine.

2) The rebel test requires the input of the most unlikely data, or a known malicious input. If the real user enters the letter A, then the rebel tester will never enter a, instead of looking for meaningless input values.

There are three ways to achieve rebel behaviour:
A. The reverse test method inputs the most unlikely data each time in order to test the application's error handling capabilities.
B. Rogue test method Enter some data that should not be present. Testers who violate input in a test cause a lot of error messages, entering data that breaks the limit
C. The mis-order test requires testers to do things in the wrong order.

3) Obsessive-compulsive test method forces testers to enter the same data over and over again, and to perform the same operation in a reverse manner. For example, enter some data on the screen, and then go back and enter it again to see if the developer has written an error handler.

Three, hybrid exploration testing technology

Exploratory testing can be combined with scenario testing to help testers introduce a large and small variety of changes for a given scenario. If the scenario describes a specific action that the user wants to take, the following techniques can be used to change the order of these actions and derive a derived scene from the scene for testing different states and different code paths. This technique uses scenario-based exploratory testing primarily, and this form of exploratory testing uses a method that injects changes to the scene. A scenario can evolve many test cases by systematically considering input selection, data usage, and environmental conditions. Two main techniques are used: scene operation and roaming test.

1, through scene operation to introduce changes in order to enable testers to consider the replacement path in a more systematic and strategic way, the introduction of scene operations, is to manipulate the scene steps to inject changes to the scene.

1) The insertion step adds additional steps to the scene to make them more diverse, thus testing more functionality.

Can be used to send these types
A. Add more data
B. Using additional inputs
C. Access to the new interface
Note that these steps ultimately require the tester to return to the original scene. Our aim is to strengthen the scene rather than to radically change the basic objectives of the scene.

2) Remove the steps we can last year with redundant and optional steps, the idea of this operation is to make the scene's steps as minimal as possible. It may be that the resulting scenario lacks the steps to set initial conditions that can be used to test whether an application can recognize the presence of missing information or lack of subordinate functionality.

3) Replacement Step if some steps in the scene can be done in several ways, you can modify the scene with the scene action of the replacement step. is actually a combination of the previous two operations, which is to delete the step before inserting the step. Therefore, testers must study alternative methods to perform each step or action in the scene.

4) Repeat step scenarios often contain very clear sequence of actions. The scene action for repeating steps changes this order by repeating individual steps or repeating a set of steps to create additional changes. By repeating and rearranging the steps, we can test the new code path and discover the pitfalls that might be associated with data initialization.

5) Replace the data to understand the data sources that the application connects to and use, and ensure that their previous interactions are stable and reliable. For example, you can test the current scene by reading, modifying, or manipulating the data to replace the default database. For example, what happens if the number of existing records in the data source increases by 10 times times?

6) The basic point of the replacement environment is that the test scenario itself does not change, but the system that is used is changed when the scenario is executed on the software.

Factors to consider:
A. Replace hardware changes the hardware that is running on the measured application. You can take advantage of virtual machine technology to accomplish this task.
B. Replacing the container requires ensuring that the test scenario can run in all major containers that the user is likely to use, such as a different browser.
C. How does the replacement version of the application run on the old version?
D. Modify the local settings test does the application use cookies or write files on the user's machine? Does it use the local registry? What happens if a user modifies browser settings to restrict such activities? What happens if the user changes the application's registry settings directly? As a tester, it's a good idea to know how to handle the situation before the application is published.

2. The introduction of the change scene through the roaming test focuses on small, incremental changes and unnecessary steps in the scene, while roaming can actually create a fairly long and wide range of derivative scenarios.

1) The selling point test simulates the user's existing work habits, adding some new features to the original scene, allowing the user to use the process to learn a feature, master it, and gradually move on to new features as they become familiar with the application.

2) Landmark test method from the scene and select a specific feature from the scene of the landmark, and then randomly disrupt the order of these landmarks, so that the resulting scene is different from the original scene. If necessary, repeat the process and run the tests in the new landmark sequence.

3) Limit test method Check and modify the scene to make the software work harder, that is, challenge the software, ask it difficult questions. Does a long string input cause problems?

4) The Shenxiang test method focuses on the most unlikely or least-used features.

5) The OCD test method repeats each step in the scene two or three times. For example, moving data around the software has historically been an effective way to find important flaws.

6) Overnight test method when the test scene can be automated or can be recorded playback, the most suitable is the overnight test method, only need to repeatedly run the scene without the need to exit the application under test.

7) The destructive test method destroys activity at the resource call when the scene is being tested. If the scene requires data to be transferred over the network, you can unplug the cable before or during the procedure.

8) The Collector tests the execution scenario and the derived scene with documentation of each observed output.

9) The Super Mode test method only focuses on the interface when running the scene. Make sure that all elements are in the place where they should be, and that the interface should be well designed, with particular attention to usability issues.

10) Supporting test method testers do not perform the function described by the test script, but instead find the nearest neighbor function to perform.

11) The Cancel test not only takes full advantage of the Cancel button (click when the scene is running), but also performs the start and stop functions. If you start to perform certain functions, cancel by canceling or ESC.

12) The mixed-ticket method jumps from one scene to another, combining two or more scenes into a scene with mixed purpose. Check all the scenarios and find out the generic data, focusing on common functionality.

Static scene testing and exploratory (roaming) testing do not conflict. Scenarios can represent a great starting point for exploratory testing, and exploring can add valuable changes to the scene, otherwise the scenario will be limited. A sensible tester will combine the two approaches to better cover the application and get more input sequences, code paths, and changes in data usage.

Summary: Those who have studied exploratory software testing will definitely find that this is a kind of can give you more testing ideas, you can find more software defects, so as to ensure the correctness of the software, stability and so on. Of course, because of the many ways of thinking, it is not to look at one or two can be realized, need to test people to read and try in practice the method, not to use all, only part of it, I would like to be sure to bring a great feeling. As the master said, software testing is an essential but difficult thing, hope that all the testers have a heavy task!

Exploratory software Testing