Experience in functional and performance testing

It can be said that whoever has mastered the essence of functional testing and performance testing can occupy the technical commanding heights in the testing outsourcing market. This article provides suggestions for such software service-oriented enterprises and reference for testing technical decision-making.
Although functional testing cannot be avoided by the vast majority of software, most development companies do not understand it. Therefore, the testing outsourcing market is so prosperous and growing. Currently, functional testing has crossed the process of manually tapping the keyboard and clicking the mouse, and is moving towards automation and intelligence. Automation means that various testing tools have been widely used. intelligence means that testers need to have a deep understanding in script preparation, operation, debugging, result analysis, and test solution improvement.
The importance of performance testing has evolved with the development of network applications. Due to the complexity and diversity of the network environment, database environment, application server environment, system platform and technology, software performance is very difficult to control. Although improving system performance is not accomplished by performance testing alone, performance testing is still a very effective way to control performance, it plays an important role in Software Capability Verification, capability planning, performance tuning, and defect repair.
Selection of functional testing tools
Then, how can we test functions efficiently? Selecting an appropriate functional testing tool and training a team of high-quality tools is undoubtedly crucial. Although there are a few software service enterprises engaged in Function Testing outsourcing projects that do not use any functional testing tools at this stage. In the short term, the profitability of such enterprises is acceptable, but in the long term, they are likely to be replaced by software service enterprises with a high degree of automation.
Currently, many tools and software are used for functional testing, and tools for software of different architectures are constantly updated. Here we will focus on a typical automated testing tool, winrunner of mercury.
Winrunner is a method used to test applications. Program Can enterprise-level software functional testing tools run on schedule. By automatically capturing, detecting, and simulating user interaction operations, winrunner can identify the vast majority of software functional defects, this ensures that applications that span Multiple Functional points and databases do not encounter any functional faults during release.
Winrunner has the following features: compared with traditional manual tests, winrunner can perform Function Point Tests quickly and in batches. It can perform the same actions for the same test script, this eliminates understanding errors caused by manual testing. In addition, it can repeat the same action, and the most boring part of the test can be done by machines. It supports Program-style testing scripts, A high-quality test engineer can use it to perform extremely complex tests. By using wildcards, macros, condition statements, and cyclic statements, the test script can be reused; it targets most Programming Language It provides a good integration and support environment with Windows technology, which brings great convenience for functional testing of applications based on Windows platform.
The workflow of winrunner can be roughly divided into the following six steps:
1. Identify the application GUI
In winrunner, we can use GUI spy to identify various GUI objects. After identification, winrunner will store them in the GUI map file. It provides two GUI map file modes: Global GUI map file and GUI map file per test. The biggest difference is that the latter generates a GUI file for each test script, which can be automatically created, stored, and loaded. This mode is recommended for beginners. However, this mode is not easy to change the description object, and its efficiency is relatively low. Therefore, it is a better choice for an experienced tester, it generates only one shared GUI file, which makes the test script easier to maintain and more efficient.
2. Create a test script
When creating a test script, you can record the script first, and then manually add the required TSL (similar to the C language) to the recorded script ). The recording script has two modes: context sensitive and antive. The selection is based on whether to simulate the mouse track. analog is generally used for playback. During the recording process, the two modes can be switched through the F2 key.
You only need to look at the scale of modern software and the number of functional points to understand that functional testing has already crossed the stage of manual typing and mouse clicking. Performance testing is an effective means to control system performance. It plays an important role in Software Capability Verification, capability planning, performance tuning, and defect repair.
3. debug the test script)
There is a special debug toolbar in winrunner for debugging the test script. You can use step, pause, and breakpoint to control and track test scripts and view variable values.
4. Execute the test script in the new application
When a new version of the application is released, we will test various features of the application, including new features. At this time, of course, it is impossible to record and write all the test scripts again. We can use existing scripts to run these test scripts in batches to test whether the old Function Points work normally. You can use a call command to load test scripts. You can also add various TSL scripts in the call command to increase the batch capability.
5. Analyze test results
The analysis of the test results is the most important throughout the test process. Through analysis, you can find various functional defects of the application. After a test script is run, a test report is generated. From this test report, we can find the functional defects of the application and see the differences between the actual results and the expected results, and various dialogs generated during the test.
6. Return defect (defect)
After the test report is analyzed, various defects of the application should be returned according to the test process, and the defects should be sent to the designated person for modification and maintenance.
Three steps of Performance Testing
The first step of preparation and organization is the first step in the performance testing process. At this stage, it is necessary to clarify the performance testing objectives and requirements, and organize appropriate personnel to develop a performance testing plan.
In general, performance testing applications are divided into four areas: Capability Verification, capacity planning, performance tuning, and defect repair. Capacity verification indicates that the purpose of the test is to verify whether the system capability meets the expected performance standards. Capacity planning is to evaluate the scalability of the system. performance optimization is to find the performance bottleneck of the system, it provides a basis for performance optimization. defect repair aims to identify defects in the system, such as concurrency.
The goal is to classify performance testing into the corresponding application fields, while determining the requirements is to determine the benchmark for Performance Testing in more detail. The source of performance testing requirements for products is the software requirements, design documents, user memos, and other design and demand-related documents. Of course, not all performance testing requirements are identified in these documents in a clear way. At this time, we need to further refine the description according to the not very clear document description. Our experience is to pay close attention to all performance-related descriptions during document reviews, for example, "The operation response time is required to be less ......" , "Requirement ...... Fast ......" , "Requirement ...... Support ...... User Access Requirements ...... It can run quickly and stably, and further refine it to serve as the basis for testing.
Performance testing involves many devices, environments, technologies, and tools. The organization of performance testers must also consider these aspects. A performance test group should preferably include System Engineers (responsible for building test environments, configuring servers and application servers) and network engineers (responsible for maintaining and verifying the network environment) performance analysis engineers (responsible for drafting test plans, analyzing performance test results, and providing performance test reports) and automation Engineers (responsible for writing test scripts and implementing test tools) and database Engineers (responsible for locating performance issues at the database layer ). Where conditions permit, it can also include development engineers and customer representatives to assist in the analysis and validation of performance test results.
The performance test plan is the main document used to guide the performance test process. In the test plan, in addition to specifying the testing objectives and requirements of this test, A clear test exit condition and test time and resource plan need to be provided in the test plan.
The design of the second step is also the main content of performance testing. The test design is generally based on the test scenario. A test scenario is the section of a user's actual system.
In the performance testing process, it is very important to clarify the number of participants, proportions, and specific behaviors in each scenario. These are the basis of the performance testing script. Based on experience, you can analyze user behavior from application server logs. For example, for an OA system, we analyze the logs from AM to AM ~ There are 200 page views on the mail page within the period of, and the viewing time is basically concentrated in the first 10 minutes ~ During the period, the number of views on the bug display page is 300, and the average access to the page is allocated throughout the period, we can create two scripts, the previous script simulates the mail operation (Script 1), the other script simulates the bug operation (Script 2), and considers a 15-minute test scenario, you only need to run script 1 in the first five minutes, run script 2 throughout the process, and adjust think time to bring the page view to the actual value.
Of course, not every different user application profile needs to be designed as a test scenario. In most cases, you can analyze the probability, importance, and risks of a test scenario, finally, determine the test scenario to be designed. After the scenario is clarified, different performance testing methods can be used to achieve the performance testing goal based on different performance testing application fields. In addition, it should be noted that the performance test design should also include the design of the test environment and test data, because there are many factors that affect the system performance, it is very important to maintain the controllability of the environment and data during the testing process.
Step 3 Performance Test Result Analysis is the most difficult and important step in the performance test process. It requires analysts to have an accurate understanding of the data in the test results and clarify the relationship between indicators. If there is no obvious relationship between various data indicators, in most cases, we need to consider a variety of factors to reach the final conclusion.
Based on experience, the most common problems in the performance testing process are database access layer problems, application server configuration problems, and network problems. Therefore, we recommend that you eliminate network problems first, analyze application server configurations, and then perform performance analysis at the database access layer according to the principle of "simplicity to complexity, the focus is on the analysis of indexes, database cache, deadlocks, and other issues. When we confirm that none of these factors are performance bottlenecks Code Analyze and check to find out the cause of performance problems.
You only need to look at the scale of modern software and the number of functional points to understand that functional testing has already crossed the stage of manual typing and mouse clicking. Performance testing is an effective means to control system performance. It plays an important role in Software Capability Verification, capability planning, performance tuning, and defect repair.