Let LoadRunner go down the altar (all)

LoadRunner is undoubtedly a powerful stress testing tool. Its scripts can be recorded and automatically associated. test scenarios can be metric-oriented and multi-party monitoring. Test results can be displayed in charts and split into combinations. I believe someone may think like this: taking a performance indicator standard list and comparing it with the test data, like the pH test paper, blue in case of alkali, red in case of acid, clear at a glance, then I can shout out: I found the performance bottleneck of the software system!
However, no matter how many powerful and intelligent adjectives we add before LoadRunner, we should not forget that the final modifier is just a noun "tool ". The westward journey also has a brilliant conclusion: officers and soldiers? A maximum of officers and soldiers with hemorrhoids! Comparing LoadRunner to the officers and soldiers who have developed hemorrhoids is a bit vulgar, but LoadRunner is a tool. Whether the performance bottleneck can be identified depends on the people who use the tool, rather than the tool itself. To perform a successful performance test, it is not enough to understand and master the LoadRunner user manual. You also need to understand all aspects of the tested software system, such as the software architecture, network topology and other knowledge. This is like a skilled carpenter. It is not because he is familiar with the chisel, the manual of the hammer, but because he can combine the texture and size of the wood, use the tools such as chisel and hammer to make a delicate chair. In performance testing, where is the intelligence activity of people reflected?
1. First, performance testing is also a kind of testing, which means that testing cases should also be written for performance testing. Whether your performance tests are sufficient to identify performance testing bottlenecks is important to your initial design of test cases. I have written no less than 10 performance test cases for a software system. When I run it later, I found that I had to add several cases to find the bottleneck, in fact, more than a dozen cases have been repeated. If you want to write a good non-repeated performance test case, you have to have a certain understanding of the tested software system.
Here, by the way, there is always a debate in the testing field that testers do not need to understand programming or have development experience, forget the purpose of the tester. The purpose of the test is to write a good test case. A good test case is to discover a software bug that was not found before. Then, whether a tester's knowledge system is sufficient is to write a good test case. For different types of tests, the depth of knowledge is different, some do not require programming knowledge, such as interface testing; some must have development experience, such as interface testing, cannot be generalized.
For performance testing, I personally think that testers do not have to have development experience, but should have a comprehensive understanding of the software architecture. Why? When performing a performance test, if you apply pressure from the client once, the performance indicators will be met. If this happens, you will be happy, because in your indicator scenario, there will be no performance bottlenecks in the software system, you don't have to bother looking for it. However, in most cases, when performing performance tests, we are not able to achieve performance indicators. The server response may have timed out, or the user may have died, A bunch of errors are thrown in the log, which is very common. Therefore, when designing a performance test scheme, we should consider designing test cases to find performance bottlenecks. In this case, we need to know which nodes exist in the entire software system and where bottlenecks may exist, for example, a B/S system has a pressure stream string such as IE client → physical network → Web Server → app server → dB. Each module of each node may become a bottleneck. The bottleneck may be caused by the module configuration or the module itself. We need to design test cases to discover this. Generally, one of the methods that I often use is to design a set of performance test cases to verify whether a node has a bottleneck. In this case, try to keep other nodes unchanged, to change the configuration of the node and monitor various metrics of the node. There are a lot of things to be said here, not to be clear in a single word. You can find a topic to discuss with you later.
Ii. Generate a script using the VU of LoadRunner. There are two script generation methods: Self-writing or embedded source code, and recording generation. It is often said that the recording script is preferred in the two modes because it is simple and intelligent. But I personally think it is better to write scripts because:
1. good readability, clear process, clear meaning of checkpoint interception. Business-level code is easier to read and maintain than protocol-level code. A script library can be created if necessary. Most of the recorded Code has no maintenance value.
2. the handwritten program can simulate application running more realistically than the recorded script. Because the recorded script intercepts the Network Package, generates protocol-level code, and omitted the processing logic of the client. For example, if you use VU to record an IE behavior that runs scripts and applets, only HTTP APIs are generated. The Applet and script running in IE are not simulated, this is not a complete system.
3. the handwriting program is more technical than the recording script. Why not improve both development and testing capabilities? LoadRunner provides Java User, VB user, C user, and other language-type scripts, which are used for development scripts rather than recording.
Whether recorded or handwritten, the script should be selected based on the actual effectiveness of the script simulation program, considering the project progress, development difficulty, and other factors.
Here I want to say that developing a good script is necessary for a successful performance test. A good script should be able to reproduce client program behavior to the maximum extent. As in the above example, the script only simulates part of the client's behavior. If some of them are not simulated, the bottleneck of the client may be ignored. I once suffered a loss and wrote a Java Socket script to connect to the server. However, I ignored the business logic on the client interface and passed the performance test using my script. All of them are OK, however, when a real User goes online, the client terminal interface receives a large amount of server information, leading to the death of the client process. Yes.
3. Set up and execute performance testing scenarios.
From the successful running of VU to the successful running of the Controller, the following steps are generally required (I also saw it on the English forum, I think it is good to share it ):
1. Confirm Susi in Vu (single user single cycle times single user & single iteration)
2. Confirm SUMI in Vu (single user multi-cycle times single user & multi iteration)
3. Confirm the Musi in the Controller (multi user & single iteration)
4. Confirm mumi in the Controller (multi-user multi-cycle times multi user & multi iteration)
It makes sense to divide such a step. The first step is to verify that the script is written correctly, and the second step can verify that the data pool is operating properly. The third step is to verify the concurrency function. The fourth step is to verify the performance of the software system. Some of the questions raised by some friends on the Forum are described here. If a problem occurs during the runtime scenario in the controller, you must first ensure that the runtime is successful in vu. This is a clear logic. In software engineering, we need to develop a proccess for all the actions of software development, that is, the process, and performance testing. We can debug scripts and scenarios based on the process to detect and locate problems as soon as possible. Unless you are a master and familiar with your mind, you can surpass proccess without any problems.
The scenario is to organize virtual users and transaction numbers according to certain rules to simulate real-world business behavior. This actually copies and connects the behavior of a single user. The organization of a scenario usually has a lot to do with the business rules of the real world.
The following may not be an appropriate metaphor:
Scripts are like actors. scenes are like the stage of performances. The test engineer is the director. The Director determines how many actors can perform performances on the stage, and the story cannot be outrageous, out of reality, or else it will be ruined. Note that the director is not only responsible for ensuring the smooth completion of the performance, but also to observe and collect feedback from the audience to confirm whether the performance is successful.
Similarly, the performance tester not only checks whether the scenario is successfully executed, but also collects feedback from various servers to check whether the performance of the software system is normal.
In the real world, it is necessary to analyze and calculate the conversion of user business rules to operational performance indicators. Of course, this is usually done by market demand analysts, But I think testers should understand these indicators during performance tests and understand why they should do so. Sometimes some performance indicators are unclear and accurate, and need to be analyzed by testers. For example, a performance indicator requires the software system to support login behavior of 700 users per minute. This is an inaccurate performance requirement for testers. This means that 700 of the instantaneous concurrent users log on to the system at a one-minute response time requirement? Or can 700 users log on to the software system in one minute? These two scenarios have different pressure on the software system. The first is obviously large, and the second is smaller. Even some performance requirements are to support 50000 registered users. This requirement needs to be analyzed and some business probability algorithm models need to be introduced. This is no longer the responsibility of the performance tester. However, because many software companies do not have standardized procedures or fail to execute the procedures, these tasks must be done by the tester, this is exactly the opportunity to exercise.
4. analyze the result data to find the performance bottleneck of the Software System
As mentioned above, we have done so much to find the performance bottleneck of the software system in this step.
I personally think that finding a performance bottleneck is a very challenging task. Chairman Mao once said that a good CAPTCHA is able to develop a combat plan based on the existing objective situation, and then in the course of combat, locate the inconsistency between the plan and the execution, analyze the plan, and adjust the combat plan to reduce the deviation between the plan and the battle potential. It is a process of combining theory with practice, a process of combining subjective thoughts with objective reality (Note: I am a loyal fans of the old family of Chairman Mao ).
In the performance test, the test plan is our combat plan, and the execution of the performance test is our battle battlefield. Unexpected problems may be found in performance tests. Of course, an experienced performance testing expert may be able to fully consider before the test, so that the test scheme matches the actual situation of the test execution, and find out the performance bottleneck. I am not an expert at sunshinelius. Of course, I have to deal with and analyze problems in performance tests in a rush, and may modify the test scheme, add necessary test cases, and delete useless test cases. In short, performance testing is a process of constantly modifying the test scheme and executing test case repeatedly until the performance bottleneck is approaching. In this process, you need to know a lot of knowledge. The more you know, the closer you get to the truth about the running of the software system, and the more you can find out the performance bottleneck.
For example, if LoadRunner calls a programmer's program, the occupation of server resources may be a clue to trace the bottleneck, but if it is a standard middleware, it is not that simple, for example, if an Oracle database parameter is set incorrectly, it will cause a database bottleneck. If the application calls the database API incorrectly, for example, if no soft resolution variable is used, it may also lead to a database bottleneck. None of these bottlenecks are reflected in indicators such as CPU and memory usage.
In this case, you must have a certain understanding of the middleware, know which parameters are useful, and how to adjust them. In addition, you may also use the proprietary monitoring tool of middleware for analysis. LR performance counters are not enough.
In my personal experience, it is difficult to find the bottleneck, from easy to difficult
Server hardware bottleneck> network bottleneck> application bottleneck> Server Operating System Bottleneck (parameter configuration)> middleware bottleneck (parameter configuration, database, web server, etc)
I had a deep memory. I used lR for a two-day performance test, but I couldn't go up the indicators. Later I used the graphical performance detection tool of the Oracle database to find that the query processing time for a table was too long, the original index was not properly created. After the index of the table was changed, the performance was slightly improved, but it was still difficult to go up. Check again and find that there is a problem with writing an SQL statement in the application, which is long and time-consuming. After the change, the test indicators will still fail.
After at least four rounds of troubleshooting, it was done. The last bottleneck to be removed was the operating system problem (I didn't think of it at first. Then I checked the system message and found that an error was reported)
5. Let me talk about it again.
Performance testing for each system is a new challenge!
From: http://bbs.51testing.com/thread-6154-1-1.html