Let LoadRunner go down the altar again.

1. The LoadRunner hinderedPerformance TestingPeople's understanding of the communication process
I hope that the person who does the performance test can forget the tool. We all know that Vugen has a recording function, in fact, recording this function is a very bad choice for testing, is to follow the scene implementation brings a lot of uncertainty. Because some people do not understand the script, or do not know what the meaning of the script, resulting in some performance problems, helpless. And I don't know how to find out why. So I think the performance Tester handwriting script is the best choice, but isn't the recording feature available? Of course not, but if the record, you must know the meaning of the functions of the script, to thoroughly understand what these functions want to accomplish what function, what can be achieved, what cannot be achieved. In order to know how to solve some problems when they occur. And the process of solving the problem depends onotherKnowledge, not the LR, find help, you can solve it. So rely on tools to have a degree, or do performance testing is not credible.
We all know that LR supports a lot of network protocols, and that the specific network protocols that derive their own dedicated language, this should be its greatest advantage, but LR is not the language that it declares the support of the very good. We know that in the 8.0 version, LR already has the Java AJAX protocol, but if you do not modify the configuration file is not displayed. That's a 9.5 version, which has already been published, but it feels like a lot of problems with this protocol. Similarly, there are other agreements. Tools, and performance tests, are definitely two different things, so don't rely too much on LR. We all know that.MercuryThe concept of BTO is presented, so many of the conceptual designs in LR are explained from the business perspective. Testing professionals, to understand its concepts and our specific environment of the relationship between, or can only copy the condom. So it can be said that the focus of LR is the degree of mastery of the Protocol, not necessarily all, but to be particularly proficient in some of their own testing closely related. In fact, many of our testers are not very familiar with the above-mentioned AJAX architecture, so even if the performance test is "stop in the appearance, not the other", then wasted resources.
Again, LR'sDatabaseThe support. All along, we know that in LR it is troublesome to directly face the database test (relative to the HTTP protocol). A few days ago, looked at the other tools, some tools and database communication made the corresponding module, operation, very simple, need to write less code. This function is better than what is done in LR. But we also have to understand that LR is a product under the BTO concept. It is important to note that the development of a lot of classes will be used to map data patterns for the corresponding table operations (such as Hibernate), so that in the performance test need to pay special attention to the application server process settings, there may be such a scenario, the database server is not pressure, the application server has been "killed" in advance.

2. LoadRunner simplifies the performance testing process
As you can see from the Mercury performance test execution flow, it is divided into the following sections: Plan tests, create scripts, create scenes, execute scenes, analyze results. This method can be said to represent the main part of the performance testing process. But this process can also result in confusion. First of all, the performance test needs research, and the need for research is very thin, probably in most of the company did not do so, but this does not mean that the research part can be ignored. We often run into performance tests, and we're talking about performance testing requirements. This is not an uncomfortable thing for us to say. There is also modeling, LR's methodology and execution flow are not mentioned in the modeling process, and in the actual application environment, we still have to consider this process, only this process is the premise of the scene setup. It should be said that modeling is not done in LR, but it should be mentioned in the execution flow and methodology. In a complete performance testing process, there are a lot of management constraints, of course, this can not rely on tools. We'll talk about the related issues later.
In the LR market is so strong, I think it should only be considered as a tool, rather than a universal product that can complete the performance test throughout the process. I met a lot of novice performance tests and performance tests that have been done for a while, often asked questions like: I will use LR, I can do the performance test? Our company has LR, can we promote the performance test? Or more seriously, with LR, the performance test has a sense of security. It's not imperial sword, we can chop anyone we take. In fact, know how to cut or how to cut is the most important, bamboo is also a sharp weapon.
I've seen this happen several times, and the client thinks it's time for two days or a day to get a performance test done because you're using a tool and it automatically generates results. Often, a person who is very, very familiar with the tool, is not very clear about the process and the results of the analysis. and write a report can only describe the phenomenon of the surface. Such a performance test can only say that there is a warning effect on the actual system quality improvement is not very significant. A colleague who began to write a program in the 80 's said: "This kind of performance testing method, the quality of the system is not a guarantee, but the customer." "and system-level performance testing has not been able to change anything from the overall situation, can only look at the shortcomings of the system, but also do not have any constructive guidance on the tuning." The design of LR is primarily about system-level performance testing, so we use it to understand what it can bring to us. Maybe someone is going to be in a dead alley, I'll use it for finer code and integrated performance testing. Can not say completely can not, with sledgehammer kill Chicken also line, is waving overweight, do not kill chicken not good but add shoulder periarthritis.

3. LoadRunner's monitoring weaknesses
A few days ago I was asked about LR's monitoring of database and application servers, and I've been advising them to do it with other tools. Because LR monitoring is only a shell, and personally think, is not a very high efficiency of the shell. For Oracle's monitoring, we all know that LR after the connection of Oracle, there will be two tables to choose, there are many counters, for performance test engineers, choose what counters, is very difficult, because do not know what these counters mean. So for the most part, we see some people recommending that they choose the default ones (some instructions in Help). It would be lucky if we had a problem with the counter that we chose. But this phenomenon is as unstable as walking on the road and picking up 100 dollars. So we have to go through the scene repeatedly to determine the bottleneck of the problem. This is a very painful thing for us in performance testing. Because some scenarios can be performed for hours, days, or even a week. Monitoring WebLogic is also the same, I would definitely use weblogicscripting tool to write script monitoring, nor is it recommended to use LR monitoring function. Even if the exclusion of the compatibility between the tools does not say, I think that LR does the monitoring than the performance test process to achieve a lot less efficiency. Of course, some business tools have done quite well, and resource usage is acceptable. The same is true for UNIX monitoring, and if we do it with LR, we may not know when the RPC will collapse. We had to start all over again, which was deadly damage to our long-time scenario execution. Using LR to monitor Unix, try not to use it. Using Nmon or UNIX-brought performance monitoring tools can be, what top ah glance ah some of us all, but at the end of the reminder, they are also required to run the application of host resources.
Because many people like to see all the data in one result, in order to ensure the synchronization of data, so spare no effort in the LR to complete such a monitoring function. However, no matter how much data in the results, the analysis of the results to rely on their own sharp sense of smell and rich experience, and the implementation of LR in the monitoring point in fact, the efficiency is not high, so I recommend that the least resources to do the most things. Don't rely too much on a tool. In my current job, doing a complete performance test will use a lot of tools to combine the results of these tools and analyze them together. And some features have a much stronger real-time viewing capability than LR. Plus, the results of LR directly generated by the report, nor can it be used as our final report, so let all the data in the parser practice, only have a meaningless sense of personal integrity, the actual results are not significant.

4. LR is a front-end tool
(The front-end tool here is not referring to the presentation of the page, but to emphasize the location of the LR tool in an application)
In general, the problem that can be manifested in LR has been thrown onto LR after a series of processing in the system, so we'll discuss some of the error codes of LR and the string of red strings that have been printed. It is also necessary to further analyze the entire communication process of the application in order to find the final problem essence. For example, some programmers like to put primitiveSQLStatements are written directly into the code, millions of lines where can you find them? Performance comes out is very difficult to see, LR will definitely tell you the machine IO throughput, and then subdivide the reason for nothing, and so it is not as good as writing Performance Analyzer yourself. Just like the one you just gave a friend yesterday. When you put a different error inside and outside the for loop, if you just look at LR, you can only describe the problem, and you can't find the cause of the problem.
In the process of LR scenario execution, all the data obtained by default is obtained from the tool itself, which means that the data cannot directly reflect the performance of the server. When we analyze this data, we must give the corresponding server-side data to prove that our data is meaningful. Say separately how much TPS has, how much response time, how high throughput, no sense. Because this data is a prerequisite, and these premises, LR can not provide.
Of course, all the pressure-generating tools are also front-end tools. Take it out alone. The reason for this is that performance testing is not a front-end tool that can be done. It can only be a bearer of the phenomenon of the tool. The real thing to do is the analysis behind it.

5. About Analysis and tuning
In analyzing this piece, analysis or give a lot of useful tools, of course, these tools on the data processing, there are certain reasons and laws. We also need to understand this, in order to do a very good analysis of the results. Take a simple example of why the response time provided in LR is the maximum, minimum, average, standard variance, 90% values in summary? We need to understand these reasons, and then do a detailed analysis, so that we can perfect the front-end data analysis. However, none of these analyses can be the key data in the report, as there is also a need to analyze all aspects of a system to write a suggested part of the report. We do performance tests, not to say, what might be the cause of the problem, and I often see such reports. Such a report, if it was written to me, I must have called back to rewrite. We face the customer, there will be such a problem, a while ago just done a project, another colleague wrote the report, let me change from beginning to end, because the original report is only a list of data.
In contrast, LR is the best in the list of data. However, LR will not tell you what the results of your test are, and of course you can set up SLAs, but it is not a good idea, since this setting is only a comparison of SLA and test results to make the report look nice when you know your goal. Of course, when analyzing data including DB, APP, middleware and so on, we also need to look at the corresponding monitoring results. Analysis is a process of clues, must not break, broken can only describe the process, and not get the results. Above said monitoring, I stressed that with some manufacturers of their own monitoring functions, then there will be very good results. We can't let LR tell us top 5 of SQL, but Statspack can tell us. Other monitoring tools are similar. So if you want to do a good job of analyzing, or using better monitoring tools, here's what it means: LR is not the best monitoring tool in many ways.
Tuning, likewise, LR is definitely not tuned because it can't even analyze it. Only by ourselves following the vine and touching the melon, we will find a way to take it off. If this process with LR monitoring function, it is easy to break, so tuning rely on these data, will be more tired.

6. Software Performance Engineering
A few days ago, when I modified my performance partition name in the 7-point test Forum, I couldn't remember to take a proper name for this large partition of performance testing. Finally, we finally locate a word: Software performance Engineering.
In each software performance test project, this is a project, the project will have a role, division of responsibilities, time schedule control, risk control, and so on content. And these are not a tool can be represented, glimpse, never know what it looks like. We can't go around with LR and say, I can do performance tests. In the whole software performance engineering, we need to do more management and control work, these are the focus of our performance testing. A performance test engineer with an LR recording script on it, I think it's definitely not a good performance test engineer. But everyone has to take the tool to record the script to pressure this process to learn seriously. Also need to carefully think and understand every operation and step, in order to improve the understanding of software performance Engineering. Perhaps one day in the performance testing section to join some real system experts, it is really a blessing in the field of performance testing.
Software performance Engineering, is a perfect concept of the whole, need to have bigger picture understanding. To focus on performance from the point of need to the final product delivery phase (which does not include operations), it is also clear to know the location of the entire software performance engineering that the performance test occupies. From frame design, module design, database design, coding are to pay attention to performance, often found performance bottlenecks, due to the overall system design problems, can not be changed, so performance can not be tuned, can only play a role in evaluation. In such a time, even if the performance tester ability is strong, also can not turn out what tricks.
In addition, software performance engineering is a need to cooperate, not a performance test engineer with tools can do, the above has said that the project to have a role division, responsibility division, then our software performance engineering needs what? Software Performance Engineering requires:Operating SystemPersonnel, database personnel, network personnel, middleware personnel, application server personnel and so on, the process, where there is a problem, it is necessary for the corresponding people to check the cause and solve the problem. And some companies and customers think that one or a few of the tools of performance test engineers can complete a complete performance engineering, I can only say that these people are all-rounder (such people should be valuable). Application test knowledge and technical means, under the multi-role coordination, makes the software performance engineering implementation is more operable, a software system performance can be more secure.

In this article, the relationship between performance tests and loadrunner that I understand is recorded.

Reprint: http://www.51testing.com/html/48/202848-130049.html

Let LoadRunner go down the altar again.