Web services are at the core of distributed computing, and the interaction between them is usually difficult to test. Distributed development, large developer teams, andCodeIncreasingly componentized expectations may make Web service development more and more easy to hide errors. These types of errors are extremely difficult to detect. Stress testing is an effective way to detect such code errors, but it can only be used when the stress system is designed to be effective. This article will give you an in-depth understanding of the basic requirements of this pressure system. Test Method Traditional testing methods include some simple formsUnit TestUsually executed by developers. Designing these tests requires an internal understanding of the software, and these tests are almost always targeted at very small and specific parts of the product. These types of tests are suitable for simple Web services that have little or no interaction with other code components. Function verification (Functional Verification)It is also a testing process.Source codeUnderstand limited designers to perform tests to confirm the core features of products or services. This test is designed to prove that this core function complies with a certain specification. For example, does my online auction show the correct bidding price? Has my insurance broker found the cheapest offer? If these tests fail, it usually means that a basic problem of the product is detected (this problem can be fixed directly ). This test is also suitable for simple web services, so that you can check whether the service can correctly execute its functions. System Test)It is usually completed in the function verification phase, after verifying the core functions. It tends to take the entire system as a whole to find out the problem-how the Web Service operates as a part of the system, and how the Web Service interacts with each other. Because system testing is performed at the end of the development lifecycle, it is usually not possible to allocate enough time for completion. In addition, due to the tight release schedule and the backward movement of all important stages of development, the system test phase is often ignored, and some common and rare errors cannot be detected. Even if such errors are discovered, it is too late to determine the cause of the errors and try to fix them. Therefore, when looking for code errors, you must design the system test application as efficiently as possible. System Testing usually consists of three parts:
- Performance ):This involves the process of determining relevant product statistics. For example, how many messages are there per second? How many users can a service accept at the same time?
- Case (scenario ):This is the exact configuration process required to recreate the customer. Therefore, any problems found in the case can be detected before the customer uses the product.
- Stress (or workload balancing ):It is different from the other two parts because it is designed to overload the software by applying a large workload. If stress testing is performed effectively by maintaining a high intensity of use of the product (but not exceeding the limits determined by performance statistics), it will often find many hidden errors, these errors cannot be found using any other technology mentioned above (these errors are often the most difficult to fix ).
In terms of Checking code errors, it can be proved that the most efficient of the three system test components isStress Testing. However, this process is often confused with other elements or functional tests of the system, so the methods involved in this process have not been correctly processed or implemented. Errors under pressure With stress testing, you may find many errors that are more difficult to find using other testing methods. There are two error types:
- Memory leakage (Memory Leak ):It is a very difficult phenomenon to detect. Memory leaks often occur in released products for simple reasons. It is difficult to design test cases to detect them. Using a simple function test, we can hardly find the memory leakage problem, because we did not use the product much before the product was completed. Memory leakage usually requires that the Operation repeat a large number of times to make the memory consumption more noticeable. HoweverProgramming Language(Such as C/C ++) compared to JavaProgramIt is more difficult to introduce Memory leakage errors, but as long as the program still keeps referencing the object, the object may still be instantiated and the memory occupied by it will never be released.
- Concurrency and synchronization ):Stress Testing is outstanding in finding concurrency, because it applies many different code paths and timing conditions in any test lifecycle. The general rule is that the longer the stress test runs, the more code path combinations and timing conditions are involved and applied. Of course, this makes it difficult to reproduce these problems (errors can occur in five minutes or five days ). Deadlocks, thread leaks, and any common synchronization problems can only be detected during stress testing. These types of problems are hard to be identified by executing unit tests. Developers will not always consider that their or her code will interact with code elsewhere (the Code may not have been written during unit testing.
Existing stress testing tools Many tools that claim to be able to perform stress tests on the product are currently under development. Tools widely used for Web Services. However, many of these tools are simple HTML/soap generators that simulate many client connections and therefore generate high loads on the Web server (this is useful for finding problems with the Web server, but it is not so useful for searching for Web Services ). These tools are useful for basic stress tests, but they are often used to repeatedly execute the same functional task only in the extended function verification phase. If enough time and resources are available, you can create a custom stress testing system to perform more effective tests. Because the stress system designers generally have a better understanding of the products and web services to be tested, they will be able to ensure the specific code areas that the stress system can use. Design Pressure Application When designing a stress testing system that tries to conduct stress tests on web services, let them run code in a specific way. These styles go beyond functional verification to find out if the tested web service can not only do what we think it can do, in addition, it continues to run normally when some high-intensity pressure is applied. Stress Testing requires four basic conditions for Web service applications. Many established stress systems apply these conditions. An effective stress testing system will apply the following key conditions:
-
- Repetition ):Or the most obvious and understandable stress condition is the repetition of the test. In other words, the repetition of the test is to execute an operation or function over and over again, for example, repeatedly calling a web service. The function verification test can be used to determine whether an operation can be executed normally. The stress test determines whether an operation can be executed normally and whether the operation can continue to run normally at each execution. This is important to infer whether a product is applicable to a certain production situation. Customers usually reuse the product, so stress testing should discover code errors before the customer. Many of the simplest stress systems only implement this condition, but repeated expansion function verification tests cannot constitute an effective stress test. When combined with the following principles, you can repeat to discover many hidden code errors.
- Concurrency (concurrency ):Concurrent sending is a behavior that executes multiple operations at the same time. In other words, multiple tests are executed at the same time. For example, many Web services are called simultaneously on the same server. This principle does not necessarily apply to all products (such as stateless services), but most software has a certain concurrent behavior or multi-thread behavior element, this can be measured only by executing multiple code examples. Function Testing or unit testing will hardly work with any concurrent design. The stress system must go beyond functional testing and traverse Multiple Code paths at the same time. How to achieve this depends on the specific product. For example, a Web service stress test needs to simulate multiple clients at a time. Web services (or any multi-threaded code) usually access some shared data between multiple thread instances. The added complexity due to additional programming usually means that the Code has many errors caused by concurrency. Because the introduced concurrency means that the code in one thread may be interrupted by the code in other threads, the error is only in a specific sequence in one instruction set (for example, with a specific timing condition) it will only be detected during execution. You can apply a lot of code paths by combining the original and repeated principles.AndTiming conditions.
-
- Magnitude (magnmagnitude ):The stress system should be applied to another condition of the product considering the load volume in each operation. You can perform one operation at a time in the stress test, but the operation itself should also increase the burden on the product as much as possible. For example, a Web service allows a client to input a message. You can simulate a client that inputs a long message to enable this separate operation for high-intensity use. In other words, you increase the magnitude of this operation. This magnitude is always specific to the application, but you can determine it by finding the value of the product that can be measured and modified by the user-for example, data size, delay length, transfer of funds, input speed, input changes, and so on. Individual high-intensity operations may not discover code errors (or functional defects), but when combined with other stress principles, you can increase the chance of discovering problems.
- Random Variation:Finally, any pressure system is less random. If you randomly use the countless variations described in the preceding stress principles, you can apply many different code paths during each test run. The following is a few examples of how to change the test in the test life cycle. When repeated operations are used, you can change the time interval between repeated operations, the number of repeated times, or change the order of Web Services before restarting or reconnecting to the service. With concurrency, you can change the number of Web services that run together, the number of Web services that run at the same time, or the decision about whether to run many different services or run many identical instances. Magnitude may be the easiest to change-variables that appear in the application can be changed each time you repeat the test (for example, sending messages of various sizes or numeric input values ). If the test is completely random, it is difficult to reproduce errors under pressure in a consistent manner, so some systems use random changes based on a fixed Random Seed. In this way, with the same seed, the chance to reproduce errors is even greater.
A stress test typically combines all of the above principles and runs as long as possible within the permitted range. The longer the test execution time is allowed, the more code paths can be traversed and more errors can be found. Of course, once an error is found, it must be diagnosed and repaired. A code error can be displayed after multiple days of stress testing, therefore, the system must ensure that all available debugging information is generated when an error occurs-otherwise, it may take the same amount of time to reproduce the error. Conclusion Testing is a vital part of software development, and stress testing is an important and often misinterpreted or ignored part. Following the principles described above, you can design and implement an effective stress testing system to find hidden problems related to your code. Whether using a pre-written tool or creating a fully dedicated stress system, stress testing is an essential method for finding problems with web services (or any other program, and ultimately improve the quality of your software products. References
- Read the Article published by Alan booth and Andrew Citron on developerworks , "stress testing your software without stressing your testers ". (March February 2001)
- you can find some resource links for Software testers at the software testing institute.
- software testing and quality engineering magazine has many useful articles.
- you can also find many useful links in the software testing and resource center.
|