Massive user-high concurrency SaaS product testing on-line process

Massive user high concurrency SaaS product testing on-line SaaS product testing on-line process-take Web plugin product as Example 1 overview

In Internet products, IT companies pay more attention to the collaboration between product functions, and the products of SaaS form play a more and more important role.

A typical communication process for a SaaS service that is fully hosted by a host agent is as follows:

Such products generally have the following characteristics:

• The provision of specialized services by third parties is generally
• Service is usually provided as a network medium.
• With embedded client functionality
• Third-party service-side capabilities
• Generally not in the form of independent products directly oriented to customers
• Generally need to integrate "parasitic" in the host product to face the customer

The main products of SaaS form are:

• Weather Widget
• Webmaster statistics tools, such as Cnzz
• Forum Plugin
• Online advertising, such as: Baidu ads, Google ads
• Search engine in the station
• Share social plugins, such as: Jiathis
• Mobile app Stats plugin, such as: Friends League
• Mobile App Ad Plugin

As a SaaS service provider, there are advantages and disadvantages to the development and maintenance of such products.

The advantages are as follows :

• The cost of publishing behavior is very low, directly using the network to publish
• Low maintenance cost, unified maintenance on the whole network
• Small, fast iteration releases prevent major problems from being generated
• Direct transparent ground to customers to ensure the rapid access to data and feedback

disadvantages are as follows :

• High server concurrent capacity

  Because this service is provided to the B-end users, B-end user packaging to the C-end users, this amount of growth is very considerable. The server must have high performance requirements

• High server and Client interface stability (with version baggage)

  After the increase in usage, as a SaaS product, there must be an ongoing, fast iteration of the upgrade. Since this product is integrated with the host product, it is equivalent to "Yiyanjichu" , and thousands of users may already be using this interface. Must be aware of the interface compatibility before and after

• Strong client compatibility (with platform baggage)

  Generally for the front-end Web products, IE6/IE7/IE8 is a PC under the historical burden of the past. If the Web product also involves the mobile browser side, then the status of the Android fragmented device, different mainstream models are also a historical burden around the past

As a whole, the main test-on-line process for a complete product with server and client is as follows:

1. Server/Client Individual unit tests
2. Service-side Interface testing
3. Server-side Performance testing
4. End-to-end functional Testing (integration testing)
5. Platform compatibility test
6. All-Grid grayscale testing

2 Measurable architecture

As mentioned in the previous article, a system suitable for testing is bound to be a system with better testability on the architecture. At present, the more typical and mainstream Internet applications of the system basically have a relatively good testability, the typical architecture as shown:

The most typical feature of this architecture is that the effective separation of servers and clients, and now with the diversification of the platform, the client has been but not limited to the following:

• App

  • Android
  • Ios
  • Windows Phone

• Browser

• Client

Note

In order to facilitate the description, the above structure can be referred to as A/S , b/s, c/S , the overall structure diagram can be considered "typical abc/s network product architecture Diagram"

3 Unit Test

Unit tests are included within each subproject and are done by the developer. The main content is the function of the first layer, the developer needs to the main algorithm function input and output verification, so that in the refactoring or code modification, you can make effective regression test, to prevent the new changes affect the old code.

Unit tests are necessary, and the more unit tests are done, the more timely the problem will be discovered. However, in the actual operation, taking into account the ROI, you can only the main core function function of the unit test of the automated code writing, that only 20% of the cost to produce 80% of the return of this function function. Using the 2/8 principle, there is no worry about the slip, because there are other tests that will be effectively supplemented.

Note

The 2/8 principles mentioned above refer to the principle of unit test Automation Code Development, not the principle of unit testing, unit testing is actually done in the development process, to verify the correctness of the development Code, unit testing is actually always in use, but not written in automated code, Regression is poor. For functions that require strong regression, write the automated unit test code as much as possible.

4 Interface Test

Interface testing is strictly a category of white-box testing . In the system architecture above, the server provides the client with interface services, that is, the client initiates a request (HTTP or socket) to the server, and then the server side makes the appropriate action on the request and returns the corresponding stream (typically encoded json/xml string or picture binary stream).

The interface hierarchy is the easiest to automate relative to other levels, and is the highest return on investment (ROI) . Because the UI of the top level to the customer is essentially driven by data from the business layer. The most direct manifestation of the data of the business layer here is the interface of the service side.

In the current mainstream service-side interface (through HTTP communication, JSON as a character codec) as an example, it is easy to use some automation framework, such as: Pyunit, at the data level to simulate the business flow generated by the interface operation, the data of the automated comparison of judgment.

The usual technical means are:

• PYUNIT Automated test Framework organization use cases
• Requests Library Impersonation HTTP request
• Encoding and decoding of the JSON library
• Pyunit's assert function to judge

The specific implementation results are as follows:

Note

Using the visual Python IDE to run the results from the Pyunit program, the left tip if all is green, indicates that all unit test cases in this file pass.

The main content of the interface test:

• Request interface, to determine the input and output content of a single interface
• Simulating business operations to determine the business flow of multi-interface implementations

As the focus of this article is to tell the process, the use of specific tools, follow-up articles will be described in detail, omitted from the table.

Basically, if the test case is sufficient and basically automated, more than 90% of the problems on the server side can be exposed at a very low test cost at this stage.

5 Performance Testing

After the server has passed the interface test, it is not enough to perform a performance test before it is officially released on the production servers. Because the previous interface test just simulates a single client, it's not just a matter of code logic when it comes online, it also involves code performance issues.

The more typical is the 12306 train ticket online purchase website, although in business logic is walk through, no problem. But as long as the holiday, from all over the country to buy tickets extremely dense, but the load capacity of the server did not keep up, the direct result of service collapse. So before going online, we need to conduct performance testing, can draw some specific performance indicators, and then compare with the actual production environment, do some planning work.

The plan includes but is not limited to the following points:

• Web Application Tuning
• Web Server kernel Tuning
• Middleware caching and so on tuning
• Add a Web cluster machine

At this point, the server side of the test work is basically completed.

6 Functional Testing

Not all functions can be tested on the interface. Interface testing only detects the ability of the server-side to provide services at the data level, but the user-oriented program, after all, is human rather than code, so there are issues such as interface display and so on that involve end-user interaction levels that can only be addressed through end to end functional testing to solve the problem.

End-to-end testing is mainly to the interface layer of the detection, due to the interface test, end-to-end functional testing of the workload will be significantly reduced.

In the first end-to-end functional testing, there may be coverage, and interface testing use cases have a lot of repetition, but as time passes and the number of tests increased, the testers will become experienced, the work of functional testing in later iterations will be significantly reduced. Experienced testers will gradually pick out the functional test cases that are equivalent to the interface test, leaving only a few of the functional test cases that the interface test is not complete.

Because end-to-end functional testing is to validate a product or service as a whole, it is a test that simulates a real user scenario and is basically done by hand, so it is the most time-consuming and testing strategy, and the regression is poor, it is generally not recommended to use this strategy extensively. Recommendations can be found and resolved in the unit testing or interface testing phase, as far as possible in those stages of discovery and resolution, do not stay at the user level. User-level functional testing is only for validation, not for discovery.

Note

The functional test described here is not a platform compatibility test, but on the basis of interface testing, the interactive interface layer of functional transparent operation testing, only in the most mainstream peace platform to verify. For example, Web apps typically use chrome as the mainstream authentication platform

7 Compatibility test

The advantage of SaaS products is that there is no need to consider the whole network version of the unified problem, but the shortcomings are obvious, because the audience is the entire network of users, so as far as possible to ensure the compatibility of the mainstream platform.

The current popular application platforms are:

• PC-side Browser

  • Ie6/7/8/9/10/11
  • Chrome
  • Firefox
  • Safari
  • Sogou/Cheetah/360/qq Browser

• Mobile-Browser

  • Uc
  • /Hand Q/qq Browser
  • Chrome
  • FireFox
  • Safari
  • Andoird Native Browser
  • iOS native browser
  • Sogou/Cheetah/360

• PC System

  • Xp/win8/win8 or more
  • Mac OS
  • Linux Desktop

• Mobile platform

  • Android
  • Ios
  • Windows Phone

Testers are required to perform compatibility testing on these platforms as appropriate.

These are just a few different platforms that have not taken into account fragmented devices, such as the Android ecosystem, which is known for its fragmentation , and the expressiveness of different models will vary.

In general, experienced compatibility testers will know some of the platform's problem equivalence to avoid polling-style testing as much as possible.

Share some of the experience of compatibility testing below:

• PC Browser

  • Basically IE6 problems most of the oldest antique
  • IE6/7/8 can be divided into a large class, IE7 and IE8 are basically prone to problems.
  • Chrome on the Test no problem, IE9 and above basically no problem
  • Cache problems need to be noted, to prevent the test environment is not set up a good error detection

• Mobile browser

  • basically chrome is OK, no other browsers are having problems

    Because mobile smart terminals are more likely to appear Late, from the time of birth is based on the WebKit kernel to generate the browser

  • attention to move Mobile browser   cloud acceleration   features

    Such a cache can cause a test environment to be built Error detection

  • Mobile QQ and built-in browser requires a separate focus test

    These two apps use a large number of users, but also for the ordinary browser has made a lot of changes, many of the current IM sharing pages are directly embedded in the app inside

• Mobile side

  • Android and iOS different ways to get the width of webview, there are front-end layout compatibility issues
  • Not all devices can support the H5 perfectly.
  • Android/ios/windowsphone the touch response to the web is a bit different

8 Grayscale Testing (release)

The previous tests were either in the development environment or in the test environment, but the end-service was needed to generate value on-line to the production server. Because of the migration to the production server, the environment changes, and the use of grayscale testing (publishing) has the following benefits:

• Early detection of code environment-level issues and early resolution

  The production environment and the development environment cannot be guaranteed to be fully consistent, so there may be a problem with the code environment principle level . Although it has been agreed to try to keep the development environment and production environment consistency, but obviously also can not be fully consistent (such as the production server generally use Linux-server, but developers may use linux-desktop or Mac-os)

• Identify scale -of-magnitude issues early and address them in advance

  The production-oriented code has issues such as performance and network nodes, and so on scale magnitude. For concurrency, the actual bearing value and the theoretical value may have a gap; user access due to geographical problems, there may be a network is not smooth or cache problems caused by the service is not normal

Gray-scale testing (release) can avoid "non-black and white" this absolute release situation, can effectively avoid the new version of the network after all the user function is not normal use of the situation, to the problem before the expansion, to find and solve the problem.

about how to do a good job of gray-scale testing (release) is also a specific method of strategy, where the first only to explain the process, the method is also described.

Gray-scale testing of the specific implementation of the need for software engineering, specific business processes, application deployment, system architecture must have a considerable understanding to design a better application for their own grayscale process, there are the following considerations:

• Application system has grayscale

  This is about the same as testability, which needs to be adjusted in the results and can be configured to perform A/b separate testing for users at the same time.

• Application system needs to have a certain rollback capability

  Once there is a significant problem in the grayscale process, there should be remedial rollback measures when the drawdown is needed.

The above considerations, are required to have a good technical ability to achieve good.

9 Summary

Different test strategies apply to the non-current test phase:

• unit test

• interface test

  after design, the reference design document is written by the test developer and used after the server production code is built and deployed in the test environment. module testing and troubleshooting for interface level. Basically, most of the problems on the service side are to be found and resolved here, and the latter process is only for the auxiliary verification work.

• performance test

• functional test

  Complete the tuning on the client and server side, build the client and use it at the user level for functional integrity verification

• grayscale test

  Validate the migration of the production environment Test to form a smooth publication

As of here, this article is only the basic testing theory and process of the SaaS product introduction, the follow-up concrete implementation method, after all, the light has a good model is not enough, must have the corresponding advanced methods to practice the ground, in order to produce a huge productivity .

Author:	Harmo ha mo
Author Introduction:	Https://zhengwh.github.io
Technical Blog:	Http://www.cnblogs.com/beer
Email:	[Email protected]
Qq:	1295351490
Time:	2015-10
Copyright Notice:	Welcome to learn to exchange for the purpose of the reader to reprint, but please "annotated source"
Support this article:	If you are inspired by the article, you can click the button in the lower right corner of the blog to "recommend"

Massive user-high concurrency SaaS product testing on-line process