Performance tests based on user experience: Chapter 1 Introduction

Copyright:Based on the spirit of sharing, this article can be reproduced to benefit more test peers. Please retain this copyright statement upon reprinting and ensure the integrity of the article, but shall not be used for any commercial or other purposes for profit purposes.

Original article name: User Experience, Not Metrics
Original Author: Scott Barber
Source: http://www.perftestplus.com/resources/UENM1.pdf

Translation name: Performance Test Based on User Experience
Translation: pent
Http://www.cnblogs.com/pent/archive/2007/07/01/802096.html.

 

Chapter 1Introduction

As more and more people rely on the Internet to engage in daily business operations, the performance of applications becomes crucial to successful e-commerce solutions. To ensure success, many companies have developed tools and methods to test and adjust application performance. However, these tools and methods focus on system measurement optimization rather than user experience. The articles in the User Experience, Not Metrics series will show you how to determine the real User Experience and how to use the Rational Suite TestStudio test tool, this article discusses a set of effective methods for application procedural optimization, focusing on the performance experience of end users. This section describes the concepts and terminologies in the entire series of articles.

1. Introduction

How many times have you been forced to terminate the task and select other sites because the webpage speed is too slow? About 46% of consumers choose to leave because the site is too technical or performance-oriented. In other words, if your site is too slow, customers will leave. This is well known to all Internet users. At this time, your first thought is not "Oh, I don't know how the site's throughput is", but "it's too slow! I don't have time to wait here. Go somewhere else ". Now, do people leave your site because of performance problems?

Based on this, users do not care about your throughput, bandwidth, click rate per second, and other indicators. They only need a good user experience. There are a lot of books on the market discussing how to design good performance, and more books focus on how to make the site more intuitive, vivid, pleasant, and easy to operate. We have also discussed the benefits of speed. But how can we really predict and optimize the system to improve user experience? This is a direct user experience test. There are two ways to do this. You can directly invest your site in a production environment that can be used for data collection and system optimization, and pray that your site will not be too slow or crash. Another wise choice is to simulate real multi-user activities, perform repeated tests and optimization, and then invest in the production environment. This sounds like a simple choice, but how can we accurately simulate real multi-user activities? This is the question that this article tries to answer.

2. terms and concepts

Understanding the following terms and concepts is important for subsequent articles.

Performance Testing: determines and eliminates Bottlenecks of applications or systems by running the expected user model. See figure 1.

 

Workload Distribution. Taking the usage of a retail site as an example, most users are shopping, some are searching for specific products, some are checking out and finally leaving the site, at the same time, an administrator may update the product price. The load distribution is based on the proportion of users performing specific functions within a period of time. In the preceding example, the load distribution may be: shopping-83%, product search-5%, checkout-10%, and background management-2%.

From the perspective of user experience, Performance Goals can represent the maximum response time allowed when a predetermined number of concurrent users are executed.

Response Time is measured from the end user's perspective. For example, the time from when the user clicks the logon button to when the next page is fully loaded.

Session Duration is the total Duration of a single user's access to a site. The Session Duration varies with the user type.

The number of Concurrent Users is The total number of Users who actually access The site or active sessions at a specific time point.

The Total Usage per hour (Total Hourly Usage) is the expected number of users who visit the site within one hour. This is calculated by multiplying the session duration by the number of concurrent users.

Baselines is understood as a single-user, single-Script test at the time comparison start point.

User Delay is a period of waiting time for the script to be inserted, so that the script can be played back at the same pace as the actual User.

In this series of articles, different people may use different terminologies. These terms are described as follows.

Load Test is a multi-user Test that includes waiting time and accurately simulating operations on the target user group. Server Load balancer can be used to execute different user loads to obtain information about the maximum number of users supported by the site, and to determine whether the user meets the expected performance objectives.

Stress Test is a Test that consists of multiple scripts and does not contain waiting time under high user load. This type of test is useful for evaluating the stability and functionality of the system under a certain load, but it is not suitable for determining the user experience.

Benchmark (Benchmarks) to collect metric indicators related to system hardware and supporting software. For example, you can access a large image through a benchmark test to determine the Web server throughput and click rate per second.

3. Overview of articles

A new article will be published in this series every month. All articles will cover the discussion of practical application methods and technologies, examples and (or) code samples in reality, and exercises for "Now you try it. Each article is divided into junior, intermediate, or expert levels, which usually refer to the complexity of the Code required for technical implementation discussions. The concepts mentioned in each article will apply to all levels of professional knowledge. The first 12 articles are briefly described below.

3.1. Simulate Real Users (Modeling Real Users)

One of the keys to determining the real user experience is to effectively simulate the actual users and user groups. The first three articles in this series will discuss how to use Rational Test Studio to accurately simulate a single user or the entire user base from the application perspective. Related topics include:

Part 2: Modeling Individual User Delays

Part 3: Modeling Individual User Patterns

Part 4: Modeling Groups of Users

3.2. (capture) meaningful time

After simulating actual users, it becomes quite urgent to obtain the response time of the system (Application) from these users. It is not enough to simply capture the time. Unless you can interpret the time pattern, the time is meaningless. The next three articles discuss how to use Rational Test Studio to capture and explain real time experiences. Related topics include:

Part 5: What shocould I time and where do I put my timers?

Part 6: What is an outlier and how do I account for one?

Part 7: publish lidating and interpreting Times

3.3. Report to Stakeholders (Reports to Stakeholders)

I have to admit that stakeholders and decision makers need test reports. We cannot simply tell them that the test results are "OK" or "no ". We need to collect a large amount of data and form a concise and meaningful report. This article will discuss which Test types will bring more value to stakeholders and decision makers, and how to use Rational Test Manager to collect data and generate various abstract information. Related topics include:

Part 8: What Tests add value to stakeholders?

Part 9: Summarizing upload SS multiple tests with accuracy

Part 10: Creating a Degradation Curve

3.4. advanced topics

The final topic of this series will be discussed on some advanced issues that have plagued the author. These articles are in the form of case studies. Each case study summarizes the special requirements of the customer's problems, the author's thought process for forming a solution, the key points of possible solutions, and describes the selection methods in detail. Related topics include:

Part 11: Handling Secure Session ID's

Part 12: Conditional user path navigation (intelligent surfing)

Part 13: Working with Unrecognized Protocols

4. Summary

This series of articles is clear: compared with today's metering habits, the user point of view is a more reliable way to measure website performance. This series of articles aims to teach readers how to use Rational Test Studio to simulate multi-user activities and a verified method to optimize the performance experience of end users. This article will share some useful information about how to effectively use methodologies and use Rational testing tools, and even some useful Tips (Tips) to circumvent the issues that once made experts feel embarrassed. I hope everyone will like it and stay tuned.

5. Appendix (word)

User experience

Application's perspective Application perspective

Fraction of an hour: the unit of hour.

Stakeholders

Concise and meaningful report

Imperative urgent and necessary

Returned topic: [translation] performance tests based on user experience