Another discussion post: How can I test whether a portal website supports simultaneous online use of 0.1 million users?

Document directory

• Feedback

The content of this post is typical. If you are interested, consider it.

First, the landlord raised the following question:

A recent project in the company is a portal website that requires performance testing. According to the characteristics of the project, the main test items and test solutions are developed.
One is to test the maximum number of concurrent requests acceptable to several common pages (username parameterization, set a collection point Policy)
One is to test whether the user can operate normally under the server's long-term pressure (username parameterization, iterative running script)
Another method is to test whether the server can accept 0.1 million users for online operations at the same time, but the LoadRunner license can only support 10 thousand users. How can I develop this solution?

Follow your reply below:

NetizensXingcyxReply:

1. It is useless to find 10 computers, and only 10000 licenses are supported.
2. Find HP support. Of course, the premise is that you have enough money.
3. 10000 user concurrency is measured. In my opinion, there are usually 10000 concurrent users that support 100000 online users, and there is no problem.

NetizensJacklooReply:

In general, this type of performance indicators has no practical significance for most software, but more importantly, they require hardware.
If IIS is used as the application server, the maximum number of concurrent jobs per server cannot reach 0.1 million, so you must use a cluster to achieve Load Balancing through multiple machines;
If an application server such as Websphere is used, the maximum number of concurrent jobs that a single server can afford can reach 0.1 million, but the cluster must be used for performance consideration, load Balancing is achieved through multiple machines;
So, as long as you have enough servers in the cluster, the number of concurrent jobs reaches 0.1 million.
There is usually a simple calculation method. One connection generates one session, and each session has a memory size setting on the server, which is 3 m on NT, therefore, 0.1 million concurrency requires GB of memory. Of course, in actual use, other programs also occupy the memory. Therefore, the number of prepared memory needs to be greater than this.
Another 0.1 million users are online at the same time, which is totally different from the 0.1 million concurrency. This upstairs has already been said. But how can we convert 0.1 million concurrent online users into how many concurrent users?
This requires a large amount of historical log information. System Logs must contain the number of online users at the same time and the number of user operations. The ratio of the two data records is the ratio of the number of concurrent users that you want to change.
In addition, based on empirical statistics, for a Java-developed web system (I have not counted anything else and cannot provide data ), generally, a single dual-CPU, 2 GB memory server can support a maximum of 500 concurrent connections (in this case, most operations are time-out errors and the server is prone to downtime, in fact, there is no practical significance). The maximum number of concurrent operations that can be normally used (the waiting time for a single-step non-large data volume operation cannot exceed 20 seconds) cannot exceed 300.
Assume that your 0.1 million concurrent online user conversion concurrency is 9000, so you need at least 18 such machines. We recommend that you set at least 30 machines.
Of course, if you buy a large server with 200 CPUs, 0.1 million GB of memory, and GB of optical fiber bandwidth, even if it is concurrent users, the speed is definitely too high.

Reply from the landlord:

Thank you, jackloo!
How high are the server requirements if I want to test 10000 users to perform common operations online at the same time (add a user every two seconds and add it to 10000?

NetizensJacklooReply:

Apply a classic line "high, really high"
Haha. In addition, it takes nearly six hours for all the settings to enter the running status.
You can press a system to see the details. The following situations may occur:
1. Server down;
2. Client downtime;
3. The server rejects the request at a certain time. All errors are displayed on the client;
4. Barely completed the test, but the network is congested or the test results are displayed for a long time. Assume that the bandwidth between the client and the server is 10000 Mbit/s/= 10 KB, then each user can only get 10 KB. This speed is close to that of one 64 kB modem for Internet access;
In addition, all the above analyses did not take into account the background of the system, such as databases and middleware.
I have never met such a performance requirement as you said, and I have to feel the same:
1. Server: As mentioned above, 50 PC servers are required;
2. Network: 50 K for each user, at least five hundred megabytes of bandwidth exclusive, it is estimated that only the network latency is about one second;
3. If there is a database, at least Oracle, it is best to use sysbase, and SQL server will definitely not survive. The database server requires at least 10 machines with 4 CPUs and 16 GB of memory;
4. If there is a CORBA, prepare at least 10 machines with 4 CPUs and 16 GB memory;
In addition, Server Load balancer, firewalls, routers, and various software have no investment of 10 million.

NetizensMybasswoodReply:

What should we do if we use 0.1 million users.
For example, for VoIP, if there are 0.1 million users, the server requires that each client should report to the server at least once within 3600 seconds, otherwise the server will be canceled.
The client is registered every 60 seconds.
Therefore, it is estimated that within 3600 seconds, each client will report at least one successful report, which is the minimum standard. Only 0.1 million users will be able to be eaten by the server within 3600 seconds. This is the minimum requirement.
The highest requirement is: all 0.1 million users can register within 60 seconds. If the server can be eaten in 60 seconds, the average concurrency is about 3334 per second.
The minimum requirement is that all 10 users can register within 3600 seconds. If the server can be eaten within 3600 seconds, the average number of concurrent users per second is almost 60. another problem is that the client must send at least 60 requests within 3600 seconds, at least once successfully. in addition, if these users are distributed all over the world, the value will change.

 

The following is my opinion:

Give the landlord a suggestion.

Your testing environment in the company is certain. You need to check the actual processing capability of the system in the current environment. If there are resources available, perform a scaling test.
The test results are analyzed to describe the processing capability and scalability of the system.

Of course, describe your test environment in the report.

Another netizenRobustMessage:

Do you mean to use the test results of 10000 users to speculate on 0.1 million users?
Or, as some old man said, test what kind of scalability tests, and then submit a report. I just want to use 10 thousand to speculate on 0.1 million cases? (Note: if that is the case, you need to perform performance tests. You only need to calculate the performance results .)
Or, as some old man said, this type of performance indicator has no practical significance for most software, and is it more about hardware requirements? (Note: if that is the case, you need to perform performance tests and Performance Tuning. You only need to calculate the performance and add hardware .)
In fact, "practice is the only criterion for testing truth! "This sentence is the truth. It is only known if it has been tested. Any speculation is only speculation and does not reflect the actual situation.
As for performance testing tools, LR only has a high penetration rate (high market share) and is not an advantage in performance indicators. there are many tools in the world that are better than it. For example, avalanche2500 of siprent Communications Company and performance testing tool provided by large-scale computer laboratories. supports recording/playback and test result analysis. it can simulate the protocol from the data layer to the application layer (including http-web, of course) and support a single 1 million concurrent connection. you can also test the concurrent performance of 1 million.

 

Even the reply:

The opinions mentioned above are good, but the understanding of performance testing is somewhat different.

Let's leave aside the performance testing tool. In fact, this question is about how to do a performance test.

For example, if you want to know the effect of a new vaccine on humans, do you want to find all the people on Earth and give them a try? Of course not. We can only calculate a model through testing and sampling, and then calculate the overall features through the sample performance. This is the field of statistics research ,. However, please note that statistics does not include the same content as the old man mentioned above: you only need to calculate the performance result.

The same is true for performance tests.

The performance requirement mentioned by the landlord should be the stress that the system may face after it is launched. We will not discuss whether the requirement is accurate and effective. We should assume that it is effective. Therefore, since it is necessary to verify whether the system is able to cope with the situation that 0.1 million users are online at the same time after it is launched, it is natural to use the production environment for testing. If yes, OK. You can perform this test. As for tools, developers can help write some simple scripts for pressurization, and then collect test data through other third-party tools.

But what if there is no production environment and there is only one dual-CPU, 2850 servers with 3G memory? This is like the example above. The feasible method is to use different levels of load on this server for testing, and obtain the optimal load and maximum load of the system in this environment based on the test data, estimate the load and resource consumption based on the test data and find the relationship between them.

Generally, a large portal website does not use only one super server to handle all the loads, because using Server Load balancer and cluster technology can better solve this problem, it must be that multiple servers are distributed in different places, and the content is synchronized to each server through the content delivery network. During access, users are actually routed to a suitable server by the application layer or front-end device. Therefore, it is necessary to test the scalability. When the number of clusters increases, do you need to know whether the system's response capability can be linearly increased, that is, whether the system can withstand more pressure, provide services to more users.

Finally, for performance testing, we need to confirm the system's response capability and check whether the system can meet the performance requirements.

If you have different opinions, welcome to PK discussion.

Reply to me:

To jackloo


The requirements for hardware and software resources you mentioned are not completely accurate. The actual resource consumption is calculated based on the business type provided by the website.
For most portal websites, most of them are static pages. During user access, the system only returns a static page to the user and does not need to maintain the session, in this case, the load is mainly concentrated on I/O and network traffic-that is why large portal websites are deployed in a distributed manner. Of course, if cache is used, the memory usage will increase with the server running time, but the CPU is usually not a key resource.

This is the difference between portal websites and enterprise applications or online games.

Or even:

 

To


As mentioned above, you need to further clarify whether your testing requirements are valid and reasonable.

Performance requirements must be measured based on the specific business types provided by the website. As mentioned above, only static page access is provided? Or are there other businesses?

Distinguish between registered users, online users, and concurrent users.

In addition, what you need to worry about most urgently is not the license issue of LR, but whether the tested system can support hundreds or thousands of concurrent users. If you cannot even support this, in this case, tens of thousands of concurrent accesses are not considered.

 

I hope you can write down your different opinions and comments. Let's discuss and make progress together.

Feedback # 1st Floor[Poster] Reply Reference View

ByJackei I will continue to discuss with the author again: 0.1 million users are the expected number of customers after the official launch. Here I would like to ask what is the configuration required for 10 thousand million users online? (For example, a total of 10 thousand users are loaded every two seconds.) I replied again: it seems that the landlord still does not understand.

1. What kind of configuration should be evaluated through your test;
2. Is there a reference basis for your 10000 users and loading a user every two seconds to simulate the actual scenario?
3. You mentioned how many pages to test. What is the user distribution on these pages?# Floor 2[Poster]ReplyReferenceView

ByJackei If you are interested, go to 51 to participate in the discussion. ^_^

Http://bbs.51testing.com/thread-48563-1-1.html# Third FloorReplyReference

Generally, Server Load balancer is used for large websites. For example, 30 servers of a shopping website are divided into WWW, SSL, and www, in this way, the WWW page can be accessed by-million users, and the SSL part can be placed by a maximum of-users at the same time.
At the same time, the database server must reach a certain scale.# 4th floor[Poster]ReplyReferenceView

ByJackei @ Oscarxie
Thank you. The data you provided is more helpful for your understanding of the above discussion.# 5th floor[Poster]ReplyReferenceView

ByJackei Continue to follow up with netizensRobustReply:

Quote:
Original Post published by jackei

In medicine, bioengineering, and other fields, it is extremely difficult for many experiments to obtain samples. It is also reasonable to use hundreds of samples to obtain estimation of larger overall features.

No error, because the premise is that their sampling method is scientific and reasonable. For sampling in such engineering fields, their samples will be evenly distributed. If the sampling method is incorrect, their results will also be incorrect.

In contrast, in this example, if we take a sample from 10000, we speculate that 0.1 million of such results are correct and scientific? NetizensJacklooReply: Haha. Wonderful. Learn from Jackie and robust.
However, for portal systems like LZ, most of them are static pages because of user permissions. However, as long as it is a multi-server cluster, we can use the test results of one machine to calculate the load capacity after multiple machine clusters, consider the load balancing and routing pressure at most, such as bandwidth, speed, latency, and so on. However, if all the changes occur on one machine, we can only calculate some indicators. From these indicators, we can simply determine whether the indicators are not feasible, for example, if 0.1 million concurrent users only have one hundred megabytes of bandwidth, we can calculate that each user only has 1 kb of bandwidth, which is obviously not feasible. However, the actual results still need to be tested. After all, the system pressure and the number of users do not change linearly.
In addition, I also explain why "this type of performance indicator has no practical significance for most software, and more is the requirement on Hardware ".
This category refers to the maximum number of concurrent users and the maximum number of online users. Because users operate on different functions, the pressure on the system is different. In actual use, the function changes in the overall user operation due to different times or emergencies. Therefore, the absence of uniform functional usage standards makes such indicators a gimmick. All software vendors are self-speaking. They install their own software in an extreme environment and do some non-practical operations to obtain a performance indicator that you will never reach. Just like the fuel consumption of a car manufacturer, you just need to listen to it. It is not counted in actual use.
In addition, this type of system is widely used and mature, and many software can roughly estimate the performance characteristics of the system after the scheme is designed, these results in a low proportion of system tuning in terms of software performance (of course, it is not completely ruled out that the performance will be further improved after some code and configurations are optimized later ), more are from the hardware aspect, such as increasing the memory, raid the hard disk, increasing the bandwidth, or even increasing the machine. Even reply: Hi. Robust. I'm glad someone can discuss the problem like this. ^_^

Let me talk about my opinion.

First, this continuous normal distribution, such as response time, is usually evenly distributed as long as there are enough samples, that is, it is statistically significant. -- What is a sufficient sample size? For example, it is not enough for 100 concurrent users to execute an iteration. You can consider executing 100 iterations for each virtual user to minimize the impact of random sampling on Data Authenticity. Of course, we must understand that the number of samples is limited by the receipt time and related resources. In my personal understanding, we should obtain as many sample data as possible within the range of resource restrictions.

Second, in fact, we are discussing two issues: Performance Testing and scalability testing ), in other words, the system performance is expected through capacity planning.

It should be said that performance testing is also an experiment, but in general it is not the 0.1 million users mentioned above, but the last user starting from the first user access to the system before the system goes offline. Generally, the average response time is used to evaluate the system performance. However, to obtain the most accurate average response time, it is necessary to record the response time of each user from the first user access to the last user before the system goes offline, and then average. But obviously this is unrealistic, just like the example of the vaccine I mentioned above. We can only use performance tests to obtain experimental data through different levels of stress (concurrency), and analyze these experimental data to obtain estimation of the real system performance. STD. (standard deviation), confidence interval and other statistical methods, I will be in my "LoadRunner didn't tell you" STD. the application in performance analysis is discussed in detail.

For scalability testing, it is necessary for many large websites or carrier-level systems, because considering the growth of users, therefore, it is necessary to know whether the system can improve the processing capability of the system through some simple and effective methods (such as adding a cluster) in the future, and whether a huge investment is required to purchase licenses for hardware and various software in the current phase. Generally, we need to analyze the concurrency, system performance, and consumption of software and hardware resources, and use mathematical modeling to obtain a capacity model. In this regard, you can find more information through Google.

I hope this discussion can continue ^_^ or even reply,To user jacklooHaha, we are glad to see that everyone has come back to continue discussing this topic. Now we feel that our discussion is very deep and we hope we can continue.

In addition, let's talk about some personal opinions.

First, if the system response capability changes when the cluster is used, only the data of one cluster is insufficient, we must continue to test 2/3/4 or more (if there are enough resources), and the data and analysis results will be closer to the real situation.

Second, I also agree with you that "this type of performance indicator has no practical significance for most software, and more is the requirement on hardware ". On the one hand, we can see that the performance requirements of the landlord are still debatable. However, we can also think about how much concurrency the system can support in a certain environment. For example, if the current bandwidth is 100 m, what is the concurrency supported? What should I do when I upgrade to 1000 m? Tests based on the existing environment can be used to understand possible bottlenecks of the system under different pressures and serve as a reference for future deployment.

 

# 6th Floor Reply Reference

By asqcf [unregistered users] Everyone is me and I am everyone.

With the rapid development of the Internet, more and more website space providers (iDCs)
1. How to Select a proper IDC
To choose a good IDC, you must choose a regular company, a reputable company, a company that focuses on the brand image, and a long period of time to carry out business. After all, old brands will not be caused by a customer, after years of influence on the brand image, there is also a later service guarantee. The most important thing is the speed and quality of Website access.

Ii. How to identify access speed and quality
The general practice is to ping the website you want to test on your computer. The result is that you can only test the access speed of your region and network. This test method has great limitations and cannot reflect the actual situation of the website. You cannot find the IDC you need. You cannot make the right choice.

Now, the Free Software we provide can solve the problem for you. The website Evaluation Network Testing System (www.cecela.com) provides website evaluation and network testing services to test the network performance of websites, individuals, network operators, IDCs, and regions, it mainly includes access speed, network stability, comparison and ranking. The system uses the real client mode and provides 24x7 around-the-clock data monitoring. It can be used by website application, time, region, carrier, and other dimensions, comprehensively evaluates the accessibility and access effects of WebSite Services, and assists Internet operators in improving and optimizing website application services, ultimately helping achieve business growth and brand improvement. This software is free of charge. You only need to log on to and download the client and complete the registration. When you get the data you want, other registered users are also tested. This is the true meaning of the use of this software. Address: http://www.cnblogs.com/jackei/archive/2006/11/16/561846.html