Before making the benchmark test design better, you must first check which common errors should be avoided, which may result in useless or inaccurate test results.
- Use is a subset of data, rather than a complete set. For example, an application requires hundreds of GB of data, but only 1 GB of data is used for testing. Or, you can use only the current dataset for testing, but you want to simulate performance testing after the business grows in the future.
- Use the wrong distributed data. For example, the distributed data is evenly used, while the "hot spots" is ignored in the real system. (The random data is often different from the real distribution ).
- Use non-real distribution parameters. For example, assume that all user profiles are read evenly.
- Use the single-user hypothesis for testing in multi-user applications.
- Test A distributed application on a single server.
- Does not match real user behavior. For example, the "think time" summarized on the web page )". After a real user requests a page, he or she will read it without pausing it and clicking the relevant link one by one.
- Run the same query cyclically. Actually, the query is different, which causes changes in the cache content. The same query is cached in whole or in part to some extent.
- Ignore the error check. If a test error occurs, the test result is meaningless. For example, if a slow operation suddenly becomes very fast, you need to check whether there are any errors. Otherwise, the test results may only test how MySQL can quickly check for an SQL syntax error! After the benchmark test, you must adhere to the relevant error logs, which is a principled requirement.
- The system warm-up process is ignored, for example, testing immediately after the system is restarted. Sometimes it is necessary to know how long the server can reach the normal performance capacity after restart, so pay special attention to the warm-up time. Conversely, if the purpose of the test is to analyze the performance under normal conditions, note that when the benchmark test is started immediately after the restart, many caches do not have data (it is "cold", cold), and such test results do not reflect the effect after the cache is filled with data.
- Use the default server settings.
Only by avoiding the above errors can we embark on a long way to improve the test quality.
