How to perform benchmark test test conditions
– How to avoid common mistakes? You need to run benchmarks in a real-world environment.
- Similar or identical hardware
Includes CPU, memory, network, IO system
- The same software configuration
- Similar data set size
- Similar distribution of data
- Similar access patterns
– Avoid queries and data caching
– Reconstruct the distribution of the access
- The equivalent number of threads
– Multi-user and multi-server
Record all information
- It's better to record useless information than to have important information.
- Document all steps for re-execution of benchmark tests
- Configuration: Hardware, software version and parameters, data size, data set
- Collect hardware statistics (CPU, disk, network, memory)
- Collect software statistics (GLOBAL status, INNODB status)
- Results of organization output
Isolate the benchmark environment
- Run benchmarks on systems that are not intended for other purposes
- Avoid sharing your environment (virtual machines or Sans)
- Make sure no other applications use the network
- Start up under the same conditions (warmup)
- (if possible) start the system before each benchmark test
- Disable all programs that may interfere with the benchmark (crons)
Automated benchmark Testing
Benefits of automated Testing:
-Simplifies repeatable execution
-Avoid manual errors
-Improved documentation
-Easy to re-execute
-Easy analysis
Test results Analysis
- Processing of all collected data
- Identify what data to provide useful information
- Answer the questions defined in the goal
- Documentation of the results
- Summarize test Results
- Defining Test metrics
Database optimization-benchmark test (II)
