Use Yahoo cloud service benchmark to measure cloud application performance

It is easy to compare the cost of cloud computing vendors, but it is not easy to compare the performance of the application. A performance comparison of the different types of database applications running in the same cloud or across different cloud computing environments will be a particularly challenging task. The Yahoo Cloud computing Service Benchmark (YCSB) is a test framework designed to help users understand the performance of different cloud computing databases and store data performance under actual load. The source code for this test framework is available from the GitHub download.

Yahoo Cloud Services Benchmarking tool includes two important components: a workload builder and a set of generator-generated workloads, as well as interfaces for a large number of NoSQL databases, including Cassandra, DynamoDB, Hbase, MongoDB, Redis and Oracle's NoSQL database. Of course, it also comes with a JDBC interface for relational databases.

Database test benchmarks such as YCSB are particularly useful when faced with application architecture decisions. For example, an existing database application that no longer meets performance requirements might force you to consider expanding hardware resources or changing the underlying database. If additional hardware can achieve near-linear performance improvements (for example, double the server can achieve double performance), then hardware expansion will be an appropriate choice. However, hardware expansion does not always help us achieve linear improvements in performance, because there are other performance-enhancing bottlenecks in applications that do not allow administrators to leverage the extra hardware resources.

If you change your database, you will have a lot of options, especially if you choose to use the NoSQL database to replace the previous relational database. Although initial performance improvements may not be obvious, some specific types of NoSQL databases will illustrate the problem. For example, social network analysis is just right for using a graphics database. If users do not need to support complex query applications, a key-value database such as Cassandra and Amazon Dynamodb might be more appropriate than a document store like MongoDB. Test benchmarks can help you determine which database is best for your needs by testing the data.

Yahoo Cloud services Benchmark Workload management

The first step in running YCSB is to determine the type of database you want to test and the workloads that you want to run on the database. Once you have created your database, you must create a plan that is relevant to your target workload. The implementation details of the plan will vary depending on the type of database. For example, you might create a datasheet in MySQL and create a column family and a key value space in Cassandra.

Test benchmarks are a great tool for comparing database performance, but workloads must also match your actual production load, and other design considerations need to be considered in addition to test results when choosing a database. You can run the core workloads provided by YCSB or create your own. The core workload consists of a set of six workloads with different read/write operating characteristics. Some workloads are based on update upgrades, while others test performance for read operations. If you create your own workload, you must develop a Java application to extend the existing YCSB class to generate data and perform read/write operations.

The workload is set with the parameter switch, so you can run the test workload on a number of different threads and per unit time. It is divided into two phases: to create the import phase of the data, and to perform operations under a specific workload and then output a set of performance statistics. In addition to the running times and the number of times per unit of operation, there are a lot of deferred measurements, including average, maximum, minimum, number 95% delay and 99% delay.

If you have a ready-made application, it may collect data from the performance monitoring tool to determine the running data for read, write, and update operations in your application. If you use a relational database, you need to pay special attention to the most frequent and longest running query operations.

There are good candidates to use in custom workloads.

If you develop a new application, you should run a series of test benchmarks before you formally put your application into production so that you can evaluate a large number of different possible production loads. This can provide valuable information for performance evaluation of different databases under different conditions. If you want to know about the medium workload for most of the time and the running of peak workloads that occasionally occur in design requirements, you can test for two scenarios. Ideally, the database you select will perform well in all expected scenarios.

You should also choose a database based on how often you use it. If a database behaves well under most expected workloads, but behaves poorly under a design or occasional workload, you might choose this database.