Test hbase performance of ycsb

I am seriously thinking that the people in prison are not necessarily worse than those outside.-- Sanmao

This article has been in the draft box for more than two months. Today, I was suddenly told that I want to change the direction. After reading hbase for the last three months, I am so sorry.

When I first installed the hbase system and had no application, I don't know what to do with it. Okay, let's test it.

Hbase version: 0.90.5 hadoop version: 1.0.0, distributed deployment, a total of four nodes.

The test tool is ycsb and Yahoo cloud system benchmark. This tool is used to test various distributed databases. Can be downloaded on this site: https://github.com/brianfrankcooper/YCSB, personal understanding, ycsb is more used to test different distributed databases, such as Cassandra, hypertable, hbase, test the performance of different databases.

Download Test ToolLater, the version I used is: ycsb-0.1.4

1 tar xfvz ycsb-0.1.42 cd ycsb-0.1.4

Start configuring the test environment:
Copy the jar package in hbase to the hbase-binding/lib folder, and copy the/CONF/hbase-site.xml in hbase to/hbase-binding/CONF. Create the specified table usertable in hbase. The default test is to operate such a table. Such a column family is required, not necessarily three.

1 create 'usertable','f1','f2','f3'

The test process is divided into two phases: loading data first and executing transactions.
Load dataTo enter the bin directory of ycsb and start to enter the command:

1 ./ycsb load hbase -P ../workloads/workloada -p threads=10 -p columnfamily=f1 -p recordcount=10000 -s > load.dat

Command analysis: load: indicates to load data for testing.
-P: Specifies the configuration file.
Workloads/workloada: the configuration file. This workload has a mix of 50/50 reads and writes
-P: Parameter
Threads: the number of client threads

Load dataAnd some results are as follows:

1 [OVERALL], RunTime(ms), 2295.02  [OVERALL], Throughput(ops/sec), 4357.2984749455343  [INSERT], Operations, 100004  [INSERT], AverageLatency(us), 198.26185  [INSERT], MinLatency(us), 96  [INSERT], MaxLatency(us), 7888837  [INSERT], 95thPercentileLatency(ms), 08  [INSERT], 99thPercentileLatency(ms), 09  [INSERT], Return=0, 10000

After loading the data, start to execute things, mainly including read and update operations. Run the following command:

 ./ycsb run hbase -P ../workloads/workloada -threads 10 -p measurementtype=timeseries -p 
   columnfamily=f1 -p timeseries.granularity=2000 > transactions.dat 

The specific meaning of the command is basically the same as that in the previous article. You can refer to the website documentation for proper configuration.
After the command is executed, some results are displayed:

 1 [OVERALL], RunTime(ms), 541.0 2  [OVERALL], Throughput(ops/sec), 1848.4288354898335 3  [UPDATE], Operations, 502 4  [UPDATE], AverageLatency(us), 4569.788844621514 5  [UPDATE], MinLatency(us), 56 6  [UPDATE], MaxLatency(us), 373970 7  [UPDATE], Return=0, 502 8  [UPDATE], 0, 4569.788844621514 9  [READ], Operations, 49810  [READ], AverageLatency(us), 4618.51606425702811  [READ], MinLatency(us), 74212  [READ], MaxLatency(us), 37928713  [READ], Return=0, 49814  [READ], 0, 4618.516064257028 

In this way, a test process is completed, but ycsb tool is used simply according to the ycsb document. The most important thing I think is that after testing with ycsb, analyze the data to understand the hbase system.