NoSQL Product Evaluation: Cassandra, MongoDB, Couchbase and Aerospike

Recently, Thumbtack published two papers, namely ultra-high performance NoSQL benchmark and nosql failover characteristics, the former is the analysis of durability and performance trade-offs, the latter is about Aerospike, Cassandra, Couchbase and MongoDB several NoSQL fault-transfer features. All two benchmarks try to test "user-oriented applications with high throughput and low latency requirements, which can be stored using key-value forms."

Thumback used YCSB (Yahoo! Cloud serving Benchmark), the new YCSB changes are recorded in the documentation for the first paper. Before looking at the new benchmark test, let's take a look at a funny thing about the original YCSB:

Shortly after the launch of YCSB (more than 1 years ago), Hyperdex used the benchmark to test Hyperdex, Redis and MongoDB several high-performance databases, and the results were sharper--throughput seconds kill the thunder of MongoDB and Cassandra, Catch up with Redis.

To this end, "enthusiastic" netizens in the Redis community published a post Hyperdex vs. Redis, and got the redis of the father of Salvatore Sanfilippo Great God "strong" reply:

The fact doesn't sound so interesting because:

Redis and memcached have a more or less upper limit on the single core per second query, Memcached allows automatic use of multi-core technology (which may be possible in the future), while using Redis you need multiple instances, and this can only be used on a Web server. Of course, these systems use the memory processing form, and through reasonable optimization.

What I want to say is that I can also modify redis so that it always returns "Foo", which achieves a single core of 150,000 ops per second. So the real thing should be this:

1. The base e design is very rough, Redis does not support such a comparison of "nutrition" is not

2. In all other tests, they may be using a single core Redis to combat multiple core hyperdex (or Multi-node hyperdex). For example, Redis Lpush can easily insert 1 million options per second into the list, but if you use 4 instances at a time, you'll probably get 3 and 4 million writes per second of each kernel. However, this does not mean that we need to write on the first page "single core 4 million operations per second"!

3. Data sets for benchmark test cases may always be stored in memory

4. No disclosure of all methods so that the test results are of no value

Finally, I think that using wrong guidance to shape the product is also bad behavior, the first 3 months may be harvested, then what will happen?

"Anyone who tries to catch two hares at the same time will eventually reap nothing." ”

And then Salvatore Sanfilippo is the YCSB benchmark made the following comments:

This is the discussion I initiated at Hackernews on this benchmark YCSB: discussion address

Essentially, YCSB has made classic mistakes in constructing ideas, instead of using appropriate use cases to get the best performance of different databases, it uses a layer to enforce data models for different databases. For most databases, the local data model is used, but for other (such as Redis) databases, it is only simulating the use of local operations.

Similarly, the benchmark compares the performance between single core redis and multiple core hyperdex.

(Responsible editor: The good of the Legacy)