Fragmentation and Redis Cloud clusters

"Editor's note" Recently, Anshu and Rajkumar, from Aerospike, posted an article on the high Scalability website claiming that it could obtain the million TPS at a cost of USD 1.68 per hour, which was affected by Redis Labs's Itamar Haber Using a non-benchmark program, it seems to the author that the benchmark is a trap, so what about the Redis test results?

The following is the original text:

One day, I saw an article written by Anshu and Rajkumar from Aerospike company. Very like this article, I was particularly impressed by their EC2 instance of the heavy adjustment and then break through the 1 million mark, but I have been thinking--redis how to do?

I could have done a mature benchmark program, but a proven benchmark program is time-consuming and expensive. And it doesn't take into account the unexpected difficulties that began. The real benchmark test is a trap, it's just a show that accumulates on specific optimizations and steps. But I want an answer, I want to be quicker, so I'm willing to make some sacrifices. This is the near-perfect-non-benchmark test program.

My own definition of a non-standard test program is nothing like the benchmark (hence the name). Here, you can travel light. Relying on the expertise of our lab colleagues, we measured the performance of Redis cloud software without further optimization. We use the following settings to run our non-standard test program:

Single-Slice Rediscloud memory NoSQL The database server runs on an Amazon instance. 3 million objects, each object in size 100 bytes. Run the Memtier_benchmark tool client on a server-side instance, using the following command-line arguments: '--ratio=1:1-n 1000000-d 100-t 1-c--pipeline=75 '. Read and write workloads in the same proportion (we do not particularly favor one of these types of operations, and feel that the mix better reflects reality). An on-demand c3.8xlarge instance.

We don't have time to build a VPC and adjust slide-up groups to best performance, so we run everything in our standard service environment--in noisy, crowded EC2 networks. Of course, we have not adjusted the CPU behavior or the number of threads and the fragmented configuration for this experiment, allowing Redis cloud to use its default values. We did not test for multiple network configurations or add additional resilient network interfaces (ENI), simply provide a new HVM Redis Cloud server and a non-standard test program for it.

On this point, you can get the original output from our operation, but the result is slightly higher than 1.2 million TPS (exactly 1228432). Of course, this amazing result is really exciting to me, and I immediately asked for a mature, comprehensive, optimized, thorough, exhaustive benchmark to truly test the limits that Redis can reach, guess what?

Fragmentation and Redis Cloud clusters

By design, a Redis server (mostly) is a single-threaded process. In this case, fragmentation is typically used to deploy Redis databases that exceed the RAM performance of a single or stand-alone server. There are generally accepted three ways to implement fragmentation: client, agent, or cluster. Because of the fragmented Redis client and agent-based solutions, it is relatively easy to implement them independently of the actual underlying database engine, such as REDIS-RB and Nutcracker. However, few Redis cluster solutions are available today.

A fragmented Redis cluster means that the Redis server (process) is deployed to one or more compute nodes in the network. The cluster runs the Redis database, each possibly spanning many nodes and multi-core, exceeding the total RAM. A product-level cluster ensures the availability of both database performance and infrastructure management as well as database resources.

The most famous Redis cluster implementation should of course be open source, of course, Redis cluster (v3) has entered the beta version, and is expected to be officially announced within a few months. This upcoming release will provide an excellent answer to many of today's challenges. Among its many new features, the new OSS version also includes the ability to create fragmented clusters. For the entire Redis community (if you offend someone, sorry), we think Redis version 3 is an important version in every way.

In addition to open source V3, there are some other redis clusters. Some people walk in front and build their own clusters, everyone has their own reasons. I don't want to borrow anyone to elevate myself (like Twitter, Weibo or Pinterest), but one company has set up a cluster that is Redis Labs. Our Redis cloud service is technically supported by our own Redis cluster and has been used in most of its products over the past two years. In the meantime, we have been building and operating our clusters across multiple cloud and data domains.

Redis Labs is an open source contributor to the Redis project-most of our staff are great and are breathing with the company-but the solutions that our users need are not all in the scope of open source. To meet these business challenges, we developed solutions that allow us to extend the Redis database from MB to TB while the system is running. We deploy, expand, and manage clusters on four different IaaS providers and 20 data centers. Users build tens of thousands of databases, not only to maintain the availability and performance of each database, but also to juggle operational and administrative tasks.

Here's a little-known fact about the Redis Labs cluster: You don't need to make any changes in your application to start using them. Yes, you can use existing code and client libraries and still get all the scalability, availability, and operational benefits that the cluster provides. Users only need to create databases and configuration options (availability, data persistence, fragmentation, security, and so on), and they simply use a single redis URL (hostname and port) in Redis cluster. Of course, you can do things like tuning, best practices, optimizations, and so on, but (as in the non-benchmark program), even without them, our cluster is a great performer.

Original link: The 1.2M ops/sec redis Cloud Cluster single Server unbenchmark (Compile/Wei revisers/Zhonghao)

Free Subscription "CSDN cloud Computing (left) and csdn large data (right)" micro-letter public number, real-time grasp of first-hand cloud news, to understand the latest big data progress!

CSDN publishes related cloud computing information such as virtualization, Docker, OpenStack, Cloudstack, data center,     sharing Hadoop, Spark, Nosql/newsql, HBase, Impala, Large data viewpoints, such as memory calculation, stream computing, machine learning and intelligent algorithms, provide services such as cloud computing and large data technology, platform, practice and industry information.           &NBSP