Redis2.4 statistics for 30 million of data volume

First, let's talk about the work scenario. We need to provide a service that can process tens of millions of data records. The response time of a single request is less than 20 ms. Because the data type is key: list [], redis is suitable for storing data. This article is available only to test the efficiency of redis storage. Step 1: create data. The idea is as follows: (1) generate three

First, let's talk about the work scenario. We need to provide a service that can process tens of millions of data records. The response time of a single request is less than 20 ms. Because the data type is key: list [], redis is suitable for storing data. This article is available only to test the efficiency of redis storage. Step 1: create data. The idea is as follows: (1) generate three

First, let's talk about the work scenario. We need to provide a service that can process tens of millions of data records. The response time of a single request is less than 20 ms. Because the format of the stored data is key: list [], redis is suitable for storing data. This article is available only to test the efficiency of redis storage.

Step 1: create data. The idea is as follows: (1) generate 30 million keys first. To solve the problem that random functions cannot evenly distribute data, we use two steps to create keys. First, the key is generated from 1 to 30 million to solve the quantity problem. Then, use a random function to generate million data and add it to these keys. (2) to improve efficiency, 50 threads are used to concurrently create data. Each thread is responsible for receiving 0.6 million of the data. (3) Insert 10 thousand pieces of data into the keys of several lengths. [30 million data, about 3G memory is used, the physical machine memory is 8G, allocated to redis 4G]

Step 2: Calculate the access speed. To accurately reflect the query speed, 10 thousand random keys are generated and accessed, and 10000 get times are obtained. The query time is about 15625 ms, with an average of ms.

For a list with a key length of 14 and 10000 data records, run lrange (key, 0,-1). The time is 31 ms, and the key length is 10, for a list containing 10000 identical data, run lrange (key, 0,-1) for 16 ms.

At the time of insertion, when the key of the list is 14, 10000 pieces of data are inserted, which takes 23015 ms, while when the key is 10, it takes 19875 ms.

It can be seen that the length of the key increases by 4, and the query time increases from 16 ms to 31 ms, while the time for inserting a record increases from 1.9875ms to 2.3015 ms.

Three conclusions:

1. Control the key length and try to use meaningful and short words.

Ii. Control the length of the list. Do not exceed 10 thousand. If the length may exceed 10 thousand, you need to consider the expiration deletion mechanism.

III. It is recommended that the memory occupied by redis not exceed 50% of the actual memory; otherwise, there will be read timeout during insertion.

Test Environment OS: Linux x86_64 memory 8 GB, redis version: 2.4.18 [latest 2.6.14http: // redis. io]