Redis Tutorial (11): Virtual Memory Description:

Reproduced in:http://www.itxuexiwang.com/a/shujukujishu/redis/2016/0216/138.html

First, Introduction:

Like most NoSQL databases, Redis also follows the Key/value data storage model. In some cases, Redis keeps keys/values in memory to improve the efficiency of data queries and data modifications, but this is not always a good choice. In view of this, we can further optimize, that is, to keep only the keys in memory data, so as to ensure the efficiency of data retrieval, and the values of the data can be swapped out to disk when seldom used.
In practical applications, about 10% of the keys are relatively common keys, so redis can swap out the rest of the keys and values that are not commonly used by the virtual memory, and once the keys or values that are swapped out need to be read, Redis reads it back into main memory again.

Second, the application scenario:

For most databases, the best way to do this is to load all the data into memory, and the subsequent query operations can be completely based on the memory data. In reality, however, such scenarios are not common, and more often, only some of the data can be loaded into memory.
In Redis, there is a very important concept that keys are not normally exchanged, so if you have a large number of keys in your database, each of which has only a small value associated with it, then this scenario is not very suitable for using virtual memory. If the opposite is true, the database contains only a small number of keys, and each key has a very large value associated with it, then this scenario is more appropriate for using virtual storage.
In practical applications, in order to make the virtual memory more fully functional to help us improve the efficiency of the system, we can combine the keys with many smaller values into keys with a small number of large values. The most important method is to change the original Key/value mode to hash-based mode, so that many of the original keys can be used as attributes in the hash.

Three, configuration:

1). Add the following configuration entry in the configuration file to enable the current Redis server to open the virtual Memory feature at startup.
vm-enabled Yes

2). Set the maximum number of virtual memory bytes available to Redis in the configuration file. If the data in memory is larger than this value, some of the objects are swapped out to disk, where the memory consumed by the swapped-out object is freed until the memory is less than the value that is used to stop swapping out.

The code is as follows:

Vm-max-memory (bytes) #p # page Title #e#

The Redis Exchange rule is to consider the "oldest" data as much as possible, that is, data that is not used for the longest time will be swapped out. If the age of two objects is the same, the value of the larger data will be swapped out first. It is important to note that Redis does not swap keys to disk, so if only the data of the keys is filled with the entire virtual memory, then the data model will not be suitable for using the virtual storage mechanism, or the value is set to larger to accommodate the entire keys data. In practical applications, if you consider using Redis virtual memory, we should allocate as much memory as possible to Redis for use in order to avoid frequent swapping in and out.

3). Set the number of pages in the configuration file and the number of bytes that are consumed per page. In order to transfer the in-memory data to disk, we need to use the swap file. These files are not related to data persistence, and Redis will remove them all before exiting. Because most of the access to the interchange file is random access, it is recommended that the swap file be stored on a solid-state disk, which can greatly improve the system's operational efficiency.

The code is as follows:

Vm-pages 134217728
Vm-page-size 32

In the above configuration, Redis divides the interchange file into Vm-pages pages, where each page occupies a byte of vm-page-size, then the size of the swap file that Redis eventually available is: Vm-pages * vm-page-size. Because a value can be stored on one or more pages, but one page cannot hold more than one value, in view of this, we need to take full account of this feature of Redis when setting up vm-page-size.

4). There is a very important configuration parameter in the Redis configuration file, namely:
#p # pagination Title #e#

The code is as follows:

Vm-max-threads 4

This parameter represents the maximum number of threads that Redis applies when performing IO operations on a swap file. In general, we recommend that the value is equal to the CPU cores of the host. If this value is set to 0, Redis will perform this operation synchronously when it interacts with the swap file for IO.
For Redis, if the operation swap file is performed synchronously, when a client is accessing the data in the interchange file, another client attempts to access the data in the interchange file, and the client's request is suspended until the end of the previous operation. Especially when reading large data values on relatively slow or busy disks, the impact of this blocking is even more abrupt. However, synchronous operation is not useless, in fact, from the perspective of global execution efficiency, synchronous mode is better than asynchronous mode, after all, synchronization saves thread switching, inter-thread synchronization, and thread pull up and other operations resulting in additional overhead. In particular, when most of the frequently used data can be read directly from the main memory, the performance of the synchronization mode will be more excellent.
If your real-world application is the opposite, there is a lot of swapping in and out, and your system has a lot of cores, because of this, you do not want the client to have to block a short period of time before accessing the swap file, if it is true, I think the Async method may be more suitable for your system.
As for the final choice of configuration, the best answer will come from constant experimentation and tuning.

Redis Tutorial (11): Virtual Memory Description: