Architecture Design: System Storage (--REDIS) Basic concepts and installation use

1. Overview

Redis is a memory database, the so-called memory database refers to the main media it stores data is memory rather than the traditional disk, the latter is only used for accessibility. Redis can be used as a NoSQL database, cache and message broker, and currently the most used scenarios in industry practice use Redis as a cache subsystem, such as storing online users ' logins, storing orders submitted within 1 hours, caching image paths or image content, etc. Second, more scenarios are used as message agents, such as Dubbo support for event subscriptions and notifications using Redis.

The originator of Redis is Salvatore Sanfilippo, which was originally developed to address the fast storage and querying of common friend relationship data on social networking sites. Currently VMware is funding the development and maintenance of REDIS projects, the latest version of Redis is 3.X (version 3.2.5 at the time of writing), with built-in support for many useful data storage structures such as String, hashes, lists, set, Sorted sets, etc., also provides a lot of practical high-performance, high-reliability features, such as set operations, LRU cache Content Management, master-slave synchronization and so on.

In this discussion of Redis, you will first spend less space on the basic installation and use of Redis, and then drill down into the data structures supported by Redis to explain the redis underlying design support for these data structures, and then introduce the major configuration optimizations for Redis, Finally, we introduce the cluster building method of Redis (based on 3.X version) and the implementation case.

2. Quick installation and basic configuration instructions 2-1. Quick Installation

Redis official website very good to remember: https://redis.io/, installation is very simple. Please go to the official website to download the latest released stable version: Http://download.redis.io/releases/redis-3.2.5.tar.gz, after decompression can be directly make & make install. Note that an older version of the operating system, similar to CentOS 6.X, may report some errors during installation, and then fix it in turn:

• Make:cc:Command not found make:

makemakemake***

Encountered this error stating that the compilation tool was not installed. Yum installs GCC:

# yum install -y gcc

• You need TCL 8.5 or newer in order

# make testerror8.5orinorder

This is because the version of the software is too old to download an installation on the line:

# wget http://downloads.sourceforge.net/tcl/tcl8.6.1-src.tar.gz......解压......# cd ./unix/......安装即可......

• The installation process may have other problems, according to the error resolution can be

Once these issues are resolved, you can install the version of Redis 3.2:

# make install......Hint: It‘s a good idea to run ‘make test‘ ;)......INSTALLinstallINSTALLinstallINSTALLinstallINSTALLinstallINSTALLinstall......

2-2. Client Connection Test

Redis's client components are seamlessly integrated with the spring framework and are compatible with a variety of development languages: PHP, Go, C #, Python, C + +, Nodejs, and more. Below we simply use Redis's Java client component, Jedis, to complete a simple Redis server connection and write a Key-value value to the servers. We use the Jedis version of 2.8, and you first need to introduce this component in the MAVEN project:

......<dependency>     <groupId>redis.clients</groupId>    <artifactId>jedis</artifactId>    <version>2.8.0</version></dependency>......

The following are the main code snippets for the Java test program:

......=new Jedis("192.168.61.140"6379);// 写入一个K-V值redisClient.set("key1""value");redisClient.close();......

With the above code, we write a k-v key-value pair to the Redis server, is this operation successful? We can use Redis's own client function to view:

# redis-cli -h 192.168.61.140192.168.61.140:6379get key1"value"

The client features of Redis are very powerful, including writing, querying, monitoring, debugging, and so on, from the results of the "get" command, the Java client code write operation is successful. But that's the simplest write operation, the many complex data structures that Redis supports, and how they are organized, which we'll discuss in more detail later in this article.

3. Basic configuration Parameters

In the Redis 3.2 release, the most important configuration file is the file named redis.conf that exists in the installation directory. The template format clearly illustrates the main information and implications that can be configured in the Redis 3.2 release. It mainly includes several aspects: Network configuration item, general configuration item, snapshot configuration item, replication (highly available) configuration item, cluster configuration item, security Configuration item, resource limit configuration item, Lua script configuration, slow log configuration, monitoring configuration, event notification configuration, data structure advanced configuration, etc.

This section will first discuss some of the basic configuration items, followed by the introduction of Redis support for various data structures, and then introduce the corresponding configuration items, introduction of the Redis data snapshot principle of the snapshot configuration items, introduction of Redis high-availability scenarios and cluster scenarios when the corresponding configuration options are described.

3-1. Network configuration items and general configuration items

• Bind: The host address of the binding, not set by default, will process all requests. Because a host may have multiple IPs, if you set 192.168.1.100 here, Redis will only process data requests that reach this IP.

• Protected-mode: Sets whether to turn on network protection mode, by default Yes (on). This option works by if you do not have the bind parameter set and if you do not have a secure login password, Redis allows only locally connected clients to access it when this parameter is turned on. That is, if you do not set the bind parameter and require other clients to access it over the network, then set this parameter to No (close).

• The work port for the Port:redis service, which defaults to 6379.

• Tcp-backlog: There is a configuration parameter "Net.core.somaxconn" on the Linux server, which is the backlog limit for the socket Listener (listen) parameter. A backlog is a queue of requests that has completed a TCP three handshake confirmation with the server, but has not yet been processed by the upper-level application. The default value for this parameter at the operating system level is 128, which is obviously a little bit smaller on a system with very high pressure. You can change to 2048 or larger, and the tcp-backlog configured for Redis should be less than or equal to the maximum limit you set on the operating system (otherwise the Redis settings are no larger). Net.core.somaxconn parameter adjustment can be found in other materials.

• Timeout: This parameter indicates that the REDIS server actively shuts down this client connection when a client is connected and idle for a timeout (in seconds). The default value of this configuration parameter is 0, which means that this function is turned off.

• Tcp-keepalive: The health check of the client TCP connection, if not set to 0, means that the Redis server will periodically send the so_keepalive heartbeat mechanism to detect feedback from the client. The default value for this configuration is 300 seconds, which is detected at 300 seconds. The benefit of a health check is that the Redis server can discover this problem and proactively shut down the peer channel in the event that the client shuts down abnormally. This parameter is recommended to open.

• Daemonize: When yes, runs in daemon mode. The default value of this parameter is no, the main purpose is to facilitate debugging in the test environment, when running in a production environment, you can configure this option to Yes.

• Pidfile: The configuration options for this PID file do not work when Redis is not running as a daemon. When Redis is running as a daemon, the files that this configuration option sets are created and used.

• LogLevel: The Redis log level configured, which defaults to notice. This option allows you to set these values:Debug, a lot of information whether useful or useless will be displayed or recorded, generally used for development and test State;verbose, much less than debug, but also very little to show useful information; Notice, a moderate amount of information and generally do not miss the useful data, can be used in the production environment;warning, only very useful and dangerous information is displayed or logged, it is recommended to use the warning level logging mode.

• LogFile: the location where the log file exists, default to an empty string setting, and the log is printed to the standard output device.

• Syslog-enabled: The default is no (on), and when configured to Yes, Redis logs are output to the system log.

• Databases: Starting with a very early version of Redis, it supports the simultaneous existence of multiple independent memory databases. This value sets the number of databases, the default database is 16, and you can use the Select <dbid> command to specify the database ID on the connection.

3-2. Security and resource restriction configuration items

• Requirepass: This option specifies the authentication password that the client must use before executing any other commands after the connection is completed. This is useful in situations where the technical team needs to deny access to the Redis service by some untrusted clients.

• Rename-command: In a production environment, there are some redis commands that are very dangerous, such as the Flushall command and the FLUSHDB command. So Redis server provides a renaming function for the production environment's instruction security, which allows us to change some instructions that need to be avoided. For example, the following settings can change the flushall instruction to a null character:

  ......rename-command FLUSHALL ""rename-command FLUSHDB ""......

  This is the client that uses Redis to execute the above command, and you receive the following prompt:

  ......// 开启了密码功能，并设置密码为123456# redis-cli -h 192.168.61.140 -a 123456192.168.61.140:6379> FLUSHDB(error) ERR unknown command ‘FLUSHDB‘192.168.61.140:6379> FLUSHALL(error) ERR unknown command ‘FLUSHALL‘......

• MaxClients: Sets the number of clients currently connected to this Redis service by default of 10000. Note that the setting of this value is also associated with the operating system file limit parameter, the actual situation is 10,000 simultaneous normal client connection is fully sufficient, if the actual situation needs to be adjusted, it is necessary to confirm that the setting value does not exceed the number of file descriptor limits set on the operating system.

• MaxMemory: This option tells Redis how much physical memory is used to start rejecting subsequent write requests, which can protect your redis from the use of too much physical memory, which in turn will seriously affect performance or even crash. This parameter is set to bytes, the recommended setting range should not exceed 3/5 of the operating system disposable memory, such as your operating system can control 8GB of memory, then the recommended setting value is 5GB, that is 5242880k, 5120MB, 5GB. In the configuration of large hours, Redis supports a variety of units shorthand, that is, 1GB 1Gb 1gB is a meaning. If you do not set this parameter, there is no limit, but this is not recommended in a production environment.

• Maxmemory-policy: This parameter is important, and it sets the memory purge policy when memory reaches the maximum available quantity . This option sets the following values:volatile-lru Setting value, if you are using Redis, it is recommended that you set the expiration time of key expire if you do not have a special storage requirement. Once the expiration time is set, Redis places these keys in an LRU queue (the content of the LRU queue algorithm can refer to the articles prior to this topic), and Redis cleans up expired keys in the LRU queue using "lazy delete" + "periodic delete". Redis also initiates a purge operation when the memory reaches full state. The allkeys-lru setting ignores the expiration time setting for key, placing all keys in the LRU queue and purging stale data. volatile-random This option takes the "Pick up" algorithm for the expired key value to clear. The Allkeys-random option is similar to the VOLATILE-RANDOM option, except that the key selection extends to all keys, not just the key that sets the expiration time expire. Volatile-ttl, this option clears the currently expiring key (the minimum TTL algorithm). noeviction, if the purge policy sets this value, no purge policy will be started with Redis memory full, and subsequent writes are rejected directly. Noeviction is a default policy, so it is recommended that you make changes to the purge policy in the production environment.

• Maxmemory-samples: The above mentioned LRU algorithm and the TTL algorithm in the Redis implementation is not fixed, the Redis support technician according to the actual situation in the algorithm performance and accuracy between the adjustment. The Maxmemory-samples configuration item is actually a sampling value of a key, the higher the value the higher the accuracy of the algorithm. For example, when set to 10, the LRU algorithm in Redis basically handles the invalid key is very accurate, but it needs to consume more CPU resources, when set to 3, although the processing failed key precision is not high, but processing speed is very fast. The default value for this option is 5.

3-3. Log and monitor configuration items 3-3-1, slow log

Redis provides a slow logging capability from an earlier version, called Slow logging. When an internal operation in Redis takes more time than a set limit threshold, the operation is recorded in memory by the slow log function. Note that the internal operating time is mentioned here and does not include the possible network I/O operation time.

Clients through Redis use Slowlog get, Slowlog len, Slowlog Reset, and so on to see the slow log conditions of the Redis server record. The Redis configuration information provides two parameters for the technical team to adjust the slow log function: Slowlog-log-slower-than and Slowlog-max-len.

• Slowlog-log-slower-than: This parameter is the time threshold for a given slow log, in subtle units. That means 1000000 equals 1 seconds. The default for this parameter is 10000. Note that if the value of this parameter is set to a negative number, the slow log feature is disabled, and if set to 0, each action log is logged.

• Slowlog-max-len: This parameter stores the total length of slow logs that can be recorded by the Redis server. There is no limit to this value, just the technical team is aware that this consumes valuable memory resources, and the default of 128 is actually a more practical setting.

3-3-2, delay monitoring function

The Latency monitor feature is an important monitoring feature that starts with Redis Version 2.8.3, which reflects its latency status. The task is that when a technician sets a time threshold for a delay operation, any operation that exceeds this threshold time will be considered a slow operation to be recorded. After logging in, Redis clients can control/query LATENCY monitors using commands such as LATENCY LATEST, LATENCY history, LATENCY RESET, LATENCY GRAPH, Doctor LATENCY. Latency monitor features are described in more detail in the official Redis documentation: Https://redis.io/topics/latency-monitor.

Latency-monitor-threshold, this parameter is used to set the operating time threshold, if the delay monitoring function is turned off, you can set this value to 0. In addition, the client connection can set this threshold independently, only after the successful connection of the Redis server, the "CONFIG SET latency-monitor-threshold XXXX" command can be executed.

4, the following article introduction

Later in this article we will discuss the following basic knowledge of Redis:

A, event functions and configuration items
B. Basic data structure
String
List
Hash
Set
Zset
C, in-depth data structure and memory optimization
D. Redis Cluster case
Cluster scenarios (1)
Cluster Scenarios (2)

Architecture Design: System Storage (--REDIS) Basic concepts and installation use