Redis Primer Basics and Deployment Configuration explanations

1.redis Introduction

REmote DIctionary Server (Redis) is a key-value in-memory data structure storage System written by Slvatore Sanfilippo, and is open source (BSD-licensed), which can be used as a database, cache, and message middleware.

2.redis composition

• It supports multiple types of data structures such as strings (string), hashes (hashes), lists (lists), collections (sets), ordered collections (sorted sets) with range queries, bitmaps, Hyperloglogs and geo-spatial (geospatial) index RADIUS query;

• Redis has built-in replication (replication), LUA scripts (Lua Scriptiong), LRU-driven events (LRU eviction), transactions (transactions), and different levels of disk persistence (persistence);

• High availability is provided through Redis Sentinel (Sentinel) and automatic Partitioning (Cluster) (Hi availablility).

3.redis Advantages

• High performance: Redis supports read frequency of more than 100k per second;

• Rich data types: Redis supports binary string,lisst,hashes,sets and ordered sets and other data type operations;

• Atom: All operations of Redis are principled, while Redis also supports the principled execution of several operations.

• Rich features: Redis also supports publiss/subscribe, notifications, key expiration and other features, Redis supports master-slave replication

4.redis Application Scenarios

• All of the best usage scenarios for Redis are data in-memory

• Redis more scenes are used as a substitute for memecached

• Redis is more appropriate when you need more data type support than Key/value

• Redis is more appropriate when stored data cannot be raised

5.redis Summary 5.1redis Database Summary

• Improves the scalability of the DB by placing new data on the newly added server.

• Increased availability of DB, affecting only users who need access to data on the Shard server

• Improve the maintainability of the DB, the upgrade and configuration of the system can be done by Shard, the impact on the service is small

• Small data inventory with small query pressure, faster query, better performance

5.2redis Application Summary

• To configure the Master-slave, you can support switching when a service failure occurs

• Disable data persistence on the master side, just configure data persistence on slave

• Physical memory + virtual memory is not enough, this dump a deadly, long time the machine hangs, in this case is a disaster

• When Redis physical memory uses more than 3/5 of the total memory capacity, it starts to be more dangerous, and it starts to swap, memory fragmentation

• When maximum memory is reached, the key with expiration time is emptied back, even if the key does not expire

• Redis and DB synchronous write problem, first write db, then write Redis, because the memory is basically no problem

6.redis Deployment Setup

Download the latest stable version of the Redis Operations Command on the Redis official website (http://www.redis.io) as follows:

6.1. Get the Redis installation package and unzip

wget Http://download.redis.io/releases/redis-3.0.7.tar.gztar Xzf redis-3.0.7.tar.gz

6.2. Enter the catalogue and view the Readme file

CD redis-3.0.7less README

6.3. Configuring the Redis installation package

Make Manifesto=jemalloc && make prefix=/application/redis-3.0.7 Install

6.4. Create a soft connection

Ln-s/application/redis-3.0.7//application/redis

6.5. Configure the Environment language

Lang=en_us. UTF-8

6.6. View the directory of REDIS commands

Tree /application/redis/bin/
     The results are as follows:
/application/redis/bin/
├── redis-benchmark
├── redis-check-aof
├── redis-check-dump
├── redis-cli
├── redis-sentinel -> redis-server
└── redis-server

After the command is executed, five executable files are generated in the /applicaion/redis/bin/ directory, which are:
Redis-benchmark redis-check-aof redis-check-dump redis-cli redis-sentinel redis-server
Their role is as follows:
Command description
Redis-server Redis server daemon launcher
Redis-cli Redis command line operation tool, of course, can also be operated with telnet and its plain text protocol
Redis-benchmark Redis performance testing tool to test Redis read and write on your system and your configuration
Redis-check-aof update log check
Redis-check-dump term local database check

7. Configure and start the Redis service

A. Configure the environment variables, the command is as follows

Echo ' path=/application/redis/bin/: $PATH ' >>/etc/profilesource/etc/profile

View the execution results as follows:

Tail-1/etc/profile Tip: path=/application/redis/bin/: $PATH

To see if it is being imported into the global path

Which Redis-server

Execution results are as follows

/application/redis/bin/redis-server

B. Viewing command help

[[email protected] ~]# redis-server --help
Usage: ./redis-server [/path/to/redis.conf] [options]
       ./redis-server - (read config from stdin)
       ./redis-server -v or --version
       ./redis-server -h or --help
       ./redis-server --test-memory <megabytes>
Examples:
       ./redis-server (run the server with default conf)
       ./redis-server /etc/redis/6379.conf
       ./redis-server --port 7777
       ./redis-server --port 7777 --slaveof 127.0.0.1 8888
       ./redis-server /etc/myredis.conf --loglevel verbose

Sentinel mode:
       ./redis-server /etc/sentinel.conf --sentinel

c. Start Redis Service Operations Command:

Cp ~/redis-3.0.7/redis.conf /application/redis /# The default configuration file containing redis in the installation package
a) echo vm.overcommit_memory=1 >> /etc/sysctl.conf
b) sysctl vm.overcommit_memory=1 or execute echo vm.overcommit_memory=1 >>/proc/sys/vm/overcommit_memory

Redis-server /application/redis/conf/redis.conf &
Tip: After the startup, there will be a command line that is stuck, but in fact, just press Enter, because here is the carriage return background execution.
Tip: Check if the port exists to determine whether redis is successfully started.
[[email protected] ~]# netstat -lntup|grep -w 6379
Tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 5896/redis-server *
Tcp 0 0 :::6379 :::* LISTEN 5896/redis-server *

D. Turn off Redis

REDIS-CLI shutdown or Killall redis-server

8.redis Main configuration file

[[email protected] ~]# grep -vE "#|^$" /application/redis/redis.conf
1.redis is not running as a daemon, you can modify it with this configuration, use yes to enable the daemon
Daemonize yes
2. When redis runs as a daemon, redis will write the pid to the /var/run/redis.pid file by default, which can be specified by pidfile.
Pidfile /var/run/redis.pid
3. Specify the redis listening port. The default port is 6379. Modify the port for security.
Port 6397
4. In a high concurrent environment, a backlog value is needed to avoid slow client connection problems. Note that the Linux kernel silently reduces this value to the value of /proc/sys/net/core/somaxconn, so you need to make sure to increase the aomaxconn and tcp_max_syx_backlog values to express the desired effect.
Tcp-backlog 511
5. Bind the host address
Bind 127.0.0.1
6. When the client is idle for a long time, close the connection. If it is specified as 0, it means to disable the function.
Timeout 0
7. tcp connection keep-alive policy, can be set by tcp-keepalive configuration item, the unit is wonderful, if set to 60 seconds, the server will initiate an ACK request to the idle client every 60 seconds to check the client. Whether the terminal has been hanged or not, the client that is not responding will close its connection, so it takes up to 120 seconds to close a connection. If it is set to 0, no keep-alive detection is performed. Generally reasonable value is 60 seconds
Tcp-keepalive 0
8. Specify the log level. Redis supports a total of four levels: debug/verbose/notice/warning. The default is verbose.debug records a lot of information for development and testing. verbose: a lot of streamlined useful information, unlike debug records. Many; notice ordinary verbose, often used in production environments; warning only very important or serious information will be recorded to the log
Loglevel notice
9. Specify the log file name. You can also use "stdout" to force redis to write the log information to the standard output. The default is standard output. If redis is configured as the daemon, the log mode is configured here. Standard output, the log will be sent to /dev/null
Logfile ""
10. Set the number of databases, the default data is 0, you can use the select <dbid> command to specify the database ID on the connection
Databases 16
11. Specify how many times the update operation is performed, and the data is synchronized to the data file, which can be coordinated with multiple conditions.
Save <seconds> <changes>
Three conditions are provided in the redis default configuration file:
Save 900 1
Save 300 10
Save 60 10000
Represents 1 change in 900 seconds (15 minutes), 10 changes in 300 seconds (5 minutes), and 10000 changes in 60 seconds.
12. If the RDB snapshot function is enabled, if redis persists data to disk, if there is a failure, by default redis will stop accepting all write requests. The advantage of this is that the user can know the memory in a clear way. The data and the data on the disk are already inconsistent. If redis ignores this inconsistency and continues to accept the request, it may cause some catastrophic consequences. If the next RDB persistence is successful, redis will automatically respond to the request. Of course, if you don't care about this data inconsistency or there are other ways to discover and control this inconsistency, you can turn off this feature to ensure that redis continues to accept new requests when the snapshot write fails.
Stop-writes-on-bgsave-error yes
13.When exporting to .rdb database, use LZF to compress string object. The default setting is "yes". If you want to save CPU, you can set this to "no", but if there is a compressed key but there is no compression, then The data file will get bigger.
Rdbcompression yes
14. Because the version 5 RDB has a CRC64 algorithm and put it at the end of the file. This will make the file format more reliable but there is a performance penalty (about 10%) when producing and loading RDB files, so you can turn it off to get the best new energy, and generate a closed validation RDB file with 0. Verification and, it will tell the load code to skip checking
Rdbchecksum yes
15. Database file name and storage path
Dbfilename dump.rdb
16. Working directory, the local database will be written to this directory, the file name is the value of "dbfilename" above, the accumulated file is also placed here, note that you must specify the directory here, not the file name.
Dir ./
17. When a slave loses its connection to the master, or synchronization is in progress, there are two possibilities for slave behavior: 1) If slave-server-stale-data is set to "yes" (the default), the slave will continue to respond to the client. The end request may be normal data or empty data that has not yet obtained a value.
2) If slave-server-stale-data is set to "no", the slave will reply "SYNC with master in progress" to handle various requests, except for the INFO and SLAVEOF commands.
Slave-serve-stale-data yes
18. You can configure whether the slave instance accepts writes. A writable slave instance may be useful for storing temporary data (because the data written to slave# will be easily deleted after synchronization with the master), but if the client is writing due to a configuration error, it may cause some problems. Redis2.6 defaults all slaves to read-only. Note: Read-only slaves are not designed to be exposed to untrusted clients on the Internet. They are just a layer of protection against instance misuse. A read-only slave supports all management commands such as config, debug, and more. To limit you can use "rename-command" to hide all administrative and dangerous commands to enhance the security of read-only slaves.
Slave-read-only yes
19. Replication set synchronization strategy: disk or socket, new slave connection or old slave can not only receive different when reconnecting, you have to do a full synchronization. A new RDB file needs to be dumped and passed from the master to the slave. There may be two situations: 1) disk-backed: the master creates a new process dump RDB, which is incremented by the parent process (ie the main process) to the slave after completion; 2) based on socket(diskless): The master creates a new process that directly dumps the RDB to the slave's socket, without going through the main process, without going through the hard disk. Based on the hard disk, after the RDB file is created, once it is created, it can serve more slaves at the same time. Based on the socket, after the new slave comes, it has to be queued (if it has not come beyond repl-diskless-sync-delay), the next one will be done next. When using diskless, the master waits for a repl-diskless-sync-dely number of seconds. If there is no slave, it will be transmitted directly. Later, it will be queued up, otherwise it can be transmitted together, the disk is slower, and the network is more When you are fast, you can use diskless (default is disk-based)
Repl-diskless-sync no
20. If set to 0, the transfer starts with ASAP.
Repl-diskless-sync-delay 5
21. Is TCP_NODELAY disabled after the slave socket sends SYNC? If you choose yes redis will use less TCP packets and bandwidth to send data to the slave, but this will delay the transfer of data to the slave, the default of the Linux kernel will reach 40 milliseconds, if the delay of no data transmission to the slave is selected Will reduce but use more bandwidth by default we will optimize for low latency, but when the high traffic situation or the number of hops between the master and the slave is too much, it is a good choice to set this option to yes.
Repl-disable-tcp-nodelay no
22. The priority of the slave is an integer displayed in the info output of redis. If the master is no longer working properly, sentinel will use it to select a slave promotion = as the master. A small-priority slave will be promoted to master first, so for example, if there are three slave priorities of 10, 100, and 25, sentinel will pick the slave with the lowest priority number of 10. 0 is a special priority, indicating that the slave cannot As the master, a slave with a priority of 0 will never be promoted to master by sentinel. The default priority is 100.
Slave-priority 100
23. Require the client to verify the identity and password when processing any commands. This feature is useful in environments where other clients you don't trust can access the Redis server. This should be commented out for backward compatibility. And most people don't need an ID card (for example, they run on their own server). Warning: Because redis is too fast, outsiders can try a password of 150k per second to try to crack the password. This means you need a strong password, otherwise it is easy to crack too easy.
Requirepass 898redis
24. By default, redis is asynchronous to export data to disk. This mode is good enough in many applications, but a redis process may cause a write loss for a period of time (depending on the configured save command). AOF is a more reliable alternative persistence mode, such as using the default data write file strategy, in the case of a server power outage or single write situation, redis its own process problems but the operating system is still running normally At the time of the event, redis can lose 1 second of write operation, AOF and RDB persistence can be started at the same time and there will be no problem. If AOF is enabled, redis will load the AOF file at startup, which will ensure data reliability.
Appendonly no
25. Append file name, default appendonly.aof
Appendfilename "appendonly.aof"
26. The fsync() system call tells the operating system to write data to disk instead of waiting for more data to enter the output buffer. Some operating systems will actually pull the data to disk immediately; some will try to do this quickly.
Redis supports three different modes:
No: Do not brush immediately, only when the operating system needs to brush. faster;
Always: Writes to the AOF file immediately after each write operation. Slow, but the safest
Everysec: Write once per second. A compromise solution.
The default "everysec" usually provides a good balance between speed and data security. Based on your understanding, if you can relax the configuration to "no" for better performance (but if you can tolerate some data loss, consider using the default snapshot persistence mode), or conversely, use "always" "It will be slower but more secure than everysec."
Appendfsync everysec
27. If the AOF synchronization policy is set to "always" or "everysec", and the background storage process (background storage process or write AOF log) will generate a lot of disk I / 0 overhead. Some Linux configurations will cause Redis to block for a long time because of the fsync() system call. Note that there is currently no perfect fix for this situation, even fsync() for different threads will block our synchronous write(2) calls. To alleviate this problem, you can use the following option. It can organize fsync() when BGSAVE or BGREWRITEAOF is processed. This means that if a child process is saving, redis is in an "unsynchronizable" state. This actually means that in the worst case, 30 seconds of log data may be lost (default Linux settings). If this is set to yes, it will cause no delay, and it will remain no. This is the best way to save persistent data. Safe way.
No-appendfsync-on-rewrite no
28. Automatically rewrite the AOF file. If the AOF log file is increased to the specified percentage, redis can automatically rewrite the AOF log file through BGREWRITEAOF. How it works: redis remembers the size of the AOF file that was last rewritten (if there is no write operation after restarting, the AOF size when starting directly). This base size is compared to the current size. If the current size exceeds the specified scale, the rewrite operation will be triggered, and the minimum size of the rewritten log will need to be specified, thus avoiding rewriting if the specified percentage is reached but the size is still small. Specifying a percentage of 0 disables the AOF auto-rewrite feature.
Auto-aof-rewrite-percentage 100
Auto-aof-rewrite-min-size 64mb
29.AOF file may be incomplete at the end (the last system shutdown problem, especially mount ext4The file system was not loaded with the data=ordered option. It only happens when the OS dies, redis itself will not be incomplete). When redis restarts, there is a problem when loading into memory. When it happens, you can choose redis to start the error, or load as much normal data as possible. If aof-load-truncated is yes, it will automatically issue a log to the client and then load (default). If it is no, the user must manually redis-check-aof to repair the AOF file.
Aof-load-truncated yes
30. If the maximum time limit (milliseconds) is reached, redis will record a log and then return error. When a script exceeds the maximum time limit, only script kill and shutdown nosave can be used. The first one can kill without calling write. Kill with the second command, set to 0 or a negative value, the time limit is unlimited.
Lua-time-limit 5000
31. The redis slow query log can log queries that exceed the specified time. The running time does not include various I/O times, such as connecting the client, sending response data, etc., and only calculating the actual time the command is executed (this is just the command execution phase in which the thread is blocked and cannot serve other requests at the same time). Query log configuration; two parameters: one indicates the redis timeout (in subtle) to record the command that exceeds the time, and the other is the slow query log length. When a new command is written to the log, the oldest record is removed from the queue. The time unit below is subtle, so 1000000 is 1 second. Note that a negative time will disable the slow query log, while a 0 will force all commands to be logged.
Slowlog-log-slower-than 10000
32. There is no limit to this length, just to consume memory. Can recycle memory by slowlog reset
Slowlog-max-len 128
33. Delay monitoring is disabled by default because it is basically not needed, in milliseconds
Latency-monitor-threshold 0
34. redis can notify the pub/Sub client about the occurrence of the key space. For example, if the key space event notification is turned on, and the client executes the del command on the key foo of the database No. 0, two messages will be posted through the Pub/Sub. :
[email protected]_:foo del
[email protected]_:del foo
You can select the type of event that redis will notify in the table below. The event type is indicated by a single character:
K key space notification, prefixed with [email protected]<db>_
E key event notification, prefixed with [email protected]<db>_
Notification of g DEL, EXPIRE, RENAME and other types of unrelated general commands. . .
$ String command
l List command
s Set command
h Hash command
z ordered collection command
x expired event (generated each time the key expires)
e eviction event (when the key is generated when the memory is full)
A g$lshzce alias, so "AKE" means all events
Notify-keyspace-events takes a string of 0 or more characters. An empty string means that the notification is disabled. Example: Enable list and general event notification: notify-keyspace-events Elg Example 2: To get the expired key notification subscription to the channel named [email protected]_:expired, use the following configuration notify-keyspace-events Ex default notification It is disabled because the user usually does not need this feature and the feature has a loss of performance. Note that the notifications used are disabled because the user usually does not need this feature and the feature has a loss of performance. Note that if you do not specify at least one of k or E, no events will be sent.
Notify-keyspace-events ""
35. When there is a large amount of data, it is suitable to use hash coding (this will require more memory), and the upper limit of the number of elements cannot exceed the given limit. The redis hash is a hashmap inside the value. If the number of members of the map is small, the map will be stored in a compact format similar to one-dimensional linearity, which saves the memory overhead of a large number of pointers. Exceeding the set value will be converted into a real hashmap. When the value of this map does not exceed more than one member, it will be stored in a linear compact format. The default is 64, that is, there are 64 members inside the value that use linear compact storage. Automatically convert to the current hashmap
Hash-max-ziplist-entries 512
36. When the value of each member inside the value map does not exceed the number of bytes, linear compact storage is used to save space.
Hash-max-ziplist-value 64
37. Similar to the hash-max-zipmap-entries hash, if there are fewer data elements, you can use another way to encode and save a lot of space. List data type how many nodes will use the compact storage format of the pointer
List-max-ziplist-entries 512
38.list data type node value is smaller than how many bytes will be in compact format
List-max-ziplist-value 64
39. There is also the case of such a special encoding: the data is all a string of 64-bit unsigned integer numbers. The following configuration is used to limit the maximum limit for using this encoding in this case.
Set-max-intset-entries 512
40. Similar to the first and second cases, the ordered sequence can be processed in a special coding manner, which saves a lot of space. This encoding is only suitable for ordered sequences whose length and elements meet the following restrictions:
Zset-max-ziplist-entries 128
Zset-max-ziplist-value 64
41. Regarding the Hyperloglogs sparse representation limit setting, if its value is greater than 16000, it still uses a dense representation, because densely indicates that the memory can be used more effectively, the recommended value is 3000.
Hll-sparse-max-bytes 3000
42. Hash refresh, which takes 1 millisecond every 100 CPU milliseconds to refresh Redis' main hash table (top-level key-value mapping table). The hash table implementation used by redis (see dict.c) uses a delayed hash refresh mechanism: the more you operate on a hash table, the more frequent the hash refresh operation; conversely, if the server is very inactive then use the point Save the hash table in memory. The default is to perform a hash table refresh 10 times per second to refresh the dictionary and then free up memory as soon as possible. Recommendation: If you care about the delay, use "activerehashing no", and it is not good to delay each request by 2 milliseconds. Set if you don't care too much about the delay and want to free up memory as soon as possible
Activerehashing yes
43. The client's output buffer limit can be used to force disconnection of clients that are not fast enough to read data from the server for some reason (a common reason is that a publish/subscribe client can't consume messages at the same speed Catch up with the speed at which they are produced. You can set different limits for three different clients: normal -> normal client slave -> slave and MONITOR client pubsub -> client that subscribes to at least one pubsub channel or pattern, below Is the syntax for each client-output-buffer-limit:
Client-output-buffer-limit <class><hard limit> <soft limit> <soft seconds> Once the hard limit is reached, the client is immediately disconnected, or the soft limit is reached and continues for the specified number of seconds (continuous). For example, if the hard limit is 32 megabytes and the soft limit is 16 megabytes/10 seconds, the client will immediately disconnect if the output buffer size reaches 32 megabytes, or the client reaches 16 megabytes and If the limit exceeds the limit for 10 seconds, the connection will be disconnected. The default normal client does not impose restrictions because they do not receive data when they are not actively requesting (in push mode), and only asynchronous clients may appear to request data faster than it can read. The pubsub and slave clients have a default value because the subscriber and slaves receive data in a push, setting the hard and soft limits to 0 to disable the feature.
Client-output-buffer-limit normal 0 0 0
Client-output-buffer-limit slave 256mb 64mb 60
Client-output-buffer-limit pubsub 32mb 8mb 60
44.redis calls internal functions to perform many background tasks, such as closing the client's timeout connection, clearing expired keys that have not been requested, and so on. Not all tasks are executed at the same frequency, but redis performs the inspection task according to the specified "hz" value. By default, "hz" is set to 10. Increasing this value will use more CPUs when redis is idle, but at the same time when multiple keys expire at the same time, Redis will be more responsive and timeouts can be More precise processing. The range is between 1 and 500, but values above 100 are usually not a good idea. Most users should use the default value of 10, which is only necessary to increase to 100 at very low latency requirements.
Hz 10
45. When a child process rewrites the AOF file, if the following options are enabled, the file will not be generated if 32M data is generated. This instruction is very useful for incremental writes to the hard disk and avoiding large delay peaks.
Aof-rewrite-incremental-fsync yes 

This article is from the "Dream It Man" blog, please be sure to keep this source http://runningyongboy.blog.51cto.com/8234857/1837269





Redis Primer Basics and Deployment Configuration explanations