About Redis
Redis is fully open source free and is a high-performance Key-value database.
Redis and other Key-value cache products have the following three features:
Redis supports data persistence, which saves data in memory on disk and can be loaded again for use when restarting.
Redis not only supports simple key-value types of data, but also provides storage of data structures such as List,set,zset,hash.
Redis supports backup of data, that is, Master-slave mode of data backup.
The performance is very high –redis can read the speed is 110,000 times/s, the write speed is 81,000 times/s.
Installation of Redis
wget http://download.redis.io/releases/redis-3.2.4.tar.gz
TAR–ZXVF redis-stable.tar.gz Decompression
Make compilation
Problems may occur make[3]: Gcc:command not found, GCC compilation tool not installed
Once the compilation is successful, go to the SRC folder and perform the make install for Redis installation
Make install-to/usr/local/bin directory: Redis-server,redis-cli,redis-check-aof,redis-check-dump
REDIS-SERVER–V Check if the installation is successful
Start Redis, go to the redis.conf directory, execute
Redis-server redis.conf. If not followed by redis.conf, start with the default configuration
Redis configuration file
REDIS.CONF configuration items are described below:
1. Redis is not running as a daemon by default and can be modified by this configuration item, enabling the daemon with Yes
daemonize Yes
2. When Redis is running as a daemon, Redis writes the PID to the/var/run/redis.pid file by default and can be specified by pidfile
Pidfile/var/run/redis.pid
3. Specify the Redis listening port, the default port is 6379, the author in his own article blog post explains why 6379 as the default port, because 6379 Merz the corresponding number on the phone keypad, and Merz from Italy showgirl Alessia Merz name
Port 6379
4. The host address of the binding
bind 127.0.0.1 this Ip to be set up as your server's Ip
5. If the client closes the connection after a long period of inactivity, if it is specified as 0, the function is turned off
Timeout
6. Specify logging level, Redis supports four levels in total: Debug, verbose, notice, warning, default is verbose
loglevel verbose
7. Logging mode, default to standard output, if Redis is configured to run as daemon, and this is configured as logging mode as standard output, the log will be sent to/dev/null
logfile stdout
8. Set the number of databases, the default database is 0, you can use the Select <dbid> command to specify the database ID on the connection
databases
9. Specify how many times the update operation will synchronize the data to the data file and can be combined with multiple conditions
Save <seconds> <changes>
There are three conditions available in the Redis default configuration file:
Save 1
Save
Save 10000
Represents 1 changes in 900 seconds (15 minutes), 5 changes in 300 seconds (10 minutes), and 60 changes in 10,000 seconds.
10. Specify whether to compress the data when storing to the local database, the default is Yes,redis with LZF compression, if you want to save CPU time, you can turn off this option, but it will cause the database file to become huge
rdbcompression Yes
11. Specify the local database file name, the default value is Dump.rdb
dbfilename Dump.rdb
12. Specify the local database to hold the directory
dir./
13. Set the IP address and port of the master service when this machine is a Slav service, and it will automatically synchronize data from master when Redis boots
slaveof <masterip> <masterport>
14. When the master service is password protected, the Slav service connects the password of master
Masterauth <master-password>
15. Set up the Redis connection password, if the connection password is configured, the client will need to provide the password via auth <password> command when connecting to Redis, turn off by default
Requirepass foobared
16. Set the maximum number of client connections at the same time, the default is unlimited, Redis can open the number of client connections for the Redis process can open the maximum number of file descriptors, if set maxclients 0, indicating no restrictions. When the number of client connections reaches the limit, Redis closes the new connection and returns the max number of clients reached error message to the client
maxclients
17. Specify the maximum Redis memory limit, Redis will load data into memory when it is started, Redis will try to clear expired or expiring key first, when this method processing, still reach the maximum memory setting, will no longer write operation, but still can read operation. Redis new VM mechanism will store key memory, value will be stored in swap area
maxmemory <bytes>
18. Specifies whether logging occurs after each update operation, which, by default, writes data to disk asynchronously and, if not turned on, may result in data loss over a period of time when power is lost. Because the Redis itself synchronizes data files in sync with the save conditions above, some data will only exist in memory for a period of time. Default is No
appendonly No
19. Specify the update log file name, default to Appendonly.aof
appendfilename appendonly.aof
20. Specify the update log condition, a total of 3 optional values:
No: Indicates that the data cache of the operating system is synchronized to disk (fast)
always: Represents a manual call to Fsync () to write data to disk (slow, secure) after each update operation
everysec: Indicates synchronization once per second (trade-offs, default values)
appendfsynceverysec
21. Specify whether to enable the virtual memory mechanism, the default value is no, a simple introduction, the VM mechanism to store data paging, by Redis will be less accessible pages, such as cold data swap to disk, Access to multiple pages is automatically swapped out into memory by the disk (in a later article I will carefully analyze the Redis VM mechanism)
Vm-enabledno
22. Virtual memory file path, default value is/tmp/redis.swap, cannot be shared by multiple Redis instances
Vm-swap-file/tmp/redis.swap
23. Store all data greater than vm-max-memory in virtual memory, regardless of the vm-max-memory settings, all index data is memory stored (REDIS index data is keys), that is, when the vm-max-memory is set to 0, In fact, all value is present on disk. The default value is 0
vm-max-memory0
24.Redis swap file is divided into a lot of page, an object can be saved on more than one page, but a page can not be shared by multiple objects, Vm-page-size is based on the size of the stored data to set, the author suggests that if you store many small objects, The page size is best set to 32 or 64bytes, and if you store large objects, you can use a larger page, and if you are unsure, use the default values
vm-page-size32
25. Set the number of pages in the swap file, since the page table (a bitmap that indicates that the page is idle or used) is in memory and consumes 1byte of memory per 8 pages on disk.
vm-pages134217728
26. Set the number of threads to access swap files, preferably do not exceed the machine's core number, if set to 0, then all the swap file operation is serial, may cause a relatively long delay. The default value is 4
VM-MAX-THREADS4
27. Set when replying to the client, whether to merge the smaller package into one package send, the default is to turn on
Glueoutputbufyes
28. Specifies that a special hashing algorithm is used when more than a certain number or maximum element exceeds a critical value
Hash-max-zipmap-entries64
Hash-max-zipmap-value
29. Specify whether to activate the reset hash, which is on by default (described later in the introduction of the Redis hashing algorithm)
Activerehashingyes
30. Specify other profiles that can use the same configuration file across multiple Redis instances on the same host, while each instance has its own specific configuration file
include/path/to/local.conf
Redis data types
Redis supports five types of data: string (String), hash (hash), list, set (set), and Zset (sortedset: Ordered set).
String:
Get, set. Note: a key can store up to 512MB.
Hash:
Hmset, Hmget, Hkeys, hvals
List:
Lpush, Lrange
Set
Sadd, Smembers
Zset
Zadd Collection Name Serial Number Collection Elements
Zadd Myzset 1 ZLH
Zadd MySet 2 Tom 3 Jim
Zcard Myzset View the number of members of the Zset collection
Zrange Mzyset 0-1
Zrange Mzyset 0-1 Withscores
Zrankmzyset Jim Gets the subscript position of the Zset member
Zcountmyset 1 3
Zrem Myzset ZLH To delete a specified member or multiple members
Zscore Myzset ZLH
Zincrby Myzset 4 Tom Tom's score value plus 4
Zrangebyscore Myzset 1 5
Redis Publish Subscriptions
A Redis Publish Subscription (PUB/SUB) is a message communication pattern: the Sender (pub) sends a message and the Subscriber (sub) receives the message.
Subscribe DONGNAO Client Subscription message, Dongnao for the appropriate channel
Publish Dongnao message is published, and clients subscribing to the channel receive the message
Redis transactions
Redis transactions can execute multiple commands at once, with the following two important guarantees:
A transaction is a separate quarantine operation: All commands in a transaction are serialized and executed sequentially. The transaction is not interrupted by a command request sent by another client during execution.
A transaction is an atomic operation: the commands in a transaction are either all executed or none of them are executed.
A transaction goes through the following three stages from start to execution:
Begins a transaction.
command to queue.
Executes a transaction.
MULTI Start a thing
Set Dongnao Jack
Get Dongnao
Hmset map Key1 value1 Key2 value2
Hmget Map Key1
Exec executes things
Redis Data Backup and recovery
By default, the Redis server program automatically iterates through the database every once in a while, writing memory snapshots in a file called "Dump.rdb", a persistent mechanism called snapshot. With snapshot, if the server goes down and the Redis server program restarts, Redis automatically loads the DUMP.RDB and restores the database state to the last time it was snapshot.
To manually make a memory backup:
Save
Redis Security
Configget Requirepass Get the password
ConfigSet Requirepass 123
Configget Requirepass
Auth Password Password verification
Redis Persistence
snapshotting
Snapshot Persistence
Rdb
Snapshots are the default persistence mode. This is the way in which the in-memory data is written to the binary in a snapshot, the default file name is Dump.rdb. You can automatically make snapshot persistence by configuring settings. We can configure Redis to take snapshots automatically if more than M key is modified in n seconds, the following is the default snapshot save configuration:
Save 1 #900秒内如果超过1个key被修改 to initiate a snapshot save
Save #300秒内容如超过10个key被修改, the snapshot save is initiated
Save 60 10000
The client can also use the Save or Bgsave command to notify Redis to do a snapshot persistence, and each snapshot persistence is to write the memory data to disk one at a time, not the incremental synchronization of only dirty data. If the amount of data is large, and more write operations, it will inevitably cause a large number of disk IO operations, may seriously affect performance. In addition, because the snapshot is done at a certain interval, if Redis is accidentally down, all changes after the last snapshot will be lost.
append-only
AOF
Redis appends each received write command to the file via the Write function (default is appendonly.aof). When Redis restarts, the contents of the entire database are rebuilt in memory by re-executing the write commands saved in the file.
Appendonlyyes #启用aof持久化方式
# Appendfsync always #每次收到写命令就立即强制写入磁盘, slowest, but guaranteed full persistence, not recommended
Appendfsync everysec #每秒钟强制写入磁盘一次, a good compromise in performance and durability, recommended
# Appendfsync No #完全依赖os, best performance, no guarantee of durability
Redis Master-Slave
Redis master-slave replication process: When slave is configured, slave establishes a connection with master and sends the sync command. Whether it is the first connection or reconnection, master initiates a background process, saves the DB snapshot to a file, and the master master process starts collecting new write commands and caches. After the background process finishes writing the file, master sends the file to Slave,slave to save the file to the hard disk, then loads it into memory, then master forwards the cached command to the slave, and the subsequent master sends the received write command to slave. If master receives multiple slave simultaneous connection commands at the same time, master initiates only one process to write database mirroring and then sends it to all slave. Master synchronizes data with non-blocking and can receive read and write requests from users. However, on the slave side is blocking mode, slave is not able to respond to client queries when synchronizing master data.
You can disable data persistence in master, just comment out all the save configurations in the master configuration file and configure data persistence on slave only
Redis Cluster
There are 16,384 hash slots built into the Redis cluster, and when a key-value is required to be placed in a Redis cluster, Redis first calculates a result using the CRC16 algorithm for key and then the result to 16384 for the remainder so that each key corresponds to a number 0 -16383 hash slots, Redis maps the hash slots to different nodes roughly equal to the number of nodes. Each node of the cluster is responsible for a portion of the hash slot. This structure makes it easy to add or remove nodes, and either adding or modifying a node does not cause the cluster to be unusable. The advantage of using hash slots is that you can easily add or remove nodes. When you need to add nodes, you just need to move some of the other node's hash slots to the new node; When you need to remove a node, just move the hash slot on the removed node to the other node, and at this point, we don't have to stop all redis services when we add or remove nodes later. Hash Slots
CentOS-based clustering
1, the cluster must have at least 3 nodes, each node has a master-slave structure
Create a new three folder, put the configuration file CP in, then modify the configuration file port, release the support cluster configuration, start the node
cluster-enabled Yes
Cluster-config-file nodes.conf
Cluster-node-timeout 5000
2, install Ruby, must be more than 2.0, more than 2.0 comes with RubyGems Package Manager
Yuminstall Ruby
3, Gem command install Redis package, add REDIS-TRIB.RB call Redis interface Package
Geminstall Redis
4. Manage Redis clusters with REDIS-TRIB.RB cluster tools
./redis-trib.rb Create--replicas 1 120.77.22.187:7000120.77.22.187:8000 120.77.22.187:9000 120.77.22.187:7010 120.77.22.187:8010120.77.22.187:9010
--REPLICAS1 setting the number of nodes from
Enter cluster mode to operate Redis command,-C cluster mode
Redis-cli-c-h–p
Redis Cluster Add node
The main thing is to see the phenomenon, slots transfer, the corresponding data will change?
Adding nodes
./redis-trib.rb Add-node 127.0.0.1:7006 127.0.0.1:7000
View node Status
Cluster nodes
Move hash slots, hash slot reallocation
./redis-trib.rb Reshard 127.0.0.1:7000
Enter all to randomly transfer from all the master nodes to get enough XX hash slots
Redis Cluster Delete node
Turn the hash slots out and go to the other master node
./redis-trib.rb Reshard 127.0.0.1:7000
Then delete the node
./redis-trib.rb Del-node 120.77.22.187:902094ee1415a16076f4070c5dd4b94defc14618dbb8
Redis-sentinel High Availability
Redis-sentinel is a highly available (HA) solution that is officially recommended by Redis, and if Master is down, the Redis itself (including many of its clients) does not implement automatic primary and standby switching when using Redis for master-slave high-availability scenarios. The Redis-sentinel itself is also a standalone process that monitors multiple Master-slave clusters and discovers that the master can be switched on automatically after the outage.
Its main function has the following points
Periodically monitor whether Redis works as expected;
If a Redis node is found to be running, it can notify another process (such as its client);
Ability to switch automatically. When a master node is unavailable, it is possible to elect one of the master's multiple slave (if there are more than one slave) as the new master, The other slave node changes the address of the master that it follows to the new address of the slave that is promoted to master.
It is important to note that the configuration file is dynamically modified during Sentinel operation, for example, when a primary standby switch occurs, the master in the configuration file is modified to another slave. This allows Sentinel to restore the status of the Redis cluster that was previously monitored by this configuration when it restarts.
Sentinel Monitor MyMaster 127.0.0.1 6379 1
Sentinel Down-after-milliseconds MyMaster 60000
#sentinel Can-failover MyMaster Yes
Sentinel Failover-timeout MyMaster 180000
Sentinel Parallel-syncs MyMaster 1
When Sentinel is clustered, the solution to this problem becomes simple, requiring more than one sentinel to communicate with each other to confirm that a master is really dead, and this 2 represents that when 2 Sentinel in the cluster thinks Master is dead, To really think that the master is no longer available.
Down-after-milliseconds
Sentinel sends a heartbeat ping to master to confirm that Master is alive, and if Master does not respond to pong within a "certain timeframe" or if it responds to an error message, then this Sentinel will subjectively (unilaterally) Think this master is no longer available (subjectively down, also referred to as Sdown). And this down-after-milliseconds is used to specify this "time range", in milliseconds.
Can-failover
No indicates that the current Sentinel is an observer and only participates in voting not to implement failover
At least one of the global is Yes
Parallel-syncs
When the new master is generated, the number of slave "slaveof" to the new master and "SYNC"
Default is 1, we recommend keeping the default value
Client requests are terminated when Salve performs salveof and synchronization
Failover-timeout
Failover expiration time, when failover starts, no failover action is triggered during this time
Current Sentinel will consider this failoer to be a failure
Looking at the queue technology in Redis cluster mode with publish/subscribe function (i)