The data writeback mechanism of Redis is divided into two types, synchronous and asynchronous,
Personal feeling Method 2 the way to fork the main process is clumsy, but it seems to be the only way. Hot data in memory can be modified at any time, and the memory image to be saved on disk must be frozen. Freezing will result in suspended animation. Fork a new process is equivalent to replicating a memory image then, so that the main process does not need to freeze, as long as the child process on the operation.
A fork in a small memory process does not require much resources, but when the memory space of the process is in G, fork becomes a horrible operation. Not to mention the process of fork 14G memory on a 16G memory host? It will certainly be reported that memory cannot be allocated. More exasperating is, the more frequent changes in the host fork on the more frequent, the fork of the cost of the operation itself may not be much better than suspended animation.
After you find the reason, directly modify the/etc/sysctl.conf kernel parameter vm.overcommit_memory= 1
Sysctl-p
The Linux kernel determines whether or not to release according to the parameter vm.overcommit_memory parameters.
Resources:
Redis Persistence http://redis.io/topics/persistence
Google Groups https://groups.google.com/forum/?fromgroups=#!forum/redis-db
First, the discussion and understanding of Redis persistence
There are two ways of Redis persistence: RDB and AOF
First, we should be clear about what is the use of persistent data, and the answer is for data recovery after a reboot.
Redis is an in-memory database, either an RDB or a aof, that is just a measure of its data recovery.
So Redis will read the RDB or the aof file and reload it into memory when it recovers with the RDB and aof.
An RDB is a snapshot snapshot store, which is the default persistence method.
Can be understood as semi-persistent mode, which is to save data to disk periodically according to certain policies.
The resulting data file is Dump.rdb, and the cycle of the snapshot is defined by the save parameter in the configuration file.
The following are the default snapshot settings:
save 900 1 #When a key data is changed, refresh to Disk once in 900 seconds save 300 10 #When 10 Keys data is changed, refresh to Disk once every 300 seconds save 60 10000 #When 10,000 Keys data is changed, refresh to Disk once every 60 seconds
The Redis Rdb file does not break because its write operation is performed in a new process.
When a new Rdb file is generated, the Redis-generated subprocess writes the data to a temporary file, and then renames the temporary file to an Rdb file by means of an atomic rename system call.
In this way, Redis's RDB files are always available whenever a failure occurs.
At the same time, Redis's Rdb file is also a part of the Redis master-slave synchronization implementation.
The first time slave is synchronized to master is:
Slave the sync request to master, master dumps the Rdb file, then transfers the Rdb file to slave, then master forwards the cached command to the slave, and the first synchronization is complete.
The second and subsequent synchronization implementations are:
Master sends a snapshot of the variable directly to each slave in real time.
However, no matter what causes slave and master to disconnect, the process of repeating the above two steps will be repeated.
The master-slave replication of Redis is based on the persistence of memory snapshots, as long as there is slave there will be memory snapshots.
Obviously, the RDB has its shortcomings, that is, once the database has a problem, then the data stored in our Rdb file is not entirely new.
All the data from the last Rdb file generation to Redis outage was lost.
AOF (append-only File) is more persistent than the RDB approach.
Because when using aof persistence, Redis appends each received write command to the file via the Write function, similar to MySQL's binlog.
When Redis restarts, the contents of the entire database are rebuilt in memory by re-executing the write commands that are saved in the file.
The corresponding setting parameters are:
$ vim/opt/redis/etc/redis_6379.conf
appendonly yes #Enable AOF persistence appendfilename appendonly.aof #AOF file name, default is appendonly.aof # appendfsync always #Forcibly write to disk every time a write command is received. This is the most guaranteed complete persistence, but it is also the slowest. appendfsync everysec #Force to write to disk once per second, a good compromise in performance and durability, is the recommended way. # appendfsync no #It is completely dependent on the writing of the OS, generally about once every 30 seconds, the best performance but the least guarantee of persistence, is not recommended.
The full persistence of aof also poses another problem, and the persistence file becomes bigger and larger.
For example, we call the INCR Test command 100 times, the file must save all 100 commands, but in fact, 99 is redundant.
Because you want to restore the state of the database, it is enough to save a set test 100 in the file.
To compress aof persistent files, Redis provides the bgrewriteaof command.
When this command is received, Redis will use a snapshot-like method to save the data in memory to a temporary file in the form of a command, and finally replace the original file to achieve control over the growth of the aof file.
Because it is the process of simulating a snapshot, the old aof file is not read while overriding the AoF file, but instead the entire in-memory database content is rewritten with a command to a new aof file.
The corresponding setting parameters are:
$ vim/opt/redis/etc/redis_6379.conf
no-appendfsync-on-rewrite yes #When the log is rewritten, no command append operation is performed, but it is only placed in the buffer to avoid conflicts with DISK IO caused by the append of the command. auto-aof-rewrite-percentage 100 #When the current AOF file size is twice the size of the AOF file obtained during the last log rewrite, a new log rewrite process is automatically started. auto-aof-rewrite-min-size 64mb #The minimum value of the current AOF file to start a new log rewrite process, to avoid frequent rewrites due to the small file size when Reids is just started.
What do you choose? Here are some suggestions from the official:
In general, if you want to provide high data supportability, it is recommended that you use both persistence methods at the same time.
If you can accept a few minutes of data loss from a disaster, you can use only an RDB.
Many users use aof only, but we recommend that you use an RDB as well, since the RDB can take a full snapshot of the data at times and provide a faster restart.
Therefore, we hope to unify aof and RDB into a persistence model in the future (long-term plan).
In terms of data recovery:
The RDB has a shorter startup time for two reasons:
One is that each data in an RDB file has only one record, and does not have a record of multiple operations of the data, as in the case of aof logs. So every piece of data just needs to be written once.
Another reason is that the Rdb file storage format and the Redis data in memory encoding format is consistent, do not need to do data encoding work, so the CPU consumption is much smaller than the AOF log load.
Second, disaster recovery simulation
Since persistent data is used for data recovery after a reboot, it is very necessary to perform such a disaster recovery simulation.
It is said that if the data is to be persisted and stability is to be ensured, half of the physical memory is recommended to be left blank. Because at the time of the snapshot, the sub-processes that fork out the dump operation will occupy the same memory as the parent process, the real copy-on-write, the performance impact and the memory consumption are relatively large.
At present, the usual design idea is to use the replication mechanism to compensate for aof, snapshot performance deficiencies, to achieve data persistence.
That is, master on snapshot and aof do not do, to ensure that master read and write performance, and slave on the same time to open snapshot and aof to persist, to ensure the security of data.
First, modify the following configuration on master:
$ sudo vim/opt/redis/etc/redis_6379.conf
#save 900 1 #Disable Snapshot #save 300 10 #save 60 10000 appendonly no #Disable AOF
Next, modify the following configuration on the slave:
$ sudo vim/opt/redis/etc/redis_6379.conf
save 900 1 #Enable Snapshot save 300 10 save 60 10000 appendonly yes #Enable AOF appendfilename appendonly.aof #the name of the AOF file # appendfsync always appendfsync everysec #Force to disk once per second # appendfsync no no-appendfsync-on-rewrite yes #No command append operation during log rewrite auto-aof-rewrite-percentage 100 #Automatically start a new log rewrite process auto-aof-rewrite-min-size 64mb #Minimum value to start a new log rewrite process
Start master and slave, respectively
$/etc/init.d/redis Start
Confirm that the snapshot parameter is not started in master after boot is complete
Redis 127.0.0.1:6379> CONFIG GET Save
1) "Save"
2) ""
Then generate 250,000 data in master with the following script:
[Email protected]:/opt/redis/data/6379$ cat redis-cli-generate.temp.sh
#! / bin / bash REDISCLI = "redis-cli -a slavepass -n 1 SET" ID = 1 while (($ ID <50001)) do INSTANCE_NAME = "i-2- $ ID-VM" UUID = `cat / proc / sys / kernel / random / uuid` PRIVATE_IP_ADDRESS = 10.`echo "$ RANDOM% 255 + 1" | bc`.`echo "$ RANDOM% 255 + 1" | bc`.`echo "$ RANDOM% 255 + 1" | bc` CREATED = `date" +% Y-% m-% d% H:% M:% S "` $ REDISCLI vm_instance: $ ID: instance_name "$ INSTANCE_NAME" $ REDISCLI vm_instance: $ ID: uuid "$ UUID" $ REDISCLI vm_instance: $ ID: private_ip_address "$ PRIVATE_IP_ADDRESS" $ REDISCLI vm_instance: $ ID: created "$ CREATED" $ REDISCLI vm_instance: $ INSTANCE_NAME: id "$ ID" ID = $ (($ ID + 1)) done
[Email protected]:/opt/redis/data/6379$./redis-cli-generate.temp.sh
In the process of generating data, it is clear to see that master only creates the Dump.rdb file on the first slave synchronization, and then gives slave by the way of incremental transfer of commands.
Dump.rdb file no longer grows.
[Email protected]:/opt/redis/data/6379$ LS-LH
Total 4.0K
-rw-r--r--1 root root Sep 00:40 Dump.rdb
On the slave, you can see that dump.rdb files and aof files are growing, and the aof file growth rate is significantly larger than the Dump.rdb file.
[Email protected]:/opt/redis/data/6379$ LS-LH
Total 24M
-rw-r--r--1 root root 15M Sep 12:06 appendonly.aof
-rw-r--r--1 root root 9.2M Sep 12:06 Dump.rdb
After the data is inserted, the current amount of data is acknowledged first.
Redis 127.0.0.1:6379> Info
redis_version:2.4.17 redis_git_sha1:00000000 redis_git_dirty:0 arch_bits:64 multiplexing_api:epoll gcc_version:4.4.5 process_id:27623 run_id:e00757f7b2d6885fa9811540df9dfed39430b642 uptime_in_seconds:1541 uptime_in_days:0 lru_clock:650187 used_cpu_sys:69.28 used_cpu_user:7.67 used_cpu_sys_children:0.00 used_cpu_user_children:0.00 connected_clients:1 connected_slaves:1 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0 used_memory:33055824 used_memory_human:31.52M used_memory_rss:34717696 used_memory_peak:33055800 used_memory_peak_human:31.52M mem_fragmentation_ratio:1.05 mem_allocator:jemalloc-3.0.0 loading:0 aof_enabled:0 changes_since_last_save:250000 bgsave_in_progress:0 last_save_time:1348677645 bgrewriteaof_in_progress:0 total_connections_received:250007 total_commands_processed:750019 expired_keys:0 evicted_keys:0 keyspace_hits:0 keyspace_misses:0 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:246 vm_enabled:0 role:master slave0:10.6.1.144,6379,online db1:keys=250000,expires=0
The current amount of data is 250,000 key, which consumes 31.52M of memory.
Then we kill the master Redis process directly, simulating the disaster.
[Email protected]:/opt/redis/data/6379$ sudo killall-9 redis-server
Let's check the status in slave:
Redis 127.0.0.1:6379> Info
redis_version:2.4.17 redis_git_sha1:00000000 redis_git_dirty:0 arch_bits:64 multiplexing_api:epoll gcc_version:4.4.5 process_id:13003 run_id:9b8b398fc63a26d160bf58df90cf437acce1d364 uptime_in_seconds:1627 uptime_in_days:0 lru_clock:654181 used_cpu_sys:29.69 used_cpu_user:1.21 used_cpu_sys_children:1.70 used_cpu_user_children:1.23 connected_clients:1 connected_slaves:0 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0 used_memory:33047696 used_memory_human:31.52M used_memory_rss:34775040 used_memory_peak:33064400 used_memory_peak_human:31.53M mem_fragmentation_ratio:1.05 mem_allocator:jemalloc-3.0.0 loading:0 aof_enabled:1 changes_since_last_save:3308 bgsave_in_progress:0 last_save_time:1348718951 bgrewriteaof_in_progress:0 total_connections_received:4 total_commands_processed:250308 expired_keys:0 evicted_keys:0 keyspace_hits:0 keyspace_misses:0 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:694 vm_enabled:0 role:slave aof_current_size:17908619 aof_base_size:16787337 aof_pending_rewrite:0 aof_buffer_length:0 aof_pending_bio_fsync:0 master_host:10.6.1.143 master_port:6379 master_link_status:down master_last_io_seconds_ago:-1 master_sync_in_progress:0 master_link_down_since_seconds:25 slave_priority:100 db1:keys=250000,expires=0
You can see that the status of Master_link_status is down and master is inaccessible.
At this point, the slave still works well and retains aof and RDB files.
Below we will restore the data on master through the saved aof and Rdb files on the slave.
First, the synchronization state on the slave is canceled to prevent the main library from restarting before the incomplete data is restored, thus directly overwriting the data from the library, resulting in all data loss.
Redis 127.0.0.1:6379> slaveof NO One
Ok
Confirm that there is no master-related configuration information:
Redis 127.0.0.1:6379> INFO
redis_version:2.4.17 redis_git_sha1:00000000 redis_git_dirty:0 arch_bits:64 multiplexing_api:epoll gcc_version:4.4.5 process_id:13003 run_id:9b8b398fc63a26d160bf58df90cf437acce1d364 uptime_in_seconds:1961 uptime_in_days:0 lru_clock:654215 used_cpu_sys:29.98 used_cpu_user:1.22 used_cpu_sys_children:1.76 used_cpu_user_children:1.42 connected_clients:1 connected_slaves:0 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0 used_memory:33047696 used_memory_human:31.52M used_memory_rss:34779136 used_memory_peak:33064400 used_memory_peak_human:31.53M mem_fragmentation_ratio:1.05 mem_allocator:jemalloc-3.0.0 loading:0 aof_enabled:1 changes_since_last_save:0 bgsave_in_progress:0 last_save_time:1348719252 bgrewriteaof_in_progress:0 total_connections_received:4 total_commands_processed:250311 expired_keys:0 evicted_keys:0 keyspace_hits:0 keyspace_misses:0 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:1119 vm_enabled:0 role:master aof_current_size:17908619 aof_base_size:16787337 aof_pending_rewrite:0 aof_buffer_length:0 aof_pending_bio_fsync:0 db1:keys=250000,expires=0
To copy data files on slave:
[Email protected]:/opt/redis/data/6379$ Tar Cvf/home/dongguo/data.tar *
Appendonly.aof
Dump.rdb
Upload the Data.tar to master and try to recover the data:
You can see that the master directory has an initialized slave data file, which is small and is deleted.
[Email protected]:/opt/redis/data/6379$ ls-l
Total 4
-rw-r--r--1 root root Sep 00:40 Dump.rdb
[Email protected]:/opt/redis/data/6379$ sudo rm-f dump.rdb
Then unzip the data file:
[Email protected]:/opt/redis/data/6379$ sudo tar xf/home/dongguo/data.tar
[Email protected]:/opt/redis/data/6379$ LS-LH
Total 29M
-rw-r--r--1 root root 18M Sep 01:22 appendonly.aof
-rw-r--r--1 root root 12M Sep 01:22 Dump.rdb
Start Redis on master;
[Email protected]:/opt/redis/data/6379$ sudo/etc/init.d/redis Start
Starting Redis Server ...
To see if the data is restored:
Redis 127.0.0.1:6379> INFO
redis_version:2.4.17 redis_git_sha1:00000000 redis_git_dirty:0 arch_bits:64 multiplexing_api:epoll gcc_version:4.4.5 process_id:16959 run_id:6e5ba6c053583414e75353b283597ea404494926 uptime_in_seconds:22 uptime_in_days:0 lru_clock:650292 used_cpu_sys:0.18 used_cpu_user:0.20 used_cpu_sys_children:0.00 used_cpu_user_children:0.00 connected_clients:1 connected_slaves:0 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0 used_memory:33047216 used_memory_human:31.52M used_memory_rss:34623488 used_memory_peak:33047192 used_memory_peak_human:31.52M mem_fragmentation_ratio:1.05 mem_allocator:jemalloc-3.0.0 loading:0 aof_enabled:0 changes_since_last_save:0 bgsave_in_progress:0 last_save_time:1348680180 bgrewriteaof_in_progress:0 total_connections_received:1 total_commands_processed:1 expired_keys:0 evicted_keys:0 keyspace_hits:0 keyspace_misses:0 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:0 vm_enabled:0 role:master db1:keys=250000,expires=0
You can see that 250,000 of the data has been fully restored to master.
At this point, you can safely restore the synchronization settings of the slave.
Redis 127.0.0.1:6379> slaveof 10.6.1.143 6379
Ok
To view the synchronization status:
Redis 127.0.0.1:6379> INFO
redis_version:2.4.17 redis_git_sha1:00000000 redis_git_dirty:0 arch_bits:64 multiplexing_api:epoll gcc_version:4.4.5 process_id:13003 run_id:9b8b398fc63a26d160bf58df90cf437acce1d364 uptime_in_seconds:2652 uptime_in_days:0 lru_clock:654284 used_cpu_sys:30.01 used_cpu_user:2.12 used_cpu_sys_children:1.76 used_cpu_user_children:1.42 connected_clients:2 connected_slaves:0 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0 used_memory:33056288 used_memory_human:31.52M used_memory_rss:34766848 used_memory_peak:33064400 used_memory_peak_human:31.53M mem_fragmentation_ratio:1.05 mem_allocator:jemalloc-3.0.0 loading:0 aof_enabled:1 changes_since_last_save:0 bgsave_in_progress:0 last_save_time:1348719252 bgrewriteaof_in_progress:1 total_connections_received:6 total_commands_processed:250313 expired_keys:0 evicted_keys:0 keyspace_hits:0 keyspace_misses:0 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:12217 vm_enabled:0 role:slave aof_current_size:17908619 aof_base_size:16787337 aof_pending_rewrite:0 aof_buffer_length:0 aof_pending_bio_fsync:0 master_host:10.6.1.143 master_port:6379 master_link_status:up master_last_io_seconds_ago:0 master_sync_in_progress:0 slave_priority:100 db1:keys=250000,expires=0
The master_link_status is displayed as up and the sync status is normal.
In the process of this recovery, we also copied the aof and RDB files, then which one of the files to complete the recovery of data?
In fact, when the Redis server is hung up, the data is restored to memory at the following priority level when restarted:
1. If only configure AOF, load aof file to recover data when reboot;
2. If both RDB and AOF are configured at the same time, startup is only load aof file recovery data;
3. If only the RDB is configured, startup is the load dump file to recover the data.
That is, the priority of the AOF is higher than the RDB, which is well understood, because the AOF itself guarantees the integrity of the data above the RDB.
In this case, we secured the data by enabling the AOF and Rdb on the slave, and restored the master.
However, in our current online environment, because the data are set to expire time, the use of aof is not very practical, too frequent write operations will cause the AOF file to grow to an unusually large, much more than the actual amount of our data, which will also lead to data recovery in a significant amount of time.
Therefore, you can only open snapshot on slave to localize, and you can consider the frequency of Save or call a scheduled task to perform periodic bgsave of snapshot storage to ensure the integrity of the localized data as much as possible.
In such a framework, if only master hangs, slave complete, data recovery can reach 100%.
If master and slave are hung at the same time, data recovery can also be achieved in an acceptable degree.
Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.
Details of the Redis persistence scenario
Start building with 50+ products and up to 12 months usage for Elastic Compute Service
1 on 1 presale consultation
24/7 Technical Support 6 Free Tickets per Quarter Faster Response
Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.
A comprehensive suite of global cloud computing services to power your business
Payment Methods We Support
