高可用Redis(十)：Redis原生命令搭建叢集

標籤：sla   down   data   dump   inf   auto   ec2   received   $2   

1.搭建Redis Cluster主要步驟

1.配置開啟節點2.meet3.指派槽4.主從關係分配

2.環境說明

兩台虛擬機器，IP地址分別為：192.168.81.100和192.168.81.101兩台虛擬機器作業系統均為：CentOS 7.5 64位兩台虛擬機器系統說明：    使用yum方式安裝Redis    關閉firewalld防火牆使用兩台虛擬機器的7000,7001,7002連接埠搭建三主三從的Redis Cluster    其中192.168.81.100主機三個連接埠啟動的Redis Server為主節點    192.168.81.101主機三個連接埠啟動的Redis Server為從節點

3.具體步驟3.1 在192.168.81.100虛擬機器上操作

[[email protected] etc]# cd /opt[[email protected] opt]# mkdir config[[email protected] opt]# cd config[[email protected] config]# vi redis_7000.conf             # 建立7000連接埠運行需要的設定檔，檔案內容如下    port 7000    daemonize yes    dir ‘/var/lib/redis‘    logfile ‘/var/log/redis/redis_7000.log‘    dbfilename ‘redis_7000.data‘    cluster-enabled yes    cluster-config-file nodes-7000.conf    cluster-require-full-coverage no[[email protected] config]# sed ‘s/7000/7001/g‘ redis_7000.conf > redis_7001.conf  # 把redis_7000.conf檔案中的7000替換成7001，並產生redis_7001.conf檔案[[email protected] config]# sed ‘s/7000/7002/g‘ redis_7000.conf > redis_7002.conf  # 把redis_7000.conf檔案中的7000替換成7002，並產生redis_7002.conf檔案[[email protected] config]# redis-server /opt/config/redis_7000.conf               # 指定設定檔，啟動redis server節點[[email protected] config]# redis-server /opt/config/redis_7001.conf               # 指定設定檔，啟動redis server節點            [[email protected] config]# redis-server /opt/config/redis_7002.conf               # 指定設定檔，啟動redis server節點[[email protected] config]# ps aux | grep redis-server                             # 查看redis-server進程運行情況                        root       2444  0.1  0.5 142904  5328 ?        Ssl  19:38   0:00 redis-server *:7000 [cluster]root       2475  0.0  0.5 142904  5328 ?        Ssl  19:39   0:00 redis-server *:7001 [cluster]root       2479  0.2  0.5 142904  5328 ?        Ssl  19:39   0:00 redis-server *:7002 [cluster]root       2483  0.0  0.0 112648   964 pts/0    R+   19:39   0:00 grep --color=auto redis-server

3.2 在192.168.81.101虛擬機器上進行第一步的操作，產生三個設定檔,並啟動Redis server

[[email protected] config]# redis-server /opt/config/redis_7000.conf               # 指定設定檔，啟動redis server節點[[email protected] config]# redis-server /opt/config/redis_7001.conf               # 指定設定檔，啟動redis server節點           [[email protected] config]# redis-server /opt/config/redis_7002.conf               # 指定設定檔，啟動redis server節點[[email protected] config]# ps aux  | grep redis-server                            # 查看redis-server的進程運行情況                        root       1704  0.1  0.2 142916  5344 ?        Ssl  19:41   0:00 redis-server *:7000 [cluster]root       1736  0.0  0.2 142916  5344 ?        Ssl  19:41   0:00 redis-server *:7001 [cluster]root       1740  0.0  0.2 142916  5340 ?        Ssl  19:42   0:00 redis-server *:7002 [cluster]

3.3 在192.168.81.100虛擬機器上查看cluster節點的資訊

[[email protected] config]# redis-cli -p 7000127.0.0.1:7000> set hello world             # 向當前節點執行寫入命令，失敗(error) CLUSTERDOWN Hash slot not served127.0.0.1:7000> cluster nodes               # 查看叢集節點資訊，只有當前節點的資訊，且顯示為主節點d20aa403c5d7d9507adcc4ef6132c14f3c9486ed :7000 myself,master - 0 0 0 connected127.0.0.1:7000> cluster info                # 查看cluster的資訊cluster_state:fail                          # 叢集為失敗狀態cluster_slots_assigned:0cluster_slots_ok:0                          # 當前叢集中有0個slot(槽)cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:1                       # 當前叢集中節點個數為1cluster_size:0cluster_current_epoch:0cluster_my_epoch:0cluster_stats_messages_sent:0cluster_stats_messages_received:0127.0.0.1:7000> exit[[email protected] config]# cd /var/lib/redis/[[email protected] redis]# ls              dump.rdb  nodes-7000.conf  nodes-7001.conf  nodes-7002.conf[[email protected] redis]# cat nodes-7000.conf    # 查看7000連接埠上啟動並執行redis-server的run_id與執行cluster nodes命令得到的run_id相同d20aa403c5d7d9507adcc4ef6132c14f3c9486ed :0 myself,master - 0 0 0 connectedvars currentEpoch 0 lastVoteEpoch 0

3.4 在192.168.81.100上執行meet操作

[[email protected] redis]# redis-cli -p 7000 cluster meet 127.0.0.1 7001       # 通過7000連接埠的redis server向7001連接埠的redis server發送meet操作OK[[email protected] redis]# redis-cli -p 7000 cluster nodes                     # 通過7000連接埠啟動並執行redis server查看cluster的節點資訊，已經添加7001連接埠啟動並執行redis server,且都為master節點d20aa403c5d7d9507adcc4ef6132c14f3c9486ed 127.0.0.1:7000 myself,master - 0 0 0 connected45d79e730593df3216a4236a19c21ff601949449 127.0.0.1:7001 master - 0 1539719395866 1 connected[[email protected] redis]# redis-cli -p 7001 cluster nodes                     # 通過7001連接埠啟動並執行redis server查看cluster的節點資訊，也可以看到7000連接埠啟動並執行redis serverd20aa403c5d7d9507adcc4ef6132c14f3c9486ed 127.0.0.1:7000 master - 0 1539719408423 0 connected45d79e730593df3216a4236a19c21ff601949449 127.0.0.1:7001 myself,master - 0 0 1 connected[[email protected] redis]# redis-cli -p 7002 cluster nodes                     # 通過7002連接埠的redis server查看cluster的節點資訊，7000連接埠和7001連接埠啟動並執行redis server都不在其中，因為7002連接埠啟動並執行redis server沒有meete5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a :7002 myself,master - 0 0 0 connected[[email protected] redis]# redis-cli -p 7000 cluster meet 127.0.0.1 7002       # 向7002連接埠啟動並執行redis server發送meet操作OK[[email protected] redis]# redis-cli -p 7001 cluster nodes                     # 7002連接埠啟動並執行redis server已經添加到叢集中了            e5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 127.0.0.1:7002 master - 0 1539719438785 0 connectedd20aa403c5d7d9507adcc4ef6132c14f3c9486ed 127.0.0.1:7000 master - 0 1539719439689 2 connected45d79e730593df3216a4236a19c21ff601949449 127.0.0.1:7001 myself,master - 0 0 1 connected[[email protected] redis]# redis-cli -p 7002 cluster nodes              45d79e730593df3216a4236a19c21ff601949449 127.0.0.1:7001 master - 0 1539719442308 1 connectedd20aa403c5d7d9507adcc4ef6132c14f3c9486ed 127.0.0.1:7000 master - 0 1539719441301 2 connectede5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 127.0.0.1:7002 myself,master - 0 0 0 connected[[email protected] redis]# redis-cli -p 7000 cluster meet 192.168.81.101 7002  # 向192.168.81.101:7002連接埠啟動並執行redis server執行meet操作              OK[[email protected] redis]# redis-cli -p 7000 cluster meet 192.168.81.101 7000  # 向192.168.81.101:7000連接埠啟動並執行redis server執行meet操作OK[[email protected] redis]# redis-cli -p 7000 cluster meet 192.168.81.101 7001  # 向192.168.81.101:7001連接埠啟動並執行redis server執行meet操作OK[[email protected] redis]# redis-cli -p 7000 cluster nodesc96589b19b8ff5d9d286d470b8efff9a540726d8 192.168.81.101:7000 master - 0 1539719546060 5 connected53911425f4479ce4f054b8b714f4700715f0dc64 192.168.81.101:7002 master - 0 1539719548113 3 connectedbb88f0996503bac3b222b9cc4ec25139ad34ed3a 192.168.81.101:7001 master - 0 1539719547106 4 connected45d79e730593df3216a4236a19c21ff601949449 127.0.0.1:7001 master - 0 1539719544042 1 connectedd20aa403c5d7d9507adcc4ef6132c14f3c9486ed 127.0.0.1:7000 myself,master - 0 0 2 connectede5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 127.0.0.1:7002 master - 0 1539719545050 0 connected[[email protected] redis]# redis-cli -p 7002 cluster nodes                     # 查看叢集中的節點資訊，發現6個node都已經添加到當前叢集中了45d79e730593df3216a4236a19c21ff601949449 127.0.0.1:7001 master - 0 1539719554166 1 connectedd20aa403c5d7d9507adcc4ef6132c14f3c9486ed 127.0.0.1:7000 master - 0 1539719554668 2 connectedbb88f0996503bac3b222b9cc4ec25139ad34ed3a 192.168.81.101:7001 master - 0 1539719555174 4 connected53911425f4479ce4f054b8b714f4700715f0dc64 192.168.81.101:7002 master - 0 1539719556180 3 connectede5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 127.0.0.1:7002 myself,master - 0 0 0 connectedc96589b19b8ff5d9d286d470b8efff9a540726d8 192.168.81.101:7000 master - 0 1539719557188 5 connected[[email protected] redis]# redis-cli -p 7002 cluster info         # 查看叢集相關的資訊cluster_state:fail                  # 叢集狀態為失敗cluster_slots_assigned:0cluster_slots_ok:0cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6               # 叢集中有6個已知節點cluster_size:0cluster_current_epoch:5cluster_my_epoch:0cluster_stats_messages_sent:507cluster_stats_messages_received:507

3.5 在192.168.81.101虛擬機器上查看cluster的資訊

[[email protected] config]# redis-cli -p 7000 cluster nodes        # 查看cluster的節點資訊，6個節點的資訊都顯示出來，且都為master53911425f4479ce4f054b8b714f4700715f0dc64 192.168.81.101:7002 master - 0 1539690828453 3 connected45d79e730593df3216a4236a19c21ff601949449 192.168.81.100:7001 master - 0 1539690825432 1 connectedd20aa403c5d7d9507adcc4ef6132c14f3c9486ed 192.168.81.100:7000 master - 0 1539690826437 2 connectede5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 192.168.81.100:7002 master - 0 1539690827448 0 connectedbb88f0996503bac3b222b9cc4ec25139ad34ed3a 192.168.81.101:7001 master - 0 1539690827950 4 connectedc96589b19b8ff5d9d286d470b8efff9a540726d8 192.168.81.101:7000 myself,master - 0 0 5 connected[[email protected] config]# redis-cli -p 7001 cluster nodes        # 查看cluster的節點資訊，6個節點的資訊都顯示出來，且都為master53911425f4479ce4f054b8b714f4700715f0dc64 127.0.0.1:7002 master - 0 1539690840955 3 connectedc96589b19b8ff5d9d286d470b8efff9a540726d8 192.168.81.101:7000 master - 0 1539690842968 5 connectedbb88f0996503bac3b222b9cc4ec25139ad34ed3a 192.168.81.101:7001 myself,master - 0 0 4 connectede5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 192.168.81.100:7002 master - 0 1539690844979 0 connectedd20aa403c5d7d9507adcc4ef6132c14f3c9486ed 192.168.81.100:7000 master - 0 1539690841960 2 connected45d79e730593df3216a4236a19c21ff601949449 192.168.81.100:7001 master - 0 1539690843977 1 connected[[email protected] config]# redis-cli -p 7002 cluster nodes        # 查看cluster的節點資訊，6個節點的資訊都顯示出來，且都為masterbb88f0996503bac3b222b9cc4ec25139ad34ed3a 127.0.0.1:7001 master - 0 1539690847798 4 connected53911425f4479ce4f054b8b714f4700715f0dc64 192.168.81.101:7002 myself,master - 0 0 3 connectedd20aa403c5d7d9507adcc4ef6132c14f3c9486ed 192.168.81.100:7000 master - 0 1539690846795 2 connectede5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 192.168.81.100:7002 master - 0 1539690843775 0 connectedc96589b19b8ff5d9d286d470b8efff9a540726d8 192.168.81.101:7000 master - 0 1539690844780 5 connected45d79e730593df3216a4236a19c21ff601949449 192.168.81.100:7001 master - 0 1539690845786 1 connected[[email protected] config]# redis-cli -p 7002 cluster info         # 查看cluster的資訊cluster_state:fail              # 叢集狀態為失敗狀態cluster_slots_assigned:0cluster_slots_ok:0cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6           # 叢集中添加了6個節點cluster_size:0                  # 叢集中cluster_current_epoch:5cluster_my_epoch:3cluster_stats_messages_sent:413cluster_stats_messages_received:413

3.6 在192.168.81.100虛擬機器上編寫指令碼，完成分配slot

[[email protected] ~]# cd /opt/config/[[email protected] config]# vi add_slots.sh                    # 編寫add_slots.sh指令碼    #!/bin/bash    start=$1    end=$2    port=$3    for slot in `seq ${start} ${end}`    do        echo "slot:${slot}"        redis-cli -p ${port} cluster addslots ${slot}    done                                                                                                                [[email protected] config]# sh add_slots.sh 0 5461 7000        # 運行add_slots.sh指令碼，把0到5461號槽分配給192.168.81.100:7000的redis server節點slot:0OKslot:1OKslot:2OKslot:3OKslot:4OKslot:5OK...中間省略slot:5459OKslot:5460OKslot:5461OK[[email protected] config]# redis-cli -p 7000 cluster info        # 查看cluster叢集的資訊cluster_state:ok                    # 叢集狀態為OK狀態cluster_slots_assigned:5462         # 叢集中已經有5462個資料槽cluster_slots_ok:5462               # 5462個槽的狀態為0K狀態cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6               # 叢集添加了6個節點   cluster_size:1                      # 叢集的大小為1個cluster_current_epoch:5cluster_my_epoch:2cluster_stats_messages_sent:2237cluster_stats_messages_received:2237[[email protected] config]# redis-cli -p 7000 cluster nodes        # 查看叢集中節點的資訊c96589b19b8ff5d9d286d470b8efff9a540726d8 192.168.81.101:7000 master - 0 1539720483326 5 connected53911425f4479ce4f054b8b714f4700715f0dc64 192.168.81.101:7002 master - 0 1539720484336 3 connectedbb88f0996503bac3b222b9cc4ec25139ad34ed3a 192.168.81.101:7001 master - 0 1539720485346 4 connected45d79e730593df3216a4236a19c21ff601949449 127.0.0.1:7001 master - 0 1539720482314 1 connectedd20aa403c5d7d9507adcc4ef6132c14f3c9486ed 127.0.0.1:7000 myself,master - 0 0 2 connected 0-5461                # 192.168.81.100:7000連接埠啟動並執行redis server已經分配的槽為0到5461號槽，其他節點還沒有分配槽e5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 127.0.0.1:7002 master - 0 1539720486351 0 connected[[email protected] config]# redis-cli -p 7002 cluster nodes 45d79e730593df3216a4236a19c21ff601949449 127.0.0.1:7001 master - 0 1539720507768 1 connectedd20aa403c5d7d9507adcc4ef6132c14f3c9486ed 127.0.0.1:7000 master - 0 1539720509782 2 connected 0-5461bb88f0996503bac3b222b9cc4ec25139ad34ed3a 192.168.81.101:7001 master - 0 1539720510789 4 connected53911425f4479ce4f054b8b714f4700715f0dc64 192.168.81.101:7002 master - 0 1539720508776 3 connectede5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 127.0.0.1:7002 myself,master - 0 0 0 connectedc96589b19b8ff5d9d286d470b8efff9a540726d8 192.168.81.101:7000 master - 0 1539720511799 5 connected[[email protected] config]# sh add_slots.sh 5462 10922 7001    # 運行add_slots.sh指令碼，把5462號到10922號槽分配給7001連接埠啟動並執行redis server[[email protected] config]# sh add_slots.sh 10923 16383 7002   # 運行add_slots.sh指令碼，把10923號到16383號槽分配給7002連接埠啟動並執行redis server          [[email protected] config]# redis-cli -p 7000 cluster nodes    # 查看叢集的節點資訊c96589b19b8ff5d9d286d470b8efff9a540726d8 192.168.81.101:7000 master - 0 1539720810075 5 connected53911425f4479ce4f054b8b714f4700715f0dc64 192.168.81.101:7002 master - 0 1539720807558 3 connectedbb88f0996503bac3b222b9cc4ec25139ad34ed3a 192.168.81.101:7001 master - 0 1539720804033 4 connected45d79e730593df3216a4236a19c21ff601949449 127.0.0.1:7001 master - 0 1539720809067 1 connected 5462-10922        # 7001連接埠啟動並執行redis server的槽編號為5462號到10922號槽d20aa403c5d7d9507adcc4ef6132c14f3c9486ed 127.0.0.1:7000 myself,master - 0 0 2 connected 0-5461                            # 7000連接埠啟動並執行redis server的槽編號為0號到5461號槽e5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 127.0.0.1:7002 master - 0 1539720808061 0 connected 10923-16383       # 7002連接埠啟動並執行redis server的槽編號為10922號到16383號槽[[email protected] config]# redis-cli -p 7000 cluster info     # 查看叢集的資訊cluster_state:ok                    # 叢集狀態為ok狀態cluster_slots_assigned:16384        # 叢集已經被分配16384個槽cluster_slots_ok:16384              # 叢集中狀態為ok的槽有16384個cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6               # 叢集有6個節點cluster_size:3                      # 叢集大小為3個cluster_current_epoch:5cluster_my_epoch:2cluster_stats_messages_sent:2950cluster_stats_messages_received:2950[[email protected] config]# redis-cli -p 7001 cluster info cluster_state:okcluster_slots_assigned:16384cluster_slots_ok:16384cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6cluster_size:3cluster_current_epoch:5cluster_my_epoch:1cluster_stats_messages_sent:3158cluster_stats_messages_received:3158[[email protected] config]# redis-cli -p 7002 cluster info cluster_state:okcluster_slots_assigned:16384cluster_slots_ok:16384cluster_slots_pfail:0cluster_slots_fail:0cluster_known_nodes:6cluster_size:3cluster_current_epoch:5cluster_my_epoch:0cluster_stats_messages_sent:2828cluster_stats_messages_received:2828[[email protected] config]# redis-cli -p 7000127.0.0.1:7000> set hello world     # 成功向叢集中寫入資料OK127.0.0.1:7000> config get cluster* # 擷取叢集的配置資訊1) "cluster-node-timeout"2) "15000"3) "cluster-migration-barrier"4) "1"5) "cluster-slave-validity-factor"6) "10"7) "cluster-require-full-coverage"8) "no"127.0.0.1:7000> exit

3.7 在192.168.81.101虛擬機器上操作

[[email protected] config]# redis-cli -p 7002 cluster nodes        # 查看叢集的節點資訊，都是masterbb88f0996503bac3b222b9cc4ec25139ad34ed3a 127.0.0.1:7001 master - 0 1539693824725 4 connected53911425f4479ce4f054b8b714f4700715f0dc64 192.168.81.101:7002 myself,master - 0 0 3 connectedd20aa403c5d7d9507adcc4ef6132c14f3c9486ed 192.168.81.100:7000 master - 0 1539693826743 2 connected 0-5461e5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 192.168.81.100:7002 master - 0 1539693829260 0 connected 10923-16383c96589b19b8ff5d9d286d470b8efff9a540726d8 192.168.81.101:7000 master - 0 1539693830266 5 connected45d79e730593df3216a4236a19c21ff601949449 192.168.81.100:7001 master - 0 1539693828252 1 connected 5462-10922[[email protected] config]# redis-cli -p 7000 cluster replicate d20aa403c5d7d9507adcc4ef6132c14f3c9486ed        # 使192.168.81.101:7000連接埠啟動並執行redis server做為192.168.81.100:7000連接埠啟動並執行redis server的masterOK[[email protected] config]# redis-cli -p 7001 cluster replicate 45d79e730593df3216a4236a19c21ff601949449OK[[email protected] config]# redis-cli -p 7002 cluster replicate e5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a OK[[email protected] config]# redis-cli -p 7002 cluster nodes        # 查看叢集的節點資訊bb88f0996503bac3b222b9cc4ec25139ad34ed3a 127.0.0.1:7001 slave 45d79e730593df3216a4236a19c21ff601949449 0 1539693962338 4 connected53911425f4479ce4f054b8b714f4700715f0dc64 192.168.81.101:7002 myself,slave e5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 0 0 3 connectedd20aa403c5d7d9507adcc4ef6132c14f3c9486ed 192.168.81.100:7000 master - 0 1539693958297 2 connected 0-5461e5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a 192.168.81.100:7002 master - 0 1539693960324 0 connected 10923-16383c96589b19b8ff5d9d286d470b8efff9a540726d8 192.168.81.101:7000 slave d20aa403c5d7d9507adcc4ef6132c14f3c9486ed 0 1539693961329 5 connected45d79e730593df3216a4236a19c21ff601949449 192.168.81.100:7001 master - 0 1539693963347 1 connected 5462-10922[[email protected] config]# redis-cli -p 7002 cluster slots        # 查看叢集的槽與節點的關係1) 1) (integer) 0   2) (integer) 5461   3) 1) "192.168.81.100"      2) (integer) 7000      3) "d20aa403c5d7d9507adcc4ef6132c14f3c9486ed"   4) 1) "192.168.81.101"      2) (integer) 7000      3) "c96589b19b8ff5d9d286d470b8efff9a540726d8"2) 1) (integer) 10923   2) (integer) 16383   3) 1) "192.168.81.100"      2) (integer) 7002      3) "e5ea887c98e79ef2b2205d6cc1d7ac5cfe936d9a"   4) 1) "192.168.81.101"      2) (integer) 7002      3) "53911425f4479ce4f054b8b714f4700715f0dc64"3) 1) (integer) 5462   2) (integer) 10922   3) 1) "192.168.81.100"      2) (integer) 7001      3) "45d79e730593df3216a4236a19c21ff601949449"   4) 1) "127.0.0.1"      2) (integer) 7001      3) "bb88f0996503bac3b222b9cc4ec25139ad34ed3a"

3.8 向叢集中寫入資料

[[email protected] config]# redis-cli -c -p 7000127.0.0.1:7000> set hello worldOK127.0.0.1:7000> exit

4.優點

可以深入理解Redis Cluster架構

5.缺點

容易出差錯，生產環境中不建議使用

高可用Redis(十)：Redis原生命令搭建叢集