Keepalive, Twemproxy, Redis overall cluster building record

Introduction build time: 20140724 build system: centos6.2, centos6.5 server architecture 192.168.0.221twemproxy, redismaster01, redisslave02192.168.0.220twemproxy, redismaster02, ghost

Introduction build time: middleware build system: centos 6.2, centos 6.5 server architecture 192.168.0.221 twemproxy, redis master01, redis slave02 192.168.0.220 twemproxy, redis master02, redis 255.192.168.0.10 keepalived master 192.168.0.20 keepalived slav

Introduction

Construction Time:

Build system: centos 6.2, centos 6.5

Server Architecture

192.168.0.221 twemproxy, redis master01, redis slave02
192.168.0.220 twemproxy, redis master02, redis slave01
192.168.0.10 keepalived master
192.168.0.20 keepalived slave

Service version:

Redis? : 2.4.10
Keepalived: 1.2.2
Twemproxy-master: 0.3.0

Install Redis in Redis

Decompress the redis source code package;

Complete the installation steps (make) According to README introduction)

Install the redis folder mv under/usr/local.

Configure redis master/slave

Create a redis configuration file path

Log on to the 192.168.0.221 server.

Mkdir/var/redis/master01/{conf, data, logs, run };
Mkdir/var/redis/slave02/{conf, data, logs, run };

Log on to the 192.168.0.220 server.

Mkdir/var/redis/master02/{conf, data, logs, run };
Mkdir/var/redis/slave01/{conf, data, logs, run };

Create the redis. conf file (master and slave are roughly the same, with only two lines different)

Dir/var/redis/master01 /? # Redis address
Pidfile./run/redis. pid? # The relative path address is equivalent to dir.
Logfile./logs/stdout. log
Dbfilename./data/dump. rdb
Bind 192.168.0.220 # listening service address
Daemonize yes
Requirepass 123qwe # service Password
Masterauth 123qwe? # This is only configured on the slave server, and the master does not need to be configured.
Slaveof 192.168.0.221 6601 # This only needs to be configured on the slave. The master does not need to be configured and the master Address is specified.
Databases 16
Port 6601 # service port
Timeout 0
Loglevel verbose
Save 900 1
Save 300 10
Rdbcompression yes
Slave-serve-stale-data yes # If the master is disconnected, does slave provide services?
Appendonly no
Appendfsync everysec
No-appendfsync-on-rewrite no
Auto-aof-rewrite-percentage 100
Auto-aof-rewrite-min-size 64 mb
Slowlog-log-slower-than 10000
Slowlog-max-len 1024

Start the redis service, first start the redis master service, and then start the redis slave service.

Log on to the slave server and run./redis-cli-h 192.168.0.220-p 6601-a 123qwe info.

Role: slave
Master_host: 192.168.0.221
Master_port: 6601
Master_link_status: up
Master_last_io_seconds_ago: 8
Master_sync_in_progress: 0

It indicates that redis master/slave has been configured

Install twemproxy

First download the latest twemproxy and find the latest version on github.

If you use the same system as me, you need to update autoconf first. If the current version is too low, an error will be reported during compilation.

CFLAGS = "-ggdb3-O0" autoreconf-fvi
./Configure-prefix =/usr/local/twemproxy? -Enable-debug = log
Make & make install

Configure twemproxy

The configuration here is built on two servers. Each server has a twemproxy. The configuration is the same. The frontend uses keepalived for LVS distribution.

Log on to 192.168.0.221/220 and modify the configuration of nutcracker. yml under conf as follows:

Redis1:
Listen: 0.0.0.0: 10000
Hash: fnv1a_64 # Distribution Algorithm
Distribution: ketama
Timeout: 400
Backlog: 1024
Preconnect: true
Auto_eject_hosts: true
Server_retry_timeout: 2000 # retry time
Server_failure_limit: 3 # number of failed attempts. If the limit is reached, the backend distribution permission will be revoked.
Servers:
-192.168.0.0.220: 6601: 1? # Redis instance ip address, port, weight
-192.168.0.221: 6601: 1

Start redis Service

/Usr/local/twemproxy/sbin/nutcracker-c/usr/local/twemproxy/conf/nutcracker. yml-d

Redis Testing Service

Find another server with redis installed.

./Redis-cli-h 192.168.0.220-p 10000-a 123qwe

If no error is reported, the operation is successful!

Install keepalived

Log on to 192.168.0.10

Yum install kernel-devel
./Configure-prefix =/usr/local/keepalived-with-kernel-dir =/usr/src/kernels/2.6.18-308. el5-x86_64/
Make & make install

Before make, if the following result is true, your system can complete the functions of keepalived.

Keepalived configuration
--------
Keepalived version ?????? : 1.2.2
Compiler ???????????????? : Gcc
Compiler flags ?????????? :-G-O2-DETHERTYPE_IPV6 = 0x86dd
Extra Lib ??????????????? :-Lpopt-lssl-lcrypto
Use IPVS Framework ?????? : Yes # To implement the LVS function, you must ensure that this is YES.
S sync daemon support: Yes #
S use libnl ?????????? : No
Use VRRP Framework ?????? : Yes
Use Debug flags ????????? : No

Keepalived Configuration

Modify the keepalived. conf file.

Global_defs {
Router_id wd_twemproxy
}

Vrrp_instance wd2 {
State BACKUP # master Standard MASTER
Nopreempt
Interface em2
Track_interface {
Em2
}

Virtual_router_id 70
Priority 100 # master
Advert_int 1
Authentication {
Auth_type PASS
Auth_pass css74... tfwwapmzi4r
}

Virtual_ipaddress {
192.168.0.100/16 dev em2 scope global # define VIP
}
}

Virtual_server 192.168.0.100 10000 {
Delay_loop 6
Lb_algo wrr
Lb_kind DR
Nat_mask 255.255.255.0.0
Persistence_timeout 120
Protocol TCP

Real_server 192.168.0.220 10000 {
Weight 1
TCP_CHECK {
Connect_timeout 60
Nb_get_retry 3
Delay_before_retry 3
Connect_port 10000
}
}

Real_server 192.168.0.221 10000 {
Weight 1
TCP_CHECK {
Connect_timeout 60
Nb_get_retry 3
Delay_before_retry 3
Connect_port 10000
}
}
}

After the configuration is complete, start keepalived.

/Usr/local/keepalived/sbin/keepalived-f/usr/local/keepalived/etc/keepalived. conf

Execute ipvsadm-Ln

Shown as follows:

IP Virtual Server version 1.2.1 (size = 4096)
Prot LocalAddress: Port sched1_flags
-> RemoteAddress: Port
Forward Weight ActiveConn InActConn
TCP? 192.168.0.100: 10000 wrr persistent 120
-> 192.168.0.220: 10000 ???????? Route ?? 1 ????? 0 ????????? 0
-> 192.168.0.221: 10000 ???????? Route ?? 1 ????? 0 ????????? 0

Indicates that keepalived is successfully configured.

Summary (key: Read it first)

Because I don't want to modify the above content again, I will add instructions directly in the following section. After reading the instructions, I will adjust the above configurations according to the following content!

Marathon Season 1

After you complete the configuration, you can find that the service cannot be provided. Do you want to know why? I don't know TM. Anyway, it cannot be connected.

[Mon Jul 28 10:57:29 2014] nc_proxy.c: 336 accepted c 9 on p 8 from 'unknown'
[Mon Jul 28 10:57:29 2014] nc_core.c: 201 close c 9 'unknown 'on event FF00FF eof 0 done 0 rb 0 sb 0: Connection reset by pee
[Mon Jul 28 10:57:35 2014] nc_proxy.c: 336 accepted c 9 on p 8 from 'unknown'
[Mon Jul 28 10:57:35 2014] nc_core.c: 201 close c 9 'unknown 'on event FF00FF eof 0 done 0 rb 0 sb 0: Connection reset by peer

The above is the error message after twemproxy is started. If you also use keepalived as the frontend LVS distribution function, it is estimated that the result will not be better than me.

Solution:

Stop using keepalived to use LVS, and use hot standby at most. The front end uses two twemproxy, one master, one slave, and keepalived to provide services.

Marathon Season 2

If redis uses the Master/Slave function, do not use password authentication. It is safe to directly use the Intranet address. Otherwise, twemproxy cannot provide services. Remember, remember !!!

Original article address: Keepalive, Twemproxy, Redis overall cluster building record, thanks to the original author for sharing.