Redis is the cache server for mysql (read/write splitting)

I. redis introduction Redis is a key-value storage system. Similar to Memcached, data is cached in the memory to ensure efficiency. The difference is that redis periodically writes the updated data to the disk or writes the modification operation to the append record file, and on this basis implements master-slave (master-slave) synchronization. In some cases, you can

I. redis introduction Redis is a key-value storage system. Similar to Memcached, data is cached in the memory to ensure efficiency. The difference is that redis periodically writes the updated data to the disk or writes the modification operation to the append record file, and on this basis implements master-slave (master-slave) synchronization. In some cases, you can

I. redis Introduction

Redis is a key-value storage system. Similar to Memcached, data is cached in the memory to ensure efficiency. The difference is that redis periodically writes the updated data to the disk or writes the modification operation to the append record file, and on this basis implements master-slave (master-slave) synchronization. In some cases, relational databases can be well supplemented. It provides clients such as Java, C/C ++ (hiredis), C #, PHP, JavaScript, Perl, Object-C, Python, and Ruby for ease of use.

Ii. Architecture Diagram

650) this. width = 650; "src =" http://www.68idc.cn/help/uploads/allimg/151111/1213395222-0.jpg "title =" 3.png" alt = "wKioL1UaFsii5Z8qAADOf7iND80904.jpg"/>

The general structure is read/write splitting. mysql Data is synchronized to redis through a trigger.

3. Install the LNMP Environment(Yum is used for installation to save time)

1. Modify the yum Source

[Root @ redis ~] # Vim/etc/yum. repos. d/epel. repo # Add this file [epel] name = Extra Packages for Enterprise Linux 6-$ basearchbaseurl = pairepobaseurl = Latest

2. Install yum

 [root@redis ~]# yum -y install nginx php php-fpm php-cli php-common php-gd php-mbstring php-mysql php-pdo php-devel php-xmlrpc php-xml php-bcmath php-dba php-enchant mysql mysql-server

3. Configure nginx

[Root @ redis ~] # Vim/etc/nginx. confserver {listen 80; # define the use of www.xx.com to access server_name www.xx.com; # Set the access log access_log/logs/www.xx.com. access. log main; # default Request location/{root/www/; # define the default website root directory location index of the server. php index.html index.htm; # define the name of the home index file} location ~ \. Php $ {root/www/; fastcgi_pass 127.0.0.1: 9000; fastcgi_index index. php; fastcgi_param SCRIPT_FILENAME/www/$ fastcgi_script_name; include fastcgi_params ;}}

4. Start the service

[Root @ redis ~] # Sed-I's/apache/nginx/G'/etc/php-fpm.d/www. conf [root @ redis ~] #/Etc/init. d/php-fpm start starting php-fpm: [OK] [root @ redis ~] #/Etc/init. d/mysqld start is starting mysqld: [OK] [root @ redis ~] # Mkdir/{logs, www} [root @ redis ~] # Chown-R nginx: nginx/{logs, www} [root @ redis ~] #/Etc/init. d/nginx start is starting nginx: [OK] [root @ redis www] # service iptables stopiptables: Flushing firewall rules: [OK] iptables: Setting chains to policy ACCEPT: filter [OK] iptables: Unloading modules: [OK] [root @ redis] # netstat-tnlp # view listener Active Internet connections (only servers) proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0: 80 0.0.0.0: * LISTEN 2101/nginx tcp 0 0 127.0.0.1: 9000 0.0.0.0: * LISTEN 7544/php-fpm tcp 0 0 0.0.0.0: 3306 0.0.0.0: * LISTEN 1871/mysqld

5. Authorize mysql

[root@redis ~]# mysqlmysql> grant all privileges on *.* to root@localhost identified by '123456';mysql> flush privileges;

6. Test

[root@redis ~]# vim /www/index.php 
 

Then visit the page to view the php related information. The basic environment is complete.

4. Install redis

1. Install redis

[Root @ redis ~] # Wget-c-t 0 http://download.redis.io/releases/redis-2.8.19.tar.gz [root @ redis ~] # Mkdir/usr/local/redis [root @ redis ~] # Tar xvf redis-2.8.19.tar.gz # installation is very simple, direct make can be [root @ redis ~] # Cd redis-2.8.19 [root @ redis redis-2.8.19] # make # After compilation, copy the executable files in src to the Created directory [root @ redis src] # cp redis-benchmark redis-check-aof redis-check-dump redis-cli redis-sentinel redis-server/usr/local/redis/[root @ redis redis-2.8.19] # cp redis. conf sentinel. conf/usr/local/redis/

Redis-benchmark stress testing tool

Redis-check-aof checks the integrity of the redis persistent command file

Redis-check-dump checks the integrity of redis persistent data files

Redis-cli redis client on linux

Redis-sentinel redis-sentinel is a cluster management tool mainly responsible for master-slave switchover.

Daemon Startup Program of Redis-server Redis server

2. Install the redis extension of php

[Root @ redis ~] # Wget-c-t 0 https://github.com/owlient/phpredis/archive/master.zip?root@redis ~] # Unzip master.zip [root @ redis ~] # Cd phpredis-master/[root @ redis phpredis-master] # phpize [root @ redis phpredis-master] #. /configure -- with-php-config =/usr/bin/php-config [root @ redis phpredis-master] # make & make install # modify the php configuration file, if "extension = redis. so ", add this line [root @ redis ~] # Vim/etc/php. ini extension = redis. so [root @ redis ~] #/Etc/init. d/php-fpm restart stop php-fpm: [OK] Starting php-fpm: [OK]

3. Whether the installation is successful

Access the phpinfo interface.

650) this. width = 650; "src =" http://www.68idc.cn/help/uploads/allimg/151111/1213392064-1.jpg "title =" 2.png" alt = "wKioL1UaEDeBJuP3AABbp2hjrt4347.jpg"/>

The installation is complete.

5. read/write splitting

Here is just a simple read, there is no write operation related Code, after a test, directly to the database to execute update to simulate write operations.

1. Insert some test data into mysql

[root@redis ~]# mysql -u root -p123456mysql> create database mytest;mysql> CREATE TABLE `test` (`id` int(7) NOT NULL AUTO_INCREMENT, `name` char(8) DEFAULT NULL, PRIMARY KEY (`id`)) ENGINE=InnoDB AUTO_INCREMENT=10 DEFAULT CHARSET=utf8;mysql> INSERT INTO `test` VALUES (1,'sven'),(2,'jim'),(3,'zhu'),(4,'wang'),(5,'ftd'),(6,'test'),(7,'test01'),(8,'test02'),(9,'test03');mysql> select * from mytest.test;+----+--------+| id | name   |+----+--------+|  1 | sven   ||  2 | jim    ||  3 | zhu    ||  4 | wang   ||  5 | ftd    ||  6 | test   ||  7 | test01 ||  8 | test02 ||  9 | test03 |+----+--------+

2. Compile the php test code

 Connect ('2017. 0.0.1 ', 6379) or die ("cocould net connect redis server"); $ query = "select * from test limit 8"; // For simplicity, 8 data records for ($ key = 1; $ key <9; $ key ++) {if (! $ Redis-> get ($ key) {$ connect = mysql_connect ('2017. 0.0.1 ', 'root', '123'); mysql_select_db (mytest); $ result = mysql_query ($ query); // If $ key is not found, cache the SQL query results to redis while ($ row = mysql_fetch_assoc ($ result) {$ redis-> set ($ row ['id'], $ row ['name']) ;}$ myserver = 'mysql'; break ;}else {$ myserver = "redis "; $ data [$ key] = $ redis-> get ($ key) ;}} echo $ myserver; echo"
"; For ($ key = 1; $ key <9; $ key ++) {echo" number is$ Key"; Echo"
"; Echo" name is$ Data [$ key]"; Echo"
";}?>

When redis does not have a corresponding KEY during the first access

650) this. width = 650; "src =" http://www.68idc.cn/help/uploads/allimg/151111/1213393413-2.jpg "title =" 4.png" alt = "wKiom1UaIrjC3shWAAD2e_5hT48678.jpg"/>

Access again. Now redis has related data

650) this. width = 650; "src =" http://www.68idc.cn/help/uploads/allimg/151111/1213395P2-3.jpg "title =" 5.png" alt = "wKiom1UaIuTxTtn2AAEcXGxJop8634.jpg"/>

Here, we have implemented redis as the mysql cache server, but if mysql is updated, redis will still have the corresponding KEY, and the data will not be updated, in this case, the data of mysql and redis is inconsistent. So next we will use the mysql trigger to synchronize the changed data to redis.

6. Implement Synchronization Through gearman

1. Introduction

Gearman is a distributed task distribution framework:

Gearman Job Server: core program of Gearman, which must be compiled and installed and run in the background as a daemon.

Gearman Client: it can be understood as the task requestor.

Gearman Worker: The real executor of a task. Generally, you need to write the specific logic and run it in the daemon mode. After receiving the task content transmitted by Gearman Client, Gearman Worker will process it in order.

General process:

The mysql trigger to be compiled below is equivalent to the Gearman client. When you modify a table, inserting a table is equivalent to directly issuing a task. Then, use the lib_mysqludf_json UDF library function to map the relational data to the JSON format, add the task to the gearman task queue through the Gearman-mysql-udf plug-in, and finally use redis_worker.php, that is, the worker terminal of Gearman to update the redis database.

2. Install and start

[Root @ redis ~] # Yum-y install gearmand libgearman-devel [root @ redis ~] #/Etc/init. d/gearmand start is starting gearmand: [OK] [root @ redis ~] #/Etc/init. d/gearmand statusgearmand (pid 7702) is running...

3. Install the php gearman Extension

[Root @ redis ~] # Wget-c-t 0 https://pecl.php.net/get/gearman-1.1.1.tgz [root @ redis ~] # Tar xvf gearman-1.1.1.tgz [root @ redis ~] # Cd gearman-1.1.1 [root @ redis gearman-1.1.1] # phpize [root @ redis gearman-1.1.1] #. /configure -- with-php-config =/usr/bin/php-config [root @ redis gearman-1.1.1] # make [root @ redis gearman-1.1.1] # make install # If php configuration files no extension = gearman. so, add this line [root @ redis ~] # Vim/etc/php. ini extension = gearman. so [root @ redis ~] #/Etc/init. d/php-fpm restart stop php-fpm: [OK] Starting php-fpm: [OK]

650) this. width = 650; "src =" http://www.68idc.cn/help/uploads/allimg/151111/1213396210-4.jpg "title =" 6.png" alt = "wKiom1UaJ1HheqvfAAC0WqZjDCQ519.jpg"/>

The installation is successful.

4. Install lib_mysqludf_json

Lib_mysqludf_json UDF library function maps relational data to JSON format. Generally, data ing in a database is in JSON format and is converted by a program.

[Root @ redis ~] # Wget-c-t 0 https://github.com/mysqludf/lib_mysqludf_json/archive/master.zip [Root @ redis ~] # Unzip master.zip [root @ redis ~] # Cd lib_mysqludf_json-master/[root @ redis lib_mysqludf_json-master] # gcc $ (mysql_config -- cflags)-shared-fPIC-o lib_mysqludf_json.so labels: 40: 23: Error: my_global.h: the file or directory lib_mysqludf_json.c: 41: 20: Error: my_sys.h: No file or directory lib_mysqludf_json.c: 43: 19: Error: mysql. h: The file or directory lib_mysqludf_json.c: 44: 21: Error: m_ctype.h: No file or directory lib_mysqludf_json.c: 45: 22: Error: m_string.h: there is no file or directory # Here, the compilation error is reported because the mysql Development Kit is not installed. If the source code is installed in mysql, you need to go to/etc/ld. so. conf. d/# create a file under the directory to tell the system where the mysql header file is [root @ redis lib_mysqludf_json-master] # yum-y install mysql-devel [root @ redis lib_mysqludf_json-master] # gcc $ (mysql_config -- cflags) -shared-fPIC-o lib_mysqludf_json.so lib_mysqludf_json.cmysql> show global variables like 'in in _ dir '; + --------------- + modules + | Variable_name | Value | + --------------- + modules + | plugin_dir |/usr/lib64/mysql/plugin | + ------------- + modules + # copy the module to the plug-in Directory [root @ redis lib_mysqludf_json-master] # cp lib_mysqludf_json.so/usr/lib64/mysql/plugin/# register UDF mysql> create function json_object returns string soname 'lib _ mysqludf_json.so ';

5. Install gearman-mysql-udf

This plugin is used to manage distributed queues that call Gearman.

[Root @ redis ~] # Wget-c-t 0 https://launchpad.net/gearman-mysql-udf/trunk/0.6/+download/gearman-mysql-udf-0.6.tar.gz [Root @ redis ~] # Tar xvf gearman-mysql-udf-0.6.tar.gz [root @ redis ~] # Cd gearman-mysql-udf-0.6 [root @ redis gearman-mysql-udf-0.6] #. /configure -- with-mysql =/usr/bin/mysql_config -- libdir =/usr/lib64/mysql/plugin/[root @ redis gearman-mysql-udf-0.6] # make [root @ redis gearman-mysql-udf-0.6] # make install # register the UDF mysql> create function gman_do_background returns string soname 'libgearman _ mysql_udf.so '; mysql> create function gman_servers_set returns string soname 'libgearman _ mysql_udf.so '; # view the FUNCTION mysql> select * from mysql. func; + -------------------- + ----- + response + ---------- + | name | ret | dl | type | + -------------------- + ----- + response + ---------- + | json_object | 0 | response | function | gman_do_background | 0 | role | function | gman_servers_set | 0 | role | function | + ------------------------ + ----- + ----------------------- + ---------- + # specify gearman service information mysql> SELECT gman_servers_set. 0.0.1: 4730 ');

6. Compile a mysql trigger (based on the actual situation)

DELIMITER $$CREATE TRIGGER datatoredis AFTER UPDATE ON test FOR EACH ROW BEGIN    SET @RECV=gman_do_background('syncToRedis', json_object(NEW.id as `id`, NEW.name as `name`));   END$$DELIMITER ;

7. Compile the gearman worker

[Root @ redis ~] # Vim/www/redis_worker.php
 AddServer (); $ worker-> addFunction ('synctoredis ', 'synctoredis'); $ redis = new Redis (); $ redis-> connect ('2017. 127. 0.0.1 ', 6379); while ($ worker-> work (); function syncToRedis ($ job) {global $ redis; $ workString = $ job-> workload (); $ work = json_decode ($ workString); if (! Isset ($ work-> id) {return false;} $ redis-> set ($ work-> id, $ work-> name);}?> # Run [root @ redis www] # nohup php redis_worker.php &

"$ Redis-> set ($ work-> id, $ work-> name);" This statement stores id as KEY and name as VALUE separately, it must be consistent with the access to the php test code written earlier.

8. Update mysql Data

mysql> set @RECV = 1;mysql> select @RECV;+------+| @RECV|+------+|    1 |+------+mysql> update test set name = 'ssss' where id = 1;mysql> select @RECV;+------+| @RECV|+------+| NULL |+------+

From the returned value, we can see that the trigger is successfully triggered (here @ RECV is the return value of the mysql TIGGER above ). We can view data in redis:

[root@redis redis]# ./redis-cli 127.0.0.1:6379> get 1"sven"

The data here has not changed, so we have to make a mistake.

[Root @ redis ~] # Vim/var/log/audit. log type = AVC msg = audit (1427807674.425: 107): avc: denied {name_connect} for pid = 12453 comm = "mysqld" dest = 4730 scontext = unconfined_u: system_r: mysqld_t: s0 tcontext = system_u: object_r: port_t: s0 tclass = tcp_socket # If you see such a log, you will know that selinux has blocked synchronization. # Change The selinux mode to Permissive [root @ redis ~] # Getenforce Enforcing [root @ redis ~] # Setenforce 0 [root @ redis ~] # Getenforce Permissive

After the configuration is complete, execute update again to go to redis for viewing.

127.0.0.1:6379> get 1"ssss"

Refresh the php interface.

650) this. width = 650; "src =" http://www.68idc.cn/help/uploads/allimg/151111/121339C51-5.jpg "title =" 7.png" alt = "wKiom1UaVVfTHNxBAAEJ_PKtNDU910.jpg"/>

This is basically a success. As long as the application writes data to mysql, the mysql trigger detects the update and synchronizes the data to redis through Gearman. Then, read the data directly from redis. Of course, this is only an experimental environment, but there are still many details to adjust.

7. An interesting tool on GitHub

Address: https://github.com/delano/redis-dump

This tool is backed up by redis. Maybe you will ask redis if it has its own persistence. What should I do with this tool?

The persistence inherent in redis has limitations. The illusion is that we need redis to go back to a certain time point in the past (rollback ). The persistence provided by redis must be implemented by constantly moving the dump file or aof file to another directory. Otherwise, it will be overwritten by the New persistent file.

This tool can back up the database at any time, which is very convenient.

[root@redis ~]# yum -y install rubygems.noarch ruby-devel[root@redis ~]# gem install redis-dump

In this way, the installation is complete. The usage is detailed on GitHub.

Reference: http://avnpc.com/pages/mysql-replication-to-redis-by-gearman