MySQL trigger Varnish: High-Speed HTTP cache Varnish CDN Push Platform

Sendfile (Data directly from the kernal buffer) for small files
Aio (Nginx) Full Async (The difference from event-driven IO is that the data arrives at Mem before it is notified, and no notification can be returned directly to the client) support: Mmap event Drive (handles large files)

Last time, after the database modification, the data in Redis cannot be updated automatically, we must manually delete the changed data according to the above problems we make the following improvements:
So the next step is to sync the changed data to Redis via MySQL trigger.

Configuring Gearman for Data Synchronization
Gearmanis a distributed task distribution framework that supports:
The Gearman Job Server:gearman Core program needs to be compiled and installed and run in the background as a daemon.
Gearman Client: Can be understood as a requester of a task.
Gearman Worker: The real performer of the task, generally needs to write its own specific logic and through the daemon way
Run, the Gearman Worker receives the task content passed by Gearmanclient, which is processed sequentially.

Approximate process: The following MySQL trigger to write is equivalent to the Gearman client. modifying tables, inserting tables is equivalent to directly
issued a task. The relational data is then mapped to JSON format through the Lib_mysqludf_json UDF library function, and then
The task is added to the Gearman task queue through the GEARMAN-MYSQL-UDF plug-in, and finally through
redis_worker.php, the worker side of Gearman, completes the Redis database update.

DD1:
Cd/root/redis
To install the Gearman package:
Yum Install gearmand-1.1.8-2.el6.x86_64.rpm libgearman-1.1.8-2.el6.x86_64.rpm
/ETC/INIT.D/PHP-FPM start
Netstat-antlpe
650) this.width=650; "Src=" https://s1.51cto.com/wyfs02/M00/8F/82/wKiom1jiVOTR-yC8AAEb2IV4LEs326.png-wh_500x0-wm_ 3-wmp_4-s_1420401243.png "title=" 1.png "alt=" Wkiom1jivotr-yc8aaeb2iv4les326.png-wh_50 "/>
DD1:installing PHP's Gearman extension
Tar zxf gearman-1.1.2.tgz
CD gearman-1.1.2
phpize
RPM-IVH libgearman-devel-1.1.8-2.el6.x86_64.rpmlibevent-devel-1.4.13-4.el6.x86_64.rpm libevent-doc-1.4.13-4.el6.noarch.rpmlibevent-headers-1.4.13-4.el6.noarch.rpm
./configure
Make && make install
cd/etc/php.d/
CP Mysql.ini Gearman.ini
Vim Gearman.ini

650) this.width=650; "Src=" https://s5.51cto.com/wyfs02/M00/8F/80/wKioL1jiVPHQWQMoAAA6rd766f8147.png-wh_500x0-wm_ 3-wmp_4-s_906316560.png "title=" 2.png "alt=" Wkiol1jivphqwqmoaaa6rd766f8147.png-wh_50 "/>
/ETC/INIT.D/PHP-FPM restart

To view the extensions that PHP has installed:
php-m | grep Gearman
php-m | grep MySQL
php-m | grep Redis

650) this.width=650; "Src=" https://s2.51cto.com/wyfs02/M01/8F/82/wKiom1jiVP3AkwqyAAE6Ur-8rOA619.png-wh_500x0-wm_ 3-wmp_4-s_215290316.png "title=" 3.png "alt=" Wkiom1jivp3akwqyaae6ur-8roa619.png-wh_50 "/>
CD redis/
SCP gearman-mysql-udf-0.6.tar.gz 172.25.42.12
SCP lib_mysqludf_ Json-master.zip 172.25.42.12:
SCP libevent-* libgearman-* 172.25.42.12:


Dd3:
/etc/init.d/mysqld Start
Tar zxf gearman-mysql-udf-0.6.tar.gz
Yum install mysql-devel-y
RPM-IVH libgearman-devel-1.1.8-2.el6.x86_64.rpmlibevent-devel-1.4.13-4.el6.x86_64.rpm libevent-doc-1.4.13-4.el6.noarch.rpmlibevent-headers-1.4.13-4.el6.noarch.rpm
CD gearman-mysql-udf-0.6
./configure--libdir=/usr/lib64/mysql/plugin/
make && make install
cd/usr/lib64/mysql/plugin/

Enter the MySQL environment:
Registering UDF functions
mysql> CREATE FUNCTION gman_do_background RETURNS STRING SONAME
' Libgearman_mysql_udf.so ';
mysql> CREATE FUNCTION gman_servers_set RETURNS stringsoname
' Libgearman_mysql_udf.so ';
View functions
Mysql> select * from Mysql.func;
650) this.width=650; "Src=" https://s2.51cto.com/wyfs02/M02/8F/82/wKiom1jiVQjgvfssAAEbDvW25nE368.png-wh_500x0-wm_ 3-wmp_4-s_242840592.png "title=" 4.png "alt=" Wkiom1jivqjgvfssaaebdvw25ne368.png-wh_50 "/>
install Lib_mysqludf_json
lib_mysqludf_jsonudf
Unzip Lib_mysqludf_json-master.zip
CD that is converted by the program Lib_mysqludf_ Json-maste
GCC $ (mysql_config--cflags)-shared-fpic-olib_mysqludf_ json.so
lib_mysqludf_json.c
View the module directory for MySQL:
mysql> show global variables like ' Plugin_dir '
Copy lib_mysqludf_json.so module:
CP lib_mysqludf_json.so/usr/lib64/mysql/plugin/
Register UDF function
mysql> create FUNCTION json_object RETURNS STRING SONAME
' lib_mysqludf_ Json.so '
view function
mysql> select * from Mysql.func;
650) this.width=650; "Src=" https://s4.51cto.com/wyfs02/M01/8F/80/wKioL1jiVRfDGZoFAAEhotf5aeQ938.png-wh_500x0-wm_ 3-wmp_4-s_4199231214.png "title=" 5.png "alt=" Wkiol1jivrfdgzofaaehotf5aeq938.png-wh_50 "/>
Specify service information for Gearman
mysql>SELECT gman_servers_set (' 172.25.42.10:4730 ');
writing mysql triggers Test.sql

650) this.width=650; "Src=" https://s4.51cto.com/wyfs02/M02/8F/82/wKiom1jiVSLBrhlQAADbVMv5G4Y467.png-wh_500x0-wm_ 3-wmp_4-s_2661811817.png "title=" 6.png "alt=" Wkiom1jivslbrhlqaadbvmv5g4y467.png-wh_50 "/>
MySQL < Test.sql
View triggers
Mysql> SHOW TRIGGERS from test

650) this.width=650; "Src=" https://s2.51cto.com/wyfs02/M02/8F/80/wKioL1jiVTCyNak7AADmPj-JVY0142.png-wh_500x0-wm_ 3-wmp_4-s_420658226.png "style=" Float:none; "title=" 7.png "alt=" Wkiol1jivtcynak7aadmpj-jvy0142.png-wh_50 "/>

650) this.width=650; "Src=" https://s1.51cto.com/wyfs02/M02/8F/80/wKioL1jiVTGwnIDwAADcWSWDWd8026.png-wh_500x0-wm_ 3-wmp_4-s_1226200224.png "style=" Float:none; "title=" 8.png "alt=" Wkiol1jivtgwnidwaadcwswdwd8026.png-wh_50 "/>

Dd1: writing The worker side of the Gearman
nohup php worker.php &>/dev/null & ( background run)
650) this.width=650; "Src=" Https://s5.51cto.com/wyfs02/M00/8F/82/wKiom1jiVT2QsUEvAADXLRi999U109.png-wh_500x0-wm_3-wmp_4-s_508594494.png " Title= "9.png" alt= "wkiom1jivt2qsuevaadxlri999u109.png-wh_50"/>
test:
Update data in MySQL

650) this.width=650; "Src=" https://s4.51cto.com/wyfs02/M00/8F/80/wKioL1jiVUjzbLpLAACsg-Qocfo076.png-wh_500x0-wm_ 3-wmp_4-s_3616601199.png "title=" 10.png "alt=" Wkiol1jivujzblplaacsg-qocfo076.png-wh_50 "/>
mysql> Update test set name= ' Test1 ' where id=1
650) this.width=650; "Src=" https://s3.51cto.com/wyfs02/M01/8F/82/wKiom1jiVVPBxiRDAABnC_p9V1A094.png-wh_500x0-wm_ 3-wmp_4-s_1812792019.png "title=" 11.png "alt=" Wkiom1jivvpbxirdaabnc_p9v1a094.png-wh_50 "/>
mysql> Update test set name= ' Dangdang ' where id=1

650) this.width=650; "Src=" https://s1.51cto.com/wyfs02/M01/8F/80/wKioL1jiVWPCRvODAACcboIViRE263.png-wh_500x0-wm_ 3-wmp_4-s_1547998764.png "style=" Float:none; "title=" 12.png "alt=" Wkiol1jivwpcrvodaaccboivire263.png-wh_50 "/>

650) this.width=650; "Src=" https://s1.51cto.com/wyfs02/M00/8F/82/wKiom1jiVWTx_rUeAABspmQhL8M437.png-wh_500x0-wm_ 3-wmp_4-s_1954906733.png "style=" Float:none; "title=" 13.png "alt=" Wkiom1jivwtx_rueaabspmqhl8m437.png-wh_50 "/>

Varnish: High-Speed HTTP caching:
DD1:
varnish-3.0.5-1.el6.x86_64.rpm varnish-libs-3.0.5-1.el6.x86_64.rpm
Rpm–ivh

650) this.width=650; "Src=" https://s2.51cto.com/wyfs02/M00/8F/80/wKioL1jiVXDjA21yAACrKkeZn28769.png-wh_500x0-wm_ 3-wmp_4-s_2633634656.png "title=" 14.png "alt=" Wkiol1jivxdja21yaacrkkezn28769.png-wh_50 "/>
Vim/etc/sysconfig/varnish

650) this.width=650; "Src=" https://s3.51cto.com/wyfs02/M02/8F/82/wKiom1jiVX7wNlciAAEarwUXq2A945.png-wh_500x0-wm_ 3-wmp_4-s_417131323.png "title=" 15.png "alt=" Wkiom1jivx7wnlciaaearwuxq2a945.png-wh_50 "/>
cd/etc/varnish/
Vim DEFAULT.VCL

650) this.width=650; "Src=" https://s4.51cto.com/wyfs02/M02/8F/80/wKioL1jiVYmiR8ntAADu8nFQKTU032.png-wh_500x0-wm_ 3-wmp_4-s_3993461897.png "title=" 16.png "alt=" Wkiol1jivymir8ntaadu8nfqktu032.png-wh_50 "/>
/etc/init.d/varnish start

DD3:/ETC/INIT.D/HTTPD start
650) this.width=650; "Src=" https://s5.51cto.com/wyfs02/M00/8F/82/wKiom1jiVZOAJhUeAAAtZcVsNqo596.png-wh_500x0-wm_ 3-wmp_4-s_2347266704.png "title=" 17.png "alt=" Wkiom1jivzoajhueaaatzcvsnqo596.png-wh_50 "/>
### Manually clearing the cache via Varnishadm
# Varnishadm Ban.url. *$
#清除所有
# Varnishadm Ban.url/index.html
#清除 index.html Page Cache
# Varnishadm ban.url/admin/$
#清除 Admin Directory Cache

Vim DEFAULT.VCL
###viewing Cache Hit scenarios
Sub Vcl_deliver {
if (Obj.hits > 0) {
Set Resp.http.x-cache = "hit from Westos Cache";
}
else {
Set Resp.http.x-cache = "MISS from Westos Cache";
}
return (deliver);
}

Test Cache Hit:

650) this.width=650; "Src=" https://s4.51cto.com/wyfs02/M01/8F/82/wKiom1jiVaTToGcAAACRkAiKGY8831.png-wh_500x0-wm_ 3-wmp_4-s_2932784238.png "style=" Float:none; "title=" 18.png "alt=" Wkiom1jivattogcaaacrkaikgy8831.png-wh_50 "/>

650) this.width=650; "Src=" https://s4.51cto.com/wyfs02/M01/8F/80/wKioL1jiVaXz7UF8AACSVny81gU745.png-wh_500x0-wm_ 3-wmp_4-s_3306518526.png "style=" Float:none; "title=" 19.png "alt=" Wkiol1jivaxz7uf8aacsvny81gu745.png-wh_50 "/>

Define back-end servers for several different domain name sites:
Backend Default {
. Host = "172.25.42.12";
. Port = "80";
}

Backend Web {
. Host = "172.25.42.11";
. Port = "80";
}

Sub Vcl_recv {
if (req.http.host ~ "^ (www.)?" Dd.org ") {
Set req.http.host = "www.dd.org";
Set req.backend = default;
} elsif (req.http.host ~ "^bbs.dd.org") {
Set req.backend = web;
} else {error 404 "Westos cache";
}
Test: www.dd.org and dd.org are the same cache

650) this.width=650; "Src=" https://s3.51cto.com/wyfs02/M00/8F/80/wKioL1jiVbKjXO78AAAqHZNhKpQ827.png-wh_500x0-wm_ 3-wmp_4-s_3106528680.png "title=" 20.png "alt=" wkiol1jivbkjxo78aaaqhznhkpq827.png-wh_50 "/>

And bbs.dd.org is a cache

650) this.width=650; "Src=" https://s3.51cto.com/wyfs02/M01/8F/82/wKiom1jiVb2TXb0WAAA2cw9AiAg006.png-wh_500x0-wm_ 3-wmp_4-s_3635239460.png "title=" 21.png "alt=" Wkiom1jivb2txb0waaa2cw9aiag006.png-wh_50 "/>

# When accessing the www.dd.org domain name from the default data fetching, access to the bbs.dd.org domain name to the Web fetch data, access to other pages Error//NOTE: Test host plus parsing

Load balancing: (Domain load Balancing)
Two hosts respond to www.dd.org when accessing it
Avoid opening a virtual machine to DD2 do virtual host (one IP multiple domain name)
650) this.width=650; "Src=" https://s2.51cto.com/wyfs02/M01/8F/80/wKioL1jiVcrS6yRSAAD6bR8ezJ4470.png-wh_500x0-wm_ 3-wmp_4-s_2230962861.png "title=" 22.png "alt=" Wkiol1jivcrs6yrsaad6br8ezj4470.png-wh_50 "/>
DD1:

Vim DEFAULT.VCL
Director LB Round-robin {
{. backend = default;}
{. backend = web;}
}

Sub Vcl_recv {
if (req.http.host ~ "^ (www.)?" Dd.org ") {
Set req.http.host = "www.dd.org";
Set req.backend = lb;
return (pass); ( for testing purposes)
} elsif (req.http.host ~ "^bbs.dd.org") {
Set req.backend = web;
} else {error 404 "Westos cache";
}
}

650) this.width=650; "Src=" https://s1.51cto.com/wyfs02/M02/8F/82/wKiom1jiVd_Q52y4AACj8ez_h5s125.png-wh_500x0-wm_ 3-wmp_4-s_386218033.png "style=" Float:none; "title=" 23.png "alt=" Wkiom1jivd_q52y4aacj8ez_h5s125.png-wh_50 "/>

650) this.width=650; "Src=" https://s4.51cto.com/wyfs02/M02/8F/82/wKiom1jiVd_RFvoGAAAvlvNMdRU852.png-wh_500x0-wm_ 3-wmp_4-s_1082170786.png "style=" Float:none; "title=" 24.png "alt=" Wkiom1jivd_rfvogaaavlvnmdru852.png-wh_50 "/>

650) this.width=650; "Src=" https://s4.51cto.com/wyfs02/M02/8F/80/wKioL1jiVd_Ruf9HAAAu6VTqix8443.png-wh_500x0-wm_ 3-wmp_4-s_4166690004.png "style=" Float:none; "title=" 25.png "alt=" Wkiol1jivd_ruf9haaau6vtqix8443.png-wh_50 "/>

Varnish CDN push platform (empty cache)
# need to install PHP support
# Unzip bansys.zip-d/var/www/html
# vi/var/www/html/bansys/config.php
#只保留如下设置, the rest is commented out
<?php
$var _group1 = Array (
' Host ' = = Array (' 172.25.42.10 '),
' Port ' = ' 80 ',
);
Varnish Group definition
To bind a host list
$VAR _cluster = Array (
' www.dd.org ' = $var _group1,
);

Vim/etc/varnish/default.vcl
/etc/init.d/varnish Reload

Test:

650) this.width=650; "Src=" https://s2.51cto.com/wyfs02/M00/8F/82/wKiom1jiVfDwBJBpAABVRaS5GXM386.png-wh_500x0-wm_ 3-wmp_4-s_965929217.png "title=" 26.png "alt=" Wkiom1jivfdwbjbpaabvras5gxm386.png-wh_50 "/>

MySQL trigger Varnish: High-Speed HTTP cache Varnish CDN Push Platform