Varnish---Reverse proxy web Acceleration cache server and CDN push

Varnish Introduction

Varnish is a high-performance and open-source reverse proxy server and HTTP Accelerator, with a new software architecture, and now the hardware system closely, compared with the traditional squid, Varnish with higher performance, faster, more convenient management and many other advantages, Many large websites are beginning to try to replace squid with varnish, which promotes varnish's rapid development.

Norway's largest online newspaper, Verdens Gang (vg.no), used 3 Varnish instead of the original 12 Squid, which is a better performance than before, which is Varnish's most successful application case.

Workflow for Varnish file caching

Varnish is similar to the general server Software and is divided into master and child processes. The master process reads into the storage configuration file, invokes the appropriate storage type, then creates/reads the cache file of the appropriate size, then master initializes the structure that manages the storage space, then fork and monitor the child process. During the initialization of the main thread, the child process mmap the previously opened storage file into memory, creating and initializing the idle structure and suspending it to the storage management structure for allocation. The child process allocates several threads to work, mainly including some management threads and many worker threads.

Then, to start the real work, one of Varnish's responsible for receiving new HTTP connection threads begins to wait for the user, and if there is a new HTTP connection coming up, it is always responsible for receiving, then waking up a waiting thread and handing over the specific process to it. The Worker thread reads the URI of the HTTP request, finds an existing object, and returns and responds to the user directly if hit. If there is no hit, the requested content needs to be taken from the backend server, stored in the cache, and then replied to.

The process for allocating the cache is this: it creates a cache file of the appropriate size based on the size of the object that is being read. For ease of reading and writing, the program changes the size of each object to a multiple of the memory page that is closest to its size. Then find the free storage block of the most appropriate size from the existing free storage structure, and assign it to it. If the free block is not used up, the extra memory is made up of another free storage block, which is attached to the management structure. If the cache is full, the oldest object is freed according to the LRU mechanism.

The process of releasing the cache is this: there is a timeout thread that detects the lifetime of all objects in the cache, and if the pre-set TTL (time to Live) is not accessed, delete it and release the corresponding struct and storage memory. Note the free memory block before or after the block of storage memory is checked when released, and is merged into a larger chunk of memory if the preceding or subsequent free memory and the freed memory are contiguous.

The entire file cache management, regardless of the relationship between the file and memory, in fact, all of the object is considered in memory, if the system memory is not enough, the system will automatically swap it to swap space, without the need to varnish program to control.

Server11:

RPM-IVH varnish-3.0.5-1.el6.x86_64.rpm varnish-libs-3.0.5-1.el6.x86_64.rpm

Vim/etc/sysconfig/varnish varnish_listen_port=80

Vim/etc/varnish/default.vcl

  7 backend Default {
  8  . Host = "172.25.88.13"; 
  9  . Port = "80"; 
 10}  
 One 
 Backend Web {
  . Host = "172.25.4.12"; 
 +  . Port = "n"; 
 +} 
 + 
 + Sub VCL_RECV {
 if (req.http.host ~ ^) ^ (www.)? Westos.org ") {
 Set req.http.host =" www.westos.org "; 
 Set req.backend = default; 
 +} elsif (Req.http.hos T ~ "^bbs.westos.org") {
 Set req.backend = web; 
 + else {error 404 "Westos cache";     In addition to accessing these domains, the other is G G 
 "
" 
 
 
 "Sub Vcl_deliver {
" if (Obj.hits > 0) {
 Set resp.http.x-cache = "Hit fr Om westos Cache "; Hit Cache 
 [} 
 + else {
 Set resp.http.x-cache = "MISS from Westos cache"; 
 33} 
 return (deliver); 
 35} 
 

/etc/init.d/varnish start

[[email protected] ~]# NETSTAT-ANTLP |grep TCP 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1167/varnishd

Server12

/ETC/INIT.D/HTTPD start

Cat index.html

Server13manually clearing the cache via Varnishadm

Varnishadm Ban.url. *$ #清除所有
Varnishadm ban.url/index.html #清除 index.html page Cache
Varnishadm ban.url/admin/$ #清除 Admin directory Cache

first time Miss (Miss) cache

650) this.width=650; "src=" Https://s1.51cto.com/wyfs02/M01/8F/A8/wKioL1jnqc7CurrkAACWQkKYq_M177.png "title=" Screenshot from 2017-04-07 21-49-09.png "style=" Float:none; "alt=" Wkiol1jnqc7currkaacwqkkyq_m177.png "/>

first hit cache

650) this.width=650; "src=" Https://s5.51cto.com/wyfs02/M02/8F/A9/wKiom1jnqc_B14MRAACUt05cews140.png "title=" Screenshot from 2017-04-07 21-49-16.png "style=" Float:none; "alt=" Wkiom1jnqc_b14mraacut05cews140.png "/>

Varnish virtual host and wheel finder

server12:

Vim/etc/httpd/conf/httpd.conf

990 Namevirtualhost *:80
1012 <virtualhost *:80>
1013 documentroot/var/www/html
1014 ServerName server12.lalala.com
1015 </VirtualHost>

1017 <virtualhost *:80>
1018 DOCUMENTROOT/WWW1
1019 ServerName www.westos.org
1020 </VirtualHost>

1022 <virtualhost *:80>
1023 DOCUMENTROOT/WWW2
1024x768 ServerName bbs.westos.org
1025 </VirtualHost>

Server11:

vim/etc/varnish/default.vcl

  7 backend Default {
  8  . Host = "172.25.88.13"; 
  9  . Port = "80"; 
 10} 
 One 
 Backend Web {
  . Host = "172.25.88.12"; 
  . Port = "n"; 
 +} 
 + 
 + di Rector lb Round-robin {
 {. backend = default;} 
 {. backend = web;} 
 20} 
 
 A sub vcl_recv {
 req.http.host ~ ^ (www.)? Westos.org ") {
 Set req.http.host =" www.westos.org "; 
 Set req.backend = lb; 
 return (pass);   &nbs p; #为了测试方便, do not cache. 
 Elsif (req.http.host ~ "^bbs.westos.org") {
 Set req.backend = web; 
) else {error 404 "Westos cache "; 
 30} 
 31} 
 
 Vcl_deliver {
 
] set Resp.http.x-cache = "hit from Westos Cache"; > "
", "Resp.http.x-cache" 
, "MISS from Westos Cache"; 
 "
" (deliver); 
} 

650) this.width=650; "src=" Https://s5.51cto.com/wyfs02/M02/8F/A8/wKioL1jnqc_AilidAAAi7ssx1Qs222.png "title=" Screenshot from 2017-04-07 21-56-45.png "style=" Float:none; "alt=" Wkiol1jnqc_ailidaaai7ssx1qs222.png "/>

650) this.width=650; "src=" Https://s5.51cto.com/wyfs02/M00/8F/A9/wKiom1jnqdDB2nHfAAATS0KhM0M428.png "title=" Screenshot from 2017-04-07 21-56-50.png "style=" Float:none; "alt=" Wkiom1jnqddb2nhfaaats0khm0m428.png "/>

650) this.width=650; "src=" Https://s5.51cto.com/wyfs02/M00/8F/A8/wKioL1jnqdCwgBWSAAAUF7l8l4o626.png "title=" Screenshot from 2017-04-07 21-56-57.png "style=" Float:none; "alt=" Wkiol1jnqdcwgbwsaaauf7l8l4o626.png "/>

650) this.width=650; "src=" Https://s5.51cto.com/wyfs02/M02/8F/A9/wKiom1jnqdCDxJO1AAAW-nqRZnY988.png "title=" Screenshot from 2017-04-07 22-18-22.png "style=" White-space:normal;float:none; "alt=" Wkiom1jnqdcdxjo1aaaw-nqrzny988.png "/>650) this.width=650;" Src= "https://s5.51cto.com/wyfs02/M01/8F/A9/ Wkiom1jnqdcs-mwnaaboiijupwm091.png "title=" screenshot from 2017-04-07 22-15-10.png "style=" Float:none; "alt=" Wkiom1jnqdcs-mwnaaboiijupwm091.png "/>

CDN Push

When the varnish cached page is updated and needs to be re-paged, only Web developers are required to submit the changed URLs, which is our CDN push

[Email protected] html]# CD Bansys
[Email protected] bansys]# MV *..

The Bansys has two modes of operation: Telnet and HTTP.
Telnet mode needs to turn off validation of the Varnish service management port, commenting out the "-S $ in/etc/sysconfig/varnish file"
{Varnish_secret_file} "this line, restart the VARNISH service.
If this is the HTTP mode, the following settings are required for varnish:

Vim/etc/varnish/default.vcl

8 ACL Westos {
9 "127.0.0.1";
10 "172.25.88.0"/24;
11}

Sub Vcl_recv {
if (req.request = = "BAN") {
if (!client.ip ~ westos) {
Error 405 "not allowed.";
17}
Ban ("Req.url ~" + req.url);
Error "Ban added";
20}
21}

Vim config.php #只保留如下设置, the rest commented out

<?php


Varnish host List
$var _group1 = Array (
' Host ' = = Array (' 172.25.88.11 '),
' Port ' = ' 80 ',
);




Varnish Group definition
To bind a host list
$VAR _cluster = Array (
' www.westos.org ' = $var _group1,
);


Varnish version
2.x and 3.x push commands are different
$VAR _version = "3";

?>

650) this.width=650; "src=" Https://s4.51cto.com/wyfs02/M01/8F/A8/wKioL1jnqdGjJkuDAABZiQqIEPQ185.png "style=" float : none; "title=" screenshot from 2017-04-07 22-50-04.png "alt=" Wkiol1jnqdgjjkudaabziqqiepq185.png "/>

650) this.width=650; "src=" Https://s4.51cto.com/wyfs02/M02/8F/A8/wKioL1jnqdGwfKA0AABQCcuqCCM964.png "style=" float : none; "title=" screenshot from 2017-04-07 22-50-10.png "alt=" Wkiol1jnqdgwfka0aabqccuqccm964.png "/>

This article from "12049878" blog, declined reprint!

Varnish---Reverse proxy web Acceleration cache server and CDN push