Web cache varnish (to be continued)

In this section we talk about Web cache related knowledge, now compare the popular 2 of squid and varnish, their relationship is like httpd server Apache and Nginx,varnish compared with traditional squid, varnish With a higher performance, faster, more convenient management and many other advantages, many large sites have begun to try to use varnish to replace squid, which promote the rapid development of varnish. So today we're just going to talk about varnish content.

Varnish
Varnish is a Web accelerator that is installed in front of a Web application, caches Web applications, and responds to user requests, Varnish makes your Web application run faster and Varnish flexible and usable.
The varnish is divided into the master (management) process and the child (worker, the main cache work) process. The master process reads in the command, makes some initialization, and then fork and monitor the child process. The child process allocates several threads to work, mainly including some management threads and many woker threads.

The varnish frame diagram is as follows:

650) this.width=650; "style=" border-bottom:0px;border-left:0px;border-top:0px;border-right:0px; "title=" 7065d5f690934b20b13cada1d1377671 "border=" 0 "alt=" 7065d5f690934b20b13cada1d1377671 "src=" http://img1.51cto.com/ Attachment/201409/25/8733640_1411667806idta.jpg "height=" 388 "/>

Varnish's VCL State engine

Through the following state engine, the process of responding to requests from all clients is reflected.

In the VCL state engine, states are correlated, but isolated from each other, each engine uses return (x) to exit the current state and instruct varnish to enter the next state.

When varnish begins processing a request, it first needs to parse the HTTP request itself, such as getting the request method from the header, verifying that it is a legitimate htt request, and so on. When these basic analyses are finished, you need to make the first decision whether Varnish is looking for the requested resource from the cache. The implementation of this decision is done by the Vcl_recv method, and Vcl_recv is a portal to the request.

650) this.width=650; "style=" border-bottom:0px;border-left:0px;border-top:0px;border-right:0px; "title=" cf46475871284c27bca19622f7b87547 "border=" 0 "alt=" cf46475871284c27bca19622f7b87547 "src=" http://img1.51cto.com/ Attachment/201409/25/8733640_14116678066881.jpg "height=" 639 "/>

There are a total of 8 state engines:

VCL_RECV: The user's requested entry to determine if the data is cacheable.
Vcl_pipe: Non-cacheable data, direct pipeline backend processing
Vcl_hash: Can cache the data, through the hash mechanism processing, the default URL to do key, can also be customized to distinguish whether the client supports processing compressed data and determine the sensitivity of the cache.
Vcl_hit: Cache Hit
Vcl_miss: Cache Misses
Vcl_fetch: Fetch data to backend
Vcl_pass: Provides a special place to hit or miss data for special processing of the requested data
Vcl_deliver: Returns the result to the client: there are 2 scenarios: Cache hit direct return and back-end data back

Experiment:

Experimental topology:

650) this.width=650; "style=" border-bottom:0px;border-left:0px;border-top:0px;border-right:0px; "title=" image " Border= "0" alt= "image" Src= "Http://img1.51cto.com/attachment/201409/25/8733640_1411667806uP3C.png" height= "393"/ >

Installation configuration for varnish

Here I use the next good to compare the new version of the RPM package to install, this installation of varnish is 3.0.5 version

If you want to compare the new version, here is a website http://repo.varnish-cache.org/there are a lot of system version, only need to download the same as their own system OK!

Okay, so here we go. Install the configuration.

650) this.width=650; "style=" border-bottom:0px;border-left:0px;border-top:0px;border-right:0px; "title=" image " Border= "0" alt= "image" Src= "Http://img1.51cto.com/attachment/201409/25/8733640_1411667807lrdE.png" height= "532"/ >

After the installation, let's look at what files are generated by the installation:

[Email protected] 3.0.5]# RPM-QL varnish
/etc/logrotate.d/varnish
/etc/rc.d/init.d/varnish Our program script
/etc/rc.d/init.d/varnishlog
/etc/rc.d/init.d/varnishncsa
/etc/sysconfig/varnish
/etc/varnish
/ETC/VARNISH/DEFAULT.VCL Our configuration files
/usr/bin/varnish_reload_vcl
/usr/bin/varnishadm
/usr/bin/varnishhist
/usr/bin/varnishlog
/usr/bin/varnishncsa
/usr/bin/varnishreplay
/usr/bin/varnishsizes
/usr/bin/varnishstat
/usr/bin/varnishtest
/usr/bin/varnishtop
/usr/lib64/varnish
/usr/lib64/varnish/libvarnish.so
/usr/lib64/varnish/libvarnishcompat.so
/usr/lib64/varnish/libvcl.so
/usr/lib64/varnish/libvgz.so
/usr/lib64/varnish/vmods
/usr/lib64/varnish/vmods/libvmod_std.so
/usr/sbin/varnishd
/usr/share/doc/varnish-3.0.5
/usr/share/doc/varnish-3.0.5/changelog
/usr/share/doc/varnish-3.0.5/license
/usr/share/doc/varnish-3.0.5/readme
/usr/share/doc/varnish-3.0.5/readme.redhat
/usr/share/doc/varnish-3.0.5/examples
/usr/share/doc/varnish-3.0.5/examples/default.vcl
/usr/share/doc/varnish-3.0.5/examples/zope-plone.vcl
/usr/share/man/man1/varnishadm.1.gz
/usr/share/man/man1/varnishd.1.gz
/usr/share/man/man1/varnishhist.1.gz
/usr/share/man/man1/varnishlog.1.gz
/usr/share/man/man1/varnishncsa.1.gz
/usr/share/man/man1/varnishreplay.1.gz
/usr/share/man/man1/varnishsizes.1.gz
/usr/share/man/man1/varnishstat.1.gz
/usr/share/man/man1/varnishtest.1.gz
/usr/share/man/man1/varnishtop.1.gz
/usr/share/man/man3/vmod_std.3.gz
/usr/share/man/man7/varnish-cli.7.gz
/usr/share/man/man7/varnish-counters.7.gz
/usr/share/man/man7/vcl.7.gz
/var/lib/varnish if you want to store the cache storage location as a file
/var/log/varnish

Modifying a configuration file

vim /etc/sysconfig/varnish# 编辑启动脚本配置文件，修改如下项

VARNISH_LISTEN_PORT=80    # varnish监听端口改为80端口

VARNISH_STORAGE_SIZE=64M   # 此值根据自身情况调整，测试环境可调低此值

VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"#使用malloc(即内存)作为缓存对象存储方式；

650) this.width=650; "style=" border-bottom:0px;border-left:0px;border-top:0px;border-right:0px; "title=" image " Border= "0" alt= "image" Src= "Http://img1.51cto.com/attachment/201409/25/8733640_1411667808UKI4.png" height= "204"/ >

Start the service

650) this.width=650; "style=" border-bottom:0px;border-left:0px;border-top:0px;border-right:0px; "title=" image " Border= "0" alt= "image" Src= "Http://img1.51cto.com/attachment/201409/25/8733640_14116678112pFl.png" height= "59"/ >

Prepare the back-end server

Test whether 2 nodes are ok first

650) this.width=650; "style=" border-bottom:0px;border-left:0px;border-top:0px;border-right:0px; "title=" image " Border= "0" alt= "image" Src= "Http://img1.51cto.com/attachment/201409/25/8733640_14116678118VIS.png" height= "181"/ >

650) this.width=650; "style=" border-bottom:0px;border-left:0px;border-top:0px;border-right:0px; "title=" image " Border= "0" alt= "image" Src= "Http://img1.51cto.com/attachment/201409/25/8733640_1411667811OC8z.png" height= "234"/ >

Ok. are normal. Here's how to modify the configuration file in the cache server

Modifying the varnish configuration file

Back up before the usual changes.

[Email protected] ~]# Cp/etc/varnish/default.vcl/etc/varnish/default.vcl.bak

OK, start modifying

[Email protected] ~]# VIM/ETC/VARNISH/DEFAULT.VCL

Backend Default {
. Host = "192.168.1.108"; Here we'll start with a backend to test.
. Port = "80";
}

Sub Vcl_recv {
Set req.backend = default;

return (OK);
}

Now read the configuration file: Here we need to know a command varnishadm command

[Email protected] ~]# varnishadm-s/etc/varnish/secret-t 127.0.0.1:6082

Enter help after entering to see assistance information

200
-----------------------------
Varnish Cache CLI 1.0
-----------------------------
Linux,2.6.32-431.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbit
varnish-3.0.5 Revision 1a89b1f

Type ' help ' for command list.
Type ' quit ' to close CLI session.

Varnish> Help
200
Help [command]
ping [timestamp]
Auth response
Quit
Banner
Status
Start
Stop
Vcl.load <configname> <filename>
Vcl.inline <configname> <quoted_VCLstring>
Vcl.use <configname>
Vcl.discard <configname>
Vcl.list
Vcl.show <configname>
Param.show [-l] [<param>]
Param.set <param> <value>
Panic.show
Panic.clear
Storage.list
Backend.list
Backend.set_health Matcher State
Ban.url <regexp>
Ban <field> <operator> <arg> [&& <field> <oper> <arg>] ...
Ban.list

This is actually some order.

Let's start executing the command to make the configuration file effective.

varnish> vcl.load rest1 default.vcl loading command
106
Message from Vcc-compiler:
Invalid return "OK"
(' Input ' line 124 Pos 17)
return (OK);
----------------##--

... in subroutine "Vcl_recv"
(' Input ' line Pos 5)
Sub Vcl_recv {
----########--

Which is the "Vcl_recv" method
Legal returns is: "Error" "Lookup" "Pass" "pipe"
Running Vcc-compiler failed, exit 1
VCL Compilation failed note that when there is a syntax error, loading is not successful, and will jump out of the message, then we will go to change it.

varnish> vcl.load Rest1 DEFAULT.VCL
200
VCL compiled. OK, after the modifications are complete, he will show up.

varnish> vcl.use Rest1 using commands
200

Okay, let's test this out.

Let us enter the address of varnish:

650) this.width=650; "title=" c73a7aa0403445618ebd46f4b03558c8.jpg "src=" http://s3.51cto.com/wyfs02/M02/4A/5B/ Wkiol1qkwv3cqbsaaaj4lx--rc8174.jpg "alt=" Wkiol1qkwv3cqbsaaaj4lx--rc8174.jpg "/>

OK, success, but so we don't know whether the cache is capable of hitting, let's set it up so we can see if the cache is hit.

Here we only need to add header information to the configuration file.

[Email protected] ~]# VIM/ETC/VARNISH/DEFAULT.VCL

650) this.width=650; "title=" Clipboard.png "src=" http://s3.51cto.com/wyfs02/m02/4a/59/wkiom1qkwo6inj_- Aab-2l4ko8e082.jpg "alt=" Wkiom1qkwo6inj_-aab-2l4ko8e082.jpg "/>

Reload the configuration file

650) this.width=650; "title=" 98e1a458908347ffb23f8ac852d0982c.jpg "src=" http://s3.51cto.com/wyfs02/M02/4A/5C/ Wkiol1qkw3dyeg73aaa6ds-gsei774.jpg "alt=" Wkiol1qkw3dyeg73aaa6ds-gsei774.jpg "/>

Under test:

First forced refresh

650) this.width=650; "title=" 3b.jpg "src=" http://s3.51cto.com/wyfs02/M01/4A/5A/wKiom1QkW7myGWlEAAHY1b55rSU279.jpg "alt=" Wkiom1qkw7mygwleaahy1b55rsu279.jpg "/>

As you can see, the cache is hit

Okay, then we'll do it again.

650) this.width=650; "title=" fly. jpg "src=" http://s3.51cto.com/wyfs02/M02/4A/5C/wKioL1QkXDujdZy4AAHLNEe522o356.jpg " alt= "Wkiol1qkxdujdzy4aahlnee522o356.jpg"/>

As we did, we had a hit. However, it is important to note that the cache is time-free, and that he will have a default value if it is not set, so the cache will expire automatically after the default time.

OK, so let's see how we can load-balance based on 2 sets of backend.

[Email protected] ~]# VIM/ETC/VARNISH/DEFAULT.VCL

backend webserver1 {    .host =  "192.168.1.108";     .port =  "    .probe = {    .url =";   "/index.html";     .interval = 1s;    .window =  3;    .threshold = 1;  }}backend webserver2 {     .host =  "192.168.1.109";    .port =  "; "    .probe = {    .url =  "/index.html";     .interval = 1s;    .window = 3;    . threshold = 1;  }}director webserver random {    {  .backend = webserver1; .weight = 3; }    { . Backend = webserver2; .weight = 1; }}sub vcl_recv {      set req.backend =  webserver;} sub vcl_deliver {     if  (obj.hits > 0)  {         set resp.http.X-Cache =  "Hit from" + " " + server.ip;    } else {        set  resp.http.x-cache =  "Miss via" + " " +SERVER.IP;    } }

Okay, re-load the next

650) this.width=650; "title=" t6.jpg "src=" http://s3.51cto.com/wyfs02/M01/4A/5C/wKioL1QkXlDj6EihAAApQAR_xX4241.jpg "alt=" Wkiol1qkxldj6eihaaapqar_xx4241.jpg "/>

OK, start the test:

Since the load balancer is randomly accessed on the first visit, it is possible to access any one of them, and below I've truncated all the pictures of the situation

650) this.width=650; "style=" Float:none; "title=" 11111.jpg "src=" http://s3.51cto.com/wyfs02/M01/4A/5C/ Wkiol1qkygtj8iytaaiomg0pw74058.jpg "width=" 613 "height=" 730 "alt=" Wkiol1qkygtj8iytaaiomg0pw74058.jpg "/>

650) this.width=650; "style=" Float:none; "title=" 22225.jpg "src=" http://s3.51cto.com/wyfs02/M01/4A/5A/wKiom1QkYD_ Xv_-daahmxieh5qq650.jpg "alt=" Wkiom1qkyd_xv_-daahmxieh5qq650.jpg "/>

650) this.width=650; "style=" Float:none; "title=" 33333.jpg "src=" http://s3.51cto.com/wyfs02/M01/4A/5A/wKiom1QkYD_ Iwrasaahgejqvabc084.jpg "alt=" Wkiom1qkyd_iwrasaahgejqvabc084.jpg "/>

650) this.width=650; "style=" Float:none; "title=" 444444444444.png "src=" http://s3.51cto.com/wyfs02/M02/4A/5C/ Wkiol1qkygsidkroaai9lhqjr98419.jpg "alt=" Wkiol1qkygsidkroaai9lhqjr98419.jpg "/>

In fact, the back-end backend of the health of the machine can be viewed

Varnish> backend.list
200
Backend name Refs Admin Probe
Default (127.0.0.1,,80) 1 probe Healthy (no probe)
Default (192.168.1.108,,80) 2 probe Healthy (no probe)
WebServer1 (192.168.1.108,,80) 1 probe Healthy 3/3
Webserver2 (192.168.1.109,,80) 1 probe Healthy 3/3

Now 2 machines are online.

Once a machine goes offline, it can be viewed immediately. Let's try to stop the second node service and see

Varnish> backend.list
200
Backend name Refs Admin Probe
Default (127.0.0.1,,80) 1 probe Healthy (no probe)
Default (192.168.1.108,,80) 2 probe Healthy (no probe)
WebServer1 (192.168.1.108,,80) 1 probe Healthy 3/3
Webserver2 (192.168.1.109,,80) 1 probe Sick 0/3 Yes, you can see that right now.

Well, then the content of this section we will first talk about here, I hope that we have a lot of criticism, thank you!

This article is from the "Linux rookie it Road," blog, please be sure to keep this source http://wiggin.blog.51cto.com/8733640/1558310

Web cache varnish (to be continued)