High-performance HTTP accelerator Varnish (installation and configuration)

1. Install Varnish

The installation of varnish is very simple. Next we will introduce it step by step:
1. Preparations before installation
The following table 1 shows the installation environment of varnish:
Table 1
Host Name operating system IP Address
Varnish-server centos release 5.4 192.168.12.246
Web-server centos release 5.4 192.168.12.26
Create a varnish user and user group, and create the varnish cache directory and log directory:
[Root @ Varnish-server ~] # Useradd-S/sbin/nologin Varnish
[Root @ Varnish-server ~] # Mkdir/data/Varnish/Cache
[Root @ Varnish-server ~] # Mkdir/data/Varnish/log
[Root @ Varnish-server ~] # Chown-r varnish: Varnish/data/Varnish/Cache
[Root @ Varnish-server ~] # Chown-r varnish: Varnish/data/Varnish/log
2. Obtain varnish Software
Varnish's official site is http://varnish-cache.org, which contains the latest varnish instructions and version update history, from which you can find varnish download link in SourceForge, currently, the latest version of varnish is varnish 2.1.2. The package after the download is varnish-2.1.2.tar.gz. Here we will take this version as an example to install and configure it.
3. Install PCRE
If PCRE is not installed, the system will prompt that the PCRE Library cannot be found when compiling varnish2.0 or later versions. The PCRE Library must be installed first to ensure compatibility with regular expressions.
[Root @ Varnish-server ~] # Tar zxvf pcre-7.9.tar.gz
[Root @ Varnish-server ~] # Cd pcre-7.9/
[Root @ Varnish-server ~] #./Configure -- prefix =/usr/local/PCRE/
[Root @ Varnish-server ~] # Make & make install
4. Install Varnish
Install varnish in the/usr/local/directory as follows:
[Root @ Varnish-server ~] # Tar-zxvf varnish-2.1.2.tar.gz
[Root @ Varnish-server ~] # Cdvarnish-2.1.2
[Root @ Varnish-server ~] # Export pkg_config_path =/usr/local/PCRE/lib/pkgconfig
[Root @ Varnish-server ~] #./Configure -- prefix =/usr/local/varnish \
> -- Enable-dependency-trackin
> -- Enable-debugging-Symbols
> -- Enable-developer-Warnings
[Root @ Varnish-server ~] # Make
[Root @ Varnish-server ~] # Make install
[Root @ Varnish-server ~] # Cp RedHat/varnish. initrc/etc/init. d/Varnish
[Root @ Varnish-server ~] # Cp RedHat/varnish. sysconfig/etc/sysconfig/Varnish
"Pkg_config_path" specifies the path for varnish to find the PCRE Library. If PCRE is installed in another path, specify the corresponding path here, by default, varnish finds the PCRE Library path/usr/local/lib/pkgconfig. The last two steps are to copy some initialization script files of the varnish daemon. These scripts are used for varnish startup and shutdown management. The following sections will explain in detail.
So far, varnish has been installed.
 

Ii. Configure Varnish

1. VCL instructions
VCL, that is, varnish configuation language, is used to define varnish access policies. VCL syntax is relatively simple, similar to C and Perl, you can use the specified operator "= ", comparison operator "=", logical operator "!, &&,!!" . It also supports regular expression samples and "~" You can also use a keyword such as "set" to specify a variable.
Note that the "\" character has no special meaning in VCL, which is slightly different from other languages. In addition, VCL is only a configuration, not a real programming language and has no loops, no custom variables.
Before configuring varnish, you must first understand the varnish configuration syntax, VCL, and detailed introduction to some built-in functions and public variables commonly used by VCL.
VCL built-in functions
(1) vcl_recv Function
It is used to receive and process requests. When a request arrives and is successfully received, it is called. It determines how to process the request by judging the request data.
This function ends with the following keywords:
Pass: Indicates entering the pass mode and giving the request control to the vcl_pass function.
Pipe: Indicates entering the pipe mode and giving the request control to the vcl_pipe function.
Error Code [reason]: indicates that "code" is returned to the client and the request is not processed. "code" is the error identifier, for example, 200 or 405, "reason" is the error message.
(2) vcl_pipe Function
This function is called when it enters pipe mode. It is used to pass requests directly to the backend host. If the request and returned content are not changed, the unchanged content is returned to the client, until the link is closed.
This function ends with the following keywords:
Error Code [reason]
Pipe
(3) vcl_pass Function
This function is called when the pass mode is enabled. It is used to pass requests directly to the backend host. The backend host responds to the data and sends the data to the client without any caching, the latest content is returned for each connection.
This function ends with the following keywords:
Error Code [reason]
Pass
(4) Lookup
Indicates to search for the requested object in the cache, and give control to the function vcl_hit or function vcl_miss based on the search result.
(5) vcl_hit Function
After the lookup command is executed, if the request content is found in the cache, the function is automatically called.
This function ends with the following keywords:
Deliver: sends the found content to the client and gives control to the function vcl_deliver.
Error Code [reason]
Pass
(6) vcl_miss Function
After the lookup command is executed, this method is automatically called if the requested content is not found in the cache. This function can be used to determine whether the content needs to be retrieved from the backend server.
This function ends with the following keywords:
Fetch: gets the requested content from the backend and gives control to the vcl_fetch function.
Error Code [reason]
Pass
(7) vcl_fetch Function
Call this method after updating the cache from the backend host and obtaining the content. Then, you can determine whether to put the content into the cache or directly return it to the client by determining the obtained content.
This function ends with the following keywords:
Error Code [reason]
Pass
Deliver
(8) vcl_deliver Function
After finding the request content in the cache, call this method before sending it to the client. This function ends with the following keywords:
Error Code [reason]
Deliver
(9) vcl_timeout Function
This function is called before the cache content expires. It ends with the following keywords:
Discard: indicates to clear the content from the cache.
Fetch
(10) vcl_discard Function
This method is automatically called when the cached content expires or the cache space is insufficient. It ends with the following keywords:
Keep: indicates that the content is retained in the cache.
Discard
 

2. VCL processing Flowchart
Through the introduction of VCL functions, the reader has an understanding of the functions implemented by each function. In fact, each function is associated with each other, lists A running flowchart for varnish to process HTTP requests.

The process is roughly divided into the following steps:
(1) The receive status, that is, the entry status of request processing. Based on VCL rules, the request should be pass or pipe, or enter Lookup (local query ).
(2) lookup status. After entering this status, data is searched in the hash table. If it is found, it enters the hit status; otherwise, it enters the Miss status.
(3) pass status. In this status, a backend request enters the fetch status.
(4) Fetch Status: In fetch status, requests are obtained from the backend, requests are sent, data is obtained, and local storage is performed.
(5) Deliver status, send the obtained data to the client, and then complete the request.
3. built-in public variables
The built-in public variables of VCL can be used in different VCL functions. Based on the different stages of using these public variables, the following describes them in sequence.
When the request arrives, the public variables that can be used are shown in table 2:
Table 2
Meanings of Public variable names
Req. backend specifies the corresponding backend host
Server. IP indicates the Server IP Address
Client. IP indicates the Client IP Address
Req. request specifies the request type, such as get, Head, and post.
Req. url specifies the request address
Req. proto indicates the HTTP protocol version of the Request initiated by the client
Req. http. header indicates the HTTP header information in the corresponding request.
Req. restarts indicates the number of restart requests. The default value is 4.
When varnish requests to a backend host, the public variables that can be used are shown in table 3:
Table 3
Meanings of Public variable names
Beresp. request specifies the request type, such as get and head.
Beresp. url specifies the request address
Beresp. proto indicates the HTTP protocol version of the Request initiated by the client
Beresp. http. header indicates the HTTP header information in the corresponding request.
Beresp. TTL indicates the cache lifecycle, that is, the cache retention time, in seconds.
The public variables that can be used after obtaining content from the cache or the backend host are shown in table 4:
Table 4
Meanings of Public variable names
OBJ. Status indicates the Request status code of the returned content, such as 200, 302, and 504.
OBJ. cacheable indicates whether the returned content can be cached. That is to say, if the HTTP return is 200, 203, 300, 301, 302, 404, 410, and so on, and there is a non-zero lifetime, the returned content can be cached.
OBJ. Valid indicates whether an HTTP response is valid.
OBJ. response indicates the Request status information of the returned content.
OBJ. proto indicates the HTTP protocol version of the returned content
OBJ. TTL indicates the lifecycle of the returned content, that is, the cache time, in seconds.
OBJ. lastuse indicates the interval between the last request and the current time, in seconds.
Public variables that can be used to respond to a client are shown in table 5:
Table 5
Meanings of Public variable names
Resp. Status indicates the HTTP status code returned to the client.
Resp. proto indicates the HTTP protocol version returned to the client
Resp. http. header indicates the HTTP header information returned to the client.
Resp. response indicates the HTTP status information returned to the client.
In the above description, we only introduced common VCL built-in public variables. To learn more about and use public variables, please visit the varnish official website.

3. configure a simple varnish instance
The varnish configuration file may be written differently due to different versions. varnish2.x and 1. not only are the configuration files written differently in Version X, but the features of the new version also increase a lot. Besides many application bugs, the version described here is varnish2.1.2, the configuration file format is also based on varnish2.x.
After varnish is installed, the default configuration file is/usr/local/Varnish/etc/Varnish/default. VCL. All contents of this file are commented out by default. Here, we use this file as the template to create a new file VCL. conf and put it in the/usr/local/Varnish/etc directory. the conf file is as follows:
# Backend defines a backend host named webserver. ". Host" specifies the IP address or domain name of the backend host, and ". Port" specifies the service port of the backend host. "192.168.12.26" is a backend web server.
Backend webserver {
. Host = "192.168.12.26 ";
. Port = "80 ";
}

# Start by calling vcl_recv.
Sub vcl_recv {
If (req. http. X-forwarded-){
Set Req. http. X-forwarded-for =
Req. http. X-forwarded-for "," client. IP;
} Else {
Set Req. http. X-forwarded-for = client. IP;
}
# If the request type is not get, head, put, post, Trace, options, or delete, the pipe mode is enabled. Note that the relationship is.
If (req. Request! = "Get "&&
Req. Request! = "Head "&&
Req. Request! = "Put "&&
Req. Request! = "Post "&&
Req. Request! = "Trace "&&
Req. Request! = "Options "&&
Req. Request! = "Delete "){
Return (PIPE );
}
# If the request type is not get or head, the pass mode is enabled.
If (req. Request! = "Get" & Req. Request! = "Head "){
Return (PASS );
}

# Cache the two domain names ixdba.net or ixdba.cn. This is a wildcard domain name concept, that is, all domain names ending with ixdba.net or ixdba.cn are cached.
If (req. http. Host ~ "^ (. *) .Ixdba.net" | Req. http. Host ~ "^ (. *) .Ixdba.cn "){
Set Req. backend = webserver;
}

# End with. jsp and. Do and? To read the content from the backend server.
If (req. url ~ "\. (JSP | Do) ($ | \?) "){
Return (PASS );
} Else {
Return (lookup );
}
}
 
Sub vcl_pipe {
Return (PIPE );
}
 
Sub vcl_pass {
Return (PASS );
}
 
Sub vcl_hash {
Set Req. Hash + = Req. url;
If (req. http. HOST ){
Set Req. Hash + = Req. http. Host;
} Else {
Set Req. Hash + = server. IP;
}
Return (hash );
}
 
Sub vcl_hit {
If (! OBJ. cacheable ){
Return (PASS );
}
Return (deliver );
}
 
Sub vcl_miss {
Return (FETCH );
}
 
Sub vcl_fetch {
If (! Beresp. cacheable ){
Return (PASS );
}
If (beresp. http. Set-cookie ){
Return (PASS );
}

# When the URL contains servlet, no cache is performed.
If (req. url ~ "^/Servlet /"){
Return (PASS );
}

# When a URL contains services, no caching is performed.
If (req. url ~ "^/Services /"){
Return (PASS );
}

# If the request type is get and the request URL contains upload, cache the request. The cache duration is 300 seconds, that is, 5 minutes.
If (req. Request = "get" & Req. url ~ "^/Upload (. *) $ "){
Set beresp. TTL = 300 s;
}
# When the request type is get and the request URL ends with PNG, XSL, XML, GIF, CSS, and JS, the cache duration is 600 seconds.
If (req. Request = "get" & Req. url ~ "\. (PNG | XSL | XML | PDF | PPT | Doc | docx | CHM | RAR | zip | BMP | JPEG | SWF | ICO | MP3 | MP4 | rmvb | Ogg | mov | Avi | WMV | SWF | TXT | PNG | GIF | JPG | CSS | JS | HTML | htm) $ "){
Set beresp. TTL = 600 s;
}
Return (deliver );
}
# Add a header ID to determine whether the cache is hit.
Sub vcl_deliver {
If (obj. Hits> 0 ){
Set resp. http. X-Cache = "hit from www.ixdba.net ";
} Else {
Set resp. http. X-Cache = "Miss from www.ixdba.net ";
}
Return (deliver );
}