Lighttpd + squid + Apache to build an efficient Web Server

ArticleDirectory

• Lighttpd Configuration
• Squid Configuration
• Apache configuration

Architecture Principle

Apache is usually the preferred web server in the Open Source Field. Because of its powerful and reliable nature, Apache has a brand effect and can be applied to most applications. However, its strength is sometimes cumbersome, and the configuration file is daunting. In high concurrency, the efficiency is not very high. The lightweight Web Server Lighttpd is a rising star. Its static file response capability is much higher than that of Apache, which is said to be 2-3 times that of Apache. The high performance and ease of use of Lighttpd are enough to impress us and use it whenever possible in the field where it is competent. Lighttpd also supports PHP well and supports other languages such as Python through FastCGI.

After all, Lighttpd is a lightweight server and cannot be compared with Apache in terms of functions. Some applications are not competent. For example, Lighttpd does not support caching, but most websites currently useProgramDynamic Content is generated. Without caching, even if the program is more efficient, it is difficult to meet the needs of large traffic. It is meaningless to keep the program working on the same thing. First, the web program needs to cache the data that is used repeatedly. Even this is not enough. Simply starting a web handler costs a lot, and caching the final static page is essential. This is squid's strength. It serves as a proxy and supports efficient caching. It can be used to provide reverse proxy acceleration for websites. Put squid on the front end of Apache or Lighttpd to cache the dynamic content generated by the Web server, and the Web application only needs to set the page validity time appropriately.

Even for websites dynamically generated by most of the content, there are still some static elements, such as slice, JS scripts, CSS, etc. Placing squid on the Apache or lighttp front-end will lead to performance degradation, after all, processing HTTP requests is the strength of web servers. In addition, static content that already exists in the file system is cached in Squid, which wastes memory and hard disk space. Therefore, you can consider placing Lighttpd in front of squid to form a processing chain of Lighttpd + squid + Apache. Lighttpd is at the very beginning and is used to process static content requests, the dynamic content request is forwarded to Squid through the proxy module. If the content of the request in Squid does not expire, it is directly returned to Lighttpd. New requests or expired page requests are handled by the Apache web program. After Lighttpd and squid filtering, Apache will greatly reduce the number of requests to be processed, reducing the pressure on Web applications. At the same time, this architecture allows for different processing to be distributed across multiple computers, and the Lighttpd checks each other.

In this architecture, each level can be individually optimized. For example, Lighttpd can adopt asynchronous Io mode, squid can enable memory for caching, and Apache can enable MPM, in addition, multiple machines can be used at each level to balance the load, which is highly scalable.

Instance description

The following uses several sites in the daviesliu.net and rainbud.net domains as examples to introduce the specific practices of this solution. In the daviesliu.net domain, there are several blog sites implemented using mod_python, several PHP sites, and a small mod_python program. Several PHP and Django sites may be set up in the future. The server is very weak, the CPU is celon 500, and the memory is PC 100 384 M. Therefore, the efficiency of web servers is more important. These sites are deployed on the same port of the same machine in the form of virtual hosts.

Lighttpd serves port 80, squid runs on port 3128, and Apache runs on port 81.

Lighttpd Configuration

Multiple Domain Names use the/var/www/domain/subdomain directory structure. Use the evhost module to configure document-root as follows:

Evhost. Path-pattern = var. basedir + "/% 0/% 3 /"

Ftpsearch has a Perl script and must enable CGI support. It is used for FTP site search and cache is of little significance. It is directly processed by mod_cgi of Lighttpd:

$ HTTP ["url"] = ~ "^/Cgi-bin/" {# Only allow CGI's in this directory
Dir-listing.activate = "Disable" # disable directory listings
CGI. Assign = (". pl" => "/usr/bin/Perl", ". cgi" => "/usr/bin/Perl ")
}

BBS uses phpBB, which has a low access volume. You can place it in Lighttpd (FastCGI) or Apache (mod_php) and temporarily use Lighttpd to set FastCGI processing for all. PHP page requests:

FastCGI. server = (". PHP "=> (" host "=>" 127.0.0.1 "," Port "=> 1026, "bin-path" => "/usr/bin/PHP-cgi ")))

Blog.daviesliu.net and blog.rainbud.net are blogxp programs written in mod_python. All static content has an extension, but dynamic content has no extension. Blogxp uses a python program to generate data in XML format and then convert it to an HTML page using mod_xslt. It can only be run under Apache. This site uses the typical Lighttpd + squid + Apache method for processing:

$ HTTP ["host"] = ~ "^ Blog "{
$ HTTP ["url"]! ~ "\."{
Proxy. Server = ("" => ("localhost" => ("host" => "127.0.0.1", "Port" => 3128) # port 3128 is
}
}

Share has a static page, and also uses mod_python to process requests. In/cgi:

$ HTTP ["host"] = ~ "^ Share "{
Proxy. Server = (
"/Cgi" => ("localhost" => ("host" => "127.0.0.1", "Port" => 3128 ))
)
}

Squid Configuration

Only Local access is allowed:

Http_port 3128
Http_access allow localhost
Http_access deny all

Enable reverse proxy:

Httpd_accel_host 127.0.0.1
Httpd_accel_port 81 # Apache Port
Httpd_accel_single_host on
Httpd_accel_with_proxy on # enable Cache
Httpd_accel_uses_host_header on # enable VM support

In this direction, the proxy supports all domain names on the host.

Apache configuration

Configure/etc/CONF. d/apache2 to load the mod_python, mod_xslt, and mod_php modules:

Apache2_opts = "-D Python-d xslt-D PhP5"

Root directory of all websites:

<Directory "/var/www">
AllowOverride all # Allow. htaccess to overwrite
Order allow, deny
Allow from all
</Directory>

Domain name-based VM:

<Virtualhost *: 81>
Servername blog.daviesliu.net
DocumentRoot/var/www/daviesliu.net/blog
</Virtualhost>

The evhost configuration of Lighttpd is obviously not convenient here.

The. htaccess settings under blog.daviesliu.net (for ease of development, do not restart Apache ):

Sethandler mod_python
Pythonhandler blogxp. Publisher
Pythondebug on
Pythonautoreload on

<Filesmatch "\.">
Sethandler none # static files are directly processed by Apache
</Filesmatch>

<Ifmodule mod_polict.c>
Addtype text/XSL. XSL # prevents transformation of XSL files
Addoutputfilterbytype mod_xslt text/XML
Xsltcache off
Xsltprocess on
</Ifmodule>
Header set Pragma "cache"
Header set cache-control "cache"

In blogxp. Publisher, you also need to set the type and expiration time of the returned document:

Req. content_type = "text/XML"
Req. headers_out ['expires'] = formatdate (Time. Time () + 60*5)

With this configuration, all sites can be accessed through ports 80, 3128, and 81, and port 80 is used for external access to reduce load. Port 81 Can Be Used for debugging during development without caching.

Performance Testing

Due to limited time and energy, we only use AB2 for a non-standard performance comparison test (each item is tested multiple times on average). The evaluation indicator is the number of requests per second.
The test command is used to test the Lighttpd concurrency of 10 requests scripts/prototype. JS:

AB2-N 1000-C 10 http://blog.daviesliu.net: 80/scripts/prototype. js

Static content: Prototype. js (27kb)

Con	Lighttpd (: 80)	Squid (: 3128)	Apache (: 81)
1	380	210	240
10	410	215	240
100	380	160	230

It can be seen that the Lighttpd performance is strong in static content, while squid performance is worse than that of the other two Web servers without memory cache configuration.

Dynamic page:/RSS (31 KB)

Con	Lighttpd (: 80)	Squid (: 3128)	Apache (: 81)
1	103	210	6.17
10	110	200	6.04
100	100	100	6.24

In terms of dynamic content, squid plays an obvious role, while Lighttpd is limited by Squid's efficiency, and it also needs to be much lower. If multiple squids are used for balancing, the effectiveness of Lighttpd can be realized.
In the case of a single machine with few static content, you can place squid at the beginning without Lighttpd.