Squid + MRTG implement complete cache proxy and HTTP service acceleration proxy

Squid + MRTG implement complete cache proxy and HTTP service acceleration proxy

By mirolers

Author: zitianxing
Time: 2002-9-25
Source: www.hackerabc.com

Squid is an open source code high-performance Proxy Cache Server. It supports proxy for FTP, Gopher, and HTTP data objects. In addition, squid also supports SSL, diversified access control, and complete log functions of proxy requests. Through the lightweight Internet Cache Protocol, squid allocates memory in a mesh or hierarchical manner, which can save extra bandwidth. The HTTP service acceleration proxy is provided. The web server pushes HTTP data to the Squid cache. External requests can extract the required data through the cache to implement dynamic images of the Web server.
In addition, to monitor the squid server, MRTG is used. MRTG is short for Multi Router Traffic Grapher. It is mainly used to monitor the traffic of network service devices and generate HTML files containing images to intuitively display the traffic of network services, to provide monitoring of the Squid cache service.
You can use squid + MRTG to configure a complete proxy service provision solution. The specific configuration steps are as follows:
I. Basic preparations
Squid: http://www.squid-cache.org
Version: squid-2.4.STABLE4-src.tar.gz
MRTG: http://people.ee.ethz.ch /~ Oetiker/WebTools/MRTG/pub/
Version: mrtg-2.9.22.tar.gz
Machine configuration: MB memory, 9 GB hard disk (Private cache directory for 4 GB partition), pii433.

Ii. Installation Steps
Decompress squid (generally, include the software in the/OPT directory)
# Tar zxvf squid-2.4.STABLE4-src.tar.gz
# Cd squid
// Compile
#./Configure -- prefix =/usr/local/squid -- enable-delay-pools -- enable-SNMP -- enable-Cache-digests -- enable-auth-modules = "NCSA"
# Make all
# Make install
* Interpretation of compiled projects:
-- Prefix is the directory where squid is stored. Here it is/usr/local/squid.
-- Enable-delay-pools this option enables a latency pool to limit the rated bandwidth for certain requests.
-- Enable-SNMP: This option allows MRTG to monitor the server's traffic status using the SNMP protocol. Therefore, you must select this option to enable squid to support the SNMP interface.
-- Enable-Cache-digests enables cache summarization. Originally, this project aims to quickly discover cache objects between squid cluster services. This project is used locally to speed up requests, the speed at which the cached content is retrieved.
-- Enable-auth-modules this compilation option enables the authentication module to authorize the access proxy user.

Install MRTG
# Tar zxvf mrtg-2.9.22.tar.gz
# Cd MRTG *
#./Configure -- prefix =/usr/local/mrtg-2
# Make
# Make install
/* Notes:
1. Most MRTG programs are written in Perl scripts and must be run in Perl version 5.005 or later. RH 7.2 comes with Perl version v5.6.0, which meets the requirements. (If the version is not enough, download and install it at www.perl.com .)
2. MRTG must support three libraries for processing graphics: zlib, Gd, and LibPNG. rh7.2 meets this requirement. (If not installed, go to the http://www.boutell.com/gd/ http://www.libpng.org/pub/png/src/ http://www.gzip.org/zlib to download the installation)

Iii. Specific Configuration
Lsquid configuration (only required options are configured and described ):
Http_port 80
This option is used for the port listened by the Squid proxy. To implement the HTTP acceleration proxy, the listener port 80
Tcp_outgoing_address 255.255.255.255
This syntax specifies the IP address of the remote server. In this example, we use 255.255.255.255 to specify all IP addresses.
Tcp_incoming_address
The http_port syntax can be used to bind the port and IP address. The client request. Optional.
Cache_mem 64 MB
Because squid requires a large amount of memory for reading and writing disks, I/O channels, data storage, and so on, we can slightly increase the memory value to provide server performance.

Cache_swap_low 95
This is a baseline that represents the replacement of cached internal objects. When the disk usage of the SWAp partition exceeds this baseline, the cache object replacement begins. The percentage is used as a metric.

Cache_swap_high 98
This flag is used to replace cached objects to the highest degree. When the disk utilization of swap partitions is close to this setting, the object is updated more vigorously. Here, because my cache is set to a large value, it is set to 98%.

The following configuration tags use the default system configuration.
{Maximum_object_size 4096 KB
{Objects larger than this size are not cached.
{Minimum_object_size 0 KB
{Objects smaller than this size are not cached.
{Maximum_object_size_in_memory 8 KB
{Maximum size of objects that can be cached in memory.
{Ipcache_size 1024
{Specify the IP cache size.
{Ipcache_low 90
{Minimum baseline of the cached IP address.
{Ipcache_high 95
{Maximum Cache IP address limit.
{Fqdncache_size 1024
{Cache DNS full domain name resolution size.

Cache_replacement_policy LRU
This flag is used to clear special objects in the cache when new objects are cached. Here, LRU is used to replace objects that have not been accessed for a long time. For other policies, see the configuration document.

Memory_replacement_policy LRU
The usage is the same as above. The difference is that the memory object is replaced.

Cache_dir ufs/cache 4000 16 256
Set the cache root directory to/cache. The cache type is ufs and the cache area is 4 GB. There can be 16 sub-directories, each with 256 sub-directories.

Cache_access_log/var/log/squid/access. Log
Cache_log/var/log/squid/cache. Log
Cache_store_log/var/log/squid/store. Log
Cache_swap_log/var/log/squid/swap. Log
The above items are the path and file name of the cache log record.

Log_ip_on_direct on
Record the IP address of the client host.

Log_fqdn on
Record all DNS domain name resolution.

Mime_table/usr/local/squid/etc/mime. conf
MIME file path used by squid

Pid_filename/usr/local/squid/logs/squid. PID
Squid process ID file.

Squid@ynst.net.cn ftp_user
You can log on to the FTP server anonymously.

Ftp_list_width 32
The length of the FTP file list. If the length is exceeded, the file name is truncated.
Ftp_passive on
Allow active connection to the FTP server.

Dns_nameservers 202.98.187.1
Specifies the domain name resolution server of the proxy.

Authenticate_program/usr/local/squid/bin/ncsa_auth
/Usr/local/squid/etc/passwd
Proxy authentication program. Here, htpasswd is used in/usr/local/squid/etc to generate a password file. The specific method is as follows:
# Htpasswd-CDB passwd auth_usr1 pass1
# Htpasswd-DB passwd auth_usr2 pass2
# Htpasswd-DB passwd auth_usr3 pass3
Create the passwd file and add auth_usr1, auth_usr2, and auth_usr3 authenticated users.

Authenticate_children 5
The number of authentication processes to be generated.

Authenticate_ttl 3600.
This flag sets the authentication persistence time.

Authenticate_ip_ttl 1800
This flag sets the length of time for authentication binding IP addresses.

Authenticate_ip_ttl_is_strict on
This flag denies access from other non-authenticated IP addresses within the authenticate_ttl time.

Request_header_max_size 10 KB
Request_body_max_size 1 MB
Set the HTTP request header and data size.

Request_body_max_size 0
Here, the size of the data returned by the request is 0, meaning there is no limit.

Connect_timeout 180 seconds
Read_timeout 15 minutes
Request_timeout 20 seconds
The above items are several timeout settings of the proxy service. The default value of the configuration file is used here.

The following are some configuration items in the access control list. The access IP address, domain name, and access type are restricted by the permissions of the access proxy:
ACL acldomain dstdomain .ynst.net.cn
ACL acceleratedport port 80
ACL acceleratedhost DST 202.98.187.17/255.255.255.255
ACL aclprotocol proto HTTP FTP
ACL aclmethod method get post

ACL aclauth proxy_auth required
(Note: Here, required is used to indicate that any valid user in the passwd file is granted the access permission .)

ACL aclsnmp snmp_community secrect
MRTG can use the SNMP protocol to check the agent service running status.

ACL aclconn maxconn 20
Maximum number of parallel connections.

Enable the defined access control list
Http_access allow manager localhost
Http_access deny Manager
Http_access allow aclauth
Http_access allow acceleratedport acceleratedhost
Http_access deny all

The cache management settings are as follows:
Cache_mgr admin@ynst.net.cn
Set the receiving email address of the cache manager.

Cache_inclutive_user squidusr
Cache_effective_group squidgrp
Set the user and user group for executing squid.
{Note: you can modify the user and user group permissions as follows:
# Groupadd squidgrp
# Useradd-G squidgrp-D/home/squidusr
Modify the log permission attribute so that squidusr has the write permission.
# Chown squidusr/var/log/squid
# Chgrp squidgrp/var/log/squid
# Chown squidusr/var/log/squid/*. Log
# Chown squidgrp/var/log/squid/*. Log
}

Configure the HTTP service acceleration proxy
Httpd_accel_host 202.98.187.17
Httpd_accel_port 80
Httpd_accel_with_proxy on
(Note: If you configure proxy and acceleration services, you must enable the httpd_accel_with_proxy option. If you need to configure multiple acceleration, you can use the virtual concept .)

Snmp_port 3401
This flag sets the snmp_port to port 3401 to enable MRTG to monitor the service running status.

Snmp_access allow aclsnmp localhost
Snmp_access deny all
Snmp_incoming_address 0.0.0.0
Snmp_outgoing_address 0.0.0.0

The preceding options are used to configure SNMP settings for MRTG interaction information.

Digest_generation on
Digest_rebuild period 1 hour
Digest_rewrite_period 1 hour
Digest_swapout_chunk_size 4096 bytes
Digest_rebuild_chunk_percentage 10
Generate a cache digest and set the time required for digest reconstruction and related settings. The default settings are used here. The cache digest enables a massive cache content to form a micro-search, improving proxy performance.

Finally, we configure the latency queue pool.
The latency queue pool can restrict specific request data based on certain rules, so as to prevent malicious users from sending informal data strings to attack the server.
The following options are described only when necessary. The configuration is as follows:
Delay_pools 1 # define a pool
Delay_class 1 1 # pool is of the 1 Type
Delay_access 1 allow all # Allow all requests of type 1 to access
Delay_parameters 1 128000/128000 #128 K/s restricted access
Delay_initial_bucket_level 50 # Number of bandwidths used to initialize the pool when squid is started.
 
Run the Squid proxy service:
# Squid-z/* Create the cache directory structure used by squid
If you want to run
# Squid-ncdl
If you want to run
# Squid
Check whether squid is running
# Squid-K check

Lmrtg Configuration:
First, test whether the SNMP service is enabled:
# Snmpwalk-P 3401 hostname communitystring. 1.3.6.1.4.1.3495.1.1
If the following output is displayed:
Enterprises. nlanr. Squid. cachesystem. cachesysvmsize = 7970816
Enterprises. nlanr. Squid. cachesystem. cachesysstorage = 2796142
Enterprises. nlanr. Squid. cachesystem. cacheuptime = timeticks: (766299) 2:07:42. 99
SNMP works normally.
According to the documents of mrtg.hdl.com, you can use javasmaker and indexmaker to generate the web publishing files required by mrtg. You can create a configuration file for the configuration of mrtgusing the built-in maker, and indexmakeris the webfile generated by mrtgto create an index.html file. Here, I provide a more concise method:
(Thanks to the chris MRTG squid.htm file and index.html, you can download these two files from the http://www.psychofx.com/chris/unix/mrtg .)
The options for modifying the mrtg-squid.cfg file are as follows:
Modify workdir and loadmibs path settings
Modify all "hostname" in the file to your actual server name
Modify all the names and addresses required by the "Chris" Field
Modify community string attributes
The configuration is as follows:
Workdir:/home/www/MRTG/squid
Loadmibs:/usr/local/squid/etc/mib.txt
Modify secrect @ proxysvr for all public @ hostname

Create a directory such as/home/www/MRTG/squid.
# Mkdir/home/www/MRTG/squid
# Cd/home/www/MRTG/squid
# Cp/opt/mrtg-squid.cfg.
# MRTG mrtg-squid.cfg.
# Cp/opt/index.html.

Modify the following options for/etc/httpd/CONF/httpd. conf:
Listen 8000.
Port 8000
DocumentRoot/home/www/MRTG/squid

Because MRTG requires resources to read files, I set the read interval to 5 minutes:
Crontab-e
*/5 */usr/local/mrtg-2/bin/MRTG/home/www/MRTG/squid/mrtg-squid.cfg

In this way, we have established a complete proxy to accelerate access to the server. Through the powerful MRTG monitoring, we can count the running status of the device when the server is running. Since the above software complies with GNU, it is a powerful and economical way to build websites. Authentication control allows specific users to use the proxy service. Web acceleration servers become a convenient means of server mirroring.