It seems that squid can be used as a proxy to defend against DoS attacks. Prepare to have a try and make some squid data backup...

Squid optimization full manual (1)

Summary
This article focuses on optimizing squid performance without changing hardware conditions (12:31:57)

By ideal

I. Overview

Squid is an excellent open source proxy server software that can run on multiple system platforms. However, it has obvious disadvantages compared with other commercial products, that is, its hit rate and efficiency are relatively low.

This article focuses on optimizing squid performance without changing the hardware:

1. Compile an efficient and streamlined new kernel;
2. Use the reiserfs Log File System for the cache partition;
3. recompile squid;
4. Optimize squid configuration;

For the optimization results, I used the well-known cacheflow testing tool cfmc for testing and comparison:

First, use a script provided by cfmc to extract rul from the access. log of squid. conf, and then perform tests based on these rul.

Server Configuration:

Model: HP lh3
CPU: pii450
Memory: 256 Mbit/s 100 MHz ecc sdram dimm
Hard Disk: 9.1 GB hot-swap ultra2 Hard Disk
NIC: Ethernet Express Pro 100 10/100 m x 2
OS: RedHat 7.1

Test results before optimization:

Iteration 0: Cumulative Statistics; 933 seconds elapsed
Total objects: 72599, total object size: 513211102 bytes
Average object size: 7069 bytes
Average object Response Time: 2707 milliseconds
Objects per second: 77.81
Bytes per second: 550065, Min: 550065, Max: 879873
URLs discarded due to socket or connection failure: 6955
Redirections: 1017, cookied objects: 1036
Pragma no-Cache objects: 1656, Non-200 HTTP Response codes: 2505

Test results after optimization:

Iteration 0: Cumulative Statistics; 688 seconds elapsed
Total objects: 72599, total object size: 403833100 bytes
Average object size: 5562 bytes
Average object Response Time: 1890 milliseconds
Objects per second: 105.52
Bytes per second: 586966, Min: 586966, Max: 995582
URLs discarded due to socket or connection failure: 16372
Redirections: 1658, cookied objects: 1000
Pragma no-Cache objects: 1454, Non-200 HTTP Response codes: 3132

By comparing the above test results, we can see that:

After optimization, the objects per second is increased by 35.6%, and the average object response time is reduced by 43.2%. The overall performance is improved significantly.

2. Compile the new kernel

We use this method to compile the kernel: Cancel kernel module support and compile all the server hardware drivers into the kernel, in addition, you must also compile the support for the reiserfs file system into the kernel to increase system security while providing system performance. In versions 2.4.10 and later, the reiserfs file system has been built in.

First.

Then unlock the kernel file:

Tar xvzf linux-2.4.12.tar.gz

Enter the newly generated directory and execute:

CD Linux
Make mrproper
Make config

Based on the configuration of my server, I choose to compile the kernel as follows:

* Code maturity level options
Prompt for development and/or incomplete code/drivers (config_experimental) [Y/n/?]
* Loadable module support
Enable loadable module support (config_modules) [N/y/?]
* Processor type and features
Processor family (386,486,586/K5/5x86/6x86/6 x86mx, Pentium-classic, Pentium-mm
X, Pentium-Pro/celeon/Pentium-II, Pentium-III/celeon (Coppermine), Pentium-4, K
6/K6-II/K6-III, athlon/duron/K7, Crusoe, Winchip-C6, Winchip-2, Winchip-2A/Winch
IP-3, cyrixiii/C3) [Pentium-Pro/celeon/Pentium-II]
* General setup
Networking Support (config_net) [Y/n/?]
PCI support (config_pci) [Y/n/?]
PCI access mode (bios, direct, any) [any]
Defined config_pci_goany
PCI device name database (config_pci_names) [Y/n/?]
System v ipc (config_sysvipc) [Y/n/?]
Sysctl support (config_sysctl) [Y/n/?]
Kernel core (/proc/kcore) format (ELF, A. Out) [elf]
Defined config_kcore_elf
* Plug and Play Configuration
Plug and Play support (config_pnp) [Y/n/?]
* Block devices
Normal PC floppy disk support (config_blk_dev_fd) [Y/n/?]
* Networking options
Kernel/user netlink socket (config_netlink) [Y/n/?]
Routing messages (config_rtnetlink) [Y/n/?]
Network Packet Filtering (replaces ipchains) (config_netfilter) [Y/n/?]
UNIX domain sockets (config_unix) [Y/n/?]
TCP/IP networking (config_inet) [Y/n/?]
IP: Advanced router (config_ip_advanced_router) [Y/n/?]
IP: Policy Routing (config_ip_multiple_tables) [Y/n/?]
IP: Use netfilter mark value as routing key (config_ip_route_fwmark) [Y/n/?]
IP: fast network address translation (config_ip_route_nat) [Y/n/?]
IP: equal cost multipath (config_ip_route_multipath) [Y/n/?]
* IP: netfilter Configuration
Connection Tracking (required for masq/NAT) (config_ip_nf_conntrack) [Y/n/?]
FTP protocol support (config_ip_nf_ftp) [Y/n/?]
IP Tables support (required for filtering/masq/NAT) (config_ip_nf_iptables) [Y/n/?]
Limit match support (config_ip_nf_match_limit) [Y/n/?]
Multiple port match support (config_ip_nf_match_multiport) [Y/n/?]
Connection state match support (config_ip_nf_match_state) [Y/n/?]
Packet Filtering (config_ip_nf_filter) [Y/n/?]
Full NAT (config_ip_nf_nat) [Y/n/?]
Redirect target support (config_ip_nf_target_redirect) [Y/n/?]
Log target support (config_ip_nf_target_log) [Y/n/?]
ATA/IDE/MFM/rll support (config_ide) [Y/n/?]
* IDE, ATA and atapi Block devices
Enhanced IDE/MFM/rll Disk/CDROM/tape/Floppy support (config_blk_dev_ide) [Y/n/?]
Include IDE/ATA-2 disk support (config_blk_dev_idedisk) [Y/n/?]
Use multi-mode by default (config_idedisk_multi_mode) [Y/n/?]
Include IDE/atapi CDROM support (config_blk_dev_idecd) [Y/n/?]
* SCSI support
SCSI support (config_scsi) [Y/n/?]
* SCSI support type (disk, tape, CD-ROM)
SCSI disk support (config_blk_dev_sd) [Y/n/?]
Maximum number of SCSI disks that can be loaded as modules (config_sd_extra_devs) [8]
* Some SCSI devices (e.g. CD Jukebox) support multiple Luns
Enable extra checks in new queueing code (config_scsi_debug_queues) [Y/n/?]
* SCSI low-level Drivers
Ami megaraid support (config_scsi_megaraid) [Y/n/?]
Sym53c8xx SCSI support (config_scsi_sym53c8xx) [Y/n/?]
Default tagged command queue depth (config_scsi_ncr53c8xx_default_tags) [4]
Maximum number of queued commands (config_scsi_ncr53c8xx_max_tags) [32]
Synchronous transfers frequency in MHz (config_scsi_ncr53c8xx_sync) [80]
* Network Device Support
Network Device Support (config_netdevices) [Y/n/?]
* Ethernet (10 or 100 Mbit)
Ethernet (10 or 100 Mbit) (config_net_ethernet) [Y/n/?]
EISA, VLB, PCI and on board controllers (config_net_pci) [Y/n/?]
Etherexpresspro/100 support (config_eepro100) [Y/n/?]
* Input core support
Input core support (config_input) [Y/n/?]
Keyboard support (config_input_keybdev) [Y/n/?]
* Character Devices
Virtual terminal (config_vt) [Y/n/?]
Support for console on virtual terminal (config_vt_console) [Y/n/?]
Standard/generic (8250/16550 and compatible uarts) Serial support (config_serial) [Y/n/?]
Unix98 Pty support (config_unix98_ptys) [Y/n/?]
Maximum number of unix98 ptys in use (0-2048) (config_unix98_pty_count) [8]
* File systems
Reiserfs support (config_reiserfs_fs) [Y/n/?]
ISO 9660 CDROM File System Support (config_iso9660_fs) [Y/n/?]
/Proc file system support (config_proc_fs) [Y/n/?]
/Dev/PTS File System for unix98 ptys (config_devpts_fs) [Y/n/?]
* Console drivers
VGA text Console (config_vga_console) [Y/n/?]

It should be noted that if you want to use the transparent proxy mode, you should carefully select netfilter, because we have to use iptables to redirect the TCP packet to meet the requirements of transparent proxy.

If your server configuration is different from mine, you only need to change the corresponding hardware configuration parameters, such as the CPU type, Nic, and SCSI hard disk.

Next:

Make bzimage

After that, copy the new kernel bzimage under the arch/i386/boot/directory to the/boot/directory and change it to OPT:

Cp arch/i386/boot/bzimage/boot/
MV/boot/bzimage/boot/OPT

Edit Lilo. conf as follows:

Boot =/dev/SDA
Map =/boot/Map
Install =/boot. B
Prompt
Timeout = 50
Message =/boot/message
Default = OPT

Image =/boot/vmlinuz-2.4.2-2
Label = Linux
Initrd =/boot/initrd-2.4.2-2.img
Read-Only
Root =/dev/sda1

Image =/boot/OPT
Label = OPT
Read-Only
Root =/dev/sda1

Last run:

# Lilo
Added Linux
Added opt *

Okay, so we can start it with the new kernel. You can find that the self-compiled kernel is much smaller and faster.
Squid optimization full manual (2)

Summary
This article focuses on optimizing squid performance without changing hardware conditions (12:31:55)

By ideal

3. Use the reiserfs File System

Some tests have been conducted. Under the same conditions, if the cache partition uses the reiserfs file system, the performance is 20% higher than that of the ext2, so we will use the reiserfs File System in the cache partition. In the previous step, we have provided support for reiserfs in the kernel. Next we will re-format the original cache partition into a reiserfs file system.

First:

Tar xvzf reiserfsprogs-3.x.0j.tar.gz

Enter the newly generated directory and execute:

./Configure
Make
Make install

This will generate four reiserfs tools: mkreiserfs, reiserfsck, debugreiserfs, and resize_reiserfs.

After that, we have successfully installed the reiserfs tool. In this case, assume that the original cache partition is/dev/sda7 and the directory to be loaded is/cache. Before formatting it, we need to first umount the original partition:

Umount/cahce

Format the partition. Run the following command:

Mkreiserfs-H R5/DE/sda7

After that, modify/etc/fstab and change the/cache line:

/Dev/sda7/cache reiserfs notail, noatime 0 0

Then restart.

Iv. recompile squid

A large number of tests show that the combination of squid-2.2.STABLE5 + HNO patch is much more stable and efficient than Squid 2.3 or other versions, if you do not believe it, you can perform the test by yourself for several days. So we will use this version of squid.

First download squid-2.2.stable5-src.tar.gz from http://www.squid-cache.org/versions/v2/2.2:

Tar xvzf squid-2.2.STABLE5-src.tar.gz
Gunzip-D squid-2.2.STABLE5-hno.20000202.snapshot.gz

Then patch:

CD squid-2.2.STABLE5
Patch-P1 <../squid-2.2.STABLE5-hno.20000202.snapshot

Next, we can start to compile squid. In addition to adopting asynchronous io (multi-thread mode), we follow the principle: Remove all unnecessary functions, as shown below:

. /Configure -- prefix =/usr -- exec_prefix =/usr -- bindir =/usr/sbin -- libexecdir =/usr/lib/squid -- localstatedir =/var -- sysconfdir =/etc/squid -- Mandir =/usr/share/man -- enable-async-IO = 20 -- disable-ICMP -- disable-delay-pools -- disable-Mem-gen-trace -- disable-useragent- log -- enable-kill-parent-hack -- disable-ARP-ACL -- enable-poll -- disable-Ident-lookups

Make
Make install

Among them, -- enable-async-IO = 20 indicates that we adopt asynchronous Io and 18 threads.

After compilation, we can start configuring squid.

5. Optimize squid Configuration

My squid. conf and Related explanations are as follows:

# Remove support for proxy Arrays

Icp_port 0

# Set the location of log files and PID files

Cache_store_log none
Cache_access_log/var/log/squid/access. Log
Cache_log/var/log/squid/cache. Log
Emulate_httpd_log on
Pid_filename/var/run/squid. PID

# Set the running user and group permissions

Cache_inclutive_user squid
Cache_paitive_group squid

# Setting management information

Visible_hostname proxy.yxtc.edu.cn
Cache_mgr bye2000@yxtc.edu.cn

# Set the listening address and port

Http_port 3128
Tcp_incoming_address x. x
Udp_incoming_address x. x

# See the following supplementary description.

Cache_mem 32 MB
Cache_dir/cache 6000 14 256

# Set the cache object timeout time

Reference_age 3 months

# Access control settings

ACL mynet SRC 192.168.1.0/255.255.255.0
ACL all SRC 0.0.0.0/0.0.0.0
Http_access allow mynet
Http_access deny all

# Transparent proxy settings

Httpd_accel_host virtual
Httpd_accel_port 80
Httpd_accel_with_proxy on
Httpd_accel_uses_host_header on

# Swap Performance Tuning

Half_closed_clients off
Cache_swap_height 100%
Cache_swap_low 80%
Maximum_object_size 1024 KB

# See supplementary instructions

Refresh_pattern-I. html 1440 90% 129600 reload-into-IMS
Refresh_pattern-I. shtml 1440 90% 129600 reload-into-IMS
Refresh_pattern-I. hml 1440 90% 129600 reload-into-IMS
Refresh_pattern-I. gif 1440 90% 129600 reload-into-IMS
Refresh_pattern-I. SWF 1440 90% 129600 reload-into-IMS
Refresh_pattern-I. jpg 1440 90% 129600 reload-into-IMS
Refresh_pattern-I. PNG 1440 90% 129600 reload-into-IMS
Refresh_pattern-I. BMP 1440 90% 129600 reload-into-IMS
Refresh_pattern-I. js 1440 90% 129600 reload-into-IMS

Note:

1. cache_mem 32 MB

Note: cache_mem is not the memory size that squid can use, but the physical memory size of squid user hot object, so this value can be smaller.

2. cache_dir/cache 6000 14 256

For the calculation of level 1 and level 2 sub-directories, refer to the author's previous article "Using Linux to set up a proxy server (upper) (middle) (lower)";

3. refresh_pattern-I. html 1440 90% 129600 reload-into-IMS, etc.

These statements force control the object timeout time, which violates the spirit of the HTTP protocol. However, when the bandwidth is narrow, the system response time can be significantly improved.

4. Note that the/cache directory and log file permissions must be set to squid for all users and groups;

5. You can use the RPM package script/etc/rc. d/init. d/squid to control squid, or use the squid command to control squid. For details, see squid-H.