Nginx + apache + mysql + php + memcached + squid build cluster web Environment

Server Load Balancing Scheme with a large number of users
I. Preface
Ii. Compilation and Installation
Iii. Install MySQL and memcache
4. install Apache, PHP, eAccelerator, and php-memcache
5. Install Squid
Vi. Postscript

I. Preface and preparations
Currently, the LAMP development mode is the first choice for WEB development. How to build an efficient, reliable, and stable WEB server is always a hot topic. This article is an attempt on this topic.
The architecture diagram is as follows:

Reference ----------------------------------------------------
| Client | ==>| Server Load balancer |==>| reverse proxy/cache |==>| WEB Server |==>| Database Server |
----------------------------------------------------
Nginx Squid Apache, PHP MySQL
Preparations for eAccelerator/memcache:
Reference server: Intel (R) Xeon (TM) CPU 3.00 GHz X 2, 2 GB mem, SCISC hard drive
Operating System: CentOs4.4, kernel version 2.6.9-22. ELsmp, gcc version 3.4.4
Software:
Apache 2.2.3 (MPM mode available)
PHP 5.2.0 (this version is used because the engine of 5.2.0 is relatively more efficient)
EAccelerator 0.9.5 (accelerates the PHP engine and encrypts the PHP source program)
Memcache 1.2.0 (used to cache frequently used data)
Libevent 1.2a (required for memcache operating mechanism)
MySQL 5.0.27 (use a binary version to save compilation effort)
Nginx 0.5.4 (used as a server Load balancer)
Squid-2.6.STABLE6 (providing professional caching while doing reverse proxy)

Ii. Compilation and Installation
1) install Nginx
1) install
Nginx is pronounced as [engine x] and is a project established by the Russian Igor Sysoev Based on the BSD license. It is said that he was a member of F5, Home: http://nginx.net. Some large Russian websites have been using it for more than two years and have been doing well.
The Nginx compilation parameters are as follows:

[Root @ localhost] #./configure -- prefix =/usr/local/server/nginx -- with-openssl =/usr/include \
-- With-pcre =/usr/include/pcre/-- with-http_stub_status_module -- without-http_memcached_module \
-- Without-http_fastcgi_module -- without-http_rewrite_module -- without-http_map_module \
-- Without-http_geo_module -- without-http_autoindex_module
Here, we need to describe the support of the pcre module because I want to use regular expressions in the Nginx configuration file. I have installed the rpm package of pcre and pcre-devel, but Ngxin cannot find the. h/. so/. a/. la file correctly, so I changed it a bit:

[Root @ localhost] # mkdir/usr/include/pcre/. libs/
[Root @ localhost] # cp/usr/lib/libpcre. a/usr/include/pcre/. libs/libpcre.
[Root @ localhost] # cp/usr/lib/libpcre. a/usr/include/pcre/. libs/libpcre. la
Then, modify the objs/Makefile at the location of approximately 908 rows and comment out the following content:

./Configure -- disable-shared
Next, you can execute make and make install normally.

2) modify the configuration file/usr/local/server/nginx/conf/nginx. conf.
The following is my nginx. conf content for your reference only:

# Running user
User nobody;
# Start a process
Worker_processes 2;
# Global error logs and PID files
Error_log logs/error. log notice;
Pid logs/nginx. pid;
# Working mode and maximum number of connections
Events {
Use epoll;
Worker_connections 1024;
}
# Set the http server and use its reverse proxy function to provide Load Balancing support
Http {
# Set the mime type
Include conf/mime. types;
Default_type application/octet-stream;
# Set the log format
Log_format main '$ remote_addr-$ remote_user [$ time_local]'
'"$ Request" $ status $ bytes_sent'
'"$ Http_referer" "$ http_user_agent "'
'"$ Gzip_ratio "';
Log_format download '$ remote_addr-$ remote_user [$ time_local]'
'"$ Request" $ status $ bytes_sent'
'"$ Http_referer" "$ http_user_agent "'
'"$ Http_range" "$ sent_http_content_range "';
# Set Request Buffer
Client_header_buffer_size 1 k;
Large_client_header_buffers 4 4 k;
# Enable the gzip Module
Gzip on;
Gzip_min_length 1100;
Gzip_buffers 4 8 k;
Gzip_types text/plain;
Output_buffers 1 32 k;
Post pone_output 1460;
# Setting access log
Access_log logs/access. log main;
Client_header_timeout 3 m;
Client_body_timeout 3 m;
Send_timeout 3 m;
Sendfile on;
Tcp_nopush on;
Tcp_nodelay on;
Keepalive_timeout 65;
# Set the Server list of Server Load balancer
Upstream mysvr {
# The weigth parameter indicates the weight. A higher weight indicates a higher probability of being assigned.
# Enable port 3128 for Squid on the local machine
Server 192.168.8.1: 3128 weight = 5;
Server 192.168.8.2: 80 weight = 1;
Server 192.168.8.3: 80 weight = 6;
}
# Set Virtual Hosts
Server {
Listen 80;
Server_name 192.168.8.1 www.360mini.com;
Charset gb2312;
# Set access logs for the current virtual host
Access_log logs/www.360mini.com. access. log main;
# If you access/img/*,/js/*,/css/* resources, you can directly retrieve the local file without passing squid
# This method is not recommended if there are many files, because the squid cache works better.
Location ~ ^/(Img | js | css )/{
Root/data3/Html;
Expires 24 h;
}
# Enable Server Load balancer "/"
Location /{
Proxy_pass http: // mysvr;
Proxy_redirect off;
Proxy_set_header Host $ host;
Proxy_set_header X-Real-IP $ remote_addr;
Proxy_set_header X-Forwarded-For $ proxy_add_x_forwarded_for;
Client_max_body_size 10 m;
Client_body_buffer_size 128 k;
Proxy_connect_timeout 90;
Proxy_send_timeout 90;
Proxy_read_timeout 90;
Proxy_buffer_size 4 k;
Proxy_buffers 4 32 k;
Proxy_busy_buffers_size 64 k;
Proxy_temp_file_write_size 64 k;
}
# Set the address for viewing Nginx status
Location/NginxStatus {
Stub_status on;
Access_log on;
Auth_basic "NginxStatus ";
Auth_basic_user_file conf/htpasswd;
}
}
}
Note: The content of the conf/htpasswd file can be generated using the htpasswd tool provided by apache. The content is roughly as follows:
3) view Nginx running status
Enter the http: // 192.168.8.1/NginxStatus/address, and enter the verification account password. The following content is displayed:
Active connections: 328
Server accepts handled requests
9309 8982 28890
Reading: 1 Writing: 3 Waiting: 324

The first line indicates the number of currently active connections.
The third number in the third row indicates the total number of requests received from Nginx running to the current time. If the limit is reached, the upper limit must be increased.
The fourth line is the Nginx queue status.

Iii. Install MySQL and memcache

1) to install MySQL, follow these steps:
[Root @ localhost] # tar zxf mysql-standard-5.0.27-linux-i686.tar.gz-C/usr/local/server
[Root @ localhost] # mv/usr/local/server/mysql-standard-5.0.27-linux-i686/usr/local/server/mysql
[Root @ localhost] # cd/usr/local/server/mysql
[Root @ localhost] #./scripts/mysql_install_db -- basedir =/usr/local/server/mysql \
-- Datadir =/usr/local/server/mysql/data -- user = nobody
[Root @ localhost] # cp/usr/local/server/mysql/support-files/my-large.cnf \
/Usr/local/server/mysql/data/my. cnf

2) modify the MySQL configuration and add some optimization parameters as follows:
[Root @ localhost] # vi/usr/local/server/mysql/data/my. cnf

The main content is as follows:
[Mysqld]
Basedir =/usr/local/server/mysql
Datadir =/usr/local/server/mysql/data
User = nobody
Port = 3306
Socket =/tmp/mysql. sock
Wait_timeout = 30
Long_query_time = 1
# Log-queries-not-using-indexes = TRUE
Log-slow-queries =/usr/local/server/mysql/slow. log
Log-error =/usr/local/server/mysql/error. log
External-locking = FALSE
Key_buffer_size = 512 M
Back_log = 400
Table_cache = 512
Sort_buffer_size = 2 M
Join_buffer_size = 4 M
Read_buffer_size = 2 M
Read_rnd_buffer_size = 4 M
Myisam_sort_buffer_size = 64 M
Thread_cache_size = 32
Query_cache_limit = 2 M
Query_cache_size = 64 M
Thread_concurrency = 4
Thread_stack = 128 K
Tmp_table_size = 64 M
Binlog_cache_size = 2 M
Max_binlog_size = 128 M
Max_binlog_cache_size = 512 M
Max_relay_log_size = 128 M
Bulk_insert_buffer_size = 8 M
Myisam_repair_threads = 1
Skip-bdb
# Disable this option if innodb is not required
# Skip-innodb
Innodb_data_home_dir =/usr/local/server/mysql/data/
Innodb_data_file_path = ibdata1: 256 M; ibdata2: 256 M: autoextend
Innodb_log_group_home_dir =/usr/local/server/mysql/data/
Innodb_log_arch_dir =/usr/local/server/mysql/data/
Innodb_buffer_pool_size = 512 M
Innodb_additional_mem_pool_size = 8 M
Innodb_log_file_size = 128 M
Innodb_log_buffer_size = 8 M
Innodb_lock_wait_timeout = 50
Innodb_flush_log_at_trx_commit = 2
Innodb_file_io_threads = 4
Innodb_thread_concurrency = 16
Innodb_log_files_in_group = 3

The preceding configuration parameters must be modified as needed. Run the following command to start the MySQL server:
/Usr/local/server/mysql/bin/mysqld_safe \
-- Defaults-file =/usr/local/server/mysql/data/my. cnf &

Because MySQL is not installed in the standard directory, you must modify the location of the my_print_defaults file in mysqld_safe to pass
Mysqld_safe to start the MySQL server.
3. Install and compile memcache + libevent:
[Root @ localhost] # libevent-1.2a cd
[Root @ localhost] #./configure -- prefix =/usr/& make install
[Root @ localhost] # cd ../memcached-1.2.0
[Root @ localhost] #./configure -- prefix =/usr/local/server/memcached -- with-libevent =/usr/
[Root @ localhost] # make & make install

NOTE: If libevent is not installed in the/usr directory, copy/link the libevent-1.2a.so.1 to/usr/lib; otherwise
Memcached cannot be loaded normally. Run the following command to start memcached:
[Root @ localhost] #/usr/local/server/memcached/bin/memcached \
-L 192.168.8.1-d-p 10000-u nobody-m 128

It means to start memcached in daemon mode, listen to port 10000 of 192.168.8.1, run the user as nobody, assign it
The memory size is 128 MB.

4. install Apache, PHP, eAccelerator, and php-memcache

4. install Apache, PHP, eAccelerator, and php-memcache
2. Static php compilation is very troublesome, so dynamic module (DSO) is used here. 1) install Apache 2.2.3
[Root @ localhost] #./configure -- prefix =/usr/local/server/apache -- disable-userdir -- disable-actions \
-- Disable-negotiation -- disable-autoindex -- disable-filter -- disable-include -- disable-status \
-- Disable-asis -- disable-auth -- disable-authn-default -- disable-authn-file -- disable-authz-groupfile \
-- Disable-authz-host -- disable-authz-default -- disable-authz-user -- disable-userdir \
-- Enable-expires -- enable-module = so

Note: unnecessary modules are removed here. If you need to use these modules, remove some parameters.
2) install PHP 5.2.0
[Root @ localhost] #./configure -- prefix =/usr/local/server/php -- with-mysql \
-- With-apxs2 =/usr/local/server/apache/bin/apxs -- with-freetype-dir =/usr/-- with-png-dir =/usr /\
-- With-gd =/usr/-- with-jpeg-dir =/usr/-- with-zlib -- enable-magic-quotes -- with-iconv \
-- Without-sqlite -- without-pdo-sqlite -- with-pdo-mysql -- disable-dom -- disable-simplexml \
-- Enable-roxen-zts
[Root @ localhost] # make & make install

Note: If you do not need gd or pdo modules, remove them.
3) install eAccelerator-0.9.5
[Root @ localhost] # eAccelerator-0.9.5 cd
[Root @ localhost] # export PHP_PREFIX =/usr/local/server/php
[Root @ localhost] # $ PHP_PREFIX/bin/phpize
[Root @ localhost] #./configure -- enable-eaccelerator = shared -- with-php-config = $ PHP_PREFIX/bin/php-config
[Root @ localhost] # make & make install

4) install the memcache Module
[Root @ localhost] # memcache-2.1.0 cd
[Root @ localhost] # export PHP_PREFIX =/usr/local/server/php
[Root @ localhost] # $ PHP_PREFIX/bin/phpize
[Root @ localhost] #./configure -- enable-eaccelerator = shared -- with-php-config = $ PHP_PREFIX/bin/php-config
[Root @ localhost] # make & make install

5.) modify the php. ini configuration, modify php. ini, and add the following content:
Extension_dir = "/usr/local/server/php/lib /"
Extension = "eaccelerator. so"
Eaccelerator. shm_size = "32"; set the shared memory of the eaccelerator to 32 MB.
Eaccelerator. cache_dir = "/usr/local/server/eaccelerator"
Eaccelerator. enable = "1"
Eaccelerator. optimizer = "1"
Eaccelerator. check_mtime = "1"
Eaccelerator. debug = "0"
Eaccelerator. filter = "*. php"
Eaccelerator. shm_max = "0"
Eaccelerator. shm_ttl = "0"
Eaccelerator. shm_prune_period = "3600"
Eaccelerator. shm_only = "0"
Eaccelerator. compress = "1"
Eaccelerator. compress_level = "9"
Eaccelerator. log_file = "/usr/local/server/apache/logs/eaccelerator_log"
Eaccelerator. allowed_admin_path = "/usr/local/server/apache/htdocs/ea_admin"
Extension = "memcache. so"

Here, it is best to add the default file type cache mechanism in apache configuration, that is, using the expires module of apache, add lines similar to the following:
ExpiresActive On
ExpiresByType text/html "access plus 10 minutes"
ExpiresByType text/css "access plus 1 day"
ExpiresByType image/jpg "access 1 month"
ExpiresByType image/gif "access 1 month"
ExpiresByType image/jpg "access 1 month"
ExpiresByType application/x-shockwave-flash "access plus 3 day"

This setting is because my static files are rarely updated, so I chose the "access" rule. If the updates are relatively frequent, you can use the "modification" rule instead; you can also use the "access" rule, but when the file is updated, run the "touch" command to refresh the file time.

5. Install Squid

5. Install Squid
[Root @ localhost] #. /configure -- prefix =/usr/local/server/squid -- enable-async-I/O = 100 -- disable-delay-pools -- disable-mem-gen-trace -- disable-useragent- log -- enable-kill-parent-hack -- disable-arp-acl -- enable-epoll -- disable-ident-lookups -- enable-snmp -- enable-large-cache-files --- large-files
[Root @ localhost] # make & make install

Or use the following installation method:
[Root @ localhost] # yum install squid

If it is a 2.6 kernel, only the epoll I/O mode can be supported. The earlier kernel version can only be poll or another mode. In addition, remember to include the option to support large files. Otherwise
When the number of log files reaches 2 GB, an error is reported. Configure squid as follows:
# Set the cache directory to/var/cache1 and/var/lib/squid. The cache size for each processing is 128 MB. When the cache space usage reaches 95%
# The new content will replace the old one and will not be directly added to the directory until the space drops to 90%.
#/Var/cache1 maximum 1024 MB,/var/lib/squid maximum 5000 MB, all are 16*256 sub-Directories
Cache_dir aufs/var/cache1 1024 16 256
Cache_dir aufs/var/lib/squid 5000 16 256
Cache_mem 128 MB
Cache_swap_low 90
Cache_swap_high 95
# Setting storage policies
Maximum_object_size 4096 KB
Minimum_object_size 0 KB
Maximum_object_size_in_memory 80 KB
Ipcache_size 1024
Ipcache_low 90
Ipcache_high 95
Cache_replacement_policy lru
Memory_replacement_policy lru
# Setting timeout policies
Forward_timeout 20 seconds
Connect_timeout 15 seconds
Read_timeout 3 minutes
Request_timeout 1 minutes
Persistent_request_timeout 15 seconds
Client_lifetime 15 minutes
Shutdown_lifetime 5 seconds
Negative_ttl 10 seconds
# A maximum of 16 connections can be bound to an ip address.
Acl OverConnLimit maxconn 16
Http_access deny OverConnLimit
# Restrict baidu spider access
# Acl AntiBaidu req_header User-Agent Baiduspider
# Http_access deny AntiBaidu
# General settings
Visible_hostname cache.yejr.com
Cache_mgr 1638651355@qq.com client_persistent_connections off
Server_persistent_connections on
Cache_inclutive_user nobody
Cache_inclutive_group nobody
Tcp_recv_bufsize 65535 bytes
Half_closed_clients off
# Set non-Cache rules
Hierarchy_stoplist cgi-bin
Acl QUERY urlpath_regex cgi-bin
Cache deny QUERY
# Do not trust ETag because of gzip
Acl apache rep_header Server ^ Apache
Broken_vary_encoding allow apache
# Set access log in the same format as apache to facilitate awstats Analysis
Emulate_httpd_log on
Logformat apache %> a % ui % un [% tl] "% rm % ru HTTP/% rv" % Hs %

Initialize and start squid
[Root @ localhost] #/usr/local/server/squid/sbin/squid-z
[Root @ localhost] #/usr/local/server/squid/sbin/squid

The first command First initializes the squid cache hash sub-directory and only needs to be executed once.

Vi. Postscript

6. Note 1) to enable the changes required by squid, you can make better use of the squid cache function instead of enabling it. We need to make the following adjustments:
1. Enable the mod_expires module of apache, modify httpd. conf, and add the following content:
# Expiresdefault "modification plus 2 weeks" expiresactive
Onexpiresbytype text/html "access plus 10 minutes" expiresbytype
Image/gif "modification plus 1 month" expiresbytype image/jpeg "modification
Plus 1 month "expiresbytype image/png" modification plus 1
Month "expiresbytype text/css" access plus 1 day "expiresbytype
Application/x-shockwave-flash "access plus 3 day"
The above configuration sets the cache rules for various types of files. static files such as images and flash files are always cached and can be adjusted as needed.
2. Modify the php. ini configuration as follows:
Session. cache_limiter = nocache
The above configuration is used to cancel the cache function in php by default to avoid abnormal cache generation.
3. modify the application. For example, there is a php program page static. php stores the results of some queries to the database, and the data is not updated frequently, so we can consider its cache. Just add the following code in static. php:
Header ('cache-Control: max-age = 86400
, Must-revalidate '); header ('pragma:'); header ('Last-Modified :'.
Gmdate ('d, d m y h: I: s'). 'gmt'); header ("Expires:". gmdate ('d, d M Y
H: I: s', time () + '000000'). 'gmt ');
The code above indicates that an http header is output to let squid know that the default cache duration for this page is one day.
Ii. Brief Introduction to squidclient
* Obtain squid running status information: squidclient-p 80 mgr: info
* Obtain squid memory usage: squidclient-p 80 mgr: mem
* Obtain the squid cached list: squidclient-p 80 mgr: objects. use it carefully, it may crash
* Obtain squid disk usage: squidclient-p 80 mgr: diskd
* Force update a url: squidclient-p 80-m PURGE http://www.360mini.com/static.php
* For more information, see squidclient-h or squidclient-p 80 mgr: