Nginx + Apache + MySQL + PHP + memcache + squid build cluster Web Environment

Server Load Balancing Scheme with a large number of users

I. Preface

Ii. Compilation and Installation

Iii. Install MySQL and memcache

4. install Apache, PHP, eaccelerator, and PHP-memcache

5. Install squid

Vi. Postscript

 

I. Preface and preparations

Currently, the lamp development model is the first choice for web development. How to build an efficient, reliable, and stable web server has always been a hot topic.

This is an attempt on this topic.

The architecture diagram is as follows:

 

Reference ----------------------------------------------------

| Client | ==>| Server Load balancer |==>| reverse proxy/cache |==>| Web Server |==>| Database Server |

----------------------------------------------------

Nginx squid Apache, PHP MySQL

Preparations for eaccelerator/memcache:

Reference server: Intel (r) Xeon (TM) CPU 3.00 GHz X 2, 2 gb mem, scisc hard drive

Operating System: centos4.4, kernel version 2.6.9-22. elsmp, GCC version 3.4.4

Software:

Apache 2.2.3 (MPM mode available)

PHP 5.2.0 (this version is used because the engine of 5.2.0 is relatively more efficient)

Eaccelerator 0.9.5 (accelerates the PHP engine and encrypts the PHP source program)

Memcache 1.2.0 (used to cache frequently used data)

Libevent 1.2a (required for memcache operating mechanism)

MySQL 5.0.27 (use a binary version to save compilation effort)

Nginx 0.5.4 (used as a server Load balancer)

Squid-2.6.STABLE6 (providing professional caching while doing reverse proxy)

 

Ii. Compilation and Installation

1) install nginx

1) install

Nginx is pronounced as [engine X] and is a project established by Russian igorsysoev Based on BSD license. It is said that he was a member of F5.

, Home: http://nginx.net. Some large Russian websites have been using it for more than two years and have been doing well.

The nginx compilation parameters are as follows:

 

[Root @ localhost] #./configure -- prefix =/usr/local/Server/nginx -- With-OpenSSL =/usr/include \

-- With-PCRE =/usr/include/PCRE/-- with-http_stub_status_module -- without-http_memcached_module \

-- Without-http_fastcgi_module -- without-http_rewrite_module -- without-http_map_module \

-- Without-http_geo_module -- without-http_autoindex_module

Here, we need to describe the support of the PCRE module because I want to use regular expressions in the nginx configuration file. I have installed

The RPM package of PCRE and PCRE-devel, but ngxin cannot find the. h/. So/. A/. La file correctly, so I have made some changes.

:

 

[Root @ localhost] # mkdir/usr/include/PCRE/. libs/

[Root @ localhost] # cp/usr/lib/libpcre. A/usr/include/PCRE/. libs/libpcre.

[Root @ localhost] # cp/usr/lib/libpcre. A/usr/include/PCRE/. libs/libpcre. La

Then, modify the objs/makefile at the location of approximately 908 rows and comment out the following content:

 

./Configure -- disable-shared

Next, you can execute make and make install normally.

 

2) modify the configuration file/usr/local/Server/nginx/CONF/nginx. conf.

The following is my nginx. conf content for your reference only:

 

# Running user

User nobody Nobody;

# Start a process

Worker_processes 2;

# Global error logs and PID files

Error_log logs/error. Log notice;

PID logs/nginx. PID;

# Working mode and maximum number of connections

Events {

Use epoll;

Worker_connections 1024;

}

# Set the HTTP server and use its reverse proxy function to provide Load Balancing support

HTTP {

# Set the MIME type

Include CONF/mime. types;

Default_type application/octet-stream;

# Set the log format

Log_format main '$ remote_addr-$ remote_user [$ time_local]'

'"$ Request" $ Status $ bytes_sent'

'"$ Http_referer" "$ http_user_agent "'

'"$ Gzip_ratio "';

Log_format download '$ remote_addr-$ remote_user [$ time_local]'

'"$ Request" $ Status $ bytes_sent'

'"$ Http_referer" "$ http_user_agent "'

'"$ Http_range" "$ sent_http_content_range "';

# Set Request Buffer

Client_header_buffer_size 1 K;

Large_client_header_buffers 4 4 K;

# Enable the gzip Module

Gzip on;

Gzip_min_length 1100;

Gzip_buffers 4 8 K;

Gzip_types text/plain;

Output_buffers 1 32 K;

Post pone_output 1460;

# Setting access log

Access_log logs/access. Log main;

Client_header_timeout 3 m;

Client_body_timeout 3 m;

Send_timeout 3 m;

Sendfile on;

Tcp_nopush on;

Tcp_nodelay on;

Keepalive_timeout 65;

# Set the Server list of Server Load balancer

Upstream mysvr {

# The weigth parameter indicates the weight. A higher weight indicates a higher probability of being assigned.

# Enable port 3128 for squid on the local machine

Server 192.168.8.1: 3128 Weight = 5;

Server 192.168.8.2: 80 Weight = 1;

Server 192.168.8.3: 80 Weight = 6;

}

# Set Virtual Hosts

Server {

Listen 80;

SERVER_NAME 192.168.8.1 www.yejr.com;

Charset gb2312;

# Set access logs for the current virtual host

Access_log logs/www.yejr.com. Access. logmain;

# If you access/img/*,/JS/*,/CSS/* resources, you can directly retrieve the local file without passing squid

# This method is not recommended if there are many files, because the Squid cache works better.

Location ~ ^/(IMG | JS | CSS )/{

Root/data3/html;

Expires 24 h;

}

# Enable Server Load balancer "/"

Location /{

Proxy_pass http: // mysvr;

Proxy_redirect off;

Proxy_set_header host $ host;

Proxy_set_header X-real-IP $ remote_addr;

Proxy_set_header X-forwarded-for $ proxy_add_x_forwarded_for;

Client_max_body_size 10 m;

Client_body_buffer_size 128 K;

Proxy_connect_timeout 90;

Proxy_send_timeout 90;

Proxy_read_timeout 90;

Proxy_buffer_size 4 K;

Proxy_buffers 4 32 K;

Proxy_busy_buffers_size 64 K;

Proxy_temp_file_write_size 64 K;

}

# Set the address for viewing nginx status

Location/nginxstatus {

Stub_status on;

Access_log on;

Auth_basic "nginxstatus ";

Auth_basic_user_file CONF/htpasswd;

}

}

}

Note: The content of the conf/htpasswd file can be generated using the htpasswd tool provided by Apache. The content is roughly as follows:

3) view nginx running status

Enter the http: // 192.168.8.1/nginxstatus/address, and enter the verification account password. The following content is displayed:

Active connections: 328

Server accepts handled requests

9309 8982 28890

Reading: 1 writing: 3 waiting: 324

 

The first line indicates the number of currently active connections.

The third number in the third row indicates the total number of requests received from nginx running to the current time. If the limit is reached, the upper limit must be increased.

.

The fourth line is the nginx queue status.

 

Iii. Install MySQL and memcache

 

 

 

1) to install MySQL, follow these steps:

[Root @ localhost] # tar zxf mysql-standard-5.0.27-linux-i686.tar.gz-C/usr/local/Server

[Root @ localhost] # mv/usr/local/Server/mysql-standard-5.0.27-linux-i686/usr/local/Server/MySQL

[Root @ localhost] # cd/usr/local/Server/MySQL

[Root @ localhost] #./scripts/mysql_install_db -- basedir =/usr/local/Server/MySQL \

-- Datadir =/usr/local/Server/MySQL/data -- user = nobody

[Root @ localhost] # cp/usr/local/Server/MySQL/support-files/my-large.cnf \

/Usr/local/Server/MySQL/data/My. CNF

 

2) modify the MySQL configuration and add some optimization parameters as follows:

[Root @ localhost] # vi/usr/local/Server/MySQL/data/My. CNF

 

The main content is as follows:

[Mysqld]

Basedir =/usr/local/Server/MySQL

Datadir =/usr/local/Server/MySQL/Data

User = nobody

Port = 3306

Socket =/tmp/MySQL. Sock

Wait_timeout = 30

Long_query_time = 1

# Log-queries-not-using-indexes = true

Log-Slow-queries =/usr/local/Server/MySQL/slow. Log

Log-error =/usr/local/Server/MySQL/error. Log

External-locking = false

Key_buffer_size = 512 m

Back_log = 400

Table_cache = 512

Sort_buffer_size = 2 m

Join_buffer_size = 4 m

Read_buffer_size = 2 m

Read_rnd_buffer_size = 4 m

Myisam_sort_buffer_size = 64 m

Thread_cache_size = 32

Query_cache_limit = 2 m

Query_cache_size = 64 m

Thread_concurrency = 4

Thread_stack = 128 K

Tmp_table_size = 64 m

Binlog_cache_size = 2 m

Max_binlog_size = 128 m

Max_binlog_cache_size = 512 m

Max_relay_log_size = 128 m

Bulk_insert_buffer_size = 8 m

Myisam_repair_threads = 1

Skip-bdb

# Disable this option if InnoDB is not required

# Skip-InnoDB

Innodb_data_home_dir =/usr/local/Server/MySQL/data/

Innodb_data_file_path = ibdata1: 256 m; ibdata2: 256 M: autoextend

Innodb_log_group_home_dir =/usr/local/Server/MySQL/data/

Innodb_log_arch_dir =/usr/local/Server/MySQL/data/

Innodb_buffer_pool_size = 512 m

Innodb_additional_mem_pool_size = 8 m

Innodb_log_file_size = 128 m

Innodb_log_buffer_size = 8 m

Innodb_lock_wait_timeout = 50

Innodb_flush_log_at_trx_commit = 2

Innodb_file_io_threads = 4

Innodb_thread_concurrency = 16

Innodb_log_files_in_group = 3

 

The preceding configuration parameters must be modified as needed. Run the following command to start the MySQL server:

/Usr/local/Server/MySQL/bin/mysqld_safe \

-- Defaults-file =/usr/local/Server/MySQL/data/My. CNF &

 

Because MySQL is not installed in the standard directory, you must modify the location of the my_print_defaults file in mysqld_safe.

To pass

Mysqld_safe to start the MySQL server.

3. Install and compile memcache + libevent:

[Root @ localhost] # libevent-1.2a CD

[Root @ localhost] #./configure -- prefix =/usr/& make install

[Root @ localhost] # CD ../memcached-1.2.0

[Root @ localhost] #./configure -- prefix =/usr/local/Server/memcached -- With-libevent =/usr/

[Root @ localhost] # Make & makeinstall

 

NOTE: If libevent is not installed in the/usr directory, copy/link the libevent-1.2a.so.1 to/usr/lib

. Otherwise

Memcached cannot be loaded normally. Run the following command to start memcached:

[Root @ localhost] #/usr/local/Server/memcached/bin/memcached \

-L 192.168.8.1-D-P 10000-u nobody-M 128

 

It means to start memcached in daemon mode, listen to port 10000 of 192.168.8.1, run the user as nobody

Allocate

The memory size is 128 MB.

 

4. install Apache, PHP, eaccelerator, and PHP-memcache

 

 

 

4. install Apache, PHP, eaccelerator, and PHP-memcache

2. Static PHP compilation is very troublesome, so dynamic module (DSO) is used here. 1) install Apache 2.2.3

[Root @ localhost] #./configure -- prefix =/usr/local/Server/Apache -- disable-userdir -- disable-

Actions \

-- Disable-negotiation -- disable-Autoindex -- disable-filter -- disable-include -- disable-status \

-- Disable-asis -- disable-auth -- disable-Authn-default -- disable-Authn-file -- disable-authz-

Groupfile \

-- Disable-authz-host -- disable-authz-default -- disable-authz-user -- disable-userdir \

-- Enable-expires -- enable-module = so

 

Note: unnecessary modules are removed here. If you need to use these modules, remove some parameters.

2) install PHP 5.2.0

[Root @ localhost] #./configure -- prefix =/usr/local/Server/PHP -- With-mysql \

-- With-apxs2 =/usr/local/Server/Apache/bin/apxs -- With-FreeType-Dir =/usr/-- With-PNG-Dir =/usr /\

-- With-Gd =/usr/-- With-JPEG-Dir =/usr/-- With-zlib -- enable-magic-quotes -- With-iconv \

-- Without-SQLite -- without-PDO-SQLite -- With-PDO-mysql -- disable-Dom -- disable-simplexml \

-- Enable-roxen-ZTS

[Root @ localhost] # Make & makeinstall

 

Note: If you do not need gd or PDO modules, remove them.

3) install eAccelerator-0.9.5

[Root @ localhost] # eAccelerator-0.9.5 CD

[Root @ localhost] # exportphp_prefix =/usr/local/Server/PHP

[Root @ localhost] # $ php_prefix/bin/phpize

[Root @ localhost] #./configure -- enable-eaccelerator = shared -- With-PHP-

Config = $ php_prefix/bin/PHP-config

[Root @ localhost] # Make & makeinstall

 

4) install the memcache Module

[Root @ localhost] # memcache-2.1.0 CD

[Root @ localhost] # exportphp_prefix =/usr/local/Server/PHP

[Root @ localhost] # $ php_prefix/bin/phpize

[Root @ localhost] #./configure -- enable-eaccelerator = shared -- With-PHP-

Config = $ php_prefix/bin/PHP-config

[Root @ localhost] # Make & makeinstall

 

5.) modify the php. ini configuration, modify PHP. ini, and add the following content:

Extension_dir = "/usr/local/Server/PHP/lib /"

Extension = "eaccelerator. So"

Eaccelerator. shm_size = "32"; set the shared memory of the eaccelerator to 32 MB.

Eaccelerator. cache_dir = "/usr/local/Server/eaccelerator"

Eaccelerator. Enable = "1"

Eaccelerator. optimizer = "1"

Eaccelerator. check_mtime = "1"

Eaccelerator. DEBUG = "0"

Eaccelerator. Filter = "*. php"

Eaccelerator. shm_max = "0"

Eaccelerator. shm_ttl = "0"

Eaccelerator. shm_prune_period = "3600"

Eaccelerator. shm_only = "0"

Eaccelerator. Compress = "1"

Eaccelerator. compress_level = "9"

Eaccelerator. LOG_FILE = "/usr/local/Server/Apache/logs/eaccelerator_log"

Eaccelerator. allowed_admin_path = "/usr/local/Server/Apache/htdocs/ea_admin"

Extension = "memcache. So"

 

Here, it is best to add the default file type cache mechanism in Apache configuration, that is, using the expires module of Apache to add a similar

The following lines:

Expiresactive on

Expiresbytype text/html "Access plus10 minutes"

Expiresbytype text/CSS "access plus 1day"

Expiresbytype image/jpg "Access 1 month"

Expiresbytype image/GIF "Access 1 month"

Expiresbytype image/jpg "Access 1 month"

Expiresbytype application/X-Shockwave-flash "access plus 3 day"

 

This setting is because my static files are rarely updated, so I chose the "access" rule. If the updates are relatively frequent,

You can use the "modification" rule or the "access" rule, but when updating files, execute the "Touch" command.

, Refresh the file time.

 

 

 

5. Install squid

 

 

 

5. Install squid

[Root @ localhost] #./configure -- prefix =/usr/local/Server/squid -- enable-async-IO = 100 -- disable-

Delay-pools -- disable-Mem-gen-trace -- disable-useragent-log -- enable-kill-parent-hack --

Disable-ARP-ACL -- enable-epoll -- disable-Ident-lookups -- enable-SNMP -- enable-large-Cache-Files

-- With-large-Files

[Root @ localhost] # Make & makeinstall

 

Or use the following installation method:

[Root @ localhost] # Yum install squid

 

Only the 2.6 kernel can support the epoll I/O mode. The earlier kernel version can only select poll or other modes. In addition, remember to include

With the option of large files, otherwise in the access

When the number of log files reaches 2 GB, an error is reported. Configure squid as follows:

# Set the cache directory to/var/cache1 and/var/lib/squid. The cache size for each processing is 128 MB. When the cache space usage reaches 95%

# The new content will replace the old one and will not be directly added to the directory until the space drops to 90%.

#/Var/cache1 maximum 1024 MB,/var/lib/Squid maximum 5000 MB, all are 16*256 sub-Directories

Cache_dir aufs/var/cache1 1024 16 256

Cache_dir aufs/var/lib/squid 5000 16 256

Cache_mem 128 MB

Cache_swap_low 90

Cache_swap_high 95

# Setting storage policies

Maximum_object_size 4096 KB

Minimum_object_size 0 KB

Maximum_object_size_in_memory 80 KB

Ipcache_size 1024

Ipcache_low 90

Ipcache_high 95

Cache_replacement_policy LRU

Memory_replacement_policy LRU

# Setting timeout policies

Forward_timeout 20 seconds

Connect_timeout 15 seconds

Read_timeout 3 minutes

Request_timeout 1 minutes

Persistent_request_timeout 15 seconds

Client_lifetime 15 minutes

Shutdown_lifetime 5 seconds

Negative_ttl 10 seconds

# A maximum of 16 connections can be bound to an IP address.

ACL overconnlimit maxconn 16

Http_access deny overconnlimit

# Restrict Baidu spider access

# ACL antibaidu req_header user-agentbaidusp

# Http_access deny antibaidu

# General settings

Visible_hostname cache.yejr.com

Cache_mgr webmaster@yejr.com

Client_persistent_connections off

Server_persistent_connections on

Cache_inclutive_user nobody

Cache_inclutive_group nobody

Tcp_recv_bufsize 65535 bytes

Half_closed_clients off

# Set non-Cache rules

Hierarchy_stoplist cgi-bin

ACL query urlpath_regex cgi-bin

Cache deny Query

# Do not trust etag because of Gzip

ACL Apache rep_header server ^ Apache

Broken_vary_encoding allow Apache

# Set access log in the same format as Apache to facilitate AWStats Analysis

Emulate_httpd_log on

Logformat Apache %> A % UI % UN [% TL] "% RM % Ru HTTP/% RV" % hs %

 

Initialize and start squid

[Root @ localhost] #/usr/local/Server/squid/sbin/squid-z

[Root @ localhost] #/usr/local/Server/squid/sbin/squid

 

The first command First initializes the Squid cache hash sub-directory and only needs to be executed once.

 

Vi. Postscript

 

 

6. Note 1) If you want to enable the changes required by squid, you can make better use of the Squid cache function instead of enabling it.

We need to make the following adjustments:

1. Enable the mod_expires module of Apache, modify httpd. conf, and add the following content:

# Expiresdefault "modification plus 2 weeks" expiresactive

Onexpiresbytype text/html "Access plus10 minutes" expiresbytype

Image/GIF "modification plus 1 month" expiresbytype image/JPEG "Modification

Plus 1 month "expiresbytype image/PNG" modification plus 1

Month "expiresbytype text/CSS" access plus 1 day "expiresbytype

Application/X-Shockwave-flash "accessplus 3 day"

The above configuration sets the cache rules for various types of files. static files such as images and Flash files are always cached.

You need to make appropriate adjustments.

2. Modify the php. ini configuration as follows:

Session. cache_limiter = nocache

The above configuration is used to cancel the cache function in PHP by default to avoid abnormal cache generation.

3. modify the application. For example, there is a PHP program page static. php, which stores the results of some queries to the database and updates the data.

It is not frequent, so we can consider its cache. Just add the following code in static. php:

Header ('cache-control: Max-age = 86400

, Must-revalidate '); header ('pragma:'); header ('Last-modified :'.

Gmdate ('d, D m y h: I: s'). 'gmt'); header ("expires:". gmdate ('d, D m y

H: I: s', time () + '000000'). 'gmt ');

The code above indicates that an HTTP header is output to let squid know that the default cache duration for this page is one day.

Ii. Brief Introduction to squidclient

* Obtain squid running status information: squidclient-P80 Mgr: info

* Obtain squid memory usage: squidclient-P80 Mgr: Mem

* Obtain the cached squid list: squidclient-P80 Mgr: objects. Use it carefully, it may crash

* Obtain squid disk usage: squidclient-P80 Mgr: diskd

* Force update a URL: squidclient-P 80-mpurge http://www.yejr.com/static.php