Analysis and optimization of small and medium Web site architecture

First look at the site architecture diagram:

650) this.width=650; "Src=" Http://s4.51cto.com/wyfs02/M01/83/AF/wKiom1d6Ma7gWzEWAAHVOh56AxA285.jpg-wh_500x0-wm_3 -wmp_4-s_2403683709.jpg "title=" small and medium-sized website analysis and optimization. jpg "alt=" wkiom1d6ma7gwzewaahvoh56axa285.jpg-wh_50 "/>

The above site structure widely used in large-scale Web site, this article from the framework of each layer of analysis of the mainstream technology and solutions to help the first into the site operations friends, further understanding of the site structure, so that they form a set of architectural concepts.

First tier: CDN

The distribution of the main domestic network South Telecom North Unicom, resulting in a large number of cross-regional access delays, for a certain amount of traffic sites, increase CDN (content distribution network) layer can effectively improve this phenomenon, but also the best choice for website acceleration. The CDN caches the site pages to a nationally distributed node, where users access data from the nearest computer room, which greatly reduces the network access path. If you want to build your own CDN, it is not recommended to do so, because what? In fact, frankly speaking, it is what things do not stop on the operational dimension. CDN Architecture deployment is not complex, the effect of the factors are many, the latter management and maintenance is more complex, want to achieve the desired effect is not easy, this is a thankless of life, the last boss or feeling is you lack of capacity. Recommended to do CDN Company, the cost is not expensive, anti-flow attack ability, the effect is very good, operation and maintenance of a lot less things, why not!

Second layer: Reverse proxy (Web cache)

If the CDN does not cache the data to be requested, the request is made to this layer, and the proxy server configures the cache function (local), which locates the local cache for the data of the CDN request and returns it directly to the CDN if there is no request back-end load Balancing The device then forwards the Web server back to the proxy server, and the proxy server returns the result to the CDN. Proxy server generally caches static pages that do not change frequently, such as image, JS, CSS, HTML, etc., the mainstream cache software has squid,Varnish, Nginx.

Tier Three: Load Balancing

Load balancing is used for sites with a large number of visits, as this is the best way to solve a single server performance bottleneck. The reverse proxy forwards the request to the load balancer, which is processed by the backend Web service based on the algorithm (rotation, load case selection backend, etc.) and returns the data directly to the reverse proxy server after the Web service finishes processing. Load balancing a reasonable allocation of requests to the backend multiple Web servers, reduce the concurrent load on a single server, and ensure service availability. The mainstream load balancer software has LVS, HAProxy, Nginx.

Layer Fourth: Web services

Web services are processing user requests, Web Services processing efficiency, directly affect the speed of access, in order to avoid this layer of factors caused by slow access, should be tuned, so that Web services to the best of the state. Common Web Services are Apache and Nginx.

Apache Optimization:

1). mod_deflate Compression Module

To see if loading:

# Apachectl M |grep deflate

If not installed use APXS to compile in:

#/USR/LOCAL/APACHE/BIN/APXS C I A Apache source directory/modules/mod_deflate.c

Deflate Configuration parameters:

DeflateCompressionLevel6 #压缩等级 (1-9), the greater the value, the higher the efficiency, the higher the CPU consumption setoutputfilterdeflate #启用压缩AddOutputFilterByTypeDEFLATE t ext/html text/plain text/xml #压缩类型AddOutputFilterByTypeDEFLATE css js html htm xml php

2). Mod_expires Cache Module

To see if loading:

# Apachectl M |grep expires

If not installed use APXS to compile in:

#/USR/LOCAL/APACHE/BIN/APXS C I A Apache source directory/modules/mod_expires.c

Then enable the module in httpd.conf: LoadModule expires_module modules/mod_expires.so

There are three ways to use the caching mechanism: Global, directory, and virtual host

Global configuration, added at the end of the configuration file:

Expiresactiveon #启用有效期控制, the expired cache is automatically cleared, and the new ExpiresDefault "Accessplus 1 days" is obtained from the server #默认任意格式的文档都是1天后过期ExpiresByTy petext/html "Access plus months" expiresbytypeimage/jpg "Access plus months" #jpg格式图片缓存12月

3). Working mode selection and optimization

Apache has two common modes of operation, worker and Prefork, the default is worker, is a hybrid MPM (multi-processing module), support multi-process and multi-threaded, by the thread to process the request, so can handle more requests, improve the concurrency capability, The system resource cost is also less than the process-based MPM, because the thread uses the process memory space and the process crashes causing its threads to crash. While Prefork is a non-threaded MPM, the process takes up more resources than the worker, and because of the process connection, it is more stable in productivity than the worker. The current mode of operation can be viewed through apache2 L, using the-WITH-MPM parameter at compile time to specify the operating mode. According to their business needs to choose different modes of work, and then appropriately increase the relevant parameters of the working mode, can improve processing capacity.

Configuration parameter Description:

Startservers 8 #默认启动8个httpd进程MinSpareServers 5 #最小的空闲进程数MaxSpareServers #最大的空闲进程数, if greater than this value, Apache will automatically ki ll some processes Serverlimit #服务器允许进程数的上限MaxClients the #同时最多发起多少个访问, exceeding the queue to wait for Maxrequestsperchild 4000 #每个进程启动的最大 Thread

Nginx Optimization:

1). gzip Compression Module

HTTP {..... gzip on;   Gzip_min_length 1k;   #允许压缩的页面最小字节数, the default is 0, most of the compression, less than 1k may backfire gzip_buffers 4 16k;  #gzip申请内存的大小, 4 times times the size of the data to apply for memory Gzip_http_version 1.0;    #识别http协议版本 Gzip_comp_level 2;  #压缩级别, the 1 compression ratio is minimal, the processing speed is the fastest, 9 compression ratio is maximum, the processing speed is the slowest gzip_types text/plainapplication/x-javascripttext/css application/xml image/jpg;      #压缩数据类型 gzip_vary on; #根据客户端的http头来判断, whether compression is required}

2). Expires Cache Module

server {location ~. *. (   GIF|JPG|PNG|BMP|SWF) $ #缓存数据后缀类型 {expires 30d; #使用expires缓存模块, cache to client 30 days} location ~. *.    (JSP|JS|CSS)? $ {Expires 1d; }}

3). fastcgi optimization

Nginx does not support the direct invocation or parsing of the dynamic Program (PHP), it must be started through the FASTCGI (Universal Gateway Interface) to start the PHP-FPM process to parse the PHP script. This means that the user requests to Nginx,nginx and then the dynamic resolution to fastcgi,fastcgi start PHP-FPM parsing php script. Therefore, it is necessary to optimize the parameters of fastcgi and php-fpm properly.

Http {    ......    fastcgi_cache_path/usr/local/nginx/fastcgi_cache  levels=1:2 keys_zone=TEST:10m inactive=5m;      #  FASTCGI cache Specifies a file path, directory structure level, keyword area storage time, and inactive delete time     fastcgi_connect_timeout 300;      #指定连接到后端FastCGI的超时时间     fastcgi_send_timeout 300;       #指定向FastCGI传送请求的超时时间     fastcgi_read_timeout 300;       #指定接收FastCGI应答的超时时间     fastcgi_buffer_size 64k;        #指定读取FastCGI应答第一部分需要多大的缓冲区     fastcgi_buffers 4 64k;        #指定本地需要用多少盒多大的缓冲区来缓冲FastCGI的应答请求     fastcgi_busy_buffers_size 128k;        fastcgi_temp_file_write_size 128k;  # Indicates how much data block is used when writing to the cache file, the default value is twice times of Fastcgi_buffers     fastcgi_cache test;          # Turn on the Fastcgi_cache cache and specify a test name     fastcgi_cache_valid 200 302 1h;     Cache of #指定200, 302 response codes 1 hours     fastcgi_cache_valid 301 1d;       #将301应答代码缓存1天     fastcgi_cache_valid any 1m;       #将其他应答均缓存1分钟 {

php-fpm.conf Configuration parameters:

PM =dynamic #两种控制子进程方式 (static and dynamic) pm.max_children= 5 #同一时间存活的最大子进程数pm. start_servers= 2 #启动时创建的进程数pm. Min_ spare_servers= 1 #最小php-fpm process number pm.max_spare_servers= 3 #最大php-fpm processes

4). Proxy_cache Local Cache module

Http {        ......   proxy_temp_path  /usr /local/nginx/proxy_cache/temp;     #缓存临时目录    proxy_cache_path /usr/ local/nginx/proxy_cache/cache levels=1:2 keys_zone=one:10m inactive=1d max_size=1g;     #缓存文件实际目录, levels defines a hierarchy of directories, 1:2 Description 1 is a first-level directory, 2 is a two-level directory, Keys_zone stores metadata, and allocates 10M of memory space. Inctive represents a 1-day cache that has not been accessed and is deleted by default for 10 minutes. Max_size is the largest allocated disk space    server {      listen 80;       server_name 192.168.1.10;      location  / {        proxy_cache one;    #调用缓存区           #proxy_cache_valid  200 304 12h; # Different cache times can be set based on HTTP status code         proxy_cache_valid any  10m;      #缓存有效期为10分钟       }       #清除URL缓存 to allow IP from which network segment to clear the cache (third-party modules need to be installed Ngx_ Cache_purge "), clear the URL cache method: Access http://192.168.1.10/purge/file name       location ~  /purge (/.*) {        allow 127.0.0.1;         allow 192.168.1.0/24;        deny  all;        proxy_cache_purge cache_one$host$1$is_args$args ;       } }

Summary:

Enabling the compression module saves some of the bandwidth and increases the web-side CPU processing, but in the Web site architecture, the web-side-enabled compression module does not work because the network is transmitted to the upper layer. For a directly user-facing schema or to enable. The web also does not have to enable the Expires module, because there is a reverse proxy server and CDN, so not to the user's browser, open does not work.

If the reverse proxy using Nginx proxy, you can open the Expires module, the static file cache to the user browser, the browser initiates the request, the first to determine whether the local cache has the requested data, if you have to determine whether to expire, if not expire directly browse the cache data, even if the server resources have changed, Therefore, it is reasonable to set the expiration time according to business conditions.

5. Improve code execution efficiency with PHP cache

PHP program in the absence of the use of the cache, each time the PHP page is requested, PHP will compile this page code, which means that the duplication of the compilation work will increase the server load. With a cache, each compiled data is cached into shared memory, and the next access directly uses the buffer-compiled code to avoid a repetitive compilation process to expedite its execution. So the PHP site uses the cache is absolutely necessary! The main PHP buffers are: Eaccelerator, XCache

Fifth floor: Static and dynamic separation

Static and static separation, as the name implies, is to separate the dynamic page and the rest of the page processing on different servers, such as the use of the Web is Nginx, you can let fastcgi deployment to a separate server, specifically parsing PHP dynamic page, static page default by Nginx processing, and good cache strategy. Another example of a mall site, there will be a large number of images, you can consider increasing the file server group, will request pictures and upload pictures to the file server processing. File server mainstream use of NFS, there is a single point of failure, can be DRBD+HEARTBEAT+NFS deployment high availability, if the single pressure is too large, consider the use of distributed file systems, such as Glusterfs, Moosefs and so on.

"DRBD + heratbeat + NFS High-availability file-sharing storage": http://blog.jobbole.com/94718/

Layer Sixth: Database caching

Using the cache technology, the hot data is cached in memory, if the requested data in the cache, it is returned directly, otherwise go to the database, and update the data to update to the cache system, improve read performance, reduce database pressure. The cache implementation has both local and distributed caches, and the local cache caches the data in the local server memory or in the file. Distributed cache is to cache the data in memory, is distributed, can cache massive data, extensibility is good. The mainstream distributed cache system has memcached and redis,memcached performance stability, fast, QPS up to about 8w. If you want to persist data, choose Redis, and the performance is no less than memcached.

Layer Seventh: Database

This layer in the entire site architecture plays a leading role, directly determine the user experience, relative structure optimization is also more complex, please refer to the blog: "Operation and maintenance perspective on MySQL Database Optimization"

core idea: reduce the request layer, as far as possible to let the front-end layer return user requested data, reduce the back-end server access frequency, the most important is the database layer.

This article from "There is nothing, know in Providence" blog, please be sure to keep this source http://yangsj.blog.51cto.com/8702844/1795698

Analysis and optimization of small and medium Web site architecture