PHP high concurrency high load system architecture

Transferred from: https://my.oschina.net/angelangel/blog/338456

Constraint conditions of high concurrency and high load

1. Hardware

2. Deployment

3. Operating system

4. Web Server

5. Php

6. Mysql

7. Test

second, the way to solve-- Hardware Chapter

Improvement of processing capacity: Deploy multiple CPUs, choose Multi-core, with higher computing frequency, larger cache CPU;

The most direct response to the improvement of processing power is the efficiency of the processing of Web requests and the execution efficiency of PHP programs.

Memory bandwidth and Capacity: greater memory bandwidth and capacity;

The most direct response to the increase in memory bandwidth and capacity is to deal with large amounts of data exchange in the database.

Disk Search and I/O capability: Select higher rpm, larger HDD cache, component disk array (RAID);

The most direct response to the improvement of disk search and I/O capability is the large number of queries and reads and writes of the database.

The factors that can be taken into account include greater bandwidth, multi-line access, and exclusive bandwidth.

Server in the case of heavy load, the network bandwidth consumption is very considerable.

Strategy: Hardware facilities are the basis for coping with large loads, and the investment in hardware facilities can be based on actual pressure and budget.

third, the way to solve-- Deployment Chapter

3.1. Server separation

32. Database Cluster and library table hash

3.3. Mirror

3.4. Load Balancing

Classification:

1), DNS round robin

2) proxy Server load Balancing

3) Address Translation Gateway load Balancing

4) NAT Load Balancer

5) Reverse Proxy load Balancing

6) Hybrid load-balancing

Strategy: According to the hardware investment and business requirements, choose a reasonable deployment plan.

Deployment Scenario 1 :

Scope of application: static content as the main web site and application system, high security requirements for the system of Web sites and applications.

Main Server : Primary Server

The main body of the hosting program is under pressure to handle dynamic requests in the Web site or application system;

Push a static page to multiple publishers;

Push the attachment file to the file server;

High security requirements, static-based Web site, the server can be placed in the network of external shielding access.

DB Server : Database Server

Load database reading and writing pressure;

Data volume Exchange only with the primary server, shielding extranet access.

File/video Server : File / Video Server

The data flow which occupies the system resource and the bandwidth resource is larger in the bearer system;

Storage and reading and writing warehouses as large attachments;

As a video server will have the ability to automate video processing.

Publishing Server Group:

Only responsible for the release of static pages, hosting the overwhelming majority of Web requests;

Load balanced deployment via Nginx.

Deployment Scenario 2:

Scope of application: Web site or application system with dynamic interactive content as the main body; a website or application system with large load-pressure and sufficient budget;

Web Server Groups:

The Web service has no master-slave relationship and is a parallel redundancy design;

Load balancing via front-end load balancing device or nginx reverse proxy;

Partition dedicated file server/video server effective separation of light/heavy bus;

Each Web server can be connected to all databases via Dec, while dividing the master and slave.

Database server Group:

Relatively balanced load-carrying database reading and writing pressure;

The data synchronization of multiple databases is realized through the mapping of database physical files.

Shared disk/disk array

Unified Read and write for data physical files

Storage warehouse for large attachments

Ensure the overall system IO efficiency and data security through the equalization and redundancy of its own physical disk;

Scenario Features:

Through the front-end load balancing, the reasonable distribution of web pressure;

Through the separation of the file/video server and the regular Web server, the reasonable allocation of light and heavy data flow;

Through the database server group, the reasonable allocation database IO pressure;

Each Web server is usually connected to only one database server, through the DEC heartbeat detection, can be automatically switched to redundant database server in a very short period of time;

The introduction of disk array greatly improves the efficiency of system IO, and enhances the data security.

Four, the solution-- Environment Chapter

A , operating system

The choice of operating system, the focus is

• Are you adapting to the environmental procedures required to build siteengine?

• The system itself occupies a resource ratio;

• system security;

• Is the system easy to operate?

Strategy: We choose FreeBSD, and it is minimized after the installation of FreeBSD.

4.2 , Web Server

A large portion of the Web server's resource footprint comes from processing Web requests, which is typically the stress that Apache generates, and Apache is the world's first Web server software to use. It can run on almost all of the widely used computer platforms.

In the case of high concurrent connection, Nginx is a good substitute for Apache server. Nginx ("Engine X") is a high-performance HTTP and reverse proxy server written by the Russians. In China, has Sina, Sohu Pass, NetEase news, NetEase Blog, Jinshan Carefree net, Kimshan word bully, Xiaonei, Yupoo album, Watercress, Thunder look at a number of websites, channels using Nginx server.

Nginx Advantages of:

High concurrent connections: The official test can support 50,000 concurrent connections, running in the actual production environment to the number of concurrent connections.

Low memory consumption: With 30,000 concurrent connections, the 10 nginx processes that are open consume 150M of memory (15m*10=150m).

Built-in Health check function: If a WEB server on the backend of Nginx proxy goes down, it will not affect the front-end access.

Strategy: Compared to the old Apache, we choose lighttpd and Nginx, which have smaller resource occupancy and higher load capacity of the Web server.

4.3. Mysql

MySQL itself has a very strong load capacity, MySQL optimization is a very complex task, because this ultimately requires a good understanding of system optimization. Database work is a large number of short-term query and read and write, in addition to program development needs to pay attention to build indexes, improve query efficiency and other software development skills, from the perspective of hardware facilities to affect the efficiency of MySQL is mainly from the disk search, disk IO level, CPU cycles, memory bandwidth.

Perform MySQL optimizations based on the hardware and software conditions on the server. The core of MySQL optimization is the allocation of system resources, which is not equal to the unlimited allocation of more resources to MySQL.

The most notable parameters in the MySQL configuration file are:

L Change Index buffer length (key_buffer)

L Change the length of the table (read_buffer_size)

L Set the maximum number of open tables (Table_cache)

L set a time limit for slow-length queries (long_query_time)

If the condition permits, the general MySQL server is installed in the Linux operating system instead of being installed in FreeBSD.

Strategy: MySQL optimization requires different optimization scenarios based on the database read-write characteristics and server hardware configuration of the business system, and can deploy MySQL's master-slave structure as needed.

4.4 , PHP

1, loading as few modules as possible;

2, install the accelerator (all by caching the PHP code precompiled results and database results to improve the execution speed of PHP code)

Eaccelerator

Eaccelerator is a free open source PHP accelerator, optimized and dynamic content caching, improving the performance of PHP script cache performance, so that PHP script in the state of compilation, the cost of the server almost completely eliminated.

Apc

Alternative PHP cache (APC) is a free, publicly available, optimized code cache for PHP. It is used to provide free, open and robust architecture to cache and optimize PHP's intermediate code.

Memcache

Memcache is a high-performance, distributed memory object caching system developed by Danga Interactive for reducing database load and increasing access speed in dynamic applications. The main mechanism is to maintain a unified huge hash table in memory, memcache can be used to store data in various formats, including images, videos, files, and database retrieval results.

Strategy: Install the Accelerator for PHP.

4.5 , Proxy server (cache server)

Squid cache (squid) is a popular free software (GNU General Public License) proxy server and Web cache server. Squid has a wide range of uses, from caching related requests as Web server cache servers to increasing the speed of Web servers, to sharing network resources for a group of people and caching the World Wide Web, domain name systems and other network searches, to help network security by filtering traffic, to LAN through proxy Internet. Squid is primarily designed to operate on Unix-type systems.

Strategy: Install Squid reverse proxy server, can greatly improve server efficiency.

5 , the solution of the road--siteengine article

Siteengine (website engine) refers to a set of programmatic language frameworks designed for application-system programs. At present, the most mainstream is based on multi-layer architecture, hierarchical processing, modular design framework, support MVC Dynamic page presentation, core program layered processing, add static engine, with Ajax asynchronous efficient processing of requests.

6 , the way to solve-- Test Article

6.1. Test method

6.2. Test Cases

6.3. Pressure test

Stress testing is a basic quality assurance behavior that is part of every important software testing effort. The basic idea of stress testing is simple: not to run manual or automated tests under normal conditions, but to run tests with fewer computers or poor system resources. The resources that are typically used for stress testing include internal memory, CPU availability, disk space, and network bandwidth. Usually use concurrency to do stress testing.

Pressure test tools: webbench,apachebench, etc.

6.4. Vulnerability Testing

The vulnerabilities in our system include: SQL injection vulnerability, XSS cross-site scripting attack, etc. Security aspects also include system software, such as operating system vulnerabilities, MySQL, Apache and other vulnerabilities, can generally be resolved through the upgrade.

PHP high concurrency high load system architecture