PHP high Concurrent high load system Architecture example

1. Why do high concurrent and high load research

2, high concurrency and high load constraint conditions

3, the solution--Hardware Chapter

4, the Solution-Deployment Chapter

5, the solution--environmental article

6, the solution of the road--siteengine article

7, the solution--Test article

1, why to carry out high concurrency and high load research

1.1, the needs of product development

1.2, the need for the development of the company

1.3, the current form of decision

2, high concurrency and high load constraint conditions

2.1, Hardware

2.2, deployment

2.3. Operating system

2.4, Web Server

2.5, PHP

2.6, MySQL

2.7, testing

3, the solution--Hardware Chapter

Processing capacity of the upgrade: the deployment of multiple CPUs, the choice of multiple cores, with higher operating frequency, a larger cache of CPUs;

The most direct response to the increase in processing power is the processing efficiency of Web requests and the execution efficiency of PHP programs.

Memory bandwidth and Capacity: greater memory bandwidth and capacity;

The most direct response to the increase in memory bandwidth and capacity is to deal with a large number of data exchange in the database.

Disk Search and I/O capabilities: Select higher rpm, larger hard disk cache, component disk array (RAID);

The most direct response to the increase in disk search and I/O capability is the database's large number of queries and reads and writes, and file reading and writing.

The promotion of network bandwidth can be considered factors include: greater bandwidth, multiple line access, exclusive bandwidth;

Server in the case of heavy load, the network bandwidth consumption is very considerable.

Strategy: Hardware facilities are the basis for coping with heavy loads, and the input of hardware facilities can be based on actual pressure and budget.

4, the Solution-Deployment Chapter

4.1, Server separation

4.2, database cluster and library table hash

4.3, mirror

4.4. Load balance

Classification:

1), DNS round robin

2) proxy Server load Balancing

3) Address Translation Gateway load Balancing

4) Nat Load Balancing

5) Reverse Proxy load Balancing

6) Mixed Load-balancing

Strategy: Choose a reasonable deployment plan based on hardware input and business requirements.

Deployment Scenario 1:

Applicable scope: Static content as the main site and application system, the system security requirements of the Web site and application system.

Main server: Primary servers

The main body of the load-carrying program is pressure to handle the dynamic request in the website or application system;

Push static pages to multiple publishers;

Push the attachment file to the file server;

High security requirements, static-oriented web site, the server can be placed in the intranet screen access to the network.

DB Server: Database servers

Load database reading and writing pressure;

Only with the primary server for data volume exchange, shielding extranet access.

File/video server: File/video Server

The data stream which occupies the system resource and the bandwidth resource is large in the load system;

Storage and read-write warehouses as large attachments;

As a video server will have the ability to automate video processing.

Publishing Server Group:

Only responsible for the release of static pages, hosting the vast majority of Web requests;

Load balanced deployment through Nginx.

Deployment Scenario 2:

Applicable scope: The website or application system with dynamic interactive content as the main body, and the website or application system with large load pressure and sufficient budget;

Web server groups:

Web Service has no master-slave relationship, and it belongs to parallel redundancy design.

Load balance is realized by front-end load balancing device or nginx reverse proxy;

Partition dedicated file server/video server effectively separate light/heavy bus;

Each Web server can connect all databases through Dec, while dividing the master from the other.

Database Server Groups:

The relatively balanced load database read and write pressure;

The data synchronization of multiple databases is realized through the mapping of database physical files.

Shared disk/disk array

Unified Read and write for data physical files

Storage Warehouses for large attachments

Ensure the IO efficiency and data security of the whole system through the balance and redundancy of the physical disk;

Program Characteristics:

Through the front-end load balance, the rational allocation of web pressure;

By separating the file/video server from the regular Web server, the data stream is allocated reasonably.

Through the database server group, the database IO pressure is allocated reasonably.

Each Web server usually connects to only one database server, and can automatically switch to a redundant database server in a very short time through the DEC heartbeat detection;

With the introduction of disk array, the efficiency of system IO is greatly improved, and the data security is enhanced.

5, the solution--environmental article

5.1. Operating system

The choice of operating system, the focus is

• Are you comfortable with the environmental procedures needed to build siteengine?
• The ratio of resources occupied by the system itself;
• system security;
• Is the system easy to operate?
Strategy: We choose FreeBSD, and it is the FreeBSD after the installation is minimized.

5.2, Web server

A large portion of the Web server's resources are consumed by processing Web requests, which is typically the pressure of Apache, the world's first-ranked Web server software. It can run on almost any widely used computer platform.

In the case of high concurrent connections, Nginx is a good substitute for Apache servers. Nginx ("Engine X") is a high-performance HTTP and reverse proxy server written by the Russians. In China, there have been Sina, Sohu Pass, NetEase news, NetEase blog, Jinshan Xiaoyao net, Kimshan PowerWord, network, Yupoo albums, Watercress, Thunder look at a number of websites, channels using Nginx server.

Advantages of Nginx:

High concurrent Connection: the official test can support 50,000 concurrent connection, in the actual production environment to run to 2~3 million concurrent connection number.

Low memory consumption: Under 30,000 concurrent connections, the 10 nginx processes that are opened consume 150M of memory (15m*10=150m).

Built-in Health Check feature: If a WEB server on the back end of the Nginx proxy is down, it will not affect front-end access.

Strategy: As opposed to the old Apache, we chose lighttpd and Nginx Web servers with smaller resource usage and higher load capacity.

5.3, Mysql

MySQL itself has a strong load capacity, MySQL optimization is a very complex work, because this ultimately requires a good understanding of system optimization. We all know that database work is a large number of short queries and read and write, in addition to the program development needs to pay attention to indexing, improve query efficiency, and other software development techniques, from the point of view of hardware facilities to affect the MySQL execution efficiency most from disk search, disk IO level, CPU cycle, memory bandwidth

Based on the hardware and software conditions on the server for MySQL optimization. The core of MySQL optimization is the allocation of system resources, which is not equal to the unlimited allocation of resources to MySQL. In the MySQL configuration file we present several of the most noteworthy parameters:

Change index buffer Length (key_buffer)

Change the length of the table (read_buffer_size)

Sets the maximum number of open tables (Table_cache)

Set a time limit for slow-length queries (long_query_time)

If the conditions allow, the general MySQL server is best installed in the Linux operating system, rather than installed in the FreeBSD.

Strategy: MySQL optimization needs to be based on the business System database read-write characteristics and server hardware configuration, the development of different optimization, and can be deployed as required MySQL master-slave structure.

5.4, PHP

1, loading as few modules as possible;

2, if it is under the Windows platform, as far as possible using IIS or Nginx to replace our usual Apache;

3, installation accelerator (all through the cache of PHP code precompiled results and database results to improve the execution speed of PHP code)

Eaccelerator

Eaccelerator is a free and open source PHP accelerator, optimized and dynamic content caching, improve performance of PHP script cache performance, so that PHP script in the state of compilation, the server's overhead almost completely eliminated.

Apc

Alternative PHP cache (APC) is a free and publicly optimized code cache for PHP. It is used to provide a free, open and robust architecture to cache and optimize PHP's intermediate code.

Memcache

Memcache is a high-performance, distributed memory object caching system developed by Danga Interactive to reduce database load and improve access speed in dynamic applications. The primary mechanism is to maintain a unified, large hash table in memory, memcache can be used to store data in a variety of formats, including images, videos, files, and the results of database retrieval.

Xcache

The buffers developed by our countrymen,

Policy: Install Accelerator for PHP.

5.5, Proxy server (caching server)

Squid cache (squid for short) is a popular free software (GNU General Public License) proxy Server and Web caching server. Squid has a wide range of uses, from the Web server as a front cache server caching related requests to improve the speed of the Web server, to a group of people sharing network resources and caching the World Wide Web, domain Name System and other network search, to filter traffic to help network security, to the LAN through the proxy Internet. Squid is mainly designed to run on Unix-type systems.

Strategy: Install Squid reverse proxy server, can greatly improve server efficiency.

6, the solution of the road--siteengine article

7, the solution--Test article

7.1. Test method

7.2. Test Cases

7.3. Pressure test

Stress testing is a basic quality assurance behavior that is part of every important software testing effort. The basic idea of stress testing is simple: not to run manual or automated tests under normal conditions, but to run tests if the number of computers is low or system resources are scarce. Typically, the resources for stress testing include internal memory, CPU availability, disk space, and network bandwidth. Generally use concurrent to do stress test.

Pressure testing tools: webbench,apachebench, etc.

7.4. Vulnerability test

Vulnerabilities in our systems include: SQL injection vulnerabilities, XSS cross-site scripting attacks, and more. Security also includes system software, such as operating system vulnerabilities, MySQL, Apache and other vulnerabilities, can generally be resolved by upgrading.

Vulnerability testing Tool: Acunetix Web Vulnerability Scanner