Technical question-how many PHP/Python/Ruby Application Server worker threads do you need?

We often encounter an application server overload or crash because we do not know how to prepare the application server thread. Although such problems often occur in PHP, they also occur in other languages, all of which are the same type of problems.

These problems are usually related to queue theory and dynamic multi-layer queue behavior under low load, as well as throughput, arrival rate and service time. In this way, reasonable system configuration becomes more difficult. In addition, the default settings are relatively poor, and the BUSY system lacks intuition. Finally, the multi-core CPU structure is hard to understand and real-time troubleshooting is also difficult.

However, if you carefully read the following explanation, you will find it easy to solve this problem.

First, you need to understand the cause of the problem. When the application server is under high load, either the CPU is insufficient, or the ram is insufficient or both are insufficient, although the ram problem is usually a CPU problem. Whether Ram is not enough or the CPU is not enough, it is disastrous because they will cause server crashes, and the server with poor configuration will be worse.

There are two important factors that cause this problem.

The first factor is: many applicationsProgramIt is not well written and runs slowly on the CPU. It usually takes 1 or several seconds to start execution. However, this slow execution speed is usually not significant in fast testing and development servers or online systems with powerful hardware. However, when the system is at a low load and its computing power is low, it is indeed a disaster.

The second factor is that many applications consume a lot of RAM, mainly because of useless large dB result cache or memory leakage, resulting in a large memory usage. Every day, we will find that some programs running on the machine occupy an average of 100-256 MB of memory for each instance. The worst single process may even use more than MB of memory.

For reference, if a 16-core machine runs 32 programs, only applications will occupy at least 32 x mb or 8 gb ram. This problem may even occur when you configure the best server. Because the program occupies a large amount of space, if Ram is 4-8 GB, the situation will be worse.

If the system load is large and the delay is long, the number of processes increases. This is because if the application scheduler in Apache finds that all processes are busy, it will create more processes until the number of maxclients (or FPM sub-processes) is reached). There are usually 256 sub-processes, multiplied by 256 MB, which will reach 64 gbram, far exceeding the application server's Ram.

The direct consequence of this problem is that the system starts switching when the number of processes is small, which will lead to a process delay and an increase in the number of processes. This type of endless loop is typically characterized by CPU overload, large number of processes, low ram, and excessive use of swap, which leads to website crash. The worst consequence is that because swap is used too much, you cannot even log on to the system via SSH to solve the problem.

Another cause of system failure is that when there is enough RAM, the CPU is heavily overloaded, but the system frequently exchanges among hundreds of Apache processes. This will stop all throughput, not to mention logging in.

Finally, we found that the database performance seriously affected the overall system performance. No matter whether the application server is busy or not, slow database response will increase the number of processes, resulting in insufficient server Ram.

The solution is straightforward: set the maximum number of processes to 2 times the number of cores. At the same time, make sure that you have enough RAM to process the process when the maximum memory usage is reached. In this case, when the CPU load is very large, the system will slow down. However, in this configuration, the process will be ranked in the queue with a reasonable load.

We set it to 2 times the number of cores because the process is delayed. For example, if you want to connect to a remote database, the process will be paused during this connection. Therefore, during this period, you must have additional programs to maximize CPU utilization.

However, an exception is that when you have RPC requests outside the server, such as using SE or SOLR in the query system, or a real remote system such as Facebook. These situations are not discussed in this blog, but they adopt more complex configurations and monitoring to ensure that the system can still run successfully at high load.

To sum up, set the application server process to 2 xcores and ensure that there is sufficient Ram. This will not overload your system, optimize your system, reduce CPU and RAM usage, and you can scale on the same hardware. Otherwise, you will surely encounter a bottleneck of system expansion.

(Authored by Steve mushero/CEO of chinanetcloud & CTOClick here to view)