"Reprint" PHP-FPM High CPU Usage Troubleshooting method

1. CPU Usage Monitoring

Sar-p All 1 100

The output results are as follows:
CPU%user%nice%system%iowait%steal%idle
All 85.54 0.00 5.69 0.00 0.00 8.76
0 74.75 0.00 25.25 0.00 0.00 0.00
1 98.00 0.00 2.00 0.00 0.00 0.00
2 89.22 0.00 3.92 0.00 0.00 6.86
3 91.00 0.00 2.00 0.00 0.00 7.00
4 75.00 0.00 9.00 0.00 0.00 16.00
5 94.95 0.00 5.05 0.00 0.00 0.00
6 95.00 0.00 4.00 0.00 0.00 1.00
7 87.88 0.00 4.04 0.00 0.00 8.08
8 93.94 0.00 3.03 0.00 0.00 3.03
9 88.00 0.00 3.00 0.00 0.00 9.00
10 89.11 0.00 2.97 0.00 0.00 7.92
11 82.35 0.00 3.92 0.00 0.00 13.73
12 73.27 0.00 7.92 0.00 0.00 18.81
13 81.44 0.00 4.12 0.00 0.00 14.43
14 77.23 0.00 6.93 0.00 0.00 15.84
15 78.79 0.00 4.04 0.00 0.00 17.17

2. Turn on PHP-FPM slow log

Configure output PHP-FPM slow log with a threshold of 2 seconds:

2= log/$pool. Log.slow #设置慢日志的存放位置, view: Tail-f Xxx.slow.log

Analyze summary php-fpm Slow logs with the Sort/uniq command:

[Email protected] log]#grep-v "^$" www.log.slow.tmp | cut-d ""-F 3,2 | Sort | uniq-c | SORT-K1,1NR | Head-n
5181 Run ()/www/test.net/framework/web/filters/cfilter.php:41
5156 filter ()/www/test.net/framework/web/filters/cfilterchain.php:131
2670 =/www/test.net/index.php
2636 run ()/www/test.net/application/controllers/survey/index.php:665
2630 Action ()/www/test.net/application/controllers/survey/index.php:18
2625 Run ()/www/test.net/framework/web/actions/caction.php:75
2605 Runwithparams ()/www/test.net/framework/web/ccontroller.php:309
2604 runaction ()/www/test.net/framework/web/filters/cfilterchain.php:134
2538 run ()/www/test.net/framework/web/ccontroller.php:292
2484 runactionwithfilters ()/www/test.net/framework/web/ccontroller.php:266
2251 Run ()/www/test.net/framework/web/cwebapplication.php:276
1799 translate ()/www/test.net/application/libraries/limesurvey_lang.php:118
1786 Load_tables ()/www/test.net/application/third_party/php-gettext/gettext.php:254
1447 Runcontroller ()/www/test.net/framework/web/cwebapplication.php:135

Parameter explanation:

----Take the first 10 rows of data

3. Tracking process with strace

1) Use Nohup to move the strace to the background until the PHP-FPM process on the attach dies:

Nohup strace-t-P 13167 > 13167-strace.log &

Parameter description:

2) can also use the-c parameter to make Strace help summary, very convenient very powerful!

[Email protected] log]# STRACE-CP 9907
Process 9907 Attached-interrupt to quit
Process 9907 Detached
% time seconds Usecs/call calls errors Syscall
------ ----------- ----------- --------- --------- ----------------
56.61 0.016612 5 3121 Read
11.11 0.003259 1 2517 715 stat
8.04 0.002358 7 349 BRK
6.02 0.001767 1 1315 poll
4.28 0.001255 6 228 recvfrom
2.71 0.000796 1 671 Open
2.54 0.000745 0 2453 Fcntl
2.37 0.000696 1 1141 Write
1.69 0.000497 1 593 Access
1.37 0.000403 0 1816 Lseek
0.89 0.000262 1 451 sendto
0.56 0.000163 1 276 208 Lstat
0.49 0.000145 0 384 getcwd
0.31 0.000090 0 1222 Fstat
0.28 0.000082 0 173 Munmap
0.26 0.000077 0 174 mmap
0.24 0.000069 2 socket
0.23 0.000068 0 725 Close
0.00 0.000000 0 Rt_sigaction
0.00 0.000000 0 Rt_sigprocmask
0.00 0.000000 0 1 Rt_sigreturn
0.00 0.000000 0 Setitimer
0.00 0.000000 0 Connect
0.00 0.000000 0 2 Accept
0.00 0.000000 0 Recvmsg
0.00 0.000000 0 Shutdown
0.00 0.000000 0 Bind
0.00 0.000000 0 GetSockName
0.00 0.000000 0 SetSockOpt
0.00 0.000000 0 GetSockOpt
0.00 0.000000 0 8 Getdents
0.00 0.000000 0 ChDir
0.00 0.000000 0 1 Futex
------ ----------- ----------- --------- --------- ----------------
100.00 0.029344 18000 986 Total

PS: You can use Strace to learn the explanation execution process of PHP interpreter

4. Accelerating PHP Interpretation execution

If your own program does not have a problem, just do too much work, can not be optimized. Consider using a PHP accelerator such as APC or XCache to reduce the time it takes for the CPU to interpret PHP files. These PHP accelerators generate intermediate code opcode when the php file is first interpreted, so the execution is much faster and reduces some CPU operations. Take XCache, for example, to see how to install and configure.
XCache: is an open source opcode cache/optimizer, which means that he can improve PHP performance on your server. By buffering the compiled PHP data to shared memory to avoid a repetitive compilation process, he is able to use the buffer-compiled code directly to speed up. You can typically increase your page generation rate by 2 to 5 times times, reducing server load.

Installation Xcache,php.ini is configured as follows, generally recommended xcache.size depending on how much php file, Xcache.count and CPU core number is the same:

[Xcache.admin]
Xcache.admin.enable_auth = Off
Xcache.admin.user = "XCache"
Xcache.admin.pass = ""

[XCache]
Xcache.shm_scheme = "Mmap"
xcache.size=1024m
Xcache.count =16
Xcache.slots =8k
Xcache.ttl=0
Xcache.gc_interval =0
xcache.var_size=16m
Xcache.var_count =1
Xcache.var_slots =8k
Xcache.var_ttl=0
Xcache.var_maxttl=0
Xcache.var_gc_interval =300
Xcache.test =off
Xcache.readonly_protection = Off
; xcache.readonly_protection = On
Xcache.mmap_path = "/dev/zero"
; Xcache.mmap_path = "/tmp/xcache"
Xcache.coredump_directory = ""
Xcache.cacher =on
Xcache.stat=on
Xcache.optimizer =off

[Xcache.coverager]
;; Xcache.coverager =on
;; Xcache.coveragedump_directory = ""

Problems:

Error when starting PHP-FPM: Cannot open or create file set by Xcache.mmap_path, check the path permission or check xcache.size/var_size Against system limitation This is because/tmp/xcache is a file and cannot be created as a directory.

After restarting the PHP-FPM service, use the top command to observe that each worker process virt (including the Swap area) is xcache.size size, but req becomes very small. Using the above configuration makes the peak time for CPU usage shorter, but the peak is still all cores will reach more than 90%, do not know where is not configured to. In addition to high concurrency,/dev/zero This configuration often results in Nginx 502 errors. /tmp/xcache and open readonly_protection are stable.

"Reprint" PHP-FPM High CPU Usage Troubleshooting method