PHP multi-process concurrency control test Cases

Turn http://blog.s135.com/post/311/

Recently encountered a problem, Linux under the PHP command-line program as a daemon, the need to read from the queue file a row of data, through the TCP protocol sent to the field of the receiving server, read the next line of data, and then sent. When local and foreign network conditions are bad, sometimes it takes a long time to send a piece of data, which accumulates easily to cause queue congestion and delay.

So I'm going to use the PHP command-line program to generate multiple subprocess and turn the serial processing into parallel processing. The easiest way to do this is to push a shell command line into the background in PHP using the exec () or popen () function, for example: <?php
EXEC ("/bin/sh/opt/zhangyan.sh &");
?> 's final & indicates that the shell script is pushed to the background to execute.

However, there is a problem, if too many processes pushed to the background, may cause the server system resources exhaustion and crash, so must control the number of processes.

I wrote a PHP program (/opt/zhangyan.php), a shell program (/opt/zhangyan.sh) as a test case.

　　The logic of the program:
1, set/opt/zhangyan.php to allow up to 500 sub processes;
2, when the/opt/zhangyan.php read a data, will allow the generation of the number of child processes minus 1 (Idle process number $p_number=500-1=499), and then the data to/opt/zhangyan.sh to the background processing, do not wait for/opt/ zhangyan.sh processing end, continue to read the next data;
3. When the number of child processes allowed to be generated is reduced to 0 o'clock (Idle process Count $p_number=0),/opt/zhangyan.php waits 1 seconds, and then checks that the number of/opt/zhangyan.sh subprocess in the background is not finished;
4, if 1 seconds later/opt/zhangyan.php found the background of the number of/opt/zhangyan.sh subprocess or 500 (Idle process number $p_number=0), will continue to wait 1 seconds, so repeated;
5, if/opt/zhangyan.php found that the background has not processed the end of the/opt/zhangyan.sh number of child processes reduced to 300 (idle process number $p_number=500-300=200), then/opt/ Zhangyan.php will be pushed back to the background to push 200/opt/zhangyan.sh subprocess;
The /opt/zhangyan.php code is as follows:

<?php

function Run ($input)

{

global $p _number;

if ($p _number <= 0)

    {

$p _number = worker_processes ($p _number);

    }

$p _number = $p _number-1;

$out = Popen ("/bin/sh/opt/zhangyan.sh/" {$input}/"&", "R");

Pclose ($out);

}

 

function worker_processes ($p _number)

{

$limit = 500;//Maximum number of processes allowed to be pushed to the background

While ($p _number <= 0)

    {

$cmd = Popen ("Ps-ef | grep/"/opt/zhangyan.sh/" | Grep-v grep | Wc-l "," R ");

$line = fread ($cmd, the);

Pclose ($cmd);

$p _number = $limit-$line;

if ($p _number <= 0)

        {

Sleep (1);//pause for 1 seconds

        }

    }

return $p _number;

}

 

$input = "http://blog.s135.com";//simulate the data read from the queue file

For ($i = 1; $i <= 1000; $i + +)

{

run ($input);

echo "Idle process number:". $p _number. "/n";

}

?>

(The/opt/zhangyan.php program is used to simulate reading 1000 rows of data from the queue file and handing it to the subprocess/opt/zhangyan.sh.) )

 

The /opt/zhangyan.sh code is as follows:

#!/bin/sh

echo $ (date-d "Today" + "%y-%m-%d%h:%m:%s") $ >>/opt/zhangyan.log

sleep_time=$ (expr $RANDOM% 4 + 1)

Sleep $sleep _time

 

(the/opt/zhangyan.sh script is used to simulate sending data to a foreign receiving server.) The $ (expr $RANDOM% 4 + 1) is used to generate random numbers between 1~5 to suspend the program for 1-5 seconds. Pause 1 seconds indicates network condition is good, send data smooth, pause 2-6 seconds to indicate network condition is bad, send process need 1-5 seconds. )

 

Execution procedure:
/usr/local/php/bin/php/opt/zhangyan.php (/usr/local/php/bin/php because of the path of the PHP parser)

　View the first and last lines of log files that/opt/zhangyan.sh has laid:
head-n 1/opt/zhangyan.log 2007-11-16 07:54:13 http://blog.s135.com
tail-n 1/opt/zhangyan.log 2007-11-16 07:54:18 http://blog.s135.com

As you can see, the 500 process is processing these 1000 data concurrently for 5 seconds. According to the original serial mode, processing each piece of data, even if only the shortest 1 seconds, it will take 1000 seconds, about 16 minutes to complete.


　PS: The PHP program as a way to protect the Linux process:
nohup/usr/local/php/bin/php/opt/zhangyan.php 2>&1 >/dev/null &
(The Nohup command can still execute the program after the user exits the terminal, "2>&1 >/dev/null" indicates that standard output and error output are not displayed, and the last & is pushed to the background.) )