php多進程插入資料

個人在虛擬機器centos7，單核，1G記憶體

/** * 類比並發請求，10萬次寫入資料庫 * 拆分為10個進程，每個進程處理一萬條插入 */$total = 10000;$num   = 10;$per   = $total/$num;$sql  = '';$child = '';echo 'start '.microtime(true).PHP_EOL;for($i = 1; $i<= $num; $i++){    $pid = pcntl_fork();    if($pid == -1) {        die('fork error');    }    if($pid > 0) {        //$id = pcntl_wait($status,WNOHANG);        $child[] = $pid;    } else if ($pid == 0) {        $link  = mysqli_connect('localhost','root','root','yii2advanced');        $start = ($i-1)*$per + 1;        $end   = $start + $per;        for($j = $start; $j< $end; $j++){            $time = microtime(true);            $sql = 'insert pcntl_test (rank,time) values ('.$j.','.$time.')';            mysqli_query($link,$sql);        }        mysqli_close($link);        $id = getmypid();        echo 'child '.$id.' finished '.microtime(true).PHP_EOL;        exit(0);    }}while(count($child)){    foreach($child as $k => $pid) {        $res = pcntl_waitpid($pid, $status, WNOHANG);        if ( -1 == $res || $res > 0) {            unset($child[$k]);        }    }}echo 'end '.microtime(true).PHP_EOL;

當$total=10000，$num = 10;執行結果如下:

start 1491903371.5548child 19860 finished 1491903417.2113child 19857 finished 1491903417.6909child 19864 finished 1491903417.7793child 19855 finished 1491903417.8695child 19859 finished 1491903417.9162child 19861 finished 1491903418.0089child 19856 finished 1491903418.0532child 19863 finished 1491903418.0842child 19862 finished 1491903418.1474child 19858 finished 1491903418.4341end 1491903418.4424總時間為46.88759994506836秒

當$total=10000,$num = 100時，執行結果如下：

start 1491904334.1735child 20085 finished 1491904337.0712child 20086 finished 1491904337.144……child 20262 finished 1491904341.5602child 20264 finished 1491904341.5803end 1491904341.5869總時間為7.413399934768677

當$total=10000,$num = 1000時，執行結果如下：

start 1491904562.0166child 20282 finished 1491904562.1191child 20277 finished 1491904562.1268child 20279 finished 1491904562.1352...child 21586 finished 1491904576.6954child 21582 finished 1491904576.7024child 21584 finished 1491904576.7226end 1491904576.7297總時間為14.71310019493103，相比100個子進程，耗時更長了。進程切換太多，影響了了效率應該是原因之一。

當$total=100000 ,$num=100時，十萬條記錄，100個進程插入

start 1491905670.2652child 21647 finished 1491905725.4382child 21651 finished 1491905725.4595child 21642 finished 1491905725.5402....child 21810 finished 1491905729.7709child 21812 finished 1491905729.8498child 21811 finished 1491905729.9612end 1491905729.9679總時間為59.70270013809204

單進程插入1萬條資料，耗時18秒，相對10個進程插入1萬記錄來說，耗時少些。
而單進程插入10萬條記錄，耗時187.40066790581，相對來說，是挺慢的了。三分鐘。。。

不過，本人再fork 1000個進程，來插入10萬記錄時，成功的情況下36秒左右，也可能會出現錯誤，mysqli_connection返回false，是不是串連數受限制了？

fork 一萬個子進程，插入一百萬資料，這時，出現串連錯的情況就很多了。最後耗時360秒，資料表中插入了945300條記錄,成功率94.53%。於是查看資料庫的相關配置資訊

mysql>  show global status like '%connect%';+-----------------------------------------------+---------------------+| Variable_name                                 | Value               |+-----------------------------------------------+---------------------+| Aborted_connects                              | 0                   || Connection_errors_accept                      | 0                   || Connection_errors_internal                    | 0                   || Connection_errors_max_connections             | 628                 || Connection_errors_peer_address                | 0                   || Connection_errors_select                      | 0                   || Connection_errors_tcpwrap                     | 0                   || Connections                                   | 16519               || Locked_connects                               | 0                   || Max_used_connections                          | 501                 || Max_used_connections_time                     | 2017-04-12 15:19:54 || Performance_schema_session_connect_attrs_lost | 0                   || Ssl_client_connects                           | 0                   || Ssl_connect_renegotiates                      | 0                   || Ssl_finished_connects                         | 0                   || Threads_connected                             | 4                   |+-----------------------------------------------+---------------------+mysql>  show global variables like '%connect%';+-----------------------------------------------+--------------------+| Variable_name                                 | Value              |+-----------------------------------------------+--------------------+| character_set_connection                      | utf8mb4            || collation_connection                          | utf8mb4_general_ci || connect_timeout                               | 10                 || disconnect_on_expired_password                | ON                 || init_connect                                  |                    || max_connect_errors                            | 100                || max_connections                               | 500                || max_user_connections                          | 0                  || performance_schema_session_connect_attrs_size | 512                |+-----------------------------------------------+--------------------+修改 myqsql 設定檔，/etc/my.cnf把max_connections 改為10000，然後重啟mysql實際MySQL伺服器允許的最大串連數16384；結果然並卵，虛擬機器好像掛了了。

並發量大的時候，問題就出在了串連mysql這裡。
可以通過一個串連池來嘗試解決該問題。

個人在虛擬機器centos7，單核，1G記憶體

/** * 類比並發請求，10萬次寫入資料庫 * 拆分為10個進程，每個進程處理一萬條插入 */$total = 10000;$num   = 10;$per   = $total/$num;$sql  = '';$child = '';echo 'start '.microtime(true).PHP_EOL;for($i = 1; $i<= $num; $i++){    $pid = pcntl_fork();    if($pid == -1) {        die('fork error');    }    if($pid > 0) {        //$id = pcntl_wait($status,WNOHANG);        $child[] = $pid;    } else if ($pid == 0) {        $link  = mysqli_connect('localhost','root','root','yii2advanced');        $start = ($i-1)*$per + 1;        $end   = $start + $per;        for($j = $start; $j< $end; $j++){            $time = microtime(true);            $sql = 'insert pcntl_test (rank,time) values ('.$j.','.$time.')';            mysqli_query($link,$sql);        }        mysqli_close($link);        $id = getmypid();        echo 'child '.$id.' finished '.microtime(true).PHP_EOL;        exit(0);    }}while(count($child)){    foreach($child as $k => $pid) {        $res = pcntl_waitpid($pid, $status, WNOHANG);        if ( -1 == $res || $res > 0) {            unset($child[$k]);        }    }}echo 'end '.microtime(true).PHP_EOL;

當$total=10000，$num = 10;執行結果如下:

start 1491903371.5548child 19860 finished 1491903417.2113child 19857 finished 1491903417.6909child 19864 finished 1491903417.7793child 19855 finished 1491903417.8695child 19859 finished 1491903417.9162child 19861 finished 1491903418.0089child 19856 finished 1491903418.0532child 19863 finished 1491903418.0842child 19862 finished 1491903418.1474child 19858 finished 1491903418.4341end 1491903418.4424總時間為46.88759994506836秒

當$total=10000,$num = 100時，執行結果如下：

start 1491904334.1735child 20085 finished 1491904337.0712child 20086 finished 1491904337.144……child 20262 finished 1491904341.5602child 20264 finished 1491904341.5803end 1491904341.5869總時間為7.413399934768677

當$total=10000,$num = 1000時，執行結果如下：

start 1491904562.0166child 20282 finished 1491904562.1191child 20277 finished 1491904562.1268child 20279 finished 1491904562.1352...child 21586 finished 1491904576.6954child 21582 finished 1491904576.7024child 21584 finished 1491904576.7226end 1491904576.7297總時間為14.71310019493103，相比100個子進程，耗時更長了。進程切換太多，影響了了效率應該是原因之一。

當$total=100000 ,$num=100時，十萬條記錄，100個進程插入

start 1491905670.2652child 21647 finished 1491905725.4382child 21651 finished 1491905725.4595child 21642 finished 1491905725.5402....child 21810 finished 1491905729.7709child 21812 finished 1491905729.8498child 21811 finished 1491905729.9612end 1491905729.9679總時間為59.70270013809204

單進程插入1萬條資料，耗時18秒，相對10個進程插入1萬記錄來說，耗時少些。
而單進程插入10萬條記錄，耗時187.40066790581，相對來說，是挺慢的了。三分鐘。。。

不過，本人再fork 1000個進程，來插入10萬記錄時，成功的情況下36秒左右，也可能會出現錯誤，mysqli_connection返回false，是不是串連數受限制了？

fork 一萬個子進程，插入一百萬資料，這時，出現串連錯的情況就很多了。最後耗時360秒，資料表中插入了945300條記錄,成功率94.53%。於是查看資料庫的相關配置資訊

mysql>  show global status like '%connect%';+-----------------------------------------------+---------------------+| Variable_name                                 | Value               |+-----------------------------------------------+---------------------+| Aborted_connects                              | 0                   || Connection_errors_accept                      | 0                   || Connection_errors_internal                    | 0                   || Connection_errors_max_connections             | 628                 || Connection_errors_peer_address                | 0                   || Connection_errors_select                      | 0                   || Connection_errors_tcpwrap                     | 0                   || Connections                                   | 16519               || Locked_connects                               | 0                   || Max_used_connections                          | 501                 || Max_used_connections_time                     | 2017-04-12 15:19:54 || Performance_schema_session_connect_attrs_lost | 0                   || Ssl_client_connects                           | 0                   || Ssl_connect_renegotiates                      | 0                   || Ssl_finished_connects                         | 0                   || Threads_connected                             | 4                   |+-----------------------------------------------+---------------------+mysql>  show global variables like '%connect%';+-----------------------------------------------+--------------------+| Variable_name                                 | Value              |+-----------------------------------------------+--------------------+| character_set_connection                      | utf8mb4            || collation_connection                          | utf8mb4_general_ci || connect_timeout                               | 10                 || disconnect_on_expired_password                | ON                 || init_connect                                  |                    || max_connect_errors                            | 100                || max_connections                               | 500                || max_user_connections                          | 0                  || performance_schema_session_connect_attrs_size | 512                |+-----------------------------------------------+--------------------+修改 myqsql 設定檔，/etc/my.cnf把max_connections 改為10000，然後重啟mysql實際MySQL伺服器允許的最大串連數16384；結果然並卵，虛擬機器好像掛了了。

並發量大的時候，問題就出在了串連mysql這裡。
可以通過一個串連池來嘗試解決該問題。