php多進程插入數據(pcntl 學習筆記二)

我的在虛擬機centos7，單核，1G內存

/**
 * 模擬併發請求，10萬次寫入數據庫
 * 拆分爲10個進程，每一個進程處理一萬條插入
 */

$total = 10000;
$num   = 10;
$per   = $total/$num;

$sql  = '';
$child = '';

echo 'start '.microtime(true).PHP_EOL;
for($i = 1; $i<= $num; $i++)
{
    $pid = pcntl_fork();
    if($pid == -1) {
        die('fork error');
    }
    if($pid > 0) {
        //$id = pcntl_wait($status,WNOHANG);
        $child[] = $pid;
    } else if ($pid == 0) {
        $link  = mysqli_connect('localhost','root','root','yii2advanced');
        $start = ($i-1)*$per + 1;
        $end   = $start + $per;
        for($j = $start; $j< $end; $j++){
            $time = microtime(true);
            $sql = 'insert pcntl_test (rank,time) values ('.$j.','.$time.')';
            mysqli_query($link,$sql);
        }
        mysqli_close($link);
        $id = getmypid();
        echo 'child '.$id.' finished '.microtime(true).PHP_EOL;
        exit(0);
    }
}

while(count($child)){
    foreach($child as $k => $pid) {
        $res = pcntl_waitpid($pid, $status, WNOHANG);
        if ( -1 == $res || $res > 0) {
            unset($child[$k]);
        }
    }
}
echo 'end '.microtime(true).PHP_EOL;

當$total=10000，$num = 10;執行結果以下:

start 1491903371.5548
child 19860 finished 1491903417.2113
child 19857 finished 1491903417.6909
child 19864 finished 1491903417.7793
child 19855 finished 1491903417.8695
child 19859 finished 1491903417.9162
child 19861 finished 1491903418.0089
child 19856 finished 1491903418.0532
child 19863 finished 1491903418.0842
child 19862 finished 1491903418.1474
child 19858 finished 1491903418.4341
end 1491903418.4424
總時間爲46.88759994506836秒

當$total=10000,$num = 100時，執行結果以下：

start 1491904334.1735
child 20085 finished 1491904337.0712
child 20086 finished 1491904337.144
……
child 20262 finished 1491904341.5602
child 20264 finished 1491904341.5803
end 1491904341.5869
總時間爲7.413399934768677

當$total=10000,$num = 1000時，執行結果以下：

start 1491904562.0166
child 20282 finished 1491904562.1191
child 20277 finished 1491904562.1268
child 20279 finished 1491904562.1352
...
child 21586 finished 1491904576.6954
child 21582 finished 1491904576.7024
child 21584 finished 1491904576.7226
end 1491904576.7297
總時間爲14.71310019493103，相比100個子進程，耗時更長了。進程切換太多，影響了了效率應該是緣由之一。

當$total=100000 ,$num=100時，十萬條記錄，100個進程插入

start 1491905670.2652
child 21647 finished 1491905725.4382
child 21651 finished 1491905725.4595
child 21642 finished 1491905725.5402
....
child 21810 finished 1491905729.7709
child 21812 finished 1491905729.8498
child 21811 finished 1491905729.9612
end 1491905729.9679
總時間爲59.70270013809204

單進程插入1萬條數據，耗時18秒，相對10個進程插入1萬記錄來講，耗時少些。
而單進程插入10萬條記錄，耗時187.40066790581，相對來講，是挺慢的了。三分鐘。。。

不過，本人再fork 1000個進程，來插入10萬記錄時，成功的狀況下36秒左右，也可能會出現錯誤，mysqli_connection返回false，是否是鏈接數受限制了？

fork 一萬個子進程，插入一百萬數據，這時，出現鏈接錯的狀況就不少了。最後耗時360秒，數據表中插入了945300條記錄,成功率94.53%。因而查看數據庫的相關配置信息

mysql>  show global status like '%connect%';
+-----------------------------------------------+---------------------+
| Variable_name                                 | Value               |
+-----------------------------------------------+---------------------+
| Aborted_connects                              | 0                   |
| Connection_errors_accept                      | 0                   |
| Connection_errors_internal                    | 0                   |
| Connection_errors_max_connections             | 628                 |
| Connection_errors_peer_address                | 0                   |
| Connection_errors_select                      | 0                   |
| Connection_errors_tcpwrap                     | 0                   |
| Connections                                   | 16519               |
| Locked_connects                               | 0                   |
| Max_used_connections                          | 501                 |
| Max_used_connections_time                     | 2017-04-12 15:19:54 |
| Performance_schema_session_connect_attrs_lost | 0                   |
| Ssl_client_connects                           | 0                   |
| Ssl_connect_renegotiates                      | 0                   |
| Ssl_finished_connects                         | 0                   |
| Threads_connected                             | 4                   |
+-----------------------------------------------+---------------------+


mysql>  show global variables like '%connect%';
+-----------------------------------------------+--------------------+
| Variable_name                                 | Value              |
+-----------------------------------------------+--------------------+
| character_set_connection                      | utf8mb4            |
| collation_connection                          | utf8mb4_general_ci |
| connect_timeout                               | 10                 |
| disconnect_on_expired_password                | ON                 |
| init_connect                                  |                    |
| max_connect_errors                            | 100                |
| max_connections                               | 500                |
| max_user_connections                          | 0                  |
| performance_schema_session_connect_attrs_size | 512                |
+-----------------------------------------------+--------------------+

修改 myqsql 配置文件，/etc/my.cnf
把max_connections 改成10000，而後重啓mysql
實際MySQL服務器容許的最大鏈接數16384；
結果真並卵，虛擬機好像掛了了。

併發量大的時候，問題就出在了鏈接mysql這裏。
能夠經過一個鏈接池來嘗試解決該問題。