Multi-thread collection in stream mode. please analyze the slow speed of the collected content. I have been very busy. recently I have been studying multi-thread collection. below I have posted a comparison code. There are two problems, first, the length of the obtained results is a bit inconsistent; second, is the efficiency not high enough? Help analysis and test! PHPcode & lt ;? Php $ timeStart = multi-thread collection in the microt stream mode.
The collection speed is slow, and I have been very large. recently I have been studying multi-threaded collection. The following is a comparison code. There are two problems: first, the length of the obtained results is a bit inconsistent; 2. is efficiency not high enough? Help analysis and test!
PHP code
';}$timeEnd = microtimeFloat();echo sprintf("Spend time: %s second(s)\n", $timeEnd - $timeStart),'
';$timeStart = microtimeFloat();$timeout = 30;$status = array();$retdata = array();$sockets = array();$userAgent = $_SERVER['HTTP_USER_AGENT'];foreach($urls as $id => $url) { $tmp = parse_url($url); $host = $tmp['host']; $path = isset($tmp['path'])?$tmp['path']:'/'; empty($tmp['query']) or $path .= '?' . $tmp['query']; if (empty($tmp['port'])) { $port = $tmp['scheme'] == 'https' ? 443 : 80; } else $port = $tmp['port']; $fp = stream_socket_client("$host:$port", $errno, $errstr, 30); if (!$fp) { $status[$id] = "failed, $errno $errstr"; } else { $status[$id] = "in progress"; $retdata[$id] = ''; $sockets[$id] = $fp; fwrite($fp, "GET $path HTTP/1.1\r\nHost: $host\r\nUser-Agent: $userAgent\r\nConnection: Close\r\n\r\n"); }}// Now, wait for the results to come back inwhile (count($sockets)) { $read = $write = $sockets; //This is the magic function - explained below if (stream_select($read, $write = null, $e = null, $timeout)) { //readable sockets either have data for us, or are failed connection attempts foreach ($read as $r) { $id = array_search($r, $sockets); $data = fread($r, 8192); if (strlen($data) == 0) { if ($status[$id] == "in progress") { $status[$id] = "failed to connect"; } fclose($r); unset($sockets[$id]); } else { $retdata[$id] .= $data; } } }}foreach($retdata as $data){ $data = trim(substr($data, strpos($data, "\r\n\r\n") + 4)); echo strlen($data),'
';}$timeEnd = microtimeFloat();echo sprintf("Spend time: %s second(s)\n", $timeEnd - $timeStart);?>
------ Solution --------------------
You can try curl_multi _... concurrent execution
This can minimize the number of php commands. Php cannot solve this problem.
------ Solution --------------------
Of course, file_get_contents () is of the blocking type, so if you execute multiple capture tasks, of course, it will be slow.
Socket _ * (), fsockopen (), stream _ * () are not blocked.
------ Solution --------------------
To what extent?
Try to add this:
$ Context = stream_context_create (array ('http '=> array ('header' => 'connection: close ')));
File_get_contents ("...", false, $ context );