Implementation of Map-reduce in program based on Swoole task function

The swoole extension comes with a very powerful task process that can be used to implement a variety of complex business logic. This article mainly introduces the use of Task/finish function to implement the Map-reduce concurrent task processing within the program. A chat service often has group chat needs, my group and group members, in addition group members need to sort by points, similar to such a function can be used swoole simple implementation.

Traditional multithreaded Solutions

Create 2 global variables Map,group_map to group_id as key and store member set. User_map stores all the group that the current user joins with the UID key.

In the multi-threaded environment can not actually directly manipulate the 2 map, must be locked. When you add a user to a group or the user exits a group, you need to manipulate the 2 maps, which must be locked. If the operation is very frequent, the collision of the lock is actually very serious, this part of the operation will become serial. At the same time, only one thread can manipulate the map. The scramble for locks also leads to a lot of thread switching wasting a lot of CPU resources.

Lock.lock (); Group_map[group_id].append ([UID, score]); User_map[uid].append (group_id); Group_map.sortbyscore (); Lock.unlock ();

Swoole-based task function

Based on the Swoole task function, you can slice and then hash the tasks into different task processes to complete the task. The sorting function can be implemented directly using PHP's splheap , the time complexity is O (logn), if you want to implement the query function, such as the query according to the UID users join all groups, according to GroupID query which members. The hash can be calculated to find the corresponding task process, and then send instructions through task/taskwait, directly read the process variables to find information.

 $serv->set (Array ("Task_worker_num"), $serv->task ("cmd" = "User", "UID" and "$uid", "gid" = $gid, "score" and "+ $score", $gid% $task _worker_num); $serv->task (Array ("CMD" =&G T  "Group", "uid" = $uid, "gid" = $gid), $uid% $task _worker_num); class Mymaxheap extends splheap{public function    Compare ($value 1, $value 2) {return ($value 1[' score ')-$value 2[' score ']);    }}function Ontask ($serv, $taskId, $srcWorkerId, $data) {Static $userMap = array ();        Static $groupMap = Array (); if ($data [' cmd '] = = ' Group ') {if (!isset ($groupMap [$data [' gid ']]) {$groupMap [$data [' gid ']]        = new Mymaxheap ();        } $heap = $groupMap [$data [' gid '];    $heap->insert ("UID" + $data [' uid '], "score" = [' score ']);    } elseif ($data [' cmd '] = = ' user ') {$userMap [$data [' uid ']][] = $data [' gid ']; }}

Because the task process has only array operations, it is non-blocking and only requires the same number of processes as the number of CPU cores to be opened. No lock scrambling between processes, very good performance. The Swoole task process communication uses Unixsocket, which is the full memory communication provided by the kernel without any IO and a single read process up to 1 million per second. Although there is no direct read variable speed, but the performance is enough.

———— – The Great Dividing line —————

PHP rice Grain (phpfamily) by a group of reliable people to build, is willing to bring some phper to taste the spiritual food!

This article by Rango Exclusive authorized PHP Rice Grain released, reproduced please indicate the original source information and the following QR code (long press to identify the two-dimensional code attention):

