A cache callback and auto-triggering technique based on PHP

Background

When using Memcache or Redis in PHP, we typically encapsulate memcache and Redis, write a cache class individually, as a proxy for Memcache live Redis, and typically a singleton mode. In the business code, when using the cache class, the basic sample code for the operation is as follows

Cache Key$key = ' this is key '; $expire = 60;//Timeout///cache Instance $cache = Wk_cache::instance (); $data = $cache->fetch ( $key);//Judge Dataif (Empty ($data)) {    //if empty, call the db method    $db = new wk_db ();    $data = $db->getxxx ();    $cache->store ($key, $data, $expire);} Processing $data related data return $data;

The basic process is
The first step is to assemble the query key, the cache query value, if present, continue processing, enter the third step; if it does not exist, enter the second step.
The second step, according to the request, to the DB, query the relevant data, if the data exists, put the data into the cache
The third step is to process the data returned in the cache or DB

Problem

The above process will basically appear in the part of each call to the cache, the cache query, do not call the DB or third-party interface, obtain data, re-deposited to the cache, continue to data processing. Multiple invocations are both a problem and should be encapsulated in a more basic way. Instead of repeating this logic every time, in addition to the problem of encapsulation, there are other issues, we have a unified list of

First: Duplication of code from a design perspective requires a lower level of logic encapsulation.
Second: Key assembly, cumbersome, the actual situation, may be a variety of parameters assembled into the maintenance of the time, do not dare to modify.
Third: Set the expire time-out, scattered throughout the logic code, it is difficult to count the cache time.
IV: Since the Cache->store method is executed after the call to DB, if there are other logical processes after the DB, it is possible to forget to put the data into the cache, resulting in data.
Five: In the high concurrency system, the cache failure that moment, there will be a large number of requests directly through the rear, resulting in DB or third-party interface pressure rise, response slowed, further affect the stability of the system, this phenomenon is "dogpile".

Of the above problems, the simplest is 2, 3, for the expire time-out dispersion problem, we can be resolved through a unified configuration file, for example, we can create such a configuration file.

"Test" =>array (//namespace, convenient grouping             "keys" = = Array (                 "good" =>array(//Defined key, This key is non-final into the cache key, into key needs and params assembled into a unique key                     "Timeout" =>600,//here to define the time-out                     "params" =>array ("Epid" =>1, "Num" =>1),//Through this method, describes the need to pass parameters to assemble the final cache key                     "desc" and "description"                     ,                 "Top_test" =>array (//Defined key, This key is not the final cache key, and the key needs to be assembled with the params into a unique key                     "Timeout" =>60,//here to define the time-out                     "TTL" =>10,//auto-trigger Time                     " Params "=>array" (' site_id ' =>1, ' boutique ' =>1, ' offset ' =>1, ' rows ' = 1, ' uid ' =>1, ' tag_id ' =>1, ' Type ' =>1),//Through this method, describes the need to pass parameters to assemble the final cache key                     "desc" and "description",                     "author" = "Ugg"                     ,))

as shown above, with an algorithm, we can assemble the site_top_feeds and params into a unique storage key, the assembled key, presumably this site_top_feeds_site_ Id=12&boutique=1&offset=0&rows=20&uid=&tag_id=0&type=2 in this way, we prevent workers from assembling key themselves, thus eliminating the second problem, At the same time in this configuration file, we also set a timeout, so that when the store is called, we can read directly from the configuration file, thus avoiding the third problem. After modifying it as above, our cache method also makes the appropriate adjustments, which are called examples below.

$siteid = 121; $seminal = 1; $tag _id = n; $tag _id = n; $data =  fetch (' site_top_feeds ', Array (' site_id ' = = $siteid, ' Boutique ' + $seminal, ' offset ' = ' 0 ', ' rows ' and ' + ', ' uid ' =>null, ' tag_id ' + $tag _id, ' type ' + $type), ' Feed '), if (Empty ($data)) {//DB related operation $db = new wk_db ();    $data = $db->gettopfeeds ($site _id, $seminal, 0,20,null, $tag _id, $type);//  $data data Other processing logic here ... $cache->store ( ' Site_top_feeds ', $data, Array (' site_id ' = $siteid, ' boutique ' + $seminal, ' offset ' = ' 0 ', ' rows ' = ' 20 ', ' UID ' =>null, ' tag_id ' = = $tag _id, ' type ' = + $type), ' feed ');}

Through the above scenario, I did not see that timeout time-out is gone, key assembly is not, for the outer call is transparent. But we can tell by the configuration file what the timeout of Site_top_feeds is, and through the encapsulated algorithm, know what the assembled key is.
This way, and does not solve the first and fourth problems, encapsulation; To complete the encapsulation, the first thing to do is the callback function, as a school-based language, PHP does not have a perfect function pointer concept, of course, to execute a function does not need pointers. There are two ways in which PHP supports callback functions Call_user_func,call_user_func_array.
However, I have made two examples, found that the above method, the implementation efficiency than the original method, a lot of difference

native:0.0097959041595459scall_user_func:0.028249025344849scall_user_func_array:0.046605110168457s

The example code is as follows:

$s = Microtime (true); for ($i =0; $i < 10000; + + $i) {    $a = new A ();    $data = $a->aaa ($array, $array, $array);    $data = A::BBB ($array, $array, $array);} $e = Microtime (true); echo "Native:". ($e-$s). " S\n "; $s = Microtime (true); for ($i =0; $i < 10000; + + $i) {    $a = new A ();    $data = Call_user_func (Array ($a, ' AAA '), $array, $array, $array);    $data = Call_user_func (Array (' A ', ' BBB '), $array, $array, $array);} $e = Microtime (true); echo "Call_user_func:". ($e-$s). " S\n "; $s = Microtime (true); for ($i =0; $i < 10000; + + $i) {    $a = new A ();    $data = Call_user_func_array (Array ($a, ' AAA '), Array (& $array,& $array,& $array));    $data = Call_user_func_array (Array (' A ', ' BBB '), Array (& $array,& $array,& $array));} $e = Microtime (true); echo "Call_user_func_array:". ($e-$s). " S\n ";

In PHP, knowing an object and a method is actually a simple way to invoke it, like the example above

$a = new A (); $data = $a->aaa ($array, $array, $array); $obj = $a; $func = ' aaa '; $params = Array ($array, $array, $array); $obj $func ($params [0], $params [1], $params [2]);//can be directly executed in this way

How does the performance of this approach, after our comparative test found

Native:0.0092940330505371scall_user_func:0.028635025024414scall_user_func_array:0.048038959503174smy_callback : 0.11308288574219s

In the addition of a large number of method strategy validation, the performance loss is relatively low, the time consumption is only 1.25 times times the native method, far less than the call_user_func of 3 times times more than Call_user_func_array, 5 times times more, the specific encapsulated code

Switch (count ($params)) {case                0: $result = $obj->{$func} ();                Case 1: $result = $obj->{$func} ($params [0]);                Case 2: $result = $obj->{$func} ($params [0], $params [1]);                Case 3: $result = $obj->{$func} ($params [0], $params [1], $params [2]);                Case 4: $result = $obj->{$func} ($params [0], $params [1], $params [2], $params [3]);                Case 5: $result = $obj->{$func} ($params [0], $params [1], $params [2], $params [3], $params [4]);                Case 6: $result = $obj->{$func} ($params [0], $params [1], $params [2], $params [3], $params [4], $params [5]);                Case 7: $result = $obj->{$func} ($params [0], $params [1], $params [2], $params [3], $params [4], $params [5], $params [6]); break;                Default: $result = Call_user_func_array (Array ($obj, $func), $params);  break;            }   

before using this method, consider using create_function to create anonymous functions, perform function callbacks, test create_function can only create global functions, cannot create class functions and object functions, and then discard.

Once you have completed the above preparation, you can use the callback mechanism and call the business code again

....//correlation variable Assignment $db = new wk_db (); $callback [' obj '] = $db;            $callback [' func '] = ' gettopfeeds ';            $callback [' params '] = Array (' site_id ' = $siteid, ' boutique ' + $seminal, ' offset ' = ' 0 ', ' rows ' = ' + ', ' uid ' =>null, ' tag_id ' = $tag _id, ' type ' and ' = $type ');            $top _feed_list = $cache->smart_fetch (' site_top_feeds ', $callback, ' feed ');

The above method is used to encapsulate the cache call, while ensuring performance is efficient, thus solving the first and fourth problems.

The first four questions have been completed to achieve the cache encapsulation and effectively avoid the second, third, and fourth issues mentioned above. However, for the fifth problem, dogpile problem, not resolved, the best way to solve this problem is that the cache is about to expire, there is a process to actively trigger the DB operation, get the DB data into the cache, Other processes normally fetch data from the cache (because the cache is not invalidated at this time); Fortunately there is a redis cache, we can use the two features of Redis to solve this problem, first introduce the next two interfaces
TTL method: Returns the remaining lifetime (TTL, Time to live) for a given key in seconds, and returns 2 when key does not exist. Returns 1 when key exists but no remaining lifetime is set. Otherwise, returns the remaining lifetime of the key, in seconds. Obviously, by this method, we can easily know the remaining life time of key, through this method, we could do something before the key expires, but this method does not work, we need to ensure that only the process execution, not all processes are done, just use the following method.
Setnx method: Set the value of key to value when and only if key does not exist. If the given key already exists, then SETNX does not do any action. Setnx is a shorthand for "set if not EXists" (set if it does not exist). Return value: Set successfully, return 1. Setting failed, returns 0. By this method, the distributed lock is simulated to ensure that only one process is executed, while the other processes are handled normally. Combining the features of the Redis method above, solve the fifth kind of problem, instance code.

...//variable initialization $key = "This is key"; $expiration = 600; $recalculate _at = n, $lock _length = $data = $cache->fetch ($key); $ttl = $cache->redis->ttl ($key); if ($recalculate _at>= $ttl && $r->setnx ("Lock:". $key, True) {$r->expire ("Lock:". $key, $lock _length) ; $db = new wk_db ();  $data = $db->getxxx ();  $cache->store ($key, $expiration, $value);}

Solution Solutions

OK, the key core code is as follows
1:function Callback Part code

public static function callback ($callback) {//Security check if (!isset ($callback [' obj ']) | |!isset ($callback [' func ']) || !isset ($callback [' params ']) | |        !is_array ($callback [' params ']) {throw new Exception ("Callback array Error");        }//Use reflection to determine if objects and functions exist $obj = $callback [' obj '];        $func = $callback [' func '];        $params = $callback [' params '];        method to Judge $method = new Reflectionmethod ($obj, $func);        if (! $method) {throw new Exception ("CallBack Obj not Find func"); }//Method property to determine if (! ( $method->ispublic () | |        $method->isstatic ())) {throw new Exception ("CallBack Obj func Error");        }//Number of parameters to judge (not to detect each item) $paramsNum = $method->getnumberofparameters ();        if ($paramsNum < count ($params)) {throw new Exception ("CallBack Obj params Error");   }//6 parameters, one call, more than 6, directly call Call_user_func_array $result = false;     Determine the static class method if (!is_object ($obj) && $method->isstatic ()) {switch (count ($params)) { Case 0: $result = $obj:: {$func} ();  Case 1: $result = $obj:: {$func} ($params [0]);                Case 2: $result = $obj:: {$func} ($params [0], $params [1]);                Case 3: $result = $obj:: {$func} ($params [0], $params [1], $params [2]);                Case 4: $result = $obj:: {$func} ($params [0], $params [1], $params [2], $params [3]);                Case 5: $result = $obj:: {$func} ($params [0], $params [1], $params [2], $params [3], $params [4]);                Case 6: $result = $obj:: {$func} ($params [0], $params [1], $params [2], $params [3], $params [4], $params [5]); Case 7: $result = $obj:: {$func} ($params [0], $params [1], $params [2], $params [3], $params [4], $params [5], $params [6]);                Break  Default: $result = Call_user_func_array (Array ($obj, $func), $params);            Break                }}else{switch (count ($params)) {case 0: $result = $obj->{$func} ();                Case 1: $result = $obj->{$func} ($params [0]); Case 2: $result = $obj-&Gt {$func}                ($params [0], $params [1]);                Case 3: $result = $obj->{$func} ($params [0], $params [1], $params [2]);                Case 4: $result = $obj->{$func} ($params [0], $params [1], $params [2], $params [3]);                Case 5: $result = $obj->{$func} ($params [0], $params [1], $params [2], $params [3], $params [4]);                Case 6: $result = $obj->{$func} ($params [0], $params [1], $params [2], $params [3], $params [4], $params [5]); Case 7: $result = $obj->{$func} ($params [0], $params [1], $params [2], $params [3], $params [4], $params [5], $params [6]);                Break  Default: $result = Call_user_func_array (Array ($obj, $func), $params);            Break }        }

2: Auto Trigger callback mechanism

Public Function Smart_fetch ($key, $callback, $namespace = "wk") {Key = $prefix. $key. $suffix;        $result = $this->_redis->get ($key);        $bttl = false;            TTL Status Determination (note cold start) if (!empty ($ttl)) {//get expiration $rttl = $this->_redis->ttl ($key);                if ($rttl > 0 && $ttl >= $rttl && $this->_redis->setnx ("lock". $key, True)) {                Set the timeout time (timeout 3 seconds) $this->_redis->expire ("lock". $key, 3);            $bttl = true; }        }How to return a value that does not exist, call the callback function, get the value, and keep the database if ($bttl | |! $result | | (Isset ($CONFIG [' Flush ']) &&!empty ($CONFIG [' Flush ']))            {//re-adjust parameter $callbackparams = array ();            foreach ($params as $k = + $value) {$callbackparams [] = $value;            } $callback [' params '] = $callbackparams;            $result = Wk_common::callback ($callback);            $expire = $key _config["Timeout"];            Storage data $status = $this->_redis->setex ($key, $expire, $result);        $result = $this->_redis->get ($key);        }//Remove lock if ($bttl) {$this->_redis->delete ("lock". $key);    } return $result; }

At this point, we use the scripting language feature, through the User_call_func_array method to complement all function callback mechanism, so as to achieve the cache encapsulation, through the configuration file definition of the rules to assemble key and the time-out of each key, By using the TTL and setnx features of Redis, only one process is guaranteed to perform DB operation, so the dogpile problem can be avoided, the cache is automatically triggered, the cache persists data, and the number of DB accesses is reduced and performance is improved.

A cache callback and auto-triggering technique based on PHP