Solr4.8.0 Source Code Analysis (21) of the solrcloud of the Recovery Strategy (ii)

• Solr4.8.0 Source Code Analysis (21) of the solrcloud of the Recovery Strategy (ii)

Preface: Former <solr4.8.0 source code analysis (20) Recovery strategy of Solrcloud (a) > mentioned recovery there are two strategies, one is Peersync and replication. This section describes the next Peersync policy.

Peeysync is SOLR's preferred strategy, and whenever a recovery is required, SOLR will always first determine if it is necessary to enter Peersync, Only when Peersync is set to skip or peersync is it found that it does not meet the criteria to enter replication. This is determined by the characteristics of Peeysync, Peeysync is a short interrupt time, need to recovery the number of document used when the strategy, so it recovery faster, less impact on SOLR. Replication is a long time to interrupt, the need to recovery the number of cases carried out, time-consuming.

The previous article has introduced the overall flow of recovery, so this article directly to introduce the Peersync process, see the following:

• First, SOLR sends a getversion request to all replica to get the latest nupdate version (100 by default).

1     //Fire off the requests before getting we own recent updates (For better concurrency)2     //This also allows us to avoid getting updates we don ' t need ... if we got our updates and then got their updates, they Would3     //newer stuff that we also had (assuming updates is going on and is being forwarded).4      for(String replica:replicas) {5 requestversions (replica);6     }7 8   Private voidrequestversions (String replica) {9Syncshardrequest sreq =Newsyncshardrequest ();TenSreq.purpose = 1; OneSreq.shards =NewString[]{replica}; ASreq.actualshards =sreq.shards; -Sreq.params =Newmodifiablesolrparams (); -Sreq.params.set ("Qt", "/get"); theSreq.params.set ("Distrib",false); -Sreq.params.set ("Getversions", nupdates); - shardhandler.submit (sreq, Replica, sreq.params); -}

• Gets the latest nupdate version of this shard (default is 100) and sorts the version.

 1  recentupdates = Ulog.getrecentupdates () ; 2  3  ourupdates = recentupdates.getversions (nupdates);  4 } finally         Recentupdates.close ();  6   7  8  Collections.sort (ourupdates, abscomparator); 

• Gets the version information startingversions before recovery. By comparing Startingversions with ourupdates, you can compare whether there is an index update during recovery.
• Check if Ourupdates and startingversions have intersection, because the number of version ourupdates and Startingversions is limited to nupdates, This is to determine whether the number of index updates is greater than nupdate. Enter replication if you need to update too many indexes, that is, ourupdates and startingversions do not intersect.

1       //Now make sure, the starting updates overlap our updates2       //there shouldn ' t is reorders, so any overlap would do.3 4       LongSmallestnewupdate = Math.Abs (Ourupdates.get (Ourupdates.size ()-1));5 6       if(Math.Abs (startingversions.get (0)) <smallestnewupdate) {7Log.warn (msg () + "Too many updates received since Start-startingupdates no longer overlaps with our currentupdates");8         return false;9}

• If Ourupdates and startingversions have intersections, merge two lists, which is the union of the two.

1       //Let ' s merge the lists2list<long> NewList =NewArraylist<>(ourupdates);3        for(Long ver:startingversions) {4         if(Math.Abs (ver) <smallestnewupdate) {5 Newlist.add (ver);6         }7       }8 9Ourupdates = NewList;

• The version of this shard is lower than the other shards and enters the replication policy. The comparison of the Shard version here does not press the maximum or minimum value of version, but rather compares the version at the 0.8 and 0.2 scales.

1     LongOtherhigh =percentile (otherversions,. 2f);2     LongOtherlow =percentile (otherversions,. 8f);3 4     if(Ourhighthreshold <Otherlow) {5       //Small overlap between version windows and ours is older6       //This means, we might miss updates if we attempted to the use of this method.7       //Since there exists just one replica that's so much newer, we must8       //fail the sync.9Log.info (msg () + "Our versions is too old. ourhighthreshold= "+ourhighthreshold +" otherlowthreshold= "+otherlow);Ten       return false; One}

• If the version of this slice is higher than the other shards, it does not require recovery to exit Peersync directly.

1     if(Ourlowthreshold >Otherhigh) {2       //Small overlap between windows and ours is newer.3       //Using This list to sync would result in requesting/replaying results we don ' t need4       //and possibly bringing deleted docs back to life.5Log.info (msg () + "Our versions is newer. ourlowthreshold= "+ourlowthreshold +" otherhigh= "+Otherhigh);6       return true;7}

• The version of this shard and the version of other shards are poor to obtain the missing version of this shard.

1      for(Long otherversion:otherversions) {2       //Stop when the entries get old enough so reorders may leads us to see updates we don ' t need3       if(!completelist && Math.Abs (otherversion) < Ourlowthreshold) Break;4 5       if(Ourupdateset.contains (otherversion) | |requestedupdateset.contains (otherversion)) {6         //We either has this update, or already requested it7         //Todo:what If the Shard we previously requested this from returns failure (because it goes8         //Down )9         Continue;Ten       } One  A Torequest.add (otherversion); - Requestedupdateset.add (otherversion); -}

• Finally, the GetUpdate command is sent to the other shards, and the corresponding document is obtained according to the processed version, thus completing the peersync process

1   Private BooleanRequestupdates (Shardresponse srsp, list<long>torequest) {2String replica = Srsp.getshardrequest (). shards[0];3 4Log.info (msg () + "requesting updates from" + Replica + "n=" + torequest.size () + "versions=" +torequest);5 6     //reuse Our original Request object7Shardrequest sreq =srsp.getshardrequest ();8 9Sreq.purpose = 0;TenSreq.params =Newmodifiablesolrparams (); OneSreq.params.set ("Qt", "/get"); ASreq.params.set ("Distrib",false); -Sreq.params.set ("Getupdates", Strutils.join (Torequest, ', ')); -Sreq.params.set ("Onlyifactive", onlyifactive); theSreq.responses.clear ();//needs to being zeroed for correct correlation to occur -  -Shardhandler.submit (Sreq, sreq.shards[0], sreq.params); -  +     return true; -}

Summarize:

This paper introduces the process of Peersync, this shows that the Peersync strategy recovery process is relatively simple, the next section will specifically introduce replication strategy, which is more Peersync complex.

Solr4.8.0 Source Code Analysis (21) of the solrcloud of the Recovery Strategy (ii)