How Redis scan is used and spring Redis pits

 

The springredistemplate is encapsulated for this scan, and the sample is used (for the latest library spring-data-redis-1.8.1.release):

Set<object>execute = redistemplate. Execute (new rediscallback<set<object>> () {@Override public set<object> Doinredis (redisconnection connection) throws DataAccessException { set<object> Binarykeys = new HashSet <> (); cursor<byte[]> cursor = Connection.scan ( new Scanoptions.scanoptionsbuilder (). Match ("test*"). Count ( ) . Build ()); While (Cursor.hasnext ()) {Binarykeys.add (new String (cursor).  Next ())); } return Binarykeys; }});

Note that the cursor must not be closed in previous versions, where the cursor needs to be closed manually, but from 1.8.0, it cannot be closed manually! Otherwise, an exception is reported.

The scanoptions has two parameters, one is match and the other is count, which corresponds to the two parameters of the Scan command, respectively.

Scan Command Source code:

/* Handle the caseof a hash table. */ht = NULL;if (o = = NULL) {Key Scan HT = c->db->dict; }Elseif (o->Type = = Redis_set && o->encoding = = redis_encoding_ht) {HT = o->ptr;}Elseif (o->Type = = Redis_hash && o->encoding = = redis_encoding_ht) {HT = o->ptr; count *=2; /* WeReturn Key/valueFor thisType. */ }Elseif (o->Type = = Redis_zset && o->encoding = = redis_encoding_skiplist) {Zset *zs = o->ptr; HT = zs->dict; count * =2; /* WeReturn Key/valueFor thisType. */ }Because of the ziplist of Redis, Intset and other types of data are very small, so it can be returned at once. The following else if do this thing. All returns a key.if (HT) {General storage, not Intset, ziplistvoid *privdata[2]; /* We Pass the PointersTo the Callback:the listTo which it'll * addNew elements,And the object containing the dictionary so, it is possibleTo fetch more dataIn atype-DependentThe. */privdata[0] = keys; privdata[1] = O;do {Scan, start with the cursor, and then call the callback function to set the data into the keys return dataset. cursor = Dictscan (HT, cursor, scancallback, privdata); }while (cursor && listlength (keys) < count); }Elseif (o->Type = = redis_set) {int pos =0; int64_t ll;while (Intsetget (O-&GT;PTR,POS++,&AMP;LL))Return all the data in this set, because it is a compressed intset, it will be very small. Listaddnodetail (Keys,createstringobjectfromlonglong (ll)); cursor =0;} else if (O-> type = = Redis_hash | | O->type = = Redis_zset) {//then it must be ziplist, the string representation of the data structure, not too large. unsigned char *p = ziplistindex (O->ptr,0); unsigned char *vstr; unsigned int vlen; long long Vll while (p) {//scans the entire key and returns all of this. and returning a cursor of 0 means nothing. In fact this is equal to not traverse ziplistget (P,&VSTR,&VLEN,&VLL); Listaddnodetail (keys, (vstr! = NULL)? Createstringobject ((char*) Vstr,vlen): Createstringobjectfromlonglong (VLL)); p = ziplistnext (o->ptr,p); } cursor = 0;} else {redispanic ( "not handled encoding in SCAN.");}   

As can be seen, the scan operation of Redis because of its overall data design, unable to provide a special scan operation, is only a "can ' t guarantee, just do my best" implementation:

• Provides the key space traversal operation, supports the cursor, the complexity O (1), the whole traverse once only needs O (N);
• provide result pattern matching;
• Supports the number of data bars returned at once, but is only a hints, and sometimes returns more;
• Weak state, all States require only the client to maintain a cursor;
• Unable to provide a complete snapshot traversal, that is, if there is data modification in the middle, there may be some changes involved in the data traversal;
• The number of data bars returned each time is not necessarily, extremely dependent on internal implementation;
• The returned data may be duplicated, and the application layer must be able to handle the reentrant logic; In the example code above, the Redistemplate.execute method is a set, which is equivalent to the return key
• Count is the number of keys per scan, not the number of result sets. Count depends on the size of the scanned data, although scan is unlocked, there is no guarantee that the search will be efficient at more than the million data volume level; count cannot be too small, network interaction becomes much, and count is as large as possible. Within search result set 10,000, it is recommended to set directly to the same size as collected

How Redis scan is used and spring Redis pits