Redis sorted sets operation learning notes

This article introduces the learning notes for redis sorted sets. For more information, see.

Sorted set is an upgraded version of set. It adds an ordered Attribute Based on set.

You can specify an element when modifying it. After each element is specified, zset automatically re-adjusts the order based on the new value. It can be understood that

Two columns of mysql table, one column of value, and one column of storage order. In the operation, the key is interpreted as the zset name.

Like set, sorted set is also a set of string elements. The difference is that each element is associated with a double

Type score. The implementation of sorted set is a mixture of skip list and hash table.

When an element is added to a set, the score ing from an element to score is added to the hash table.

The overhead of the element to obtain the score is O (1). the ing from the other score to the element is added to the skip list and ranked according to the score

So we can get the elements in the set in order. The overhead of adding and deleting operations are O (log (N) and skip list

The overhead is consistent. redis uses a two-way linked list to implement the skip list, so that elements can be retrieved from the tail in reverse order. Sorted set

We often use this as an index. We can store the fields to be sorted as scores and the Object id.

When the element is stored.

The following is an example of using Sorted Sets.

Mysql has a table named summary_data. The number of records is about 30 mb,
There is a field first_path that is varchar (256). You need to find the top 10 first_paths with the maximum number of occurrences.

Method 1) Direct SQL statement

SQL statements are easy to write:

The Code is as follows:	Copy code
SELECT first_path, COUNT (*) AS c FROM summary_data group by first_path order by c desc limit 10;

The table is indexed, but the index length is KEY 'first _ path' (255). This may be the reason why indexes cannot be used:
Id: 1
Select_type: SIMPLE
Table: summary_data
Type: ALL
Possible_keys: NULL
Key: NULL
Key_len: NULL
Ref: NULL
Rows: 28136948
Extra: Using temporary; Using filesort

This SQL statement runs for 9 minutes.

Export the first_path file and generate the input/first_path file. Each line contains a record, which means the export process is really slow.

Method 2) sort and uniq

Sort input/first_path | uniq-c | sort-nr | head-n 10
Sort the order status, that is, the group status.
Uniq-c is used to count the quantity of each group.
Sort-nr and then sort the statistical results in descending order
One minute and a half.

Method 3) redis sorted set

In this way, it is only because of the sudden thought of sorted set.

Client libary is ready to use hiredis.

Easy to install: make & make install
You can see that the library file is installed in the/usr/local/lib/directory.
The header file is installed in the/usr/local/include/hiredis/directory.

Add/usr/local/lib/to/etc/ld. so. conf.
Then ldconfig

Compile:

The Code is as follows:

Copy code

Gcc-I/usr/local/include/hiredis-lhiredis./example. c # Include <stdio. h> # Include <stdlib. h> # Include <string. h>   # Include   Int main (int argc, char ** argv ){ Unsigned int j; RedisContext * c; RedisReply * reply; Const char * hostname = (argc> 1 )? Argv [1]: "127.0.0.1 "; Int port = (argc> 2 )? Atoi (argv [2]): 6379;   Struct timeval timeout = {1, 500000}; // 1.5 seconds C = redisConnectWithTimeout (hostname, port, timeout ); If (c = NULL | c-> err ){ If (c ){ Printf ("Connection error: % sn", c-> errstr ); RedisFree (c ); } Else { Printf ("Connection error: can't allocate redis contextn "); } Exit (1 ); }   FILE * fp;     Fp = fopen (argv [3], "r "); If (! Fp) exit (1 );   Char line [256];   While (fgets (line, sizeof (line)-1, fp )! = NULL ){ Reply = redisCommand (c, "ZINCRBY myzset_firstpath 1% s", line ); FreeReplyObject (reply ); }   Fclose (fp );   Reply = redisCommand (c, "ZREVRANGEBYSCORE myzset_firstpath + inf-inf withscores limit 0 10 ");   If (reply-> type = REDIS_REPLY_ARRAY ){ For (j = 0; j <reply-> elements; j + = 2 ){ Printf ("% u) % s % sn", (unsigned int) (j/2), reply-> element [j]-> str, reply-> element [j + 1]-> str ); } }     FreeReplyObject (reply ); /* Disconnects and frees the context */ RedisFree (c );   Return 0; }

The result is returned in 16 minutes, not good enough.