Number of online users using Redis statistics

When building applications, we often need to record the user's every move, and one of the more important things is to record the users online.

This article describes four scenarios for using Redis to record online users, although they can count the number of online users, but each has its own unique set of operations, with different performance characteristics and resource consumption for each scenario.

Scenario 1: Using an Ordered collection

Whenever a user goes online, we execute the zadd command, adding the user and its online time to the specified ordered set:

Zadd "Online_users" <user_id> <current_timestamp>

By using the Zscore command to check whether a specified user ID has an associated score in an ordered set, we can know if the user is online:

Zscore "Online_users" <user_id>

By executing the zcard command, we can know that there are a total of multiple users online:

Zcard "Online_users"

The powerful thing about using an ordered collection to store online users is that it is a scenario in which all of the scenarios described in this article are able to perform the most aggregation operations, because the scheme can be aggregated by the members of an ordered collection (that is, the user's ID). You can also aggregate operations based on the score of an ordered collection (that is, the user's logon time).

First, with the Zinterstore and Zunionstore commands, we can aggregate a number of sequential collections that record online users:

# Calculate 7 days to have the user on-line, and store it in 7_days_both_online_users ordered collection Zinterstore 7_days_both_online_users 7 "Day_1_online_users" " Day_2_online_users "... "Day_7_online_users" # Calculates the total number of people on the line in 7 days Zunionstore 7_days_total_online_users 7 "day_1_online_users" ... "Day_7_online_users"

In addition, with the Zcount command, we can count how many users are online within the specified time period, while the Zrangebyscore command allows us to obtain a list of these users:

# count the number of users on-line in the specified time period zcount "Online_users" <start_timestamp> <end_timestamp># get a list of users who are online within a specified time period Zrangebyscore " Online_users "<start_timestamp> <end_timestamp> withscores

In this way, we can know the number of people on the site at different times and the list of online users, for example, we can use this method to learn the site in the morning, morning, noon, afternoon and night on-line number.

Scenario 2: Working with collections

As mentioned in the previous section, using an ordered set can simultaneously store the list of online users and the time of the individual users, but if we only want to record the online user's list and do not want to store the user's live time, then you can also use the collection instead of an ordered set to record online users.

In this case, whenever a user goes online, we execute the following SADD command to add it to the online user list:

Sadd "Online_users" <user_id>

By using the Sismember command, we can check whether a specified user is currently online:

Sismember "Online_users" <user_id>

And the work of counting the number of people online can be done by executing the SCard command:

SCard "Online_users"

By using set operations, we can aggregate the online user lists for different time periods or dates, just like an ordered set scheme. For example, with SINTER or Sinterstore commands, we can calculate the users who have been online for a week:

SINTER "Day_1_online_users" "day_2_online_users" ... "Day_7_online_users"

In addition, with the Sunion command or the Sunionstore command, we can calculate the total number of online users in a week:

Sunion "Day_1_online_users" "day_2_online_users" ... "Day_7_online_users"

By executing the Sdiff command or the Sdiffstore command, we can know which users are online today, but didn't go online yesterday:

Sdiff "Today_online_users" "Yesterday_online_users"

or the working day on the line, but the holiday is not online:

# Calculate the working day on-line list Sinterstore "Weekday_online_users" "Monday_online_users" "tuesday_online_users" ... "Friday_online_users" # Calculates the holiday line list Sinterstore "Holiday_online_users" "Saturday_online_users" "Sunday_online_users" # Calculates the list Sdiff "Weekday_online_users" "Holiday_online_users" on weekdays but not on-line

Such.

Scenario 3: Using Hyperloglog

Although the use of ordered collections and collections is a good way to record the number of people online work, but the above two scenarios have a clear disadvantage, that is, the memory consumed by these two scenarios will increase with the number of statistical users: if you have more users of the site, or you need to record more than one day/ Multiple times of online user lists and aggregated calculations, then these two scenarios can consume a lot of your memory.

On the other hand, in some cases, we only want to know the number of online users, without needing to know the specific online user list, then the ordered collection and collection of stored information will appear redundant.

In situations where we need to conserve as much memory as possible and only need to know the number of online users, we can use Hyperloglog to count online users: Hyperloglog is a probabilistic algorithm that estimates the cardinality of an element, and each hyperloglog consumes only This is the best option for a system with a very large number of users but very tight memory.

In this scenario, we use the Pfadd command to record online users:

Pfadd "Online_users" <user_id>

Use the Pfcount command to get online numbers:

Pfcount "Online_users"

Because Hyperloglog also provides the Pfmerge command to calculate the intersection, we can also use this command to calculate the total number of people on the line for a given time period or date:

# count The total number of people on the line within 7 days Pfmerge "7_days_both_online_users" "Day_1_online_users" "day_2_online_users" ... "Day_7_online_users" Pfcount "7_days_both_online_users"

Scenario 4: Using bitmaps (bitmap)

Looking back at the three scenarios described above, we can conclude that:

• Using an ordered set or collection can store a specific list of online users, but it consumes a lot of memory;
• While the use of hyperloglog can effectively reduce the amount of memory needed to count online users, it is not able to accurately record specific online user lists.

So is there a way to get a list of online users and to minimize memory consumption? This method does exist-you can do this using a Redis bitmap.

A Redis bitmap is an array of bits, and by matching each bits in the array with the user ID one by one, we can use bitmaps to record whether each user is online.

When a user is online, we use the Setbit command to set the user's corresponding bits to 1:

# The user_id here must be a number because it will be used as an index setbit "online_users" <user_id> 1

By using the Getbit command to check if the value of a BITS is 1, we can know whether the specified user is online:

Getbit "Online_users" <user_id>

With the Bitcount command, we can count how many bits in a bitmap are set to 1, or how many users are online:

Bitcount "Online_users"

As with collections, users can also aggregate multiple bitmaps-through the bitop command, the user may perform logical and logical, logical, or logical, or non-operational operations on one or more bitmaps:

# calculated 7 Days of online user bitop "and" "7_days_both_online_users" "Day_1_online_users" "day_2_online_users" ... "Day_7_online_users" # Calculates the total number of online users in 7 bitop "OR" "7_days_total_online_users" "Day_1_online_users" "Day_2_online_users" ... "Day_7_online_users" # calculates that only one day of the two-day online user bitop "XOR" "Only_one_day_online" "Day_1_online_users" "Day_2_online_users"

The Hyperloglog scheme records whether a user needs to spend 1 bits online, and for a website with 1 million users, this scenario only consumes up to a KB of memory, and for a site with 10 million users, it only takes 1.25 MB Memory.

While bitmap saving memory is less effective than hyperloglog, using bitmaps can accurately determine whether a user is on-line and can aggregate online user lists like collections and ordered collections. Using bitmaps is therefore the best choice for applications that want to save as much memory as possible but need to know exactly whether the user is online or need to aggregate computing for the user's online list.

Summarize

The following table summarizes the characteristics of the above four scenarios:

Programme	features
Ordered collection	The ability to simultaneously store a list of online users and the user's on-line time can perform a lot of aggregation calculations, but also consumes a lot of memory.
Collection	The ability to store online users ' lists, and to perform aggregation calculations, consumes less memory than ordered collections, but as with ordered collections, the memory consumed by this scheme increases with the number of users.
Hyperloglog	Regardless of the number of users that need to be counted, only a few KB of memory, but due to the characteristics of probabilistic algorithms, can only give an estimate of the number of people online, and can not obtain an accurate online user list.
Bitmap	Record the list of online users and be able to perform aggregation operations on those lists, in the case of saving as much memory as possible.

Because Redis supports a variety of data structures at the same time, a problem can often be found in redis with a variety of different solutions, and each solution has its own advantages and disadvantages, the problem described in this article is a good example.

On the way to statistics online users are introduced here, I hope these programs will bring you help and inspiration.

Number of online users using Redis statistics