Example of using Redis to count the number of online users

When building applications, we often need to keep track of what users do, and one of the more important things to do is to record users online.

This article describes four scenarios that use Redis to record online users, which can count the number of online users, but each has its own unique operations, and the performance characteristics and resource consumption of each scenario vary.

.. /.. /_images/online_users.png
Scenario 1: Using ordered collections
Whenever a user comes online, we execute the zadd command, adding this user and its online time to the specified ordered set:

Zadd "Online_users" <user_id> <current_timestamp>
By using the Zscore command to check whether a specified user ID has an associated score in an ordered set, we can know whether the user is online:

Zscore "Online_users" <user_id>
And by executing the zcard command, we can know that there are a total number of users online:

Zcard "Online_users"
The powerful thing about storing online users in an ordered set is that it is a scenario in which the most aggregated operations can be performed in all the scenarios described in this article, because this scenario can be aggregated through the members of an ordered set (also the ID of the user). You can also aggregate operations based on the score of an ordered set, which is the user's logon time.

First, through the Zinterstore and Zunionstore commands, we can aggregate computations for multiple sequential collections that record online users:

# Calculate 7 days with online users and store it in an ordered set of 7_days_both_online_users
Zinterstore 7_days_both_online_users 7 "day_1_online_users" "day_2_online_users" ... "Day_7_online_users"

# figure out how many people are online in 7 days
Zunionstore 7_days_total_online_users 7 "day_1_online_users" ... "Day_7_online_users"
In addition, through the Zcount command, we can count how many users are online within a specified time period, and the Zrangebyscore command allows us to obtain a list of these users:

# count the number of users on the line within a specified time period
Zcount "Online_users" <start_timestamp> <end_timestamp>

# Get the list of users on the line within a specified time period
Zrangebyscore "Online_users" <start_timestamp> <end_timestamp> withscores
With this approach, we can know the number of people on the Web site at different time periods and the list of online users, for example, we can use this method to learn the site in the morning, morning, noon, afternoon and night on the number of online.

Scenario 2: Using Collections
As mentioned in the previous section, using an ordered set can store both the list of online users and the time of each user, but if we only want to record the list of online users without having to store the user's on-line time, the collection can be used instead of an ordered set to record the users online.

In this case, whenever a user comes online, we execute the following SADD command to add it to the online user list:

Sadd "Online_users" <user_id>
By using the Sismember command, we can check whether a specified user is currently online:

Sismember "Online_users" <user_id>
The work of counting online numbers can be accomplished by executing the scard command:

SCard "Online_users"
With the set operation, we can compute the online user list for different time periods or dates, like an ordered set scheme. For example, by using the sinter or Sinterstore command, we can calculate a user who has been online for a week:

Sinter "Day_1_online_users" "day_2_online_users" ... "Day_7_online_users"
In addition, through the sunion command or the Sunionstore command, we can calculate the total number of online users in a week:

Sunion "Day_1_online_users" "day_2_online_users" ... "Day_7_online_users"
And by executing the Sdiff command or the Sdiffstore command, we can know which users are online today, but there was no online yesterday:

Sdiff "Today_online_users" "Yesterday_online_users"
or weekday online, but the holiday is not online:

# Calculate Weekday Online list
Sinterstore "Weekday_online_users" "Monday_online_users" "tuesday_online_users" ... "Friday_online_users"
# Calculate the holiday online list
Sinterstore "Holiday_online_users" "Saturday_online_users" "Sunday_online_users"
# Calculate weekday online but not on holiday list
Sdiff "Weekday_online_users" "Holiday_online_users"
Such.

Scenario 3: Using Hyperloglog
Although using ordered collections and collections can do a good job of recording online numbers, but all two of these programs have one obvious drawback, that is, these two programs will consume more memory with the number of users increased: If you have more users of the site, or you need to record days/ A list of online users with multiple periods of time and aggregation calculations, then these two scenarios can consume a lot of memory.

On the other hand, in some cases, we only want to know the number of online users, without the need to know the specific online user list, then ordered collection and collection of information will appear redundant.

In situations where you need to save as much memory as possible and only need to know the number of online users, we can use Hyperloglog to count online users: Hyperloglog is a probabilistic algorithm that can estimate the cardinality of an element, and each hyperloglog only costs KB memory, which is the best option for a system with a very large number of users, but with very tight memory.

Under this scenario, we use the Pfadd command to record online users:

Pfadd "Online_users" <user_id>
Use the Pfcount command to get online numbers:

Pfcount "Online_users"
Because Hyperloglog also provides a pfmerge command to compute the intersection, we can also use this command to calculate the total number of people on the line for a given time period or date:

# How many people have been online within 7 days of counting?
Pfmerge "7_days_both_online_users" "Day_1_online_users" "day_2_online_users" ... "Day_7_online_users"
Pfcount "7_days_both_online_users"
Scenario 4: Using bitmaps (bitmap)
Looking back at the three scenarios presented above, we can draw the conclusion that:

Using an ordered set or set can store a specific list of online users, but consumes a lot of memory;
While using Hyperloglog can effectively reduce the amount of memory needed to count online users, it has no way of accurately recording specific online user lists.
So is there a way to get a list of online users and try to reduce memory consumption? This approach does exist-it can be done using a Redis bitmap.

The Redis bitmap is an array of bits, and by one by one correspondence of each bits in the array with the user ID, we can use a bitmap to record whether each user is online.

When a user comes online, we use the Setbit command to set this user's corresponding bits to 1:

# The user_id here must be a number because it will be used as an index
Setbit "Online_users" <user_id> 1
By using the Getbit command to check whether a bits value is 1, we can know whether the specified user is online:

Getbit "Online_users" <user_id>
With the Bitcount command, we can count how many of the bits in the bitmap are set to 1, or how many users are online:

Bitcount "Online_users"
As with collections, users can also aggregate multiple bitmaps-through the bitop command, a user can perform logical, logical, or illogical or illogical operations on one or more bitmaps:

# Calculate 7 Days of users online
Bitop "and" "7_days_both_online_users" "Day_1_online_users" "day_2_online_users" ... "Day_7_online_users"

# figure out the total number of online users in 7
Bitop "OR" "7_days_total_online_users" "Day_1_online_users" "day_2_online_users" ... "Day_7_online_users"

# figure out only one day of the two day users online
Bitop "XOR" "Only_one_day_online" "Day_1_online_users" "Day_2_online_users"
The Hyperloglog scheme records whether a user needs to spend 1 bits online, and for sites with 1 million users, it only takes a few kilobytes of memory to use it, and for sites with 10 million users, it only takes 1.25 MB Memory.

While the bitmap saves memory less than Hyperloglog, the use of bitmaps can accurately determine whether a user is online, and can aggregate the on-line user list like a collection and an ordered set. So for applications that want to save as much memory as possible, but need to know exactly whether the user is online or need to aggregate the user's online list, using bitmaps is the best choice.

Summarize
The following table summarizes the features of the above four scenarios:

Scenarios   Features
an ordered set   can simultaneously store lists of online users as well as users ' on-line time, can perform a lot of aggregation calculation operations, but consumes a lot of memory.
Collections   The ability to store lists of online users, and to perform aggregation calculations that consume less memory than an ordered set, but like an ordered set, the memory consumed by this scenario increases as the number of users increases.
hyperloglog  can only consume KB of memory regardless of the number of users who need to count it, but because of the nature of the probabilistic algorithm, only estimates of the number of online people are given, and the exact list of online users is not available.
Bitmap   To record the list of online users, and to perform aggregation operations on these lists, as much as possible to conserve memory.
Because Redis supports multiple data structures at the same time, a problem can often be found in Redis to find many different solutions, and each solution has its own advantages and disadvantages, this article describes the problem is a good example.