Introduction to the method of SQL scaling using Redis _redis

Ease of line Competition

We used the sentry.buffers in the early morning of sentry development. This is a simple system that allows us to implement a very efficient buffer counter with a simple last Write wins policy. It is important that we use it to completely eliminate any form of durability (which is a very acceptable way of sentry work).

The operation is very simple, whenever an update comes in We do the following steps:

• Create a hashing (hash key) that is bound to the incoming entity
• Use Hincrby to increase the counter value
• Hset all LWW data (such as "last seen")
• Zadd hashing (hash key) to a "pending" set with the current timestamp

Now each timescale (10 seconds in Sentry) we want to dump (dump) These buffers and fan out write (Fanout the writes). Looks like the following:

• Use Zrange to get all key
• Initiate a job for each pending key to RABBITMQ

Now the RABBITMQ job will be able to read and clear the hash table, and the "Pending" update has popped up a set. There are a few things to note:

• We'll use a set of sorts (for example, we need those 100 old sets) in the example below that we want to just pop up a set of numbers.
• If we end a multiple-sorted job in order to process a key value, this person will get the process of no-oped due to another already existing processing and emptying of the hash.
• The system can be continuously extended on many Redis nodes simply by placing a ' mount ' primary key on each node.

After we have this model of dealing with the problem, we can ensure that "most of the time" only one row in the SQL can be updated immediately, and this approach alleviates the lock problem that we can see in advance. This strategy is useful for sentry, considering that it will handle a scene that suddenly produces and all the data that is eventually grouped together into the same counter.

Speed limit

For the Sentinel's limitations, we must end the ongoing denial of service attack. We deal with this problem by limiting the speed of the connection, one of which is supported by Redis. This is undoubtedly a more direct implementation within the scope of the Sentry.quotas.

Its logic is fairly straightforward, as shown below:

def incr_and_check_limit (user_id, limit):
  key = ' {User_id}:{epoch} '. Format (user_id, int (time)/)
 
  pipe = Redis.pipeline ()
  pipe.incr (key)
  Pipe.expire
  current_rate, _ = Pipe.execute () return
 
  int ( Current_rate) > Limit

The rate-limiting approach we have articulated is one of the most basic features of Redis in caching services: adding empty keys. Implementing the same behavior in a caching service may end up using this method:

def incr_and_check_limit_memcache (user_id, limit):
  key = ' {User_id}:{epoch} '. Format (user_id, int (time ()/60))
 
  if Cache.Add (key, 0,): Return
    False
 
  current_rate = CACHE.INCR (key) return
 
  current_rate > limit

In fact, we finally adopt this strategy to enable Sentinels to track short-term data on different events. In this case, we usually sort the user data so that the most active user's data can be found in the shortest amount of time.

Basic lock

Although Redis is not highly available, our use case locks make it a good tool for work. We're not using these at the Sentinel's core anymore, but a sample use case is that we want to minimize concurrency and simple operations if things seem to be already running. This is useful for a cron-like task that may need to be performed every once in a while without strong coordination.
This use of SETNX operations in Redis is quite simple:

From contextlib Import contextmanagerr = Redis () @contextmanagerdef Lock (Key, nowait=true):
  While not r.setnx (key, ' 1 '):
    if nowait:
      raise Locked (' Try again soon! ')
    Sleep (0.01)
 
  # limit lock seconds
  r.expire (key, ten) # do something crazy yield
 
  # explicitly Unlock
  r.delete (key)

While the lock () inside the Sentinels utilizes the memcached, there is absolutely no reason why we cannot switch to Redis.

Time Series data

Recently we have created a new mechanism for storing time series data in Sentry (contained in SENTRY.TSDB). This is inspired by the RRD model, especially the graphite. We expect a quick and easy way to store short-term (for example, one months) of time series to facilitate processing of high speed write data, especially in extreme cases where the potential short-term rate is calculated. Although this is the first model, we still expect to store data in Redis, which is also a simple example of using counters.

In the current model, we use a single hash map to store all the time series data. For example, this means that all data items will have a data type and a 1-second lifecycle for the same hashing. As shown below:

Copy Code code as follows:

{
"<type enum>:<epoch>:<shard number>": {
"<id>": <count>
}}

So in this situation, we need to track the number of events. The event type is mapped to the enumeration type "1". The time of judgment is 1s, so our processing time needs to be in seconds. The hash ultimately looks like this:

Copy Code code as follows:

{
"1:1399958363:0": {
"1": 53,
"2": 72,
}}

A modifiable model may only use simple keys and add some incremental registers to the store only.

Copy Code code as follows:

"1:1399958363:0:1": 53

We chose the hash map model for the following two reasons:

1. We can set all the keys to be disposable (this can also have a negative effect, but so far it is stable)
2. Greatly compress the key value, which is quite important to deal with

In addition, the discrete number keys allow us to map the virtual discrete key values to a fixed number of key values and allocate a single storage area here (we can use 64 to map to 32 physical nodes)

The data query is now complete by using Nydus and its map () (dependent on a workspace). The code for this operation is pretty robust, but luckily it's not huge.

def get_range (self, model, keys, start, End, Rollup=none): "" "to get a range of data for group Id=[1, 2, 3]: Start a  nd end are both inclusive. >>> now = Timezone.now () >>> Get_keys (Tsdb.models.group, [1, 2, 3], >>> start=now-timed Elta (Days=1), >>> end=now) "" "Normalize_to_epoch = Self.normalize_to_epoch Normalize_to_rollup = self.
 
  Normalize_to_rollup Make_key = Self.make_key if rollup is None:rollup = Self.get_optimal_rollup (start, end) results = [] timestamp = End With Self.conn.map () as conn:while timestamp >= Start:real_epoch = Normal
        Ize_to_epoch (timestamp, rollup) Norm_epoch = Normalize_to_rollup (timestamp, rollup) for key in keys: Model_key = Self.get_model_key (key) Hash_key = Make_key (model, Norm_epoch, Model_key) results.append (rea L_epoch, Key, Conn.hget (Hash_key, model_key)) timestamp = Timestamp-timedelta (seconds=rollup) results_bY_key = Defaultdict (dict) for epoch, key, count on Results:results_by_key[key][epoch] = Int (count or 0) for key
 , points in Results_by_key.iteritems (): Results_by_key[key] = sorted (Points.items ()) return Dict (Results_by_key)

Boils down to the following:

• The key required to build.
• Using the workspace, extract the minimum result set for all connection operations (Nydus is responsible for these).
• Gives the results and maps them to the current store based on the specified time interval and the given key values.

A simple choice

I am a person who likes to solve problems with simple solutions, and it is certainly suitable to use redis in this field. Its documentation is as surprising as it is because the threshold of its documentation is very low. Although he also has trade-offs (mostly if you use persistence), they work well and are intuitive.

So what Redis solve for you?