Spring Data Redis makes NoSQL lightning fast (2)

"Editor's note" the author of this article is Xinyu Liu, and the first part of the article focuses on the features of Redis in all its aspects. In the second part, the detailed use cases are described. The article is a domestic ITOM management platform OneAPM compiled rendering.

Use Redis as a database

Now let's take a look at the various uses of Redis as a database in the server-side Java Enterprise Edition system. Regardless of the use case, Redis helps users optimize performance, processing power, and latency, and makes the conventional Java enterprise technology stack prohibitive.

1. Global Unique increment counter

Let's start with a relatively simple use case: an incremental counter that shows how many clicks a site has been subjected to. Spring Data Redis has two classes for this utility: RedisAtomicInteger and RedisAtomicLong . And unlike Java and the contract AtomicInteger AtomicLong , these Spring classes can play a role in multiple JVMs.

Listing 3: Globally unique delta counters

RedisAtomicLong counter =     new RedisAtomicLong("UNIQUE_COUNTER_NAME", redisTemplate.getConnectionFactory()); Long myCounter = counter.incrementAndGet();// return the incremented value

Note that the integer overflow and keep in mind that the operation on these two classes requires a relatively high cost.

2. Global Pessimistic lock

From time to time, users have to contend with server clusters. Suppose you run a scheduled job from a server cluster. In the absence of a global lock, nodes in the cluster initiate redundant job instances. Suppose a chat room partition can accommodate 50 people. If the chat room is full, you'll need to create a new chat room instance to accommodate another 50 people.

If a chat room is detected to be full but does not have a global lock, each node in the cluster creates its own instance of the chat room, bringing unpredictable factors to the entire system. Listing 4 describes how you should take full advantage of setnx (set if **n**ot e**x**ists: if not present) This Redis command to perform a global pessimistic lock.

Listing 4: Global pessimistic lock

public string Aquirepessimisticlockwithtimeout (string lockname, int acquiretimeout, int lockTimeout) {              if (Stringutils.isblank (lockname) | | lockTimeout <= 0) return null;        Final String lockkey = lockname;         String identifier = Uuid.randomuuid (). toString ();        Calendar atocal = Calendar.getinstance ();        Atocal.add (Calendar.second, acquiretimeout);                Date atotime = Atocal.gettime (); while (true) {//try to acquire the lock if (Redistemplate.execute (new Redisc Allback<boolean> () {@Override public Boolean doinredis (redisconnection con Nection) throws DataAccessException {return connection.setnx (rediste                Mplate.getstringserializer (). Serialize (Lockkey), Redistemplate.getstringserializer (). Serialize (identifier)); }})) {//successfully ACQuired the lock, set expiration of the lock Redistemplate.execute (new rediscallback<boolean> () {                            @Override public Boolean Doinredis (redisconnection connection)                                Throws DataAccessException {return Connection.expire (redistemplate                    . Getstringserializer (). Serialize (Lockkey), lockTimeout);                                }                });            return identifier;                } else {//fail to acquire the lock//set expiration of the ' lock in case ' is ' not ' set yet.                                if (null = = Redistemplate.execute (new rediscallback<long> () {@Override Public Long Doinredis (redisconnection connection) throws Da                       taaccessexception {               Return Connection.ttl (redistemplate. Getstringserializer (). Serialize (Lockkey                    )); }}) {//set expiration of the lock Redistemplate.execute (new Redis                                            Callback<boolean> () {@Override                                        Public Boolean Doinredis (redisconnection connection)                                Throws DataAccessException {return Connection.expire (redistemplate                        . Getstringserializer (). Serialize (Lockkey), lockTimeout); }                    });                                 } if (Acquiretimeout < 0)//no wait return null;                     else {try {   Thread.Sleep (100l);                Wait milliseconds before retry} catch (Interruptedexception ex) {}            } if (new Date (). After (Atotime));    }} return null; The public void Releasepessimisticlockwithtimeout (string lockname, string identifier) {if (Stringutils.isblan K (lockname) | |             Stringutils.isblank (identifier)) return;        Final String lockkey = lockname; Redistemplate.execute (New rediscallback<void> () {@Override Publ                                IC Void Doinredis (redisconnection connection) throws DataAccessException { byte[] ctn = Connection.get (redistemplate. Getstringserializer (). Serialize (Lockk                        EY)); if (Ctn!=null && identifier.equals (Redistemplate.getstringserializer (). Deserialize (ctn)) Connection.del (Redistemplate.getstringserializer (). Serialize (Lockkey));                    return null;    }                }); }

If you use a relational database, the lock may never be freed once the program that first generates the lock exits unexpectedly. Redis's EXPIRE settings Ensure that locks are released under any circumstances.

3. Bitmask (bit mask)

Suppose a Web client needs to poll a Web server to query customers for more than one table in a database to specify the update content. If you blindly query all the corresponding tables to find potential updates, the cost is higher. To avoid this, you can try to save an integer as a dirty metric for each client in Redis, and each digit of the integer represents a table. Set the number of digits when the required update for the customer exists in the table. Queries against tables are not triggered during polling, unless the corresponding digits are set. Redis is very efficient in terms of acquiring and setting such a bit mask STRING .

4. Leaderboard (leaderboard)

The ZSET data structure of Redis provides a concise solution to the leaderboards of gamers. ZSETworks somewhat like Java PriorityQueue , where each object is a sorted data structure and is organized in an orderly fashion. The player's position on the leaderboard can be drained by fractions. Redis ZSET defines a rich list of commands that support flexible and efficient queries. For example, Zrange (including Zrevrange) can return a specified range of features within an ordered set.

You can use this command to list the top 100 players in the leaderboard. Zrangebyscore returns features within a specified fraction range (for example, a player listed between 1000 and 2000), ZRNK returns the rank of the features within an ordered set, and so forth.

5. Bron (Bloom) filter

The Bron filter (Bloom filter) is a probabilistic data structure with high spatial utilization, which is used to test whether an element is a member of a set. False positive matches may occur, but they will not be reported as false positives. The query can return "may be in set" or "definitely not in set".

Bron filter data structures can be useful in terms of online services and offline services, including Big data analytics. Facebook uses the filter to prompt for input and extracts friends and friends from the queries entered by the user. Apache HBase uses a fabric filter to filter out hfile block disk reads that do not contain special rows or columns, resulting in significantly improved read speed. Bitly uses a fabric filter to avoid redirecting users to malicious websites, while Quara executes a segmented filter on the backend of the subscription to filter out previously viewed content. In my own project, I use the Blum filter to track user votes on various topics.

With excellent speed and processing power, Redis blends the Bron filter very well. Search GitHub to discover a number of Redis bron filter projects, some of which also support tunable accuracy.

6. Efficient Global Notifications: publish/Subscribe Channels

The Redis publish/Subscribe channel works like a fan-out messaging system, or a topic in JMS semantics. One difference between a JMS topic and a Redis publish/subscribe channel is that messages that are published through Redis are not persistent. After the message is pushed to all connected clients, the message is deleted on Redis. In other words, subscribers must remain online to receive new messages. Typical use cases for Redis Publish/subscribe channels include real-time configuration distributions, simple chat servers, and more.

In a Web server cluster, each node can be a subscriber to the Redis Publish/Subscribe channel. Messages posted to the channel are also pushed instantly to all connected nodes. The message can be a configuration change or a global notification for all online users. This push-to-talk mode is obviously highly efficient compared to constant polling.

Redis Performance Optimization

Redis is very powerful, but it can also be further optimized from the whole and based on specific programming scenarios. Here are some tips to consider.

Survival time

All REDIS data structures have a time-to-Live (TTL) attribute. When you set this property, the data structure is automatically deleted after it expires. Taking advantage of this functionality allows Redis to maintain a low memory loss.

Pipeline Technology

Sending multiple commands to Redis in a single request is called Pipeline technology. This technology saves the cost of network round trips, which is important because network latency can be several levels higher than the Redis latency. But there is a trap: the Redis command list in the pipeline must be predetermined and should be independent of each other. If the parameters of a command are calculated from the results of the previous command, the pipeline technology will not work. Listing 5 shows an example of Redis pipeline technology.

Listing 5: Piping technology

@Overridepublic List<LeaderboardEntry> fetchLeaderboard(String key, String... playerIds) {       final List<LeaderboardEntry> entries = new ArrayList<>();    redisTemplate.executePipelined(new RedisCallback<Object>() {    // enable Redis Pipeline            @Override         public Object doInRedis(RedisConnection connection) throws DataAccessException {             for(String playerId : playerIds) {                Long rank = connection.zRevRank(key.getBytes(), playerId.getBytes());                Double score = connection.zScore(key.getBytes(), playerId.getBytes());                LeaderboardEntry entry = new LeaderboardEntry(playerId,                 score!=null?score.intValue():-1, rank!=null?rank.intValue():-1);                entries.add(entry);            }                    return null;         }    });     return entries; }

Replica Sets and sharding

Redis supports master-slave replica configuration. As with MongoDB, the replica set is also asymmetric because the slave node is read-only to share the read effort. As I mentioned at the beginning of the article, you can also perform sharding to scale out the processing power and storage capacity of Redis. In fact, Redis is very powerful, and according to Amazon's internal benchmark, a EC2 instance of type R3.4xlarge can easily handle 100,000 requests per second. The legend also has 700,000 requests per second as the benchmark. For small-to-medium applications, Redis segmentation is usually not considered. (see this very good article, "Running Redis" to learn more about Redis performance optimization and segmentation.) ）

Transactions in Redis

Redis does not support full ACID transactions like a relational database management system, but its own transactions are also very effective. In essence, Redis transactions are a combination of pipelines, optimistic locks, and deterministic commits and rollbacks. The idea is to execute a list of commands in a pipeline and then observe potential updates (optimistic locks) for a critical record. Depending on whether the record being observed is being updated by another process, the list of commands or the whole determines the commit, or a complete rollback.

The following is an example of seller inventory on an auction site. When the buyer tries to purchase a product from the seller, you are responsible for observing the seller's inventory changes within the Redis transaction. At the same time, you want to remove this item from the same inventory. Before a transaction is closed, if inventory is touched by more than one process (for example, if two buyers purchase the same item at the same time), the transaction is rolled back, or the transaction determines the commit. You can start the retry after rolling back.

Transaction traps in Spring Data Redis

I RedisTemplate redisTemplate.setEnableTransactionSupport(true) learned a painful lesson when I enabled Redis transactions in the Spring class: Redis starts returning junk data after a few days of running, causing serious data corruption. Similar cases were reported on the StackOverflow.

After running a monitor command, my team found that RedisCallback Spring did not automatically close the Redis connection after the Redis operation or, in fact, it should be turned off. If you use a connection that is not closed again, you may return junk data from an unexpected Redis key. Interestingly, RedisTemplate This problem does not occur if the transaction support is set to false in.

We found that we could first configure one (for example) in the spring context and PlatformTransactionManager DataSourceTransactionManager then use @Transactional annotations to declare the scope of the Redis transaction so that Spring automatically shuts down the Redis connection.

Based on this experience, we believe that it is good practice to configure two separate in the Spring context RedisTemplate : One of the redistemplates transactions is set to False for most Redis operations, another redistemplates transaction is activated, only for Redis transactions. Of course you have to declare PlatformTransactionManager and @Transactional , in case you return garbage values.

In addition, we found the disadvantage of the combination of Redis transaction and relational database transactions (in this case, JDBC). The performance of mixed-type transactions is not the same as expected.

Conclusion

I would like to use this article to introduce the strengths of Redis to other Java enterprise developers, especially when using Redis as a remote data cache and for volatile data. Here I present six useful use cases for Redis, share some performance optimization techniques, and explain how my Glu Mobile team has solved the problem of garbage data caused by improper configuration of Spring data Redis transactions. I hope this article will inspire your curiosity about Redis NoSQL so that you can be inspired to create a world in your Java Enterprise Edition system.

This article is compiled and collated by OneAPM engineers. OneAPM can provide you with an end-to-end Java application Performance solution, and we support all common Java frameworks and application servers to quickly discover system bottlenecks and pinpoint the root cause of the anomalies. Minute-level deployment, instant experience, Java monitoring has never been easier. To read more technical articles, please visit the OneAPM Official technology blog.

This article was transferred from OneAPM official blog

Original address: http://www.javaworld.com/article/3062899/big-data/lightning-fast-nosql-with-spring-data-redis.html?page=2

Spring Data Redis makes NoSQL lightning fast (2)