Redis application practices

Redis introduction redis is a popular open-source key-value memory database. The official website is redis. io. Redis is widely used. In my opinion, many webapps are used. Project experience I used redis as cache and pubsub in the project, and realized some of its power. Here I will summarize it. New contact with re

Redis introduction redis is a popular open-source key-value memory database. The official website is http://redis.io /. Redis is widely used. In my opinion, many web apps are used. Project experience I used redis as cache and pubsub in the project, and realized some of its power. Here I will summarize it. New contact with re

Redis Introduction

Redis is a popular open-source key-value memory database. The official website is http://redis.io /.
Redis is widely used. In my opinion, many web apps are used.

Project Experience

I used redis in the project as the cache and pubsub function, and realized some of its power. Here is a summary.

The two commands we run when we first came into contact with redis are get/set, which is very simple. set key val stores a key-Value Pair in a database of a certain number in redis, get key can be used to obtain the value corresponding to this key. There is a big difference between the mysql and other relational databases used in peacetime. Setting the value and reading the value is too concise, which is also a feature of the memory database and must be fast, add, read, and delete values in a concise manner.

In my project, redis is used as a pubsub tool and also as a cache for writing files. The following describes how to use it.

Redis as the cache for writing files

I have about four thousand files to be written, and the source data is only 58 bytes. About 1000 such data records are generated in one second. That is to say, opening and Closing 1000 files in one second and writing data are both a pressure on the system and disk. This solution is very problematic and naturally comes to mind using the cache. After a certain amount of data is cached, the data is written into the file to reduce disk operations and improve the write efficiency.

If you maintain the cache yourself, you need to maintain the cache of 4000 different files, use the program to maintain 4000 caches, and trigger file write operations based on different cache sizes, I think it is a little difficult. At this time, I naturally thought of the redis memory database.
Redis has an operation room append, which appends content to a key. If the key does not exist, it creates one and returns the changed string length. This command is really useful, all at once solved my problem.
By maintaining 4000 keys in redis, append the received data to different keys. Based on the returned Data Length, when the expected length is reached, the entire process is clear and clear.

In the process of writing a program, some details need to be processed. For example, for each append operation, the client must interact with the redis server to obtain the submission result, is there a way to reduce this operation and change it to batch submission? Yes. Redis has a pipeline command to do this. The client submits data to the server through pipeline. The redis server does not immediately return the results, but stores the data in a queue, after the client executes the execute operation, the execution results are returned in the order of submission. Then compare the returned results to find the data that matches the length. This operation can greatly reduce the time consumption and improve the program running speed and performance.
When you find the data that meets the predefined length, you can use the getset (key, "") command of redis to obtain the value in the key and assign a new value to the key, this satisfies the atomic operability, and does not occur when a process obtains data for this key, before assigning new data, another process writes data to this key, causing data loss. After finding the data that needs to be written into the file, how can I notify the program to write into the file? There are two methods: synchronous and asynchronous. synchronization is to re-write the file in the same process/thread, and perform other operations after the file is written. The advantage may be that the program implementation is relatively simple, however, in a high-performance environment, it is not necessarily an acceptable method and time-consuming! What about Asynchronization? There are many asynchronous implementation methods, one is to start a new thread to do this, the other is to use message queue. In this scenario, it is not satisfactory to start a new thread to do this, and it is not stable enough. There is a celery message queue, and the website's own introduction is almost real-time distributed message queues.

When the program detects that the length is consistent, it retrieves the data and notifies a sub client in publish mode. After the subclient receives the message, we call celery tasks to execute write operations. In this way, the entire business logic flow is serialized. See the description.

Redis usage experience

Redis is easy to use and can also cause abuse. All tools have their own application scenarios. Many people can use a useful tool to do everything. I have made similar mistakes before. I think the c language is good. To write a program for downloading a Web page, I need to write it in c. If I change the result to python, it is just a few words. My life is too short, do not go with yourself and use appropriate tools to do the right thing.

Original article address: redis application practice, thanks to the original author for sharing.