Some ideas and suggestions for using Redis based on Python

Directory

1 a Little idea of Redis usage

Before you 1.1 cache, consider

After the 1.2 is cached, you need to consider

1.3 Cache is used for a period of time

2 writing Redis Database layer specification recommendations

2.1 Choosing the right redis client

2.2 Normalization defines The name and initialization of key

2.3 Choosing the right data structure

2.4 Normalization Definition action method

2.5 began a pleasant journey of calling

1 a Little idea of Redis usage Before you 1.1 cache, consider

(1) This data is a short-term reservation, for example, only three days, seven days or one months, the proposed use of cache;

(2) The data in a certain time period of a large number of requests, the proposed use of caching;

(3) With the user, the data is constantly changing, the update operation is more frequent, at this time the proposed use of cache;

(4) If the amount of data is small, and the performance of the application is not improved, the data needs to be retained for a long time, it is not recommended to use Redis cache, directly using the relational database such as MySQL storage;

(5) If the amount of data is large, but after a period of time, the data is of little value, it is recommended to adopt a cache, and set the expiration time and scheduled cleanup of the data script, so that processing can reduce storage space, but also facilitate the optimization of the system's database layer.

after the 1.2 is cached, you need to consider

(1) Cache the key in extreme cases, how large is the system memory consumption? Or what is the order of magnitude of the records that are stored?

(2) If the key is cached, is it convenient to set the expiration time? If it is not convenient to set, can be considered in the interval to dump into a relational database such as MySQL, so as to clean up the cache, free memory space.

(3) To cache the key, think about how to manually delete the cached data script how to write, cache all the data, how to distinguish valuable data to keep, worthless convenient script for automated deletion.

(4) After using the cache, consider selecting the appropriate data structure to complete the code building. Because a suitable data structure not only makes the code more elegant, later maintenance is also very convenient.

1.3 cache is used for a period of time

If a key is found to occupy a large amount of memory, exceeding expectations, the optimization recommendations are provided:

(1) Analyze the actual function of the key, and the current needs, to see whether the subsequent cache data, add the expiration time setting;

(2) Consider when the cache data is removed, can be transferred to MySQL and other relational database, if the successful transfer can be done here immediately delete the cache data method, the subsequent fetch data, you can use the first query Redis database, did not find the MySQL and other relational database ；

(3) According to the data already cached, it is possible to filter the cached data according to the field or related attributes in the data, and delete the unimportant data by the script.

2 writing Redis Database layer specification recommendations 2.1 Choosing the right redis client

For example, the following two clients are defined:

#-*-coding:utf-8-*-" fromDjango.confImportSettingsImportredisredis_db_client=Redis. Strictredis (Host=settings. redis['Redis_db_host'], Port=settings. redis['Redis_db_port'], DB=settings. redis['redis_db_db'], Socket_connect_timeout=4, Socket_timeout=2, Decode_responses=True,) redis_hot_client=Redis. Strictredis (Host=settings. redis['Redis_hot_host'], Port=settings. redis['Redis_hot_port'], DB=settings. redis['redis_hot_db'], Socket_connect_timeout=2, Socket_timeout=2, Decode_responses=False,)

At this time, according to the design and function of key, select the appropriate client to operate. The client-side definitions above are for reference only, as different clients have different ports and specific databases. (PS: The above definition is used based on the Django framework configuration file, other Python frameworks can be defined similarly)

2.2 Normalization defines The name and initialization of key

The name of the key that defines the Redis operation is proposed in uppercase letters and underscores, and it is best to set the expiration time when initializing the key object.

Examples of classes defined by key:

class Rediskey:     """ Rediskey Class Object """    def __init__ (self, prefix, ex=None):         = prefix        = ex    #  generates key value    def__call__  (self, key):        return Self.prefix + str (key)

Then define an example of a key:

User_pull_url = Rediskey (prefix='user_pull_url:', ex=8 * 60 * 60)

Where the parameter prefix is the specific key name stored in the Redis data by the key, you can add a suffix to the key by calling the __call___ method, such as user_pull_url:1808 for a key name, and ex for the expiration time defined by the key object. When you specifically write the key to add the operation, call the parameter ex to set the specific expiration time of the key.

The file in which key is stored is stored in the specified file according to the selected client, so it is easy to view and manage.

2.3 Choosing the right data structure

The Redis database consists of string (string), hash (hash table), List (list), set (set), SortedSet (ordered collection) five data structures. Here's a brief look at the features of these five data structures:

• String (String): stored in key-value format when data is added. Key is the defined key name, as described in (2) above. Value is the specific data to be stored, and the type of the data is of type string. In the requirements, such as the need to store a user's online time, you can use the key_user_id composition key name, the length of time is stored in value, at this time can choose a string data structure to store, more convenient.
• Hash (hash table): When the data is added, it is stored in the Key-value format, but the value here represents a hash table. As a metaphor, key is compared to a table name in a relational database, and value stores data records for all rows in that relational table.
• List: When adding data, it is stored in the Key-value format, but the value here represents a specific list. The list functions like a C + + data structure list, has a table operation, calculates the list length operation, gets the function of an element according to the subscript, gets the list element within the specified interval, inserts an element from the head of the list, inserts an element in the tail, etc.
• Set (collection): When adding data, it is stored in key-value format, where the value store and hash (hash table) storage type, and set (set) difference is that when adding data records, the same worth of records will be automatically filtered out, if you insert multiple records the same data, in the set Only one record is found in the (collection) stored value.
• SortedSet (Ordered collection): When adding data, the Key-value format is used for storage. The way the storage is implemented is basically the same as set (set), but the only salient feature is that the score value corresponding to the element value is stored when the data is saved, that is, the element in value is sorted by the size of the score value. When you follow a query, you can easily return the ordering sequence of the elements in value.

2.4 Normalization definition action method

For each service tier, it is recommended to create a separate cache.py file that is specifically designed to hold the Redis database layer method, which can be analogous to models.py files.

Each key uses its key name to create a class name that conforms to the code specification, and then within the class of that key, define the redis_client and redis_key of the operation, and finally define the methods for adding, modifying, querying, and deleting related data through Cliet and key.

Finally, one of the most important suggestions: The method of action defined in the cache.py is recommended to be called only by the methods body in the other class in the service layer. This benefit allows us to have a controllable estimate of the key for future data management, and also makes code invocation more canonical.

An example is given here:

Define a key:

Key Name initialization:

User_session = Rediskey (prefix='user_session:', ex=4 * 60 * 60)

Select client:

Redis_hot_client

Classes and related operation methods defined in cache.py:

classCacheusersession:#the name of the class name and key corresponds    """raw data type: Dict storage data type: Bytes Data description: Serialize user_dict into binary data, in Redis"""DB=redis_hot_client Key_prefix=Rediskey. User_session @classmethoddefget (CLS, user_id:idint): Key= Cls.key_prefix (user_id)#the specific initialization value of keyuser_dict=cls.db.get (Key)ifuser_dict:user_dict=pickle.loads (user_dict)returnuser_dict @classmethoddefDelete (CLS, user_id:idint):#Delete method definition for keyKey=Cls.key_prefix (user_id) cls.db.delete (key) Logger.info ("Delete usersession:{}". Format (user_id)) @classmethoddefset (CLS, User_id:idint, user_dict:dict): Key=Cls.key_prefix (user_id) data=pickle.dumps (user_dict) Cls.db.set (key, data, ex=CLS.KEY_PREFIX.EX)#Note Set expiration timeLogger.info ("Set usersession:{}, {}". Format (user_id, user_dict))

Here, a good way to store data is given. In the following ways, a dictionary format data can be stored in the Redis database as a string, and the dictionary format can be re-parsed after removal. In this way, Redis can be stored as a relational database, and a data record can store the values of multiple properties.

For example, a student, including the number, name, gender, grade, professional and other attributes. If a relational database is used, the number is used as the primary key, the other attributes are designed as column names, and all the students ' information can be queried by the student number. But how can a student's multiple attribute information be stored in a Redis database and taken out for good use?

Adopted strategy: The number of unique, can be used as a key value, name, gender, grade, professional union into a dictionary, in the Redis data before the conversion to the specified structure of the string format, removed and then parsed into a dictionary format. This will be a good solution to this requirement.

Why convert to a string format into a Redis database? Because no matter what data structure is stored in the Redis database, it is fetched in string format.

Here's a look at the specific code example (the Cachestudent class implementation code in cache.py does not give OH):

ImportJSON
Value_data=json.dumps ({name: "Xiaoming", Gender: "Male", Grade: "2014 ", Profession: "Software Engineering",}) #converts a dictionary to a string of a specified format through a JSON moduleCachestudent.set (student_id, Value_data)#The specified student data is deposited by means of a well-defined writing methodStudent_data= Json.loads (Cachestudent.get (student_id))#after removing the data from the specified student number, parse the string into a dictionary format using the loads method of the JSON modulePrint(studnet_data["name"])#Print the student's name informationPrint(studnet_data["gender"])Print(studnet_data["Grade"])Print(studnet_data["profession"])

Seeing the above implementation, is it possible to discover that Redis storage can be used as a relational data store?

2.5 began a pleasant journey of calling

Called here, it is appropriate to choose when the data delete operation. For example, after taking out, you can choose to dump the final stored data into the MySQL database, and then call the Delete method in cache.py to delete the data record.

Some ideas and suggestions for using Redis based on Python