9 mistakes in Using Cache

9 mistakes in Using Cache

If you want to optimize a site or application frequently, you can say that the use of cache is the fastest and most effective way. In general, we cache some frequently-used data that requires a large amount of resources or time to make subsequent use faster.

If you really want to elaborate on the benefits of caching, there are still a lot of them, but in actual applications, it is always so unsatisfactory when using caching. In other words, if cache is used, the performance can be improved to 100 (the number here is just a metering symbol, just to give you a "amount" experience). However, many times, the improved performance is only 80, 70, or less, and may even cause serious performance degradation. This phenomenon is particularly prominent when distributed cache is used.

In this article, we will introduce the nine major problems that have caused the above problems and provide corresponding solutions. The article uses. NET as an example to demonstrate Code. It is also of reference value for friends who come and other technical platforms. just replace the corresponding code!

To make the subsequent description more convenient and more complete, Let's first look at the two cache formats: local memory cache and distributed cache.

First, for the local memory cache, the data is cached in the local memory, as shown in 1:

We can clearly see that:

• The application caches data in the local memory and directly obtains the data from the local memory when necessary.
• For. NET Applications, when obtaining data in the cache, they use object references to search for Data Objects in the memory. That is to say, if we get the data object through reference, we directly modify this object. In fact, we are actually modifying the cache object in the memory.

For distributed cache, because the cached data is stored on the cache server, or the application needs to access the distributed cache server across processes, 2:

No matter where the cache server is located, because cross-process or cross-domain access to cache data is involved, cache data must be serialized before being sent to the cache server. When cache data is used, after the application server receives the serialized data, it deserializes it. The process of serialization and deserialization consumes a lot of CPU, and many problems occur here.

In addition, if we modify the obtained data in the application, the original data in the cache server is not modified unless we save the data to the cache server again. Note: This is different from the previous local memory cache.

For each piece of data in the cache, we call it a "cache item" for the purposes described later.

After the two concepts are popularized, we will enter today's topic: nine common mistakes in Using Cache:

Next, let's take a look at each point!

Too dependent on. NET default serialization Mechanism

When we use a cross-process caching mechanism in an application, such as distributed cache memcached or Microsoft AppFabric, data is cached in a process outside the application. Each time we cache some data, the cached API will first serialize the data into bytes and then send these bytes to the cache server for storage. Similarly, when we use cached data again in an application, the cache server will send the cached bytes to the application, the cached client class library will undergo deserialization after accepting these bytes and convert them to the data objects we need.

There are also three points to note:

• This serialization and deserialization mechanism occurs on the application server, and the cache server is only responsible for saving.
• . The default serialization mechanism in NET is not optimal, because reflection is required, and reflection is very CPU-consuming, especially when we cache more complex data objects.

Based on this problem, we need to select a better serialization method to minimize CPU usage. A common method is to allow the object to implement the ISerializable interface by itself.

First, let's take a look at the default serialization mechanism. 3:

Then, implement the ISerializable interface, as shown in Figure 4:

The biggest difference between our own implementation method and the default serialization mechanism of. NET is that reflection is not used. The self-implemented speed can be hundreds of times higher than the default mechanism.

Some people may think there is nothing, not just a small serialization. Is it necessary to make a big fuss?

When developing a high-performance application (such as a website), everything from architecture to code writing and subsequent deployment needs to be optimized. A small problem, such as the serialization problem, does not seem to be a problem at the beginning. If the number of visits to our site applications is millions, tens of millions, or even more advanced, these accesses need to obtain some public cache data, which is a small problem!

Next, let's look at the second misunderstanding.

Large cache objects

Sometimes, we want to cache some large objects because it is costly to generate a large object. We need to generate a large object and use it as many times as possible to improve response.

When it comes to large objects, it is necessary to introduce them in depth. In. NET, the so-called large object refers to the object whose memory is larger than 85K. The problem is clearly explained through a comparison below.

If there is a set of Person classes, which is defined as List <Person>, each Person object occupies 1 k of memory. If this set of Persons contains 100 Person object instances, is this set a large object?

The answer is: no!

Because the set is only a reference of the contained Person object instance, that is, on the managed heap of. NET, the memory size allocated by the Person set is the size of 100 references.

Then, for the following object, it is a large object: byte [] data = new byte [87040] (85*1024 = 87040 ).

Here, let's talk about why: it is very costly to generate a large object.

Because in. in. NET, large objects are distributed on the large object hosting stack (we refer to it as a "Big Heap" for short. Of course, there is also a corresponding small heap ), the allocation mechanism of objects on this large heap is different from that on the small heap: when the large heap is allocated, it always needs to find the appropriate memory space. As a result, memory fragmentation occurs, leading to insufficient memory! Let's use a diagram to describe it, as shown in Figure 5:

Very clear, as shown in Figure 5:

• The garbage collection mechanism does not compress Large Heaps after the objects are recycled (small heaps are compressed ).
• When allocating objects, You need to traverse the large pile and find the appropriate space. traversal costs a lot.
• If some space is less than 85 KB, it cannot be allocated and can only be wasted, leading to memory fragmentation.

After talking about this, let's get down to the truth and look at the cache of large objects.

As mentioned before, objects are cached and read for serialization and deserialization. The larger the cached objects (for example, 1 MB ), more CPU is consumed throughout the process.

For such a large object, it depends on whether it is frequently used, whether it is a public data object, or every user needs to generate. Because once we cache (especially in distributed cache), we need to consume both the memory of the cache server and the CPU Of the application server. If the usage is not frequent, we recommend that you generate each time! If it is public data, we recommend that you perform a lot of tests: Compare the cost of producing large objects with the memory and CPU consumed when caching it, and select a lower cost! If it is generated by every user, check whether it can be broken down. If it cannot be broken down, the cache will be released in time!

Use the cache mechanism to share data among threads

When data is stored in the cache, multiple threads of our program can access this public region. When multiple threads access the cached data, there will be some competition, which is also a common problem in multithreading.

Next we will introduce the problems brought about by competition from two aspects: local memory cache and distributed cache.

See the following code:

For the local memory cache, after these three threads run, in thread 1, the value of item may be 1, and thread 2 may be 2, thread 3 may be 3. Of course, this is not necessarily true! Only possible values in most cases!

It is hard to say that distributed cache is used! Because data modification does not happen immediately in the memory of the local machine, but goes through a cross-process.

Some cache modules have implemented locks to solve this problem, such as AppFabric. Pay special attention to this when modifying cached data.

The data will be immediately cached after the API is called.

Sometimes, when we call the cached API, we will think that the data has been replaced, and then we can directly read the data in the cache. Although this is often the case, it is not absolute! This is the case with many problems!

Here is an example.

For example, for an ASP. NET application, if we call the cache API in the Click Event of a button, and then read the cache when the page is displayed, the Code is as follows:

The above code is correct, but problems may occur. Click the button and return to the page. Data is displayed when the page is displayed. The process is correct. However, if the server memory is insufficient and the server memory is recycled, it is very likely that the cached data will be lost!

A friend will say: How fast is memory recovery?

This mainly depends on some of our settings and processing.

Generally, the cache mechanism sets the absolute expiration time and relative expiration time. The difference between the two is clear to everyone. I will not talk about it here. For the above Code, if we set the absolute expiration time, for example, 1 minute, if the page processing is very slow and the time exceeds 1 minute, then wait until the presentation, maybe the data in the cache is gone!

Sometimes, even if we cache data in the first line of code, we may no longer cache data for reading in the third line of code. This may be because the memory pressure on the server is high, and the cache mechanism directly clears the data with minimal access. Or the server's CPU is very busy and the network is not good, so that the data is not serialized and saved to the cache server.

In addition, for ASP. NET, if the local memory cache is used, it also involves IIS configuration issues (cache memory restrictions). We have the opportunity to share this knowledge for you.

Therefore, when using cached data, you must determine whether the data exists. Otherwise, there will be many "objects not found" errors, it produces some strange and reasonable phenomena that we think ".

Caches a large number of data sets and reads some of them.

In many cases, we often cache a collection of objects. However, when reading, we only read a part of the object each time. Let's give an example to illustrate this problem (the example may not be very appropriate, but it is sufficient to explain the problem ).

In a shopping site, a common operation is to query the information of some products. In this case, if you enter a "25-inch TV", then you can find the relevant products. At this time, in the background, we can query the database and find hundreds of such data records. Then, we cache these hundreds of data records as a cache item. The Code is as follows:

At the same time, we will display 10 products by page. In fact, each paging operation retrieves data based on the cached key, selects the next 10 data records, and displays the data.

If the local memory cache is used, this may not be a problem. If the distributed cache is used, the problem arises. You can clearly describe this process ,:

I believe that after reading this figure, we should be very clear about the problem: each time we get all the data according to the cache key, and then deserialize all the data on the application server, but only 10 of them are used.

Here, we can split the data set again and divide it into cache items such as 25-0-10-products and 25-11-20-products, as shown in:

Of course, there are many ways to query and cache, and there are also many ways to split. This is a common problem!

Memory waste caused by caching large-sized objects with graphic Structures

To better illustrate this problem, we first see a class structure diagram below,

If we want to cache some Customer data, there may be two problems:

Next, let's take a look at these two questions.

First, we can see the first one. If we use the distributed cache to cache some Customer information, and we do not use the default serialization mechanism to re-serialize the Customer, the serialization mechanism is used to serialize the Customer, the object referenced by the Customer is also serialized, and other referenced objects in the serialized object are serialized. The final result is: the Customer is serialized, and the Order information of the Customer is serialized, the OrderItem referenced by Order is serialized, and the Product referenced by OrderItem is serialized.

The entire object graph is serialized. If this is what we want, there is no problem. If not, we waste a lot of resources, there are two solutions: first, implement serialization by yourself and completely control which objects need to be serialized. We have already discussed it. Second, if you use the default serialization mechanism, then, add the [NonSerialized] Mark on the object that does not need to be serialized.

Next, we can see the second problem. This problem is mainly caused by the first problem: when the Customer is cached, other information about the Customer, such as Order and Product, has been cached. However, many technical personnel do not know this, and then cache the Order information of the Customer in other cache items. The usage is based on the Customer ID, for example, the ID is used to obtain the Order information in the cache, as shown in the following code:

The solution to this problem is also obvious. Refer to the solution to the first problem!

Cache application configuration information

Because the cache has a set of data failure detection cycles (as mentioned earlier, either fixed-time or relative-time failure, many technical staff like to save some dynamic information in the cache to make full use of this feature of the cache mechanism. Among them, the configuration information of the cache program is one example.

Some configurations in the application may change. The simplest is the database connection string, as shown in the following code:

After this setting, after the cache becomes invalid for a period of time, the configuration file will be re-read. At this time, the configuration may be different from the previous one, in addition, the cache can be read elsewhere for updates, especially when the same site is deployed on multiple servers, sometimes, we did not promptly modify the information in the site configuration file on each server. In this case, how to use the distributed cache to cache the configuration information, as long as the configuration file of one site is updated, all other sites have been modified, so technicians are happy. OK, this does seem to be a good method (you can use it when necessary). However, not all configuration information must be the same, but what should be considered: if a problem occurs on the cache server and the server goes down, all the sites that use this configuration information may have problems.

We recommend that you use a monitoring mechanism for the information of these configuration files, such as file monitoring. Every time the file changes, the configuration information is reloaded.

Use many different keys to point to the same cache item

We sometimes encounter a situation where we cache an object and use a key as the cache key to obtain the data, we use an index as the cache key to obtain the data, as shown in the following code:

This is mainly because we read data from the cache in multiple ways. For example, we need to retrieve data through indexes during loop traversal, such as index ++, in some cases, we may need to use other methods, such as product names, to obtain product information.

In this case, we recommend that you combine these keys to form the following form:

Another common problem is that the same data is cached in different cache items. For example, if the user queries a 36-inch color TV, a TV product numbered 100 may be in the results. At this time, we cache the results. In addition, the user is looking for a TV with the TCL manufacturer. If the TV product numbered 100 appears in the result, we cache the result in another cache item. At this time, it is obvious that there is a waste of memory.

In this case, the method used previously is to create an index list in the cache ,:

Of course, there are many details and problems that need to be solved. Here we will not describe them one by one, depending on their respective applications and situations! You are also welcome to provide better methods.

No timely update or deletion of expired or invalid data in the cache

This should be the most common problem with caching. For example, if we get the information of all the unprocessed orders of a Customer and then cache the information, the code similar to this is as follows:

After that, a user's order is processed, but the cache has not been updated, so the data in the cache is already faulty! Of course, I will only list the simplest scenarios here. You can think of other products in your application, and there may be different cached data from the actual database.

Many times, we have tolerated such inconsistencies in a short period of time. In fact, there is no perfect solution for this situation. If you want to do this, you can implement it. For example, you can traverse all the data in the cache every time you modify or delete a data record and then perform operations, however, this often outweighs the loss. Another method of compromise is to determine the data change cycle and shorten the cache time as much as possible.