9 misunderstandings about Using Cache (I)

If you want to optimize a site or application frequently, you can say that the use of cache is the fastest and most effective way. In general, we cache some frequently-used data that requires a large amount of resources or time to make subsequent use faster.

If you really want to elaborate on the benefits of caching, there are still a lot of them, but in actual applications, it is always so unsatisfactory when using caching. In other words, if cache is used, the performance can be improved to 100 (the number here is just a metering symbol, just to give you a "amount" experience). However, many times, the improved performance is only 80, 70, or less, and may even cause serious performance degradation. This phenomenon is particularly prominent when distributed cache is used.

In this article, we will introduce the nine major problems that have caused the above problems and provide corresponding solutions. The article uses. NET as an example to demonstrate Code. It is also of reference value for friends who come and other technical platforms. just replace the corresponding code!

To make the subsequent description more convenient and more complete, Let's first look at the two cache formats: local memory cache and distributed cache.

First, for the local memory cache, the data is cached in the local memory, as shown in 1:

We can clearly see that:

• The application caches data in the local memory and directly obtains the data from the local memory when necessary.
• For. NET Applications, when obtaining data in the cache, they use object references to search for Data Objects in the memory. That is to say, if we get the data object through reference, we directly modify this object. In fact, we are actually modifying the cache object in the memory.

For distributed cache, because the cached data is stored on the cache server, or the application needs to access the distributed cache server across processes, 2:

No matter where the cache server is located, because cross-process or cross-domain access to cache data is involved, cache data must be serialized before being sent to the cache server. When cache data is used, after the application server receives the serialized data, it deserializes it. The process of serialization and deserialization consumes a lot of CPU, and many problems occur here.

In addition, if we modify the obtained data in the application, the original data in the cache server is not modified unless we save the data to the cache server again. Note: This is different from the previous local memory cache.

For each piece of data in the cache, we call it a "cache item" for the purposes described later.

After the two concepts are popularized, we will enter today's topic: nine common mistakes in Using Cache:

1. Too dependent on. NETDefault serialization Mechanism
2. Large cache objects
3. Use the cache mechanism to share data among threads
4. It is considered that the cache API is calledThe data is immediately cached.
5. Caches a large number of data sets and reads some of them.
6. Memory waste caused by caching large-sized objects with graphic Structures
7. Cache application configuration information
8. Use many different keys to point to the same cache item
9. No timely update or deletion of expired or invalid data in the cache

Next, let's take a look at each point!

　　Too dependent on. NET default serialization Mechanism

When we use a cross-process caching mechanism in an application, such as distributed cache memcached or Microsoft AppFabric, data is cached in a process outside the application. Each time we cache some data, the cached API will first serialize the data into bytes and then send these bytes to the cache server for storage. Similarly, when we need to use cached data again in application, the cache server will send the cached bytes to the application, the cached client class library will undergo deserialization after accepting these bytes and convert them to the data objects we need.

There are also three points to note:

• This serialization and deserialization mechanism occurs on the application server, and the cache server is only responsible for saving.
• . The default serialization mechanism in NET is not optimal, because reflection is required, and reflection is very CPU-consuming, especially when we cache more complex data objects.

Based on this problem, we need to select a better serialization method to minimize CPU usage. A common method is to allow the object to implement the ISerializable interface by itself.

First, let's take a look at the default serialization mechanism. 3:

Then, implement the ISerializable interface, as shown in Figure 4:

The biggest difference between our own implementation method and the default serialization mechanism of. NET is that reflection is not used. The self-implemented speed can be hundreds of times higher than the default mechanism.

Some people may think there is nothing, not just a small serialization. Is it necessary to make a big fuss?

When developing a high-performance application (such as a website), everything from architecture to code writing and subsequent deployment needs to be optimized. A small problem, such as the serialization problem, does not seem to be a problem at the beginning. If the number of visits to our site applications is millions, tens of millions, or even more advanced, these accesses need to obtain some public cache data, which is a small problem!

Next, let's look at the second misunderstanding.

　　Large cache objects

Sometimes, we want to cache some large objects because it is costly to generate a large object. We need to generate a large object and use it as many times as possible to improve response.

When it comes to large objects, it is necessary to introduce them in depth. In. NET, the so-called large object refers to the object whose memory is larger than 85K. The problem is clearly explained through a comparison below.

If there is a set of Person classes, which is defined as List <Person>, each Person object occupies 1 k of memory. If this set of Persons contains 100 Person object instances, is this set a large object?

The answer is: no!

Because the set is only a reference of the contained Person object instance, that is, on the managed heap of. NET, the memory size allocated by the Person set is the size of 100 references.

Then, for the following object, it is a large object: byte [] data = new byte [87040] (85*1024 = 87040 ).

Here, let's talk about why: it is very costly to generate a large object.

Because in. in. NET, large objects are distributed on the large object hosting stack (we refer to it as a "Big Heap" for short. Of course, there is also a corresponding small heap ), the allocation mechanism of objects on this large heap is different from that on the small heap: when the large heap is allocated, it always needs to find the appropriate memory space. As a result, memory fragmentation occurs, leading to insufficient memory! Let's use a diagram to describe it, as shown in Figure 5:

Very clear, as shown in Figure 5:

• The garbage collection mechanism does not compress Large Heaps after the objects are recycled (small heaps are compressed ).
• When allocating objects, You need to traverse the large pile and find the appropriate space. traversal costs a lot.
• If some space is less than 85 KB, it cannot be allocated and can only be wasted, leading to memory fragmentation.

After talking about this, let's get down to the truth and look at the cache of large objects.

As mentioned before, objects are cached and read for serialization and deserialization. The larger the cached objects (for example, 1 MB ), more CPU is consumed throughout the process.

For such a large object, it depends on whether it is frequently used, whether it is a public data object, or every user needs to generate. Because once we cache (especially in distributed cache), we need to consume both the memory of the cache server and the CPU Of the application server. If the usage is not frequent, we recommend that you generate each time! If it is a public data, we recommend that you perform a lot of tests: Compare the cost of producing large objects with the memory and CPU consumed when saving it, and select a lower cost! If it is generated by every user, check whether it can be broken down. If it cannot be broken down, it will be cached and released in a timely manner!

　　Use the cache mechanism to share data among threads

When data is stored in the cache, multiple threads of our program can access this public region. When multiple threads access the cached data, there will be some competition, which is also a common problem in multithreading.

Next we will introduce the problems brought about by competition from two aspects: local memory cache and distributed cache.

See the following code:

For the local memory cache, after these three threads run, in thread 1, the value of item may be 1, and thread 2 may be 2, thread 3 may be 3. Of course, this is not necessarily true! Only possible values in most cases!

It is hard to say that distributed cache is used! Because data modification does not happen immediately in the memory of the local machine, but goes through a cross-process.

Some cache modules have implemented locks to solve this problem, such as AppFabric. Pay special attention to this when modifying cached data.

　　It is considered that the cache API is calledThe data is immediately cached.

Sometimes, when we call the cached API, we will think that the data has been replaced, and then we can directly read the data in the cache. Although this is often the case, it is not absolute! This is the case with many problems!

Here is an example.

For example, for an ASP. NET application, if we call the cache API in the Click Event of a button, and then read the cache when the page is displayed, the Code is as follows:

The above code is correct, but problems may occur. Click the button and return to the page. Data is displayed when the page is displayed. The process is correct. However, if the server memory is insufficient and the server memory is recycled, it is very likely that the cached data will be lost!

A friend will say: How fast is memory recovery?

This mainly depends on some of our settings and processing.

Generally, the cache mechanism sets the absolute expiration time and relative expiration time. The difference between the two is clear to everyone. I will not talk about it here. For the above Code, if we set the absolute expiration time, for example, 1 minute, if the page processing is very slow and the time exceeds 1 minute, then wait until the presentation, maybe the data in the cache is gone!

Sometimes, even if we cache data in the first line of code, we may no longer cache data for reading in the third line of code. This may be because the memory pressure on the server is high, and the cache mechanism directly clears the data with minimal access. Or the server's CPU is very busy and the network is not good, so that the data is not serialized and saved to the cache server.

In addition, for ASP. NET, if the local memory cache is used, it also involves IIS configuration issues (cache memory restrictions). We have the opportunity to share this knowledge for you.

Therefore, when using cached data, you must determine whether the data exists. Otherwise, there will be many "objects not found" errors, it produces some strange and reasonable phenomena that we think ".

　　About the author

Wang Yang is the author of "NET application architecture design: models, Principles and Practices" by HP architects and information analysts. Chief Software Architecture expert of Shanghai Yisi R & D Management Consulting Co., Ltd., deputy head of the Software consulting group.

Link: http://www.infoq.com/cn/articles/misunderstanding-using-cache