Open source framework That thing 16: The evolution of Cache-related code

Problem Introduction

The last time I worked on a large-scale project, it was necessary to use buffering because the system required a higher TPS.

The project uses more buffers, there are memcache, Redis, some also need to provide a two-level buffer, which means that the application server layer can also set some buffering.

Of course, to see the relevant implementation of the code when, roughly the following look.

public void Savesomeobject (Someobject someobject) {    memcacheutil.put ("Someobject", Someobject.getid (), Someobject);    Here is the code for the real Save object}public someobject getsomeobject (String id) {    someobject someobject = Memcacheutil.get ("Someobject" , id);    if (someobject!=null) {         someobject=//Real get object         memcacheutil.put ("Someobject", Someobject.getid (), Someobject );    }    return someobject;}

It is clear that the buffer-related code is all coupled to the original business code.

Later, because Memcache performance is not stable, and memcache function, can also be fully implemented by Redis, so decided to remove the memcache from the system, replaced by the implementation of the Redis scheme, and then changed to look like:

public void Savesomeobject (Someobject someobject) {    redisutil.put ("Someobject", Someobject.getid (), someobject);    Here is the code for the real Save object}public someobject getsomeobject (String id) {    someobject someobject = Redisutil.get ("Someobject", ID );    if (someobject!=null) {         someobject=//Real Get Object <span></span>redisutil.put ("Someobject", Someobject.getid (), someobject);    }    return someobject;}

This pass change down, the developer has tipsy, later feel performance is not high enough, this time, to add some data two level buffer, that is, local buffer has to take local, local does not take remote buffer

So, the above code is a change, the following looks like this:

public void Savesomeobject (Someobject someobject) {    localcacheutil.put ("Someobject", Someobject.getid (), Someobject);    Redisutil.put ("Someobject", Someobject.getid (), someobject);    Here is the code for the real Save object}public someobject getsomeobject (String id) {    someobject someobject = Localcacheutil.get (" Someobject ", id);    if (someobject!=null) {        return someobject;    }    Someobject = Redisutil.get ("someobject", id);    if (someobject!=null) {         someobject=//Real get object          redisutil.put ("Someobject", Someobject.getid (), someobject);    }    return someobject;}

But this time there is a problem:

Since only one computer can be modified at a time, the local buffering of other computers is actually inconsistent with the data in the remote and the database, this time, there are two ways to achieve this, one is to use Redis's read release mechanism for data synchronization, this way, will ensure that the data can be synchronized in time.

Another option is to set the local buffer to a shorter time, allowing inconsistent data to occur over a relatively short period of time.

In any case, the function is realized, the programmer's small partner this time has been changed to black eyes, finger numbness, almost collapsed.

Obviously this way of implementation is not good, so the project team has proposed improvements, can be annotated way to annotate, so that the programmer can only declare it? Good idea, and then it turned out the following way:

@Cache (type= "Someobject", parameter= "Someobject", key= "${someobject.id}") public void Savesomeobject (someobject Someobject) {    //below is the code of the real Save object} @Cache ("Someobject", key= "${id}") Public someobject Getsomeobject (String id) {    Someobject someobject=//Real get    return someobject;}

this time, the programmer's code has been very refreshing, there is no buffer-related content, but the introduction of a new problem, is the code to deal with the annotation how to write? Containers need to be introduced, such as Spring, which must be hosted by the container, and if there is a direct new instance, there is no way to buffer it. There is also a problem: Although the programmer's workload is a savings, but still to the program code is intrusive, need to introduce these annotations, if you want to add beyond the existing annotations, or need to re-write these classes, introduce other annotations, modify the existing annotations.

So, the above is an acceptable solution, but obviously not a very good solution.

If a programmer fires up, he makes the following complaint: "I just do my business, put the buffer and I have 1 cents relationship?" Always because of buffering things let me change to change, the program to change the mess do not say, my time, my work progress have affected who to tube? After the buffer related things don't fucking bother me! "What do you think, as architects, you?" At the very least, I think he makes a very reasonable point. Let's go back and look at the most primitive code:

public void Savesomeobject (Someobject someobject) {    //The following is the code for the real Save object}public someobject getsomeobject (String ID) {    someobject someobject=//Real get    return someobject;}

Here is the clean business code, and the buffering is not a bit of a relationship. Later, due to performance requirements, need to do buffer, OK, this is the fact, but with what buffer or how to buffer, and programmers do not have any relationship, so, is not allowed to participate in the programmer, you can gracefully do add buffering function? The answer, of course, is yes.

Demand Consolidation

1. In the code, do not reflect the buffer-related content, that is, do not buffer and how to do the buffer do not affect the business code
2. Whether you take an instance from a container or a new instance, you can do the same thing, which means you don't have to rely on a specific container

Solution Ideas:

Buffering, buffering, buffering, and so on, the content is found to be a performance bottleneck at run time and then submitted to the programmer for optimization. To do this, we have designed a configuration to describe these buffer-related declarations.

Of course, the structure of this configuration file can be defined according to the buffer framework that you use.

Like what:

<redis-caches>  <redis-cache type= "Org.tinygroup.redis.test.UserDao" >     <redis-method Method-name= "Saveuser" >           <redis-expire value= "$" ></redis-expire>           <redis-string type= "User" key= "${user.id}" paramter-name= "user" ><redis-string>     </redis-method>  </ redis-cache>  <redis-cache type= "Org.tinygroup.redis.test.UserDao" >     <redis-method method-name = "GetUser" >           <redis-expire value= "$" ></redis-expire>           <redis-string type= "user" key= "${id}" ><redis-string>     </redis-method>  </redis-cache></redis-caches>

in our actual application, the configuration is better than the example above, so let me start by talking about the meaning of the above two-paragraph configuration.

in the Userdao of the Saveuser, the user data will be synchronized to the Redis buffer, buffer time of 1 seconds, the buffer data stored in the type of user, the key value is ${user.id}, that is to save the user's main health. When actually entering Redis, the health value in Redis is made up of the above type and key.

when calling Userdao's GetUser, the data of type user, key value ${id} is obtained from the buffer, if it is in the buffer, it is fetched and returned, if not in the buffer, the value is taken from the old business code and placed in the buffer, and then returned to this object.

Wow, this time, it is very cool, as long as the declaration can be done to the buffer processing, but a problem came out, how to achieve the above requirements?

through the configuration file external, did do 0 of the business code intrusion, but how to add buffer to the original business business logic? Since the requirement 2 requirements can be new, you can also get the object instance in the container, so the use of containers AOP solved the run is blocked, so you have to introduce bytecode to solve.

Specific implementation

Write a maven-based buffer code processing plug-in that adds the processing plug-in after compiling, scans the old code based on the configuration file, and modifies its bytecode to add buffer-related processing logic.

It is now possible to automatically add buffer-related logic to the class file by using Maven for compile or install.

At this point, we have analyzed the buffer code directly coupled to the code, and analyzed the shortcomings, the final evolution of the annotated mode, external configuration, and a brief introduction of the implementation method.

Concrete implementation, the use of the technology is more, there are MAVEN plug-ins, there are ASM, template engine and tiny framework of some basic projects, such as: Vfs,fileresolver and so on.

If the use of tiny framework, can be directly used, if not tiny frame, you can refer to the above ideas to do their own implementation.

Welcome to the Open source technology community: http://bbs.tinygroup.org . The Code and framework information in this example will be shared in the community. "Self-write framework" member QQ Group: 228977971, let us work together to understand the mystery of the open source framework!

4th Wave Activity: Write book reviews, send beautifully developed books!

Open source framework That thing 16: The evolution of Cache-related code