Summary of Distributed Cache learning

First, distributed cache diagram

Second, why use memcached distributed cache?

Three, memcached basic principle

Iv. memcache Download and installation

V. MencacheHelper.cs example using a combination session with the project configuration cache

Vi. Summary of differences between Redis and Memcache

First, distributed cache diagram

Second, why use memcached distributed cache?

First of all, why cache, in data-driven web development, often repeated from the database to remove the same data, this duplication greatly increased the database load. Caching is a good way to solve this problem. However, although the local cache of the page can already be implemented in ASP, it is still not flexible enough. Memcached was born.

1, high concurrent access to the pain of the database: Deadlock!

2, Disk IO pain, database reading and writing is to deal with the disk, the disk read speed is limited, the general high is 7200 rpm

3. Multi-Client shared cache

4, Net+memory >> IO

5, read and write performance perfect 1s read can reach 1w times write: 10w times

6, Super simple cluster construction clister

7. Open Source

8, does not provide the master-slave assignment function, also does not provide the disaster tolerant function (disaster tolerance: that is, data backup can make the data recovery after the accident, Memcached will not be backed up, because it is cached in memory, a power loss will lose data

), so all the code basically considers performance best. If you want to consider disaster tolerance, you can use the Redis distributed cache

9, the study cost is very low, the entry is very easy

10. Rich and successful cases. A lot of big companies use this to do distributed caching.

Note: memcached in the enterprise is generally run under Linux, in order to achieve the best performance.

Three, memcached basic principle

The underlying communication is using the socket

The cache server can be understood as the socket service side, and the Web server is understood as a client.

Iv. memcache Download and installation

Download, Baidu or directly on the CSDN to search the Windows Memcache stable version on the line, 0 points.

1, after downloading, it is such an EXE

2, installation, knock a few lines of cmd command on the line, such as, the left is our Computer Services list, you can see, has started our memcached (Memcache is the name of this open source project, add a d,memcached is the specific application, that is, the name of this EXE)

3, has now started the service, and installed it on the Windows service, so that you do not have to start each time manually, with the computer and start.

Now to test, just create a new console application

　　

Using Memcached.clientlibrary;
Using System;
Using System.Collections.Generic;
Using System.Linq;
Using System.Text;
Using System.Threading.Tasks;

Namespace test
{
public class Memcachetset
{
public static void Test ()
{
String[] ServerList = {"127.0.0.1:11211"}; List of servers, which can be separated by commas

Initialize Pool
Sockiopool pool = sockiopool.getinstance ();

Modify the following parameters according to the actual situation
Pool. Setservers (serverlist);
Pool. Initconnections = 3;
Pool. Minconnections = 3;
Pool. MaxConnections = 5;
Pool. socketconnecttimeout = 1000;
Pool. Sockettimeout = 3000;
Pool. Maintenancesleep = 30;
Pool. Failover = true;
Pool. Nagle = false;
Pool. Initialize (); Initialize the pool for memcache servers

Get the Client instance
memcachedclient mc = new Memcachedclient ();//Initialize a client
Mc. EnableCompression = false;

Console.WriteLine ("---------Test----------");
Mc. Set ("Test", "my Value");//store data to the cache server, where the string "My value" is cached, key

if (MC. Keyexists ("test"))//test cache There is a project with key to test
{
Console.WriteLine ("Test is Exists");
Console.WriteLine (MC. Get ("Test"). ToString ());//Get Project with key test in cache
}
Else
{
Console.WriteLine ("Test Not Exists");
}

Console.ReadLine ();

Mc. Delete ("test");//Remove the item with the key test in the cache

if (MC. Keyexists ("Test"))
{
Console.WriteLine ("Test is Exists");
Console.WriteLine (MC. Get ("Test"). ToString ());
}
Else
{
Console.WriteLine ("Test Not Exists");
}

Console.ReadLine ();

Sockiopool.getinstance (). Shutdown ();//close pool, close sockets
}
}
}

 1 using Memcached.clientlibrary; 2 using System; 3 using System.Collections.Generic; 4 using System.Linq; 5 using System.Text; 6 using System.Threading.Tasks;             7 8 Namespace Test 9 {Ten public class MemcacheTset11 {a public static void test () 13 {14 String[] ServerList = {"127.0.0.1:11211"};             List of servers, can be separated by commas 15 16//Initialize pools Sockiopool pool = sockiopool.getinstance (); 18 19 Modify the following parameters according to the actual situation. Setservers (serverlist), pool. Initconnections = 3;22 pool. Minconnections = 3;23 pool. MaxConnections = 5;24 pool. Socketconnecttimeout = 1000;25 pool. Sockettimeout = 3000;26 pool. Maintenancesleep = 30;27 pool. Failover = true;28 pool. Nagle = false;29 pool. Initialize (); Initialize the pool for memcache servers 30 31//Get client instance Memcachedclient mc = new M EmcacHedclient ();//Initializes a client-side MC. EnableCompression = False;34 Console.WriteLine ("---------Test----------"); Set ("Test", "my Value");//store data to the cache server, where the string "My value" is cached, Key37 (MC).                 Keyexists ("test"))//test cache There is a project with a key of "test" ("Test is Exists"); 41 Console.WriteLine (MC. Get ("Test"). ToString ());//Get key test item in cache}43 else Console.WriteLine ("T EST not Exists ");}47 console.readline (); Delete ("test");//Remove the item in the cache where key is Test (MC). Keyexists ("Test")), Console.WriteLine ("Test is Exists"), Console.writel INE (MC. Get ("Test").             ToString ());}57 else58 {Console.WriteLine ("Test Not Exists"); 60 }61 console.readline (); SockiopooL.getinstance (). Shutdown ();//close pool, close Sockets65}66}67}

If the program is working properly, our memcache service is up and running.

A little, do not forget, is our memcache drive, the same, Baidu C # memcache installation is OK.

Well, now let's actually apply it.

V. MencacheHelper.cs example using a combined project to configure the cache

For what is the project configuration, I believe everyone is familiar with, is the business of infrequently changed data such as System e-mail address, it exists in the database, easy to change.

Then, looking back at these words, "infrequently changing", we have to pay attention to, such data, we have to think of the cache, the cache is used to manage such data. Then, we can use our memcache to manage it.

Project, we can create a new Memcachehelper class to encapsulate the code, and I wrote a simplest access delete.

Using Memcached.clientlibrary;
Using System;
Using System.Collections.Generic;
Using System.Linq;
Using System.Text;
Using System.Threading.Tasks;

Namespace JOEY.BookShop.Common
{
public class Memcachehelper
{
private static readonly Memcachedclient MC;
Static Memcachehelper ()
{
String[] ServerList = {"127.0.0.1:11211"}; List of servers, which can be separated by commas

Initialize Pool
Sockiopool pool = sockiopool.getinstance ();

Modify the following parameters according to the actual situation
Pool. Setservers (serverlist);
Pool. Initconnections = 3;
Pool. Minconnections = 3;
Pool. MaxConnections = 5;
Pool. socketconnecttimeout = 1000;
Pool. Sockettimeout = 3000;
Pool. Maintenancesleep = 30;
Pool. Failover = true;
Pool. Nagle = false;
Pool. Initialize (); Initialize the pool for memcache servers

Get the Client instance
MC = new Memcachedclient ();//Initialize a client
Mc. EnableCompression = false;
}

<summary>
Save
</summary>
<param name= "Key" ></param>
<param name= "Value" ></param>
public static void Set (string key, Object value)
{
Mc. Set (key, value);
}

public static void Set (string key, object value, DateTime time)
{
Mc. Set (key, value, time);
}

<summary>
Take
</summary>
<param name= "Key" ></param>
<returns></returns>
public static object Get (String key)
{
if (MC. Keyexists (Key))
{
return MC. Get (key);
}
Else
{
return null;
}

}

<summary>
Delete
</summary>
<param name= "Key" ></param>
<returns></returns>
public static bool Delete (string key)
{
if (MC. Keyexists (Key))
{
Mc. Delete (key);
return true;
}
return false;
}
}
}

 1 using Memcached.clientlibrary; 2 using System; 3 using System.Collections.Generic; 4 using System.Linq; 5 using System.Text; 6 using System.Threading.Tasks; 7 8 namespace JOEY.         Bookshop.common 9 {Ten public class MemcacheHelper11 {$ private static readonly memcachedclient mc;13 Static Memcachehelper () {string[] serverlist = {"127.0.0.1:11211"};//server list, can be separated by commas 1             6 17//Sockiopool pool = sockiopool.getinstance (); 19 20//Modify the following parameters according to the actual situation 21 Pool. Setservers (serverlist), pool. Initconnections = 3;23 pool. Minconnections = 3;24 pool. MaxConnections = 5;25 pool. Socketconnecttimeout = 1000;26 pool. Sockettimeout = 3000;27 pool. Maintenancesleep = 30;28 pool. Failover = true;29 pool. Nagle = false;30 pool. Initialize (); Initialize the pool for Memcache Servers 31 32//GetThe client instance is the MC = new Memcachedclient ();//Initializes a client-side MC.         EnableCompression = false;35}36//<summary>38///&LT;/SUMMARY&GT;40 <param name= "key" ></param>41///<param Name= "value" ></param>42 public static void Set (string key, Object value), {.             Set (key, value);}46 public static void Set (string key, object value, DateTime time) 48 {49 Mc.         Set (key, value, time),}51//<summary>53///</summary>55  <param name= "key" ></param>56///<returns></returns>57 public static object Get (String key), (MC). Keyexists (key)).           Get (key),}63 else64 {null;66 return}67  }69//<summary>71///</summary>73//<param NA          Me= "key" ></param>74//<returns></returns>75 public static bool Delete (string key) 76 {The (MC). Keyexists (key)). Delete (key), return true;81}82 return false;83}84}85}

1 var setting = this. DbSession.SettingsDal.LoadEntities (c = c.name = = "System Email address"). FirstOrDefault (); 2 string value = setting. Value.trim (); 3 Memcachehelper.set ("Setting_" + key, value);

It is convenient to store our system email address from the database in the memcache. To take the words is:

Object obj = Memcachehelper.get ("setting_" + "system mail Address");

This is because the data accessed is a string, there is no problem of serialization, if the object type of access is not a string, such as a table model, then it has to be serialized to operate, for serialization, I use Json.NET to operate.

Let's get another helper class.

Using Newtonsoft.json;
Using System;
Using System.Collections.Generic;
Using System.Linq;
Using System.Text;
Using System.Threading.Tasks;

Namespace JOEY.BookShop.Common
{
<summary>
Json.NET serialization, for a serialized dead loop caused by a mutual reference type, you can add an attribute tag on the object [Jsonignore] If there is a foreign key in model, the problem is caused by the mutual reference between two models
</summary>
public class Serializehelper
{
<summary>
Incoming object, serialized into a string to return
</summary>
<param name= "obj" ></param>
<returns></returns>
public static string Serializetostring (Object obj)
{
return Jsonconvert.serializeobject (obj);
}
<summary>
The serialized string is passed in and deserialized into the corresponding object returns
</summary>
<typeparam name= "T" > generic type, corresponding object type </typeparam>
<param name= "Serializestr" > Serialized string </param>
<returns></returns>
public static T deserializetoobject<t> (String serializestr)
{
Return jsonconvert.deserializeobject<t> (SERIALIZESTR);
}
}
}

 1 using Newtonsoft.json; 2 using System; 3 using System.Collections.Generic; 4 using System.Linq; 5 using System.Text; 6 using System.Threading.Tasks; 7 8 namespace JOEY. Bookshop.common 9 {//&LT;SUMMARY&GT;11///Json.NET serialization, for a serialized dead loop due to a mutual reference type, you can add an attribute label on the object [Jsonignore] as in model         There are foreign keys, two models of mutual references that cause the problem//</summary>13 public class SerializeHelper14 {//<summary>16 Incoming object, serialized into a string returned by//</summary>18//<param name= "obj" ></param>19/             <returns></returns>20 public static string Serializetostring (object obj) 21 {22         return Jsonconvert.serializeobject (obj);}24//&LT;SUMMARY&GT;25//////////////26//Incoming serialization string, deserialized to corresponding object return </summary>27//<typeparam name= "T" > generic type, corresponding object type </typeparam>28//<param Nam      E= "Serializestr" > Serialized string </param>29///&LT;RETURNS&GT;&LT;/RETURNS&GT;30   public static T deserializetoobject<t> (string serializestr) {return jsonconvert.deserial Izeobject<t> (SERIALIZESTR); 33}34}35}

It's easy to read, right. Of course the driver is also needed. Also Baidu Oh.

Here's a little problem, this Assembly, Newtonsoft.json in my MVC project itself, and I used in other projects (i.e. project Common) in the use of the online, the selected version is 4.5, because the MVC project refers to the common, which seems to be inconsistent version of the situation, it seems like this, will prompt the error. So I deleted the DLL in MVC, reloaded the DLL under common (now think about why I didn't delete the common under the MVC--), so that a problem arose, failed to load the file or assembly "Newtonsoft.json, version=4.5.0.0. Estimate is the reason for the configuration items, so, Baidu, under the Web. config runtime node, add a few lines, modified to:

<dependentAssembly>        <assemblyidentity name= "Newtonsoft.json" publickeytoken= "30ad4fe6b2a6aeed" Culture= "neutral"/>        <bindingredirect oldversion= "0.0.0.0-8.0.0.0" newversion= "8.0.0.0"/>      </ Dependentassembly>

8.0.0.0 is the current version of this DLL and is estimated to be a version update notification. Specifically why this is a bit confusing. Who can advise, grateful ~ ~ ~

Vi. Summary of differences between Redis and Memcache (from Baidu know)

1. What is Redis

The result of this problem affects how we use Redis. If you think that Redis is a key value store, it might be used instead of MySQL, and if you think of it as a persistent cache, it might just save some of the temporary data that is frequently accessed.  Redis is an abbreviation for remote DIctionary server, and the subtitle on Redis on the official website is a persistent key-value database with built-in NET interface written in Ansi-c for Posix systems, this definition is biased toward key value store. There are also some views that Redis is a memory database because its high performance is based on the operation of the RAM. Others argue that Redis is a data structure server because Redis supports complex data features such as list, set, and so on. Different interpretations of the role of Redis determine how you use Redis.

Internet data is currently used in two different ways to store, relational database or key value. But these internet services do not belong to these two types of data, such as the user's relationship in the social platform, it is a list, if you want to use relational database storage needs to be converted into a multi-row record form, this form has a lot of redundant data, each row needs to store some duplicate information. If you store with key value it is cumbersome to modify and delete, you need to read out all the data and write again. Redis has designed a variety of data types in memory, allowing the business to access these data structures at high speed, and without concern for persistent storage issues, from an architecture that addresses some of the previous two types of storage needs.

2. Redis can't be faster than memcache

Many developers think that Redis is unlikely to be faster than memcached, memcached is completely memory-based, and Redis has a persistent save feature, and even if it is asynchronous, Redis is unlikely to be faster than memcached. But the test results are basically the absolute advantage of Redis. Have been thinking about this reason, now think of the reasons for these aspects.

Libevent. Unlike memcached, Redis does not have a choice of libevent. Libevent to cater to versatility, the code is huge (currently Redis code is less than 1/3 of Libevent) and has sacrificed a lot of performance on a particular platform. Redis has implemented its own Epoll event loop (4) with two file modifications in the libevent. Many developers in the industry have also suggested that Redis use another libevent high-performance alternative to Libev, but the author insists that Redis should be small and dependent. An impressive detail is that the Redis does not need to be executed before it is compiled./configure.

CAS issue. CAS is a convenient way to prevent competition from modifying resources in memcached. CAS implementations need to set a hidden CAS token,cas equivalent value version number for each cache key, each time the set token needs to be incremented, resulting in a dual overhead of CPU and memory, although these costs are small, but to stand-alone 10g+ After the cache and the QPS tens of thousands, these costs will bring some minor performance differences to each other (5).

3. Single Redis storage data must be smaller than physical memory

Redis's data is all put in memory for high-speed performance, but it also brings some unreasonable. For example, a medium-sized website has 1 million registered users, and if the data is to be stored using Redis, the capacity of the memory must be able to accommodate the 1 million users. But the business situation is 1 million users only 50,000 active users, 1 weeks to visit 1 times also only 150,000 users, so all 1 million users of data are placed in memory unreasonable, RAM needs to pay for cold data.

This is very similar to the operating system, where all the data accessed by the operating system is in memory, but if the physical memory does not hold the new data, the operating system intelligently exchanges some of the long-running data that is not accessed to disk, leaving room for new applications. Modern operating systems provide applications with not physical memory, but virtual memory concepts.

Based on the same considerations, Redis 2.0 also adds VM features. Let Redis data capacity break through the limits of physical memory. And it realizes the separation of data and heat.

4. Redis's VM implementation is a repetitive wheel build

Redis VMS Follow the previous Epoll implementation of the idea is still self-fulfilling. However, the introduction of the previous operating system mentioned that the OS can also automatically help the program to achieve cold and hot data separation, Redis only need the OS to request a large memory, the OS will automatically put hot data into physical memory, cold data exchange to the hard disk, another well-known "understanding of modern operating system (3)" Varnish is the realization of this, but also achieved a very successful effect.

Author Antirez Several reasons for explaining why to implement the VM himself (6). The main OS VM swap out is based on the page concept, such as OS VM1 page is 4K, 4K as long as there is an element even if only 1 bytes are accessed, the page will not be swap, the same reason, read a byte may be swapped into 4K of useless memory. Redis's own implementation can achieve control of the granularity of the change in. In addition to accessing the OS Swap memory area, the block process is also one of the reasons why Redis is implementing its own VMS.

5. Using Redis in Get/set mode

As a key value exists, many developers naturally use the Set/get way to use Redis, in fact, this is not the most optimized way to use. In particular, in cases where VMs are not enabled, all Redis data needs to be put into memory, which is especially important for saving memory.

If a key-value unit needs to occupy a minimum of 512 bytes, it takes 512 bytes to save only one byte. At this time there is a design mode, you can reuse the key, a few key-value into a key, value is stored as a set, so the same 512 bytes will be stored 10-100 times the capacity.

This is to save memory, it is recommended to use Redis in a hashset rather than set/get way, see the Reference (7) for details.

6. Use aof instead of snapshot

Redis has two storage methods, the default is the snapshot way, the implementation method is to periodically persist the memory snapshot (snapshot) to the hard disk, the disadvantage is that if the crash occurs after persistence, it will lose a piece of data. As a result of the perfectionist's push, the author added the AoF way. AoF that is append only mode, the Operation command is saved to the log file while writing the memory data, in a system with tens of thousands of concurrent changes, the command log is a very large amount of data, the management maintenance costs are very high, recovery rebuild time will be very long, which leads to the loss of AOF high-availability intention. What's more, Redis is a memory data structure model, and all of the advantages are based on the efficient atomic operation of complex memory structures, so that aof is a very uncoordinated part.

In fact, the main purpose of AOF is data reliability and high availability, in Redis there is another way to achieve the purpose: Replication. Because of the high performance of Redis, there is virtually no delay in replication. This is achieved by preventing single points of failure and achieving high availability.

Summary

To successfully use a product, we need to understand its features in depth. Redis performance is outstanding, if can skillfully control, for many large-scale domestic applications have a great help.

Summary of Distributed Cache learning