A tentative study of the King of Key/value memcached: Two, memcached basic operations in. Net

Original address: http://www.cnblogs.com/edisonchou/p/3855969.html

One, Memcached clientlib for. Net

First of all, it must be said that many languages have implemented the connection memcached client, in which Perl, PHP-based. Only the languages listed on the memcached website are: Perl, PHP, Python, Ruby,C #,C + +, and Lua.

Well, we act as. NET code farming, naturally using C #. Since the memcached client has a. NET version, let's download one and try it out.

Download file: Http://pan.baidu.com/s/1w9Q8I

memcached clientlib Project Address: http://sourceforge.net/projects/memcacheddotnet/

Unzip the package, which has 1.1 and 2.0 two versions, where we use the 2.0 version. (The directory address in the compressed package is: \memcacheddotnet_clientlib-1.1.5\memcacheddotnet\trunk\clientlib\src\clientlib\bin\2.0\release)

The above four DLLs is we need to introduce the project in the Assembly, with them, we can communicate with the memcached server, the cool crooked ah.

Ii. memcached Basic operations in. NET 2.1 basic memcached client Operations

(1) First, open the Windows Server 2003 virtual machine, turn on the memcached service, (not necessary, if you are on this machine, you can skip this step, just turn on the memcached service)

(2) ① open vs, Create a new C # console application named: Memcachedclientdemo.

② Create a new folder, named Lib, and then copy the client assembly DLLs you downloaded above to this folder and add references to those DLLs.

(3) Start writing code to communicate with the server through the memcached client, see the following code:

        [STAThread] static void Main (string[] args) {//memcached server list//If there are more than one server,            Comma separated, for example: "192.168.80.10:11211", "192.168.80.11:11211" string[] serverlist = {"192.168.80.10:11211"};            Initialize the Socketio pool string poolname = "Mypool";            Sockiopool Sockiopool = sockiopool.getinstance (poolname);            Add Server list sockiopool.setservers (serverlist);            Set initial number of connection pools sockiopool.initconnections = 3;            Set Connection pool minimum number of connections sockiopool.minconnections = 3;            Set Connection pool maximum number of connections sockiopool.maxconnections = 5;            Sets the socket timeout for the connection (in milliseconds) sockiopool.socketconnecttimeout = 1000;            Set socket time-out (in milliseconds) Sockiopool.sockettimeout = 3000;            Set the sleep time for the maintenance thread to run: If set to 0, then the maintenance thread will not start Sockiopool.maintenancesleep = 30;            Set the fault flag for the Sockio pool Sockiopool.failover = true; Whether to start so with the Nagle algorithmCkiopool.nagle = false;            Formal initialization of container sockiopool.initialize ();            Get memcached Client instance memcachedclient memclient = new Memcachedclient ();            Specifies that the client accesses the Sockio pool memclient.poolname = poolname;            Whether to enable compressed data: If compression is enabled, data compression longer than the threshold data will be stored in the compressed form memclient.enablecompression = false;            Console.WriteLine ("----------------------------Test begins----------------------------");            01. Simple Add and read operation Memclient.set ("Test1", "Edisonchou");            Console.WriteLine ("Test1:{0}", Memclient.get ("test1"));            02. First add after modify re-read operation Memclient.set ("Test2", "Jacky");            Console.WriteLine ("Test2:{0}", Memclient.get ("test2"));            Memclient.set ("Test2", "Edwin");            Console.WriteLine ("Test2:{0}", Memclient.get ("test2"));            Memclient.replace ("Test2", "Lousie");            Console.WriteLine ("Test2:{0}", Memclient.get ("test2")); 03. Determine if the key value exists if (MemCLient.            Keyexists ("Test2")) {Console.WriteLine ("Key:test2 is existed");            }//04. Delete the data memclient.add of the specified key value ("Test3", "memcached");            Console.WriteLine ("Test3:{0}", Memclient.get ("test3"));            Memclient.delete ("Test3");            if (!memclient.keyexists ("Test3")) {Console.WriteLine ("Key:test3 is not existed");            }//05. Set Data Expiration Time: 5 seconds expires Memclient.add ("Test4", "Expired", DateTime.Now.AddSeconds (5));            Console.WriteLine ("Test4:{0}", Memclient.get ("test4"));            Console.WriteLine ("Please waiting The sleeping Time");            System.Threading.Thread.Sleep (6000);            if (!memclient.keyexists ("test4")) {Console.WriteLine ("Test4 is expired");            } Console.WriteLine ("----------------------------Test completed----------------------------");           Close Sockio pool Sockiopool.shutdown (); Console.readkey (); }

Here, let's carefully analyze this magical code:

① first defines a string array to record the IP and port information of the memcached server , it is important to note that if you have multiple memcached servers, you need to use commas to separate them, for example: " 192.168.80.10:11211 "," 192.168.80.11:11211 "," 192.168.80.12:11211 ";

②sockiopool is a socket (socket) based connection pool, in a different way understand: memcached is actually a socket server side , it constantly receives memcached client sent read and write request command. Here we use Sockiopool.getinstance ("Mypool") to get a connection pool instance named Mypool, and seeing the static method of getinstance (), we know that this is a singleton pattern. Later we configured the accessible memcached server list, the number of connections, socket timeout, and so on, and finally called the Initialize () method to formally initialize the connection pool, waiting for the connection of the subsequent client;

PS: God horse is Socket? We can understand through a scene in life: if you want to call a friend, pick up the phone first dial, the friend hears the telephone ring to lift the telephone, then you and your friend establishes the connection, can speak. Wait until your communication is over and hang up the phone to end the conversation. So, the phone here is a socket, you call the equivalent to apply for a socket, tell the socket you want to call who (the other's phone number you know beforehand). Then, you talk to each other, which is equivalent to sending data to the socket and receiving data from the socket. Finally, after the call ends, one side hangs up the phone, which is the equivalent of closing the socket and revoking the connection.

During the connection of a computer network, the client socket typically records the server host's IP address, port number, and then connects to the server and sends and accepts data. While the server-side open a listening service, it is equivalent to using the socket to specify the listening port, and then wait for the client connection, after the client connected to produce a session. When the session is complete, the connection is closed.

③ creates a new Memcachedclient (memcached client) object and specifies the name of the socket connection pool to which you want to connect, setting whether compression is enabled (set to false here). Here we understand why to set whether to use compression: In the memcached, the data is stored in the form of key/value pairs, the length of the key is limited, memcached server internal limit key is 250 characters, this length is absolutely enough, It is recommended not to exceed the maximum length and try to control it below 200 characters. In fact, we are most concerned about the value of the limit length, the value of the limit size of 1MB, then if sometimes more than 1MB what to do? At this point, you may be able to use compression, if it is less than 1Mb after compression, it can be stored in the key. However, if it is more than 1Mb after compression, it may be split into multiple keys.

PS: Key cannot have spaces and control characters. It is recommended to use a shorter key to conserve server memory and network bandwidth. In addition, the most important point is: key cannot be repeated!

④ uses the client to provide us with a variety of read and write API methods for reading and writing tests, such as set, Get, Replace, add can be added and modified data, and keyexists can determine whether the server contains the specified key data, Delete provides an interface for deleting the specified key. Here, you can understand by looking at the code, I don't have much nonsense. It is useful to note that there is an optional parameter to the data expiration time that is invalidated when the data is stored in the server for a certain amount of time.

(4) Now we are going through the debugging to see the result of this piece of code:

2.2 Advanced memcached Client operations

(1) Clone the existing Windows Server in the virtual machine and set the two server names as: MemcacheServer1 and MemcachedServer2, The IP address is set to: 192.168.80.10 and 192.168.80.11, test whether two virtual machines and host can ping each other, to build a memcached server cluster to make a minimal preparation;

(2) Since we have two memcached servers, we have to try memcached cluster Ah, because memcached's cluster is implemented on the client, we just need to add the server's IP address and port number to the server list of string array. So, we modify the above code:

① first create a new app. config file, add a appsetting entry as follows: Generally speaking, the server address information is written in the configuration file, in order to pursue the standard, we also write in the configuration file inside

② redefine serverlist: Use the value inside the configuration file; It is important to note that to use the ConfigurationManager class, you need to add a reference to the System.Configuration DLL in the reference ;

    string[] serverlist = configurationmanager.appsettings["Memcachedservers"]. Split (', ');

(3) Now we restart the Memcached1 (192.168.80.10) memcached service, emptying the cached data content, making sure that both servers now have no data, and then rerun the code, complete the code test again, and test the results as described in the two memcached sets we configured The cluster has been configured successfully.

(4) Use Telnet in the virtual machine to see which Key/value is saved by each server, because Test3 and test4 are either deleted or invalidated, so you only need to look at the first two key/value pairs:

①memcacheserver1 (192.168.80.10): Saved the second key/value to,<test2:lousie>

②memcacheserver2 (192.168.80.11): Saved the first key/value to,<test1:edisonchou>

(5) To this, we have completed a minimal memcached cluster read and write test demo. However, in a real-world development scenario, much more than just storing a string is more of an instance object that stores a custom class. This requires serialization, so let's add a new class MyObject, which is stored in memcached as a serializable object. Note: You need to add [Serializable] features to this class!

    [Serializable]    public class MyObject    {public        int ID        {            get;            Set;        }        public string Name        {            get;            Set;        }    }

Then, add the following lines of code in the main code to increase read and write testing of the custom object:

            06. Custom Object Storage            MyObject myObj = new MyObject ();            Myobj.id = 12138;            Myobj.name = "Edison Week";            Memclient.set ("Test5", MYOBJ);            MyObject newmyobj = Memclient.get ("Test5") as MyObject;            Console.WriteLine ("Hello,my ID is {0} and Name is {1}", Newmyobj.id, Newmyobj.name);

Finally, run the code to see the results as follows:

(6) How to complete the read and write operation of the custom object successfully? Now, let's see what server this custom object is on: After querying, Test5 is stored on MemcacheServer2 (192.168.80.11).

Third, look back again memcached data access model

After a series of practical operations, we tested the read and write operations on a minimized memcached cluster built by two Windows servers. So, we can't help but want to see how the memcached data access? Don't worry, now let's take a look at it, from practice to theory, in-depth understanding.

(1) Adding new key-value pairs of data

As you can see, memcached, although called a "distributed" cache server, does not have a "distributed" function on the server side, but is fully implemented by the client library. There is no connection between the server and the data access is implemented by the client's algorithm. When the client wants to access the data, it first finds the server hash list maintained by the algorithm, finds the corresponding server, and then saves the data to the specified server. For example: In the application to add a < ' Tokyo ',data> key value pair, it with the set operation submitted to the memcached client, The client calculates a server address from the server list by a certain hash algorithm (e.g., a general redundancy function or a strong consistency hash algorithm), and then stores the key value pair in the computed server.

(2) Gets the existing key-value pair data

The application wants to get key for ' Tokyo ' (It is so hot, but also to take its value is the dry god horse?) ), it submits a GET request to the memcached client, and the memcached client queries the server list for which server has the key ' Tokyo's value (that is, choose which server is just set to), if found, then the search to the server requested to return key ' Tokyo ' data.

(3) memcached distributed core- consistency hash algorithm

Consistent hash algorithm is the core of the distributed cache theory, I also learned not to go deep, but just learned a bit, I have time to learn a bit later, and then write a separate article blog introduce it, and use C # to roughly implement this algorithm. Now I would like to briefly introduce, in fact, this part of the content I have written in my other blog "Large Web site technology Architecture reading notes of the Web site's scalable architecture", interested friends can also go to see this article.

First, the Simple routing algorithm (by using the remainder hash) does not meet the need for server expansion when the business grows: the cache hit ratio drops . For example, when 3 servers are scaled up to 4, using a common remainder hash algorithm can cause about 75% (3/4) of cached data to fail to hit correctly, which increases linearly as the size of the server cluster grows. Then, as you can imagine, when a server is added to a cluster of 100 servers, the probability of not hitting is probably 99% (n/n+1), and the result is obviously unacceptable. So, can we improve the routing algorithm so that the newly added server does not affect the correctness of most of the cached data? Take a look at the following consistent hash algorithm.

The consistent hash algorithm implements a hash mapping of the key to the cache server through a data structure called a consistent hash ring, as shown in:

The specific algorithm process is:

① first constructs a length of 0~2^32 (2 of the Power of 32) of the integer ring (also known as: consistency hash ring), according to the hash value of the node name to place the cache server node in this hash ring, such as in the Node1,node2, etc.;

② calculates its hash value according to the key value of the data that needs to be cached, such as "key" in the right half, and calculates its hash value from node2 very close;

③ in the hash ring clockwise to find the hash value of this key is the closest cache server node, complete the key to the server hash mapping lookup, such as the right side of the hash value of this key is the nearest clockwise server node is Node2, so this key will read the data in Node2 ；

When a cache server cluster needs to be scaled up, only the hash value of the newly added node name (such as NODE5) is placed in the consistency hash ring, since key always finds its closest node clockwise, so the newly joined node affects only part of the entire ring . As shown in, add node5, only affect the right counterclockwise direction of the three key/value to the data, only a small portion of the entire hash ring.

Therefore, we can compare with the previous common remainder hash: When the consistency hash algorithm is used, when 3 servers are scaled to 4, the probability of hitting the original cache data is 75%, which is much higher than the average remainder hash of 25%, and with the larger cluster size, The probability of continuing to hit the original cached data will also increase. When 100 servers increase by 1, the probability of continued hits is 99%. Although there is still a small portion of the data cache that cannot be read in the server, this ratio is small enough to access the database without causing a fatal load on the database .

Iv. Summary of Learning

In this article I first spent a great effort on how to use the Memcached client to perform common basic read and write operations in. NET and build a minimized memcached server cluster of two Windows servers through VMware Workstation. Second, I call the memcached client by using C #, save the data to the Memcached server cluster, and verify that it is saved in the cluster. Finally, we return to Memcached's data access model, from theory to practice, then back to theory, understand memcached's non-communication cluster mode and data reading and writing process, and simply understand the most core algorithm in distributed technology: consistent hash algorithm.

Unconsciously almost 1:20, today to stop the pen to shut down the machine, wash and sleep. Later, I'll show you the case of applying memcached in ASP. NET MVC to resolve the login status, that is, the distributed storage of the session object. If you feel useful or interested, then please look forward to, please also trouble point a "recommendation", let me more motivated to write down, thank you!

Reference documents

(1) Preach intelligence podcast Mullen, "Memcached Open Class", http://bbs.itcast.cn/thread-14836-1-1.html

(2) Charlee, "memcached Complete Analysis", http://kb.cnblogs.com/page/42731/

(3) Small city years, "Introduction to Distributed Cache Memcached", http://www.cnblogs.com/mecity/archive/2011/06/13/Memcached.html

(4) Water absorption technology point, "distributed cache system memcached Introduction and practice", http://www.cnblogs.com/zjneter/archive/2007/07/19/822780.html

(5) Source Studio, "Unlocking the veil of socket programming", http://goodcandle.cnblogs.com/archive/2005/12/10/294652.aspx

Accessories download

(1) Memcached clientlib:http://pan.baidu.com/s/1w9q8i

(2) Memcachedclientdemo:http://pan.baidu.com/s/1hqrduss

A tentative study of the King of Key/value memcached: Two, memcached basic operations in. Net